Today, spinning up a virtual machine in the cloud takes only a few seconds. For most developers, it feels natural, almost invisible, that hardware resources can be sliced, abstracted, and shared at the click of a button.

But the idea of virtualization is much older than cloud computing. It dates back to the 1960s, when computing resources were scarce, expensive, and shared among users who had to wait their turn.

Just as Edgar Codd’s relational model redefined how we structure data, and Roy Fielding’s REST reframed how we exchange it, virtualization reimagined the very foundation of how we run it. It turned computing from a physical constraint into an abstraction, the foundation upon which cloud infrastructure, containers, and serverless computing would later be built.

This article covers:

• The origins of virtualization;

• The technical and economic conditions that led to its rebirth;

• The engineering breakthroughs that made it possible.

Time-Sharing and Mainframes

Virtualization began not as a convenience but as a necessity. In the 1960s, computers were enormous mainframes that cost millions of dollars. Only universities and large corporations could afford them, and they needed ways to share computing power among dozens of users at once.

IBM pioneered this concept with systems like CP-40 and CP-67, which allowed multiple virtual machines to coexist on a single mainframe. Each user believed they had a full computer, complete with its own operating system, but they were actually running in isolated environments managed by a control program.

In 1972, IBM released VM/CMS, a milestone in computing history. It formalized the idea that a single physical machine could host many virtual ones. This model introduced the concept of full virtualization, simulating complete hardware environments, and set the stage for decades of innovation.

The motivation was simple but powerful: maximize the use of extremely expensive resources, provide isolation between workloads, and allow experimentation without risk.

By the 1980s, however, the computing landscape had shifted. The personal computer revolution had begun, and hardware became cheap enough that individuals could own their own machines. The need to share a single computer among many users faded.

Virtualization entered a quiet period. The industry’s focus moved toward multitasking operating systems and networked PCs. UNIX, Windows, and macOS offered process isolation and scheduling, which was “good enough” for most organizations.

In academic and enterprise labs, though, the idea of virtualizing systems never disappeared. Researchers continued exploring ways to abstract hardware, particularly for performance and scalability, laying the groundwork for what would later become the second age of virtualization.

Hardware Abstraction Returns

By the mid-1990s, data centers were facing a new kind of inefficiency. Enterprises were running hundreds of physical servers, each operating at 10–20% capacity. This “server sprawl” was expensive and hard to manage. Companies needed a way to consolidate workloads, improve utilization, and reduce hardware costs.

At the same time, research in system abstraction was making significant strides. In 1997, the paper “Disco: Running Commodity Operating Systems on Scalable Multiprocessors” by Edouard Bugnion, Scott Devine, and Mendel Rosenblum introduced a new perspective on virtualization for modern hardware.

The Disco project, developed at Stanford, proposed a thin virtual machine monitor (VMM) that could run multiple copies of commodity operating systems on a multiprocessor system. Each OS instance believed it controlled the hardware, while in reality, the VMM multiplexed CPUs, memory, and devices across them.

What made Disco special wasn’t just its performance, it was its vision. It treated the VMM not as an emulation layer, but as a resource manager. It could dynamically allocate CPU and memory to workloads, isolate failures, and support scalability on shared-memory multiprocessors.

That idea of using virtualization as a flexible abstraction layer for commodity systems directly influenced the near future.

The VMware Breakthrough

Two years later, in 1999, several of the same researchers behind Disco founded VMware and published the paper “Bringing Virtualization to the x86 Architecture with the Original VMware Workstation”. This work brought the concepts pioneered in mainframes and research labs into the mainstream world of PCs.

The challenge was formidable. The x86 architecture, designed for desktop computers, was not virtualizable by classical definitions. Certain privileged instructions behaved differently depending on the CPU’s mode, making it impossible to trap and emulate them cleanly.

VMware’s solution was brilliant: a combination of dynamic binary translation and direct execution. Instead of running guest operating systems directly on the hardware, VMware’s hypervisor scanned and modified their code on the fly. Privileged instructions were replaced with safe equivalents that could run under host control, while non-privileged code executed natively on the CPU.

This approach preserved both transparency (no OS modifications required) and performance (most instructions executed at full speed).

As the authors wrote:

“We use dynamic binary translation to modify guest instructions that access privileged state, ensuring that the virtual machine monitor maintains full control of the hardware.” (Bugnion, 1999)

VMware also introduced the hosted architecture: the hypervisor ran on top of a standard operating system (like Linux or Windows), making it accessible to any developer without specialized hardware. This architectural decision was pivotal, since it democratized virtualization.

Within months, VMware Workstation made it possible for developers to run multiple OSes side by side on a laptop. For enterprises, it marked the start of true server consolidation: multiple workloads, isolated and portable, running efficiently on commodity hardware.

From Workstation to Data Center

The success of VMware Workstation started a wave of innovation. Enterprises quickly realized that virtualization could solve not just developer convenience, but large-scale infrastructure problems.

Virtualization improved resource utilization, reduced hardware costs, and enabled rapid provisioning. Most importantly, it introduced abstraction between workloads and hardware, a concept that would become fundamental to cloud computing.

VMware’s work soon evolved into ESX Server, a bare-metal hypervisor that ran directly on hardware, eliminating the overhead of a host OS. Other players entered the market: Xen, Microsoft Hyper-V, and open-source projects like KVM extended virtualization into new domains.

By the early 2000s, virtualization was no longer a lab curiosity, it was a production necessity.

Virtualization and the Birth of the Cloud

In 2006, Amazon Web Services launched EC2, the first large-scale commercial cloud built entirely on virtualization. Each EC2 instance was a virtual machine, dynamically provisioned from a pool of servers.

The key properties of cloud, elasticity, scalability, and multi-tenancy, all depended on virtualization. By decoupling applications from hardware, AWS could sell computing as a service.

This was the moment virtualization transcended its technical origins. It became an economic model: compute as an on-demand utility.

From VMs to Containers and Beyond

Virtual machines were transformative, but not perfect. Each VM required its own operating system, consuming significant memory and storage. Startup times were slow compared to the speed of modern software delivery.

In response, engineers turned to containers, lightweight environments that isolate applications at the OS level. Technologies like LXC and later Docker (2013) offered the benefits of virtualization, with isolation, portability, reproducibility, without the heavy footprint.

Kubernetes (2014) extended this model at scale, orchestrating thousands of containers across clusters. Interestingly, most of these container workloads still run on top of virtualized infrastructure. Virtualization remained the invisible layer of the cloud.

Lasting Lessons

The history of virtualization teaches recurring truths about computing:

1. Abstraction is the ultimate scaling mechanism. By separating logic from hardware, we gain flexibility, efficiency, and resilience.

2. Constraints inspire innovation. VMware’s biggest breakthrough came precisely because the x86 architecture was not designed for virtualization.

3. Old ideas resurface in new forms. The same isolation IBM introduced in the 1970s reappeared decades later in data centers and cloud environments.

Conclusion

Like the relational model and REST, virtualization was born out of necessity, to make computing more efficient, more accessible, and more adaptable. It began as a way to share expensive mainframes, reemerged to solve server sprawl, and ultimately became the foundation of the cloud.

As new paradigms emerge, edge computing, serverless architectures, confidential VMs, and the principles remain the same: isolate, abstract, and share efficiently.

Virtualization reminds us that progress in computing often comes not from adding complexity, but from rethinking boundaries.

References

• Bugnion, E., Devine, S., Rosenblum, M., Sugerman, J., & Wang, E. (1999). Bringing Virtualization to the x86 Architecture with the Original VMware Workstation. VMware, Inc.
  PDF link

• Bugnion, E., Devine, S., Govil, K., & Rosenblum, M. (1997). Disco: Running Commodity Operating Systems on Scalable Multiprocessors. Stanford University.
  PDF link

By the end of my sixth month, I felt ready to return. That meant applying, interviewing, and once again sitting on the other side of the table. What followed was a series of processes that were as revealing as surprising.

Some left me impressed, others disappointed, and many raised questions about how we treat people in such a critical moment, when they’re considering joining the company.

In this article, I will cover:

• Highlights and lowlights of several interviewing processes I ran into;

• What you should do as a team leader (not in HR);

• and what HR should take into consideration.

The basics first: what defines a good interviewing process?

Before sharing stories from my recent months of interviews, it’s worth taking a step back. What makes an interview process good in the first place?

A good process is not about trick questions, endless rounds, or proving how “selective” a company can be. It’s about creating clarity, respect, and fairness for both sides. Research consistently shows that structured and transparent interviews lead not only to better candidate experiences but also to more reliable hiring decisions1.

The fundamentals include:

• Clarity of purpose: every stage must have a reason. If you can’t explain why a round exists, it probably shouldn’t exist.

• Consistency and structure: when interviewers are aligned and all candidates are asked the same role-relevant questions, bias is reduced and reliability improves. Unstructured interviews, by contrast, are among the weakest predictors of job performance2.

• Respect for time: processes should keep to schedules, avoid last-minute cancellations, and acknowledge the candidate’s investment. Lack of closure — never hearing back after applying or interviewing — is one of the clearest signs of poor process.

• Feedback and closure: every candidate deserves to know where they stand. Even a simple rejection email is better than silence, which communicates indifference and damages the company’s brand.

• Transparency and expectation management: timelines, criteria, and outcomes should be communicated clearly. Overselling a role or creating unnecessary highs that collapse later is worse than being upfront from the start.

• Adaptability: while structure is essential, conversations should flex to the candidate’s background. Experienced candidates won’t fit neatly into a script.

• Trained interviewers: interviewing is a skill. Less experienced interviewers should be supported with clear rubrics, preparation, and pairing with senior colleagues to avoid superficial evaluations.

When these basics are present, interviewing shifts from a filtering mechanism to a meaningful two-way evaluation, one where both the company and the candidate can genuinely decide if they are the right fit for each other.

When the basics break down

With these principles in mind, my own experience re-entering the market showed just how often they are overlooked, and how much difference it makes when they are respected.

To give you a sense of scale, here’s how my interviews spread across company types:

• Big Tech (10,000+ employees): 3 interview processes;

• Large companies / global players (1,000+ employees): 7 interview processes;

• Mid-size scale-ups (300–999 employees, high growth): 6 interview processes;

• Startups / small organizations (~300 employees, often with founders involved): 5 interview processes;

I had 4 offers in total, from different clusters.

Each cluster carried its own patterns. Big Tech processes were highly structured and consistent. Large companies varied, some had strong frameworks, others showed cracks in alignment. Scale-ups often moved fast but at the cost of structure. And startups were the most personal, but also the least predictable.

Let’s jump into the facts.

What the clusters revealed

Big Techs: the gold standard
(10,000+ employees)

Big Techs were, by far, the best places to interview. Structured, consistent, and respectful processes. Every step had a clear purpose, interviewers were aligned, and expectations were well communicated. Even in rejection, you left with the sense that your time was valued and that the company had its process under control.

Highlights:

• Process stages presented upfront with absolute clarity.

• Immediate and constant availability of the recruiter throughout the process.

• Interviewers were well-trained regarding timing, questions, and follow-up discussions.

• Clear timelines, follow-ups exactly within the expected timeframe, and detailed feedback afterward.

Lowlights:

• Processes sometimes felt too standardized, highly trained but at times overly scripted or even artificial.

Large companies: trying to balance scale and humanity
(1,000+ employees)

Large companies often sat in the middle. Some leaned closer to Big Tech in structure, others closer to scale-up chaos. In general, they seemed to be searching for the right balance between standardization and a human touch. Sometimes it worked, and sometimes it slipped into bureaucracy or inconsistency.

Highlights:

• Clearer processes, with phases presented upfront.

• Ongoing communication and availability of someone to support the candidate throughout.

• Supporting materials: study suggestions, detailed explanations about each phase, and structured feedback afterward.

Lowlights:

• Interviewers with some training but limited experience.

• Lack of adaptability during the interview. Some interviewers followed the script strictly but couldn’t adjust on the fly to the candidate’s seniority or direction of the conversation.

• Occasional false highs: moments where enthusiasm or premature optimism created expectations of an imminent offer, only for the process to end abruptly without clear closure. Recruiting shouldn’t be about creating excitement, it should be about building alignment and trust.

Mid-size scale-ups: the weakest link
(300–999 employees, high growth)

Mid-size scale-ups were, in my experience, the weakest link, mostly due to inconsistency. Repeated questions, shifting timelines, and sometimes outright disrespect. Many of these companies are growing fast, but speed without structure often turns into chaos for candidates.

Lowlights:

• Lack of clarity about the goal of each interview stage.

• Interviewers clearly had little or no formal training.

• Conversations were fluid, which can seem positive at first, but often missed essential evaluation points.

• In several cases, interviewers seemed to be chosen mainly for their seniority or tenure within the company, but not necessarily for their interviewing skills. The result was visible: unstructured conversations, biased assessments, and a lack of depth in evaluating technical and leadership capabilities. When seniority replaces preparation, the process risks becoming a reflection of internal hierarchy rather than a genuine assessment of talent.

Startups and small organizations: authentic but unpolished
(~300 employees)

Overall, startups and small organizations were a genuinely good experience. The processes were more personal, often involving founders or long-time team members. While these interviews lacked polish and standardization, they made up for it with authenticity and transparency. You could feel the company’s culture in every exchange.

Highlights:

• More intimate processes, focused on the human side, showing genuine care for the candidate.

• Participation of founders or long-standing employees, bringing context and stories that help you understand the company’s reality.

• Openness and full transparency about what matters most to the candidate.

Lowlights:

• Unpredictable process length/unclear number of steps or interviews.

• Time was not always respected, mostly because conversations lacked structure and naturally took longer than necessary.

What you can do with this information

Observing these patterns across more than twenty interviews made one thing clear: the quality of your hiring process says more about your organization than any mission statement or culture deck ever will.

But what can you actually do with that insight? Here are two perspectives, one for team leaders, and another for HR leaders, because improving interviews is everyone’s job.

If you’re a team leader

Team leaders are often the face of the company during interviews. Candidates will remember how you listened, how you structured the conversation, and how transparent you were.

• Be intentional about structure. Don’t just show up and go in without intention. Know what you’re evaluating and why. If your company doesn’t provide a structured framework, create your own rubric — even a simple one helps ensure fairness.

• Respect time and follow up. Candidates are giving you their focus and emotional energy. Always close the loop, even when the outcome is negative.

• Prepare beyond the CV. Read the candidate’s background before the call and tailor your questions. Senior candidates, especially, can sense when you’re improvising too much.

• Balance listening and probing. The goal isn’t to trap the candidate, it’s to learn how they think and how they’d fit your context.

• Represent the culture you want to see. The way you interview sets the tone for how your future team will feel.

In short, interviewing isn’t a side task, it’s part of leadership. Treat it as such.

If you’re an HR or talent leader

For HR and talent leaders, interviews are the front door of the company’s brand. Every touchpoint, from scheduling to rejection, tells a story about who you are.

• Audit your process. Map every stage, understand what each one is supposed to evaluate, and remove redundancy. If two rounds cover the same thing, that’s a signal of misalignment.

• Train your interviewers. Don’t assume people “just know how” to interview. Provide guidance, calibration sessions, and feedback. Interviewing is a skill, it requires deliberate practice.

• Ensure feedback and closure. Silence is not neutral, it’s a form of disrespect. Automate rejections if needed, but never leave candidates without closure.

• Measure candidate experience. Gather feedback from candidates, not only those who got offers. It’s the most honest data you’ll get about your process.

• Balance automation with humanity. Technology can streamline hiring, but no tool replaces empathy, clarity, and respect.

A mature hiring process isn’t about being perfect, it’s about being deliberate. Companies that care about how they interview show, in practice, that they care about people.

Culture in action

Looking back, the contrast between these experiences made one thing clear: interviewing is not just a hiring tool, it’s a cultural signal.

The way you design, structure, and run your hiring process communicates your culture more clearly than any “About Us” page or beautiful video in Linkedin. Respect, clarity, and adaptability aren’t luxuries, they are culture in action.

If companies want to attract strong talent, they need to treat interviews as more than a filter. They need to treat them as a mirror.

Introduction

Recently, in one of my consulting projects, I had to integrate a CRM system with an outdated Magento platform that still relied on a SOAP integration. It reminded me how painful it used to be, 20+ years ago, when our only choices were heavy, rigid integration methods that required verbose XML, complicated schemas, and endless debugging sessions.

Thanks to Roy Fielding, that’s not our current reality.

In this article, I’m going to cover:

• What were the options to create distributed applications and integrations in early 2000;

• The main issues that Roy Fielding addressed through his doctoral dissertation;

• What lessons we can take from the way he thought about it.

The Context Before REST

In the late 1990s, the World Wide Web had exploded in popularity, yet the underlying model of interaction was poorly understood. HTTP was already widespread, but primarily treated as a transport mechanism for documents. Meanwhile, distributed object technologies such as CORBA, DCOM, and RMI were attempting to dominate the conversation on how systems should communicate.

From a software architecture perspective, these technologies shared common ground: they all attempted to abstract distributed communication into object-oriented calls, hiding the network behind familiar programming models. CORBA allowed invoking methods on remote objects as if they were local. DCOM extended the Microsoft COM model across machines. RMI let Java applications call methods on objects located in other JVMs.

While elegant in theory, this approach assumed homogeneity, required complex middleware, and often resulted in complicated integrations. The network was abstracted away, but at the cost of interoperability, scalability, and simplicity.

SOAP, introduced in 1999, took a slightly different path by leveraging XML over HTTP. But despite its promise of platform independence, SOAP still carried the same mindset: remote calls wrapped in complex envelopes, schemas, and contracts. It treated HTTP as a tunnel rather than embracing its inherent semantics.

These approaches all shared one thing: they were built with the mindset of making the network invisible. But the web was proving that exposing the network, and designing with its constraints in mind, was the key to global scale.

A Timeline of the Late 1990s

To better understand why REST emerged, it helps to place it against the chronology of distributed system technologies:

• Early 1990s – The web takes shape with HTTP 0.9/1.0 and HTML, primarily serving static documents.

• 1994–1997 – Distributed object models rise: CORBA gains traction in enterprise systems, Microsoft pushes DCOM, and Java introduces RMI. The promise is transparency of remote method calls, but complexity and interoperability issues grow.

• 1997–1998 – The web grows explosively, challenging existing models of integration. Developers begin tunneling RPC-style calls over HTTP to bypass firewalls and proxies.

• 1999 – SOAP emerges, promoted by Microsoft and IBM as a standardized XML-based messaging protocol over HTTP. It simplifies some cross-platform concerns but adds layers of verbosity and rigidity.

• 2000 – Roy Fielding publishes his dissertation at UC Irvine, proposing REST as a distinct architectural style—leaning into the constraints of the web rather than hiding them.

This timeline shows the shift: from early web simplicity, to the complexity of distributed objects and SOAP, and then back to a simpler, constraint-based approach with REST.

What Roy Fielding Saw

Roy Fielding, deeply involved in the development of HTTP and the early web standards, noticed this mismatch. The web had succeeded not because it reinvented distributed computing, but because it relied on a few simple, robust principles:

• URIs as a global addressing mechanism;

• Stateless interactions that made each request independent;

• Caching as a built-in efficiency layer;

• A client–server separation that allowed independent evolution.

What the web lacked was a clear architectural model that explained why these choices worked and how they could be extended to future systems.

As Fielding put it:

“The World Wide Web has succeeded in large part because its software architecture has been designed to meet the needs of an Internet-scale distributed hypermedia system.” (Abstract)

His dissertation, published in 2000, was his attempt to codify the architecture of the web. Rather than following the object-oriented, RPC-based mindset of CORBA, DCOM, RMI, or SOAP, Fielding proposed a different path: embrace the web’s constraints instead of hiding them.

REST as a Historical Response

It’s important to see REST not just as a checklist of constraints, but as a historical response to the limitations of the time. Distributed object systems promised transparency, but they collapsed under the realities of latency, heterogeneity, and scale. SOAP attempted to patch this by standardizing messages, but it still carried complexity and rigidity.

REST, in contrast, was about working with the network as it is. By deliberately constraining design, it enabled:

• Scalability through statelessness;

• Evolvability through a uniform interface;

• Interoperability through resources identified by URIs;

• Resilience through layered intermediaries like proxies and caches.

The contrast was clear: while CORBA or SOAP required developers to agree on detailed contracts before integration, REST allowed systems to interact dynamically through standard operations and representations. Where the other options tried to hide the web, REST leaned into it.

Adoption of REST

REST’s adoption was gradual. In the early 2000s, most enterprises still relied heavily on SOAP, especially in large corporate and government integrations where WS-* standards promised reliability and security.

REST, meanwhile, spread organically through the open web. Companies like Amazon, eBay, and later Twitter and Facebook began exposing public APIs that leaned on REST principles. The simplicity of URIs and HTTP verbs made it accessible for web developers who were already fluent in these technologies.

By the late 2000s, REST had become the dominant style for web APIs, while SOAP retreated into more specialized enterprise use cases. This divergence highlighted a cultural as much as a technical shift: developers favored pragmatic, lightweight integrations over rigid, contract-driven systems.

Lessons Beyond REST

Looking at REST in this historical context, its greatest contribution was not just a new way of designing APIs, but a shift in mindset:

• Constraints as enablers: by limiting design choices, REST created systems that could scale to Internet levels.

• The web as the platform: instead of abstracting away the network, REST built directly on top of it.

• Simplicity over complexity: REST thrived where object-distributed models struggled, because it embraced the reality of diversity and failure in the network.

More than two decades later, many of the technologies that once competed with REST have faded into history, while REST remains a baseline for how we think about integration.

Emerging Alternatives After REST

The dominance of REST eventually sparked new explorations. By the mid-2010s, developers were facing new challenges, like mobile clients with bandwidth constraints, applications requiring complex queries, and real-time updates that REST was not designed to handle elegantly.

This led to the emergence of GraphQL (2015), which prioritized flexible queries and minimized over-fetching; gRPC, which focused on performance and type safety through Protocol Buffers, and event-driven or streaming APIs using WebSockets and Kafka.

Each of these approaches can be seen as a response to specific limitations of REST, while still building on the lessons of constraint-driven architecture and interoperability.

Together, they show how REST provided a foundation, but not the final word in distributed system design.

Conclusion

Roy Fielding’s dissertation did not just propose an architectural style, it documented a turning point. It captured the lessons of the early web and set a direction away from the complexity of distributed objects and RPC toward a simpler, constraint-driven model.

As Fielding reminded us:

“A good architecture is not created in a vacuum. All design decisions at the architectural level should be made within the context of the functional, behavioral, and social requirements of the system being designed.” (Chapter 1)

The context of the late 1990s with SOAP, CORBA, DCOM, RMI, and the growing pains of the web, was exactly that environment. REST was his answer. And it remains a powerful reminder that the best architectures are those that embrace their constraints, not those that try to escape them.

When I was working at VTEX, discussing about prioritizing and implementing the Delivery Promises feature, as the engineering leader I was advocating for getting enough data to support the decision whether we should or should not prioritize the initiative.

But how can we come up with the numbers to justify something so complex?

And the results were obvious: of course, doing something that anticipates customers’ expectations would result positively. I have seen that before, there’s a clear pattern easily recognizable. Maybe the discussion should be how much we should invest, but not if we should initially invest in that.

That’s a good example of when intuition wins over analytics, while analytics support and refine the decision over time.

Today, intuition is at risk. As AI takes on more operational and repetitive tasks, many companies are reshaping or even eliminating entry-level roles. This raises a critical question: if experience is the raw material from which intuition is created, how will the next generation of professionals build the judgment needed to grow into senior and leadership positions?

First things first: some basic concepts

Intuition is not magic. Gary Klein, in The Power of Intuition, defines it as “the way we translate our experience into action.” It is the product of pattern recognition, built through repeated exposure to real situations, feedback loops, and reflection.

Daniel Kahneman would put it as the System 1 thinking: fast, automatic, and effortless. It’s the firefighter who knows where the fire will spread before seeing the flames, the chess master who sees a checkmate five moves ahead without calculating every branch, the senior engineer who senses a cascading failure from a single metric behaving oddly.

What makes intuition powerful is that it compresses years of experience into a fraction of a second. You don’t need to consciously run the numbers — you just know something is off.

And this is not irrational: research shows that experts often make better, faster decisions under uncertainty precisely because their brains have stored thousands of cases that inform those choices. In fast-moving organizations, this can be the difference between acting in time or missing the opportunity entirely.

The challenge that arises with the evolution of AI

Here’s where the problem emerges. If intuition depends on experience, and AI is now doing the repetitive, entry-level work that used to provide that experience, where will the next generation of professionals get theirs?

Take the example of a junior SRE or support engineer a decade ago: they would triage dozens of tickets per day, learn to read logs, spot patterns, and escalate issues. Over time, they built a mental library of “failure signals.”

Now, with AI handling log analysis, auto-remediating incidents, and summarizing alerts, that engineer might only see the most complex edge cases — without the “baseline cases” that teach them what normal looks like.

This is not just a technical challenge — it’s a leadership pipeline challenge.

Tomorrow’s senior engineers, product managers, and directors will be asked to make judgment calls, but they may lack the accumulated intuition that today’s leaders take for granted.

Gary Klein warns that “intuition is only reliable when it’s based on genuine expertise.” Without the repetitions, expertise never forms, and we risk raising a generation of professionals who are great at asking AI for answers but cannot sense when the AI is confidently wrong.

How to solve it

We can’t simply turn back the clock and remove AI from the equation. But we can be intentional about designing experiences that build intuition.

One solution is Decision-Making Exercises (DMX), structured scenarios where professionals must make choices with partial information, defend their reasoning, and review outcomes. This mirrors what Gary Klein calls recognition-primed decision making, forcing participants to build mental models they can use later.

Another effective method is the same one used in SRE practices: simulations, GameDays, and live drills. Netflix’s Chaos Engineering program is a well-known example, deliberately breaking production-like environments so engineers learn to diagnose, communicate, and recover under pressure. These repetitions create the “muscle memory” that intuition depends on.

The formula is simple:

• Expose people to realistic situations.

• Allow them to fail safely.

• Debrief and reflect so the lesson sticks.

If AI removes the natural volume of learning opportunities, we must create artificial ones, otherwise the intuition skill will indeed disappear.

Food for thought

AI is rewriting the way we work. But intuition, that ability to make timely, high-quality decisions in ambiguity, remains one of the rarest and most valuable human skills.

As we embrace AI, we must ask ourselves:

• Are we still creating paths for people to accumulate the experiences that form intuition?

• Are we prioritizing efficiency at the expense of future leadership capacity?

• Are we willing to slow down enough to let people learn, fail, and grow?

If we fail to answer these questions, we might end up with organizations that are efficient but fragile, dependent on machines that lack judgment and on humans who have never developed it.

Intuition might be disappearing, but it doesn’t have to. We just have to fight for it.

If you’re 40 or under, you have probably started your database-related software development learning path with relational databases. And even more, you have worked with relational databases for most of your career without even thinking about it.

But have you ever thought about how the world was before that? Some contextual information at that time:

• Hierarchical and network data models dominated, requiring engineers to understand every detail of data storage and access paths.

• In the '60s, the cost per megabyte of storage was around $6,000 (yes, six thousand dollars).

• Software changes were complex and manual, requiring planning, coding, and manually compiling on mainframes. Debugging could take days, as any minor error required punching code onto cards and rerunning entire programs.

One strong contributor to high costs on software maintenance was how data was organized and accessible. Data management was an inflexible and intricate process. Changing one element, like adding a field or adjusting a data type, often meant reprogramming large portions of the system, which took a long time and cost.

One example is tree-structured files, a hierarchical data model. Programs that consumed this database model would suffer if the file’s structure changed.

Codd mentioned this kind of challenge as “data independence”:

Independence of application programs and terminal activities from growth in data types and changes in data representation--and certain kinds of data inconsistency which are expected to become troublesome even in nondeductive systems.

Based on all these challenges, he presented a data model in June 1970 that offered something transformative for the time: independence from data storage details.

In his article “A Relational Model of Data for Large Shared Data Banks”, Codd discussed the foundation for relational databases and the concept of a universal data sublanguage, which would become the SQL language.

Codd recognized the issue with this heavy data storage dependency and its correlation with data evolution:

Future users of large data banks must be protected from having to know how the data is organized in the machine (the internal representation).

[…] Changes in data representation will often be needed as a result of changes in query, update, and report traffic and natural growth in the types of stored information.

This abstraction was groundbreaking because it removed the need for users and programs to "know" the exact layout or storage mechanisms of data. Instead, they could query data using logical structures without regard for physical order or indexes.

The definition

Codd’s proposal relies on separating the responsibilities of data description from machine representation. This separation could facilitate software maintenance and its impact by allowing users to focus on the necessary parts rather than the whole.

From there, Codd proposed a new way to represent data and its relations through the relational data model.

Codd's concept of a relation is based on the mathematics discipline, alongside concepts such as sets, domains, and degrees, defined in heterogeneous relation.

The idea was that an n-ary relation could be represented through an array of relations. In the example below, there’s a relation “supply” of degree 4, and its rows and domains explicitly identified:

Codd also defined some very familiar concepts throughout his proposal: primary and foreign keys:

One domain (or combination of domains) of a given relation has values which uniquely identify each element (n-tuple) of that relation. Such a domain (or combination) is called a primary key.

We shall call a domain (or domain combination) of relation R a foreign key if it is not the primary key of R but its elements are values of the primary key of some relation S.

Primary keys (PK) and foreign keys (FK) were essential to another concept that Codd introduced: the normal form.

In a nutshell, Codd stated that when in a “complicated data structure”, there was a need to eliminate the nonsimple domains using the process that he called “normalization”, consisting of:

1. Starting with the relation at the top of the tree, take its primary key and expand each of the immediately subordinate relations by inserting this primary key domain or domain combination.

2. The primary key of each expanded relation consists of the primary key before expansion augmented by the primary key copied down from the parent relation.

3. Strike out from the parent relation all nonsimple domains, remove the top node of the tree, and repeat the same sequence of operations on each remaining subtree.

For instance, consider the unnormalized data sets employee, job history, children, and salary history, each with its respective domains:

After applying the normalization form, the set would become:

Of course, this is extremely well-known. Normalization is part of every database course around there. However, at that time, it was crucial to represent complex data structures while keeping the key aspects of the proposal for a relational data model.

The Linguistic Foundations of Relational Databases

The shift to a relational model changed not only how data was organized but also fundamentally how we communicate with data systems. Before Codd's work, interacting with databases required low-level, system-specific commands. The relational model, however, introduced the idea of a universal data sublanguage, which evolved into SQL language.

At its core, this universal language was built on the principles of applied predicate calculus. By employing first-order logic, it offered a framework robust enough to express any valid query or manipulation of relational data.

Once we have characteristics such as domains descriptions, the relations between them, primary and foreign keys, plus normalization process, the use of terms, formulas, quantifiers, and variables, fundamental of predicate calculus, becomes possible, giving space for creating the above-mentioned universal data sublanguage.

The universality of the data sublanguage lies in its descriptive ability.

This was revolutionary: instead of being bound by rigid, system-dependent syntax, users could describe what they wanted without specifying how to get it. Codd’s vision meant that data operations could now focus on intent.

Key features of this linguistic framework included:

1. Logical Independence: Queries could be written abstractly without regard to the underlying physical structure or storage methods.

2. Declarative Nature: Users specify desired outcomes (e.g., "retrieve all suppliers for Project X") without detailing the process.

3. Descriptive Power: The language could handle the complexity of expressing relationships, constraints, and logical dependencies through a single unified syntax.

By separating the description of relationships from their implementation, this approach not only simplified data interactions but also democratized access to powerful database capabilities.

In his paper, Codd didn’t define the SQL language itself. He limited himself to defining the foundation of the data sublanguage but also went through mentioning and relating some operations, such as permutation, projection, and join, that are connected to the usage of predicate calculus and operations regarding sets.

What happened next

Codd’s work was the initial seed for several critical advancements in relational databases, the SQL language, and the establishment of a new paradigm in software development. His paper not only created a revolutionary concept but also set the stage for practical implementations that would define the future of data management.

Following his proposal, Codd continued refining and expanding his ideas, working alongside IBM researchers to transform theory into practice.

One of the most notable projects inspired by his work was System R, an experimental database system developed in the early 1970s. This system was instrumental in proving the viability of the relational model for real-world applications.

System R’s architecture serves as a good example of a real implementation of Codd’s concepts. Codd’s idea of separating data representation from machine representation is clearly stated as the Relational Data Interface (RDI) and Relational Storage Interface (RSI).

From System R’s original paper: “The Relational Data Interface (RDI) is the principal external interface of System R. It provides high level, data independent facilities for data retrieval, manipulation, definition, and control. The data definition facilities of the RDI allow a variety of alternative relational views to be defined on common underlying data”.

And “The Relational Storage Interface (RSI) is an internal interface which handles access to single tuples of base relations. This interface and its supporting system, the Relational Storage System (RSS) , is actually a complete storage subsystem in that it manages devices, space allocation, storage buffers, transaction consistency and locking, deadlock detection, backout, transaction recovery, and system recovery. Furthermore, it maintains indexes on selected fields of base relations, and pointer chains across relations‘.

Codd’s work regarding a “universal data sublanguage” also served as the foundation for SEQUEL (Structured English Query Language), the precursor to SQL. SEQUEL’s design emphasized simplicity and usability, allowing non-specialists to interact with data in previously unimaginable ways.

Building on the success of System R and SEQUEL, IBM released DB2 in the 1980s, the first commercial relational database management system (RDBMS) based on Codd’s relational principles. This marked the beginning of the adoption of relational databases across industries.

Other companies, including Oracle, Sybase, and Informix, followed suit, creating their own RDBMS products that adhered to and expanded upon the relational model. These systems became the backbone of enterprise data management, powering applications from banking systems to e-commerce platforms.

Codd’s influence extended beyond software to shape the culture of database design and development. His principles encouraged a focus on data abstraction, independence, and normalization—concepts that remain at the core of modern database education and practice.

Understanding the Significance of Persuasion in Software Engineering

Consider this scenario: You're a software engineering leader with a game-changing concept. You could march into your team's meeting room and declare your idea as the next big thing. But, what's the likelihood that your team will jump on board with enthusiasm? Probably not very high (see Lack of Ownership).

Chelsea Troy, in her insightful post about "How to Socialize Big Changes at Work" highlights a common pitfall when introducing new ideas to a group:

Presumably, if the whole team is together, they know the topic they’re there to discuss, and they’ve agreed that it’s relevant. You show up and drag the topic onto your new thing. No one in this meeting has had the chance to agree that this thing is relevant.

So you’re co-opting time that people have given to something relevant. Co-opting people’s time without their consent annoys them, at best. It’s an emotional hurdle you don’t need in your effort to get your idea adopted.

In this context, persuasion isn't solely about selling; it's about engaging. Your team and peers need to believe in it, see its potential, and feel motivated to contribute. After all, a great idea is only as good as its execution, and that requires a committed team.

So, how to start? Feedback is the answer.

Initiating the Process with Feedback

One of the first steps in persuading your team to embrace your idea is to seek feedback. This approach may seem counterintuitive, as you might fear that criticism could derail your vision. However, feedback is vital in refining your idea and involving your team in the creative process.

Imagine presenting your idea as a rough sketch and inviting your team to contribute their thoughts. This approach fosters collaboration and helps you identify potential pitfalls early on. When people feel heard and valued, they're more likely to get on board.

In the realm of software engineering, the best way to do this is by using written communication.

By segregating the message's definition from its dissemination, you have the opportunity to meticulously craft your message to ensure that your ideas are conveyed with clarity and precision. This clarity significantly aids your audience in comprehending your perspective.

Also, written materials can be easily distributed to a wide audience, making it an efficient way to reach and persuade multiple individuals or stakeholders simultaneously, regardless of their time zones.

And last, written communication creates a permanent record of your message, which can be referred to later. In his communication course, Pat Kua highlights that writing is important to capture context.

As mentioned in his material, a good example is architecture decision records. This short text file (Michael Nygard’s definition) is used by many companies and teams to document and share the details about the trade-offs and other information that may explain the decisions that someone made over time.

When this does not happen, Pat Kua defines it as “Tribal Knowledge”:

“Tribal Knowledge” sounds fun when you’re sitting around a campfire sharing stories but other cannot benefit if that person with that “Tribal Knowledge” is no longer around.

Another important decision to make while seeking feedback is choosing the right people for it. According to Riitta Katila, your manager is definitely not the ideal candidate for initial feedback on a new idea. You should first look for people with a “creator” mindset, who are more open to fresh ideas, changes, and innovation.

Once you have started or concluded the co-creation phase using the obtained feedback, it is time to spread the message through additional voices.

Finding Internal Sponsors and Allies

Even the most revolutionary ideas need champions within the organization. These champions can be your internal sponsors and allies who support and advocate for your idea. Seek out individuals who share your vision and can help you navigate the complex landscape of organizational politics.

Think of these allies as partners in your journey. They can help you gain the support and resources you need, making it easier to persuade others. Remember, you don't have to go alone.

A productive method for identifying these supporters within your organization is to pay attention to social interactions and identify those who share similar interests. Look for potential allies through your company's Slack channels, internal communication platforms, All Hands meetings, and other opportunities for engagement.

Effective Communication

Communication is the cornerstone of persuasion. You might have a brilliant idea, but it's unlikely to gain traction if you can't articulate it clearly. Craft a compelling narrative around your idea, emphasizing its benefits and value to the team and the organization.

Your goal is to create a shared understanding and enthusiasm for the idea. There is no better strategy for this than demonstrating its value and benefits, connected with your company’s business drivers.

Every idea, no matter how innovative, must demonstrate its worth. Showcase how your idea can address pain points, improve efficiency, or drive revenue. When your team and peers can connect the dots between your idea and tangible benefits, they're more likely to buy in.

Use real-world examples or success stories to illustrate how your idea has worked in similar contexts. People are more inclined to trust and support something with a proven track record.

Dealing with Resistance and Challenges

Resistance is inevitable. Not everyone will immediately embrace your idea, and that's okay. When faced with resistance, approach it as an opportunity for growth and learning.

Listen to concerns and objections with an open mind, and be prepared to adapt your approach. Sometimes, making small adjustments can turn skeptics into advocates. Patience and persistence are your allies in overcoming challenges.

Conclusion

In the ever-evolving world of software engineering, leadership is about more than just making decisions; it's about inspiring and guiding your team toward a shared vision. Persuasion, done with empathy and authenticity, is a powerful tool that allows you to lead with influence, not authority.

Remember, you're not selling a product; you're championing an idea that has the potential to transform your team and your organization. Use feedback, allies, effective communication, and a demonstration of value to win hearts and minds. Embrace resistance as an opportunity for growth, and watch your ideas flourish in the hands of a motivated and engaged team.

In the end, it's not about selling an idea; it's about inspiring a movement.

I remember when I was a teenager and bought my first programming language book (C language). I sat down on my chair, in front of a 386 PC, opened an editor, and started reading the book hoping to write code and see everything working smoothly.

The problem is that it takes more than excitement to make things work, especially if you're taking your first steps in any context.

Stepping into a leadership role amplifies this dynamic. Guiding a team into uncharted technological waters is both invigorating and fraught with potential pitfalls.

Beyond the excitement, it is crucial for any engineering manager to be completely aware of the advantages and disadvantages of being a first-mover, especially nowadays with the efficiency mindset spread around the companies.

🌟 Perks of Being a First Mover

As someone from engineering, there is a personal satisfaction of being back to breaking new ground and harnessing the power of a tool before anyone else.

But beyond that, the innovation can skyrocket your team to unforeseen heights, giving visibility to your team’s ability on discovery and visibility to the team’s members throughout your engineering organization. You and your team become the go-to peeps for insights.

Also, early adoption can shape the tool’s path, potentially even swinging it to align closely with your team's needs. This is especially important if your team will rely on this tool for a long-term basis.

🚧 The Not-So-Glam Side

On the other hand, there is time and uncertainty. Being the inaugural adopters of a technology casts us into the murky abyss of the unknown, where systemic bugs, unforeseen complications, and a potential dearth of troubleshooting resources loom.

These unanticipated complexities can potentially morph into time and resource sinks, directing vital energies away from other pivotal pursuits. Moreover, the relative scarcity of community support and pre-established solutions often means that the team is left to its own devices, navigating through challenges with limited external guidance.

Thus, the audacity to forge ahead as trailblazers also mandates an acceptance of the potentialities of uncharted challenges and solitary problem-solving.

🤹 Juggling Innovation and Stability

The pursuit of equilibrium between innovative zeal and strategic caution forms the bedrock of sustainable first-mover approaches:

• Conducting a meticulous risk assessment, evaluating not only the glittering promises but also potential pitfalls and resource requisites of a new technology, forms the initial step;

• Incremental adoption strategy, where technologies are piloted and evaluated in controlled environments before expansive deployment, can serve as a prudent measure to ascertain their viability and utility;

• Cultivating an organizational culture that marries the boldness of innovation with the wisdom of calculated risk-taking ensures that the journey through unknown technological terrains is both exhilarating and strategically sound.

🎢 The Loop-De-Loop Conclusion

In conclusion, the journey as a first-mover in the software engineering space is indeed a multi-faceted adventure, illuminated by potential innovations and occasionally punctuated by unforeseen complexities.

A rhapsody of innovation, glimpses into the future, and occasionally, a tad bit of “Why did we jump into this again?”

See you in the next publication.
— André Nobre

Understanding Positional Power

In every company, there are several sources of power, each with its characteristics and effects within the organization. Regardless of its type, the use of power must be understood as something normal and necessary, considering its conscious use and knowledge of its consequences.

One such source, positional power, is the kind of power you have when you have a specific position or title in an organization. It is usually accompanied by legitimate power, which is the formal power to act in an organization.

But even when legitimate, if misused, it can bring problems for the organization.

In software engineering, you can identify its misuse in these situations:

• Someone in a higher position proposes a solution design that is blindly followed by the other participants;

• Team members avoid discussing ideas pitched by those in higher ranks;

• Teams agree with senior propositions just to avoid conflict;

• Priorities set by senior figures lack clear justification.

Positional power risk is equivalent to the role gap between participants in the initiative or meeting.

Awareness is the antidote to misuse. Let’s delve into the pitfalls and solutions.

Pitfalls of Misusing Positional Power

While there are several unexpected effects of misuse of positional power, this list highlights three that have the greatest impact on product development:

Lack of ownership (buy-in versus ownership)

Ownership is a cornerstone of any successful product journey. When a team has ownership, they fully embrace every idea, decision, and subsequent action. This means they are not only actively involved but also hold themselves accountable for the outcomes, whether they're successful or otherwise. It's this sense of ownership that fuels motivation and commitment, driving teams to give their very best.

On the flip side, we have buy-in. Imagine the entire thinking, ideation, and decision-making being charted out by someone else or perhaps an entirely different team. Now, the primary role of the development team becomes to be convinced of this externally derived idea and execute it. Opting for this approach might seem efficient initially, but in the long run, it's often synonymous with mediocre results. And when obstacles arise? That's when the cracks really start to show, giving rise to what many dread: a blame culture.

Chasing buy-in can be translated as the desire to keep a tight grip on things during the chaotic process of change.

Misalignment

When a team doesn't truly "own" a project, a domino effect of endless realignments begins. It's a cycle of trying to grasp the core challenge and, if lucky, eventually embracing it wholeheartedly.

But here's the twist: the journey to truly attain ownership after merely buying into an idea is as taxing as building that sense of ownership from the ground up. So, when a solution is handed down by someone atop the corporate ladder, does it genuinely offer any advantages? Or is it just an illusion of efficiency?

The need to be seen at any cost

Picture this: A high-ranking individual walks into a room, and suddenly, it feels like a golden opportunity for many. The allure is almost magnetic. For some, it might seem like their big break, their chance to shine brighter than ever.

But in this glittering rush, a dangerous shift occurs. The focus deviates from collective team goals to individual showmanship. The outcome? An atmosphere rife with conflicts and, you guessed it, further misalignment.

How to avoid the negative effects of positional power

If you find yourself in a situation of being the highest-ranking person at the table, be aware of the following approaches to avoid problems:

Don’t Lead When There’s a Big Gap in Positions

Avoid leading the discussion when there is a gap of more than 2 levels between you and the rest of the audience. During the discussion or meeting, try to avoid setting the tone with initial ideas or acting as the production blocker — when an idea is promptly blocked. Remember that it is crucial that the team has ownership and not just buy-in.

Drive based on results

It's disheartening, to say the least, when someone else dictates the fruits of your intellectual labor. Instead of being tethered to another's vision, champion an outcome-driven approach. Let key performance indicators be your compass, and don't shy away from challenging your team with thought-provoking questions. The results might just surprise you!

Use the overall process to create value

A company's process shouldn't be a straitjacket. It should be a catalyst, opening doors for every individual to lend their voice and unique insights. Dive into the existing processes headfirst, advocating for an environment that minimizes the shadows of positional power. And if you feel the process is more of a roadblock than a runway, speak up and be the catalyst for change.

Positional power as an important tool

Positional power gives you the power to make decisions and move your team forward in unison. This is extremely important in a dynamic product development organization, especially under a heavy operational load.

You can (and should) use your positional power in these circumstances:

• The decision must be taken in a timely manner, with no consensus among the team;

• When a decision includes high risk or is a strategic move that could impact the organization as a whole;

Usually, positional power is very well seen in situations where there is little time for critical operational decision-making or for direction where the team has not arrived at a vision and needs help. Even so, try to follow through with more questions than answers.

What to do next

Start raising awareness within your organization about the unexpected effects of positional power. Identify your misuse and align with your peers on appropriate actions when it happens. Give feedback promptly and work together to ensure a better place to work.

In conclusion, while positional power is an inherent aspect of hierarchical structures in software engineering organizations, its effects on creativity, communication, and diversity can be far-reaching and detrimental. By recognizing and addressing these effects, organizations can strive to create an environment that supports innovation, effective communication, and inclusivity, ultimately leading to improved software development processes and outcomes.