Hello friends, and welcome back to the Product Fundamentals podcast for episode 3: Management and Measurement.

In this season, we are tracking the evolution of how we came to make software in the weird way we do, from the earliest origins of our methods, through to today.

So far, we’ve introduced the front-loaded, documentation-heavy Waterfall methodology, as well as the incremental and iterative development paradigm that was its plucky rival.

Before we take that rivalry forward into the 1980s, we’ll use this episode to cover the ways that we’ve defined and measured success in our work. We’re discussing the origin and evolution of the success criteria that keep every product manager up at night: OKRs and KPIs.

While the concepts of Objectives & Key Results as we use them today are quite recent, they really represent the culmination of the quantification of work and business that has been ongoing since the late 1800s, and whose roots go back even earlier.

So, don’t tune out! I promise it’ll be worthwhile. We’ll get class conflict in the Industrial Revolution, early progressive reformers, an economist hunted by the Nazis, and finally we’ll reach software via the semiconductor business.

[The early days]

The idea of rigorously measuring output is not as old as many may think. Modern statistics are often traced back to Sir William Petty, an English polymath of the mid 1600s. After serving as physician-general for Oliver Cromwell's army in Ireland during the English Civil War, Petty became a landed gentleman. He became what was called a "projector" on his lands, meaning he combined his private resources with public money from the Crown, in an effort to build profitable projects like ironworks and fisheries that would also provide some public benefits.

Petty had deep interest in what we would now consider the intersection of economics and statistics, though those disciplines were not yet established, and his methods were rudimentary by today's standards. Petty used simple averages and sparse data to make estimates of populations and the size of the English economy, as well as more direct matters related to the performance of his own estates. This was basically Fermi Estimation, applying simple math over a few rough parameters, but it was a step forward. One of the burning questions Petty took one: was Paris or London the larger city?

Perry called his methods “political arithmetic,” as only the State had the scale and the need for such tools. While Petty's methods were simple by today's standards, he stood out by being quantitative. By comparison, most other leading intellectuals of the era such as Thomas Hobbes (for whom Petty had been a personal secretary) were certainly applying causal reasoning in their theories, but they weren't really doing any math.

Indeed, until the Industrial Revolution, early statistics – and even the concept of large numbers – were the near-exclusive domain of the state, and even then, mostly used for censuses and military procurement.

As an interesting aside, Petty was also the first modern writer to comment on the division of labor across specialists, when  as a young man studying in Holland, he observed how Dutch ship makers worked on multiple ships at once, rather than one at a time. Petty noted that specialized teams performing the same tasks across each of several ships delivered ships more quickly and efficiently than generalist teams that built the entire ship. These concepts would obviously show up later as the Industrial Revolution took off and large scale manufacturing began.

Back to the evolution of tracking output: As European states became more centralized and sophisticated through the 18th and early 19th centuries, the discipline of statistics matured, with national censuses and agricultural yields becoming simultaneously the source of large-scale data and the motivators for better analytical methods. Indeed, the agricultural importance of statistics was so prominent that the seal of Britain’s Royal Statistical Society, founded in 1834, was a bundle of wheat.

[Taylorism]

With the takeoff of the Industrial Revolution, analytical methods were brought into manufacturing as well. 

The pioneering figure of rigorously analyzing production was American mechanical engineer and early consultant Frederick Taylor. His work, beginning in the 1880s and 1890s, was all about applying rigorous planning and structure to manufacturing processes in order to maximize efficiency. HIs ideas, called Taylorism – or more generally, Scientific Management – had an indisputably huge effect on worker productivity in the factories of the late 19th and early 20th centuries.

Many of early Scientific Management’s precepts are intuitive to us now living in the age of the assembly line, but they were significant innovations at the time. Under Taylor’s analysis, each manufacturing process was broken down into many small individual tasks, with any redundant steps or wasteful inefficiencies identified and removed from the process. By ruthlessly optimizing and routinizing work, measuring each worker’s output, and assigning specific workers to specific tightly-defined tasks, factories could be much more productive. The division of labor that Petty had noted among Dutch shipbuilders was taken to its logical extreme.

This efficiency came at a human social cost: Taylor also introduced the “differential piece-rate system,” in which experts defined the optimal way to perform a task, and then set a production quota for workers to meet when performing that task. The worker’s compensation was tied to meeting this quota – fail to meet quota and receive lower pay; exceed the quota and receive a higher pay, but don’t be surprised if the quota goes up once you meet it.

While analytical, Taylor was statistically lightweight. It took indirect intellectual descendents of Taylor, statisticians Walter Shewhart and William George Deming, to bring statistical rigor to manufacturing. We discussed Shewart and Deming in our last episode, and how they paved the way for incremental and iterative improvements in manufacturing methods through statistical analysis of quality. Other than very generic numbers like number of widgets sold, these measures of product quality might be the first data that we could call key performance indicators, or KPIs, in the modern sense. Though, the term would have been anachronistic – that term KPI did not exist until the 1940s, and was not widely used until much later. 

[World War II]

The event that really brought rigorous measurement, analysis, and planning to every corner of the economy, though, was the Second World War.

In the late 1930s, as it became increasingly clear to British leaders that another war in Europe was inevitable, the British military initiated the new field of operations research, in which analysts and engineers were put to work using rigorous analysis to improve the efficiency and effectiveness of military operations.

While lengthy examples of their successes are a better fit for another podcast, the operations researchers’ findings made an enormous difference across the war. Often, the impact came from using analytical rigor to focus on measuring what really mattered, and following the conclusions of that analysis even when they ran counter to intuition. 

By reviewing and correlating data from vast numbers of after-action reports, Operations researchers found advantages buried in the noise. Among many other wins, their findings saved precious war materiel and untold lives by optimizing the size of merchant convoys across the Atlantic – it turns out one big slow convoy is safer than multiple smaller faster ones. They increased the kill rate of British coastal defense aircraft against German U-boats seven-fold by optimizing tactics to counter submarines that had made mistakes, rather than trying in vain to hunt the most effective submarines. 

In parallel, analysis of logistics and production was critical on the homefront in America as the country geared up from the lingering Great Depression to become the arsenal of democracy. In the United States, the War Production Board, created by Franklin Roosevelt in 1942, oversaw the allocation of resources and the production of war materiel. The Board controlled production of war materiel worth roughly 2.5% of the United States’ GDP each year. To handle that task, the Board called on mathematicians, engineers, and statisticians to leverage analytical and forecasting tools to optimize where scarce resources were sent, to sequence the production of goods, to route materials through the nation’s transportation infrastructure. Major producers of war materiel, such as Ford, General Motors, Boeing, Lockheed, and so on, had to contend with resource shortages, driving them to adopt stricter monitoring of production practices and improve efficiency. Shewhart’s and Deming’s statistical methods of quality control became essential.

When the war ended, some of the lessons of operations research and wartime production persisted and spread throughout the broader economy, leaving a lasting legacy of firms quantifying, analyzing, and goal-setting in their production processes to a greater extent than before the war.

[Peter Drucker]

One more consequence of World War 2 for the United States was that it brought “the father of OKRs,” despised by the Nazis, to American shores.

That man, Peter Drucker, was an Austrian-born Lutheran economist of Jewish ancestry. He lived and studied in Germany during the 1920s and 1930s, during which time he wrote essays that were banned and burned by the Nazis, presumably because of Drucker’s own Jewish roots, for writing that being Jewish is compatible with being a patriotic German, and for generally calling the Nazis dumb. 

Sensibly enough, Drucker fled to Britain when Hitler became chancellor of Germany and his name appears to have been included in a list of 2,300 people that the Gestapo were to round up if the Nazis had successfully conquered the UK. Drucker’s experience with the Nazis no doubt informed the strong skepticism of concentrated power that suffuses his work.

Drucker would relocate to the United States in the 1940s, and became a management consultant with General Motors and with other major American companies. 

In his 1954 book, The Practice of Management, Drucker promotes the concept of “management by objective.” In this scheme, the manager of each team at a company should be responsible for setting a business objective for the team to achieve. The senior leaders of the company should approve of the objective, but ultimately the objective belongs to the team manager, and his or her performance will be judged in part based on success against the objective.

Drucker’s management by objective concept was not itself an unprecedented breakthrough. There were certainly precursors, including the concept of replacing top-down “orders” with building a shared understanding of the business situation among all members of an organization, so that everyone would pull naturally in the same direction, which was articulated by management consultant Mary Parker Follett in her essay The Giving of Orders. But Drucker’s articulation was the one with the most lasting influence.

At a high level, this sounds like the OKRs that many of us are familiar with today, but there are a few wrinkles. Drucker has a specific set of “key areas” that the objectives must cover. He enumerated them as: Market standing, innovation, productivity, physical and financial resources; profitability; manager performance and development; worker performance and attitude; public responsibility.

Drucker thought that an organization needed clear objectives in all eight areas; consistent with his broader pro-social agenda, he thought it wildly irresponsible for firms to operate without explicit objectives for less conventional areas like worker development and public responsibility. 

While he insisted that quantified measures for each objective were important, he also admitted that only a few of them had high quality quantitative measures. He hoped that widely-available benchmarks would be forthcoming across the board. This theme of new methods being forthcoming runs throughout the book; Drucker certainly perceives that he is writing at a time of transition from the World War era into a new period of automation and organizational evolution.

In contrast to today’s world of quarterly or semi-annual OKRs, Drucker’s management objectives are multi-year goals, with some expected to be less than 5 years, and some expected to be longer-term. Because the objectives are long-term, they’re also somewhat broader in scope than the OKRs we often set now.

Drucker’s model of objectives is all about managers, with teams taking a decidedly secondary role. Each manager sets the objectives for his or her team, which will determine how the manager will be evaluated. Each manager’s objectives should be consistent with and in service of the manager above them. While the company must have goals that cover all 8 of Drucker’s key areas, each manager’s objectives may only be relevant to some of those company goals – but the manager’s objectives must not be in conflict with any of the company’s goals.

Despite its focus on managers as the primary agents in the story, Drucker does want to keep the organizational chart as shallow as possible. Each layer of management adds potential for political kingdom-making and loss of focus on the company’s overall objectives, plus it makes it harder for talented people to rise to the top of the organization. Concordantly, Drucker thinks a company is too big when one set of objectives cannot include the whole thing, or when there are too many objectives for all of their owners to form one team that can work directly with the chief executive. Still, this preference did not stop mid-century giant American corporations, including the generals (that’s General Motors, General Electric, General Foods, and so on) from adopting Drucker’s system.

This is an aside from our focus on measuring progress, but Drucker does have a comment about the risk of companies growing too large that may seem prescient, written as it was in 1954, for the tech giants of today. He writes,

“This danger is particularly great in the business that originated in a common technology, such as chemistry or electrical engineering. As the technology unfolds it creates more and more diversified products with different markets, different objectives for innovation—and ultimately even with different technologies. The point is finally reached where top management cannot know or understand what the diversified businesses require—or even what they are. The point may be reached where objectives and principles that fit one business (or group of businesses) endanger another.”

[The HP Way]

Drucker’s ideas on Management by Objective were adopted by two especially important technology companies. The first was Hewlett-Packard, which incorporated the approach into their broader management philosophy, which the company labeled “The HP Way.” For decades, starting in the 1960s, Hewlett-Packard had a sterling reputation as a highly effective and innovative company, and one that took public responsibility seriously. Management by Objectives was a key part of this identity. From as early as 1960, HP co-founder David Packard characterized MBO as key to their business, and explicitly contrasted “Management by Objectives” with “Management by Control.”

In his 1996 book, also called “The HP Way,” Packard would write,

“No operating policy has contributed more to Hewlett-Packard’s success than the policy of ‘management by objective.’”

In contrast to management by control, Packard continues, 

“MBO, as it is frequently called, is the antithesis of management by control. The latter refers to a tightly controlled system of management of the military type, where people are assigned—and expected to do—specific jobs, precisely as they are told and without the need to know much about the overall objectives of the organization. Management by objective, on the other hand, refers to a system in which overall objectives are clearly stated and agreed upon, and which gives people the flexibility to work toward those goals in ways they determine best for their own areas of responsibility. It is the philosophy of decentralization in management and the very essence of free enterprise.”

In an allusion to the Peloponnesian War that I cannot help but love, Packard adds,

“the concept of people working together under common objectives and in an atmosphere of individual freedom is nothing new. It was demonstrated by Athens against Sparta more than twenty centuries ago.”

 The HP Way took some steps to embody Drucker’s skepticism of concentrated authority, adopting two measures to counterbalance the centralizing and blinding tendency of Management by Objectives. The first is Management by Walking Around, which is exactly what it sounds like. Managers should literally walk through the factory or the office, observe what’s going on, ask questions, and embrace opportunities for productive chance encounters. This “walking around” bit was lifted directly from Drucker. The second check is an Open Door Policy expected from managers: Any worker should be able to go to their manager to discuss personal and professional issues; as needed, workers can even go straight to the open door of higher leaders.

Still, these balancing measures are pretty soft and informal, and that will hold true through the future of planning. The structure of management by objectives still contains a  natural tendency toward consolidation and centralization with decision-makers at the top; all of the thinkers we discuss today leave it up to organizational culture and individual choices from leaders to balance it out.

[Andy Grove]

Besides HP, the other key adopter of Management by Objectives in the mid-20th century tech industry was Intel, especially under its long-time executive and eventual CEO, Andy Grove. Born András István Gróf in the short-lived Kingdom of Hungary between the World Wars, Grove fled Soviet domination of Hungary during the uprising of 1956 and came to America, where he Anglicized his name. He was Intel’s first employee, and by all accounts a brilliant engineer, a highly-structured thinker, and a ruthlessly efficient operator.

Starting in 1971, Grove put Drucker’s concept of Management by Objectives to great effect as a director, president, and eventually CEO of Intel. Grove significantly reworked the concept, though, creatively-enough calling his version “Intel Management by Objectives.” While Drucker had seen his objectives as long-term, and meant to express the company’s orientation across all of the important dimensions, Grove used them much more tactically, on shorter time horizons. 

In his 1983 book, High Output Management, described this approach. First come long-term goals. The goal is then decomposed into a series of objectives, which are concrete and specific definitions of what we want to achieve. Objectives should be time-sensitive; they have a deadline. He uses the example of a construction manager at Intel, who is working on a project to expanding Intel’s factory in the Philippines. The construction manager will have an objective like, “Obtain a decision on the new factory location by October.”

The objective then has several key results, which are concrete indicators of whether work is on pace to achieve the objective on time or not. In the factory planning example, key results included doing a land study by June, completing a financial analysis, getting steering committee approval, and then getting CEO approval by October. Completing the key results necessarily completes the objective.

In contrast to a practice one sees sometimes today, objectives were always completable in their time horizon. So, there are no evergreen objectives like “Write high-quality software” that roll over from period to period.

Grove writes that the objectives of any level in the organizational hierarchy should correspond to the key results of their supervisors, ensuring alignment. So, my manager takes his objectives from the key results of the level above him, then sets his own key results. Then, my objective is one of my manager’s key results, and I set my own key results.

In Grove’s scheme, objectives and key results exist at every level, including the individual. Each employee should have their own OKRs for their own individual tasks and performance. The individual worker should have the latitude to set about half of their own objectives; the other half should be inherited from their superiors.

For Grove, objectives are meant to be tightly bound in time. Because the purpose of the Intel MBO system is to indicate when projects need adjustment, he writes, 

“an MBO system should set objectives for a relatively short period. For example, if we plan on a yearly basis, the corresponding MBO system’s time frame should be at least as often as quarterly or perhaps even monthly.” 

So, annual goals have monthly or at least quarterly objectives, which then have even more frequent key result deadlines. In practice, though, Intel actually set annual objectives for the company, and its divisions set quarterly objectives.

The benefit of this system, Grove writes, is focus: there should be a small number of objectives, and because they have such regular deadlines, they will strongly encourage us to say “no” to everything else.

Grove also writes about the importance of what he calls “trend indicators” in management. While Grove didn't use the phrase "key performance indicators" in his 1983 book, that term became common in the late 1980s, and I suspect it was in reaction to Grove. Grove describes these trend indicators as “cutting holes in the black box” of a company’s production process. Trend indicators should measure the …

To Grove’s credit, he recognized the risk of Goodhart’s law, which is often casually stated as “When a measure becomes a target, it ceases to be a good measure.” Goodhart is a British economist who was criticizing the Thatcher government in the UK for using the money supply as a target for monetary policy. This adage has made it into business, as a caution against specifying any single indicator as the key sign of success; the indicator is likely to be gamed in counter-productive ways.

Andy Grove used paired indicators to avoid the downside risk of focusing on a singular key results. So, for example, an accounting team that had have the key result of “process 10,000 vouchers” would have a counterbalancing quality goal of “have less than 1% error rate.”

[MBO and Methodology]

This scheme of management by objectives would go through one more evolution before becoming ubiquitous in modern software companies; we’ll cover that in a later episode. But at this point in our historical narrative, it’s worth considering how the developing notion of OKRs relates to the contemporaneous Waterfall and Iterative Development methodologies that we discussed in our previous episodes.

In one sense, we can treat the management theory of Management by Objectives and the methodology of executing projects as independent layers of the stack. We can certainly imagine organizations operating under any of the combinations of Objectives or Orders, Waterfall or IID.

Still, it’s hard to imagine that one level will not affect the other.

Compared to Management by Orders, Drucker’s and HP’s Management by Objectives are clearly less prescriptive, and therefore share some intellectual DNA and practical utility with iterative development. IID is premised on the idea that we don’t know exactly how to get to the goal, so we need time and structure to allow us to improvise. This can’t practically happen under a regime of strict orders. It can happen in a world of long-dated objectives.

But there’s no getting around the reality that management by objectives are a system of control built around a central authority.

Taylor and his Scientific Management followers believed that wise managers could understand the whole system in ways that workers could not, and thus saw those managers as real agents in the story; the workers were tools. Thus it was appropriate for the manager to discover and rigidly define the right way to do the job, and then to use each worker’s output relative to that one true method to decide workers’ pay.

Mary Parker Follett and Peter Drucker softened Taylor’s classism, putting the emphasis on creating a common set of goals that workers and management all shared.

As corporations with many thousands of workers proliferated in the early and mid-20th century, it became important to manage mid-level managers themselves, and not just front-line workers.

In this milieu of growing organizational complexity, Drucker’s take on “Management by Objective,” with its focus on a small number of company-wide objectives, covering all of the dimensions that a company must attend to, made sense. While Drucker did indicate that each manager should set objectives for the team, he also articulated that those objectives should be scoped to run for several years. 

Consequently, Drucker’s Management by Objective system doesn’t naturally support tightly-scoped time-sensitive key results. From a methodology perspective, Drucker’s approach has a greater humility about the ability to predict the future, and leaves more flexibility to deal with surprises. The HP Way followed Drucker’s prescription quite directly.

By the time we get to Grove, though, the emphasis has shifted.

Especially in Andy Grove’s Intel system, which is the lineage that most modern software companies inherited their approach to OKRs from, the problem being solved is really a scaling problem.

While orders might work in a sprawling bureaucracy, there’s a reason Max Weber’s archetypical bureaucracy was the Civil Service. A government might be able to afford some slowness or inefficiency due to its structure, but a profit-seeking firm in a competitive marketplace couldn’t.

There’s a sense in Grove’s world that the tragic problem is that there aren’t enough Andy Groves to do everything.

This might not be literally true – I’m sure Andy Grove knew he didn’t personally have technical mastery of every single skill needed for every role at Intel – but his book doesn’t talk about worker empowerment or anything like that. He doesn’t say Intel uses objectives to harness the intellect and the creativity of diverse brilliant minds and pointing them in the same direction. The assumption is that the senior staff knows everything important, and then that their brilliant plan should proliferate out in one direction from the top of the org to the bottom.

Sure, the objectives that are inherited from the top level may be general, but after a few layers of hierarchy, in which each worker’s objectives are his manager’s key results, the level of specificity will have increased significantly, even if the worker gets some say on their personal MBO items. In contrast to Drucker’s many-year objectives, Grove’s key results are meant to have a monthly cadence, and they’re meant to be ambitious enough that they can’t all be completed in the time allowed. The result is that workers are still largely told what to do from above, with targets changing frequently, and the worker is usually unable to complete the work in the time allotted. It might not be as strict a form of control as management by orders, but in terms of overall worldview and effect, Grove’s Intel MBO looks more like an update to Taylor than a natural evolution of Drucker, who was so skeptical of centralized authority. Grove’s “Objectives” read like “Orders lite.”

It seems to me that Grove, brilliant mid-20th-century engineer that he was, is trying to engineer the company itself to be the most effective force-multiplier for himself and his senior staff. 

Of course, I imagine that Grove – and many leaders evangelizing for his OKR system today – would be quick to say that they value feedback loops, they want to incorporate ideas that come from places beyond senior staff, and so on. But none of those concessions made it to the structure of the Intel MBO system. The only channels for information to flow upward in Grove’s model organization are one-on-one meetings between immediate subordinates and managers, staff meetings of senior leaders, written reports from department heads to the CEO, and ad-hoc conversations in hallways.

This makes Grove’s model intellectually consonant with the Waterfall software methodology, with its emphasis on a wise central planner who could see the path, articulate the plan, and through strict adherence to plan, ensure success.

[Wrapping up]

Of course, writing in 1983 Grove was not out of step with his times. The 1970s and 1980s were dominated by the Waterfall methodology. So, please join us next episode, when we’ll cover the peak of the Waterfall bogeyman as it takes over the military, and as it starts to crack in the face of great technological change.

In the meantime, your comments and feedback on this episode are very welcome. You can find a transcript, sources, and links to reach me through the show page at prodfund.com. And if you like this series, and want to hear more, support the show by sharing it with someone you think would enjoy it too.

This has been the Product Fundamentals podcast. Thank you very much for listening.