It is easy to assume that the future shape of the American health care system will be settled one way or the other by the November election. But in turns out that whatever happens in the election, and subsequently to Mr. Obama’s signature health reform, we will still be far from resolving some basic challenges facing the U.S. health care system.

One challenge is how to move away from essentially unrestrained spending and towards a workable process of cost control. Analysts on both sides of the aisle recognize the deep-seated problem, but there is little stomach for tackling it. One reason for this is that Americans, much more that the residents of most other countries, resist the idea of a real and enforceable health care budget for publicly funded health programs, especially Medicare. They oppose direct spending controls in the private sector even more fiercely. That helped create the conventional political wisdom that the best strategy for achieving major reform—nationally as well as in states like Massachusetts—is to lead with the “dessert” of coverage expansions and leave the “broccoli” of cost control for another day. The hope is that, once covered, Americans will then somehow be more open to the idea of limits. But there is little evidence that any such a public change of heart is taking place.

There’s another core issue, however, that is more subtle but even more basic: how to design a process that leads to continuous improvement and innovation in the structure of the health system itself. Without constant creativity in how we organize and deliver health services, we can’t hold down spending without curbing access or quality, and we can’t keep improving efficiency. It is not that innovation doesn’t occur in American medicine. On the contrary, when it comes to developing new treatments, pharmaceuticals and medical technology, America is a world leader. The problem is not in our research labs but in the environment for continuous structural innovation in the U.S.—such things as how we organize health insurance, or the ways physicians work together in groups and with hospitals.

To understand why this problem exists, recognize that the environment for structural innovation is heavily shaped by two factors. One is the design of public programs, like Medicare for the elderly and Medicaid for the poor. Because of the sheer size of these programs, their payment system and organization play a dominant role in the arrangement of the whole U.S. health market. The other factor is the regulatory structure at the Federal and state level. Like a computer’s operating system, this regulatory system establishes the basic operating rules for health care in America—everything from which facilities and professionals are permitted to carry out which medical tasks to how insurance must be structured and sold. Consequently, any would-be innovator must work within the constraints of the regulatory system and the large footprint of the public programs.

That may seem more of an irritant than a serious problem for an innovator, but the effect is a great deal more than just irritating. In fact, the American health care system is arguably the most regulated system in the world, despite the fact that it is overwhelmingly in private hands.

Why do we constrain health care innovators in this way? Because we try to maintain a “conscious” market, in which the government seeks to manage the market, through a framework of regulation and incentives, in order to get closer to achieving social goals, such as affordable coverage for Americans no matter their income or health condition.

Our energetic, private-based delivery system does indeed respond creatively to the incentives it is given. But as private health managers seek creative ways to operate within the constraints they are given, they will often organize in ways that are good for their own efficiency and bottom line, but not necessarily the government’s goals.

Many years ago I attended a conference of teaching hospitals. A lunchtime speaker updated the hospital administrators on new Medicare payment levels the government had just announced for a wide range of diagnoses, intended to reduce overall Medicare spending. The very next speaker told the audience that his software firm had already developed a new computer program to refine diagnosis descriptions in order to maintain hospitals’ revenues under the new payment system. The government typically responds to this kind of creativity with new rules for the industry, triggering another burst of private sector creativity to live efficiently within the new rules—again, in ways that do not necessarily achieve the result government had in mind. This leads to the next cycle of regulation and reaction. And so both regulation and frustration grow.

So we face this challenge: How can we reach social goals in health care with an industry that increasingly devotes much of its creativity to working with or around a regulatory framework?

Right now, the Federal government has adopted what might be called an “intelligent design” approach to innovation in the organization of the U.S. health care system. This approach consciously uses regulatory and statutory tools to nudge or force the system down paths that health systems analysts in government agencies perceive to be the most likely to yield the combination of social goals and efficiency. It’s not that these analysts are assumed to be inherently smarter than private managers. It’s rather that their vantage point above the day-to-day fray of health care delivery, combined with their focus on long-term social goals, supposedly gives them a critically different perspective from industry health analysts and decision-makers, who focus on quarterly performance numbers.

But with its emphasis on leading health care system improvement from the top, the intelligent design approach has its own inherent flaws. That’s because at the core it relies on a central-planning model.

There are several deep-seated problems with innovation through intelligent design when it comes to the U.S. health system. The first is scale. If America’s health care sector were a separate national economy it would be about the sixth-largest economy in the world, roughly the size of the French or the British economies. Centrally managing an economy that large in order both to achieve national goals and be on the leading edge of organizational innovation would be a heroic feat indeed—and hardly likely to come out very well. Having lived in Europe from the 1950s to the 1970s, I saw the failure of such efforts to manage innovation in large economies. It is no surprise that central planning has gone out of vogue throughout the industrialized world.

Another problem is that administrative and statutory regulations create constituencies around those rules. These constituencies have invested in accommodating to the current rules and do not want to lose their investment by having them changed. Since the structure is a product of the political system, leading stakeholder constituencies lobby to retain the basic structure even if other designs might be improvements. The result is to lock down a particular regulatory design well beyond its ideal “use by” date, thus slowing structural innovation. This is especially true in with statutory changes. We can see the pattern clearly in the history of Medicare. When created in 1965, its structure was modeled on existing private insurance and reflected the best design at the time. But despite some improvements, its organizational structure has steadily fallen behind developments in other parts of American health care. It is still a primarily a fee-for-service system, for instance, while managed care dominates the private system.

It is because of this “locking in” effect that it is hard to achieve continuous organizational improvement in much of the U.S. health system and why significant change requires periodic large, convulsive, statutory reorganizations of the entire system or large parts of it. We saw this in 1965 with the advent of the Medicare and Medicaid programs. We almost saw it again in 1993 with the aborted Clinton health plan. And after some smaller reforms along the way, such as a new Medicare drug benefit in 2003, Congress again passed a sweeping restructuring in 2010. It always seems as though the intelligent design strategy has to play catch-up and to require controversial new laws.

Another shortcoming of the intelligent design model of innovation is that it encourages a constant quest to find seeming magic bullets that can be superimposed through regulation on the rest of the system. So government officials scour the country, searching for breakthrough ideas that seem to hold the key to major improvements in quality and efficiency. One day it may be the Mayo Clinic that is the center of attention. Another day it might be the Geisinger system in Pennsylvania. In addition, various federally funded demonstration projects and grants are used to try to find other possible game-changing innovations: the recent reform legislation created a Center for Medicare and Medicaid Innovation to sponsor projects. When a new approach seems promising to government planners, the next step in the approach is to pressure or incentivize the health industry to adopt it. A favorite of the Obama Administration, for instance, are so-called accountable care organizations (ACO) for Medicare. An ACO brings together health care providers to serve groups of patients with payments linked to quality metrics. The presumption is that the health care industry is not far-sighted enough to see the potential of ACOs. So if it can be nudged to adopt this model with grants and other incentives, we will save everybody money and improve care.

But the magic bullet approach to innovation contains many of the same weaknesses as do attempts to legislate innovation. For one thing what is imagined to be a breakthrough to a government official or grant-maker is invariably clouded by an Administration’s predisposition towards that particular approach. It also assumes that the best innovations are more likely to be discovered and spread thanks to a perceptive official or analyst spotting them, rather than the way innovative ideas typically emerge, and live or die: within in a competitive market.

But there is another way we could foster innovation in the organization of a health care system oriented toward social goals as well as efficiency and medical breakthroughs. We might call this way the “evolution” model, and it takes a very different approach from intelligent design. Rather that consciously trying to discern and foster specific approaches, this approach would focus on creating the best general environment for innovation and incentives to achieve social goals without establishing a clear view or predisposition of what the result will look like. As such, it would hinge on spurring competition by ensuring reliable information with which to assess success, encouragement for sharing that information, and a more “chaotic” process of discovery.

These contrasting models of innovation were at the heart of many of the disputes over the Affordable Care Act, and they will shape the debate over the future structure of the American health care system. Indeed, the evolution model is a core part of several proposals under discussion, and the debate over these ideas is intense precisely because it reflects fundamentally different visions of how best to improve the system.

For example, there is a range of proposals for a “defined-contribution” method to pay for health care, as opposed to the “defined benefit” structure of many existing programs. Under a defined-contribution mechanism, individuals—and even states—would receive a fixed amount of money (indexed in some way in the future) that is believed to be adequate to pay for at least some broadly agreed amount of health services or insurance coverage. But the manner in which beneficiaries obtain those services or coverage, and the way the health industry organizes itself to provide those services, is left open to the buyers and sellers, albeit subject to some regulations to guarantee levels of quality etc. The Medicare “premium support” proposal developed by Representative Paul Ryan (R-WI) is an example, as are proposals to provide states with a fixed block grant to pay for Medicaid. By contrast, a defined benefit is an entitlement to specific services, with the amount of money spent depending on the amount and prices of the services used.

Defined-contribution proposals are frequently portrayed—and attacked—as being primarily budget-control mechanisms. They certainly are that. But they are also mechanisms to stimulate an evolutionary process of innovation. Under a defined-contribution model, millions of individuals have an incentive to seek new and better ways of obtaining health care services. So do states under a block grant. It can be a messy and bumpy trial and error process as different organizational ideas are tried out and either succeed or fail. But over time, those parts of the health system that figure out better ways to organize and deliver care in a way that satisfies the customer will tend to prevail. And they will do so without any conscious hand of government guessing which arrangement will turn out to be best and pushing everybody to adopt it.

To be sure, good information is critical to both visions of innovation. For the more centralized, defined-benefit, intelligent design approach, the key is to collect technical information and send it vertically up the line so that wise men and women can make the best decisions. In the defined-contribution, evolutionary model of innovation, by contrast, the key is to generate useable information and spread it horizontally so that millions of people can make decisions that will end up rewarding the most useable and effective innovations. The evolutionary approach generates user-friendly and viral information. It requires information distribution platforms much like Expedia or Consumer Reports. Intelligent design depends on peer reviewed research and government studies.

In more subtle ways, we can also see the two philosophies clashing in the debate over the best frameworks for health plans to operate within. The Obama legislation created a version of health exchanges, intended to operate like the health care equivalent of a stock exchange or a farmers market, in which Americans could enroll in plans offered within a state. But in keeping with the intelligent design view, the types of plans that can be sold are to be highly regulated, and the federal law envisions exchanges functioning in a very particular way. It is true that the legislation will permit some state flexibility in the design of exchanges, but ultimately the federal government will be calling the shots.

The alternative vision of exchanges is to allow states, and even private entities, to make their own best guess about which framework will make informed shopping easy and encourage innovative health plans to enter the market. As information flows freely on the performance of different exchange models, the view is that these state frameworks for innovation themselves would become subject to an evolutionary process of innovation, with some state-led approaches emerging as more popular models for other states’ approaches. This line of thinking leads some innovation experts to argue for competing exchanges or other frameworks within a state—just as Expedia competes with Travelocity to provide the best combination of information, service and price for customers.

To the extent that government plays a role in reaching goals and creating frameworks for innovation, supporters of the evolutionary view see the proper focus as the states, not the Federal government. Advocates of state-led health care improvement, like advocates of state-led education or welfare reform, see a competition between states—rather than “intelligent design” leadership from the federal government—as the way most likely to find the best framework to spur structural innovation throughout the health system. In areas like health insurance, for instance, we really don’t know the best way to achieve social objectives such as assuring continuous and affordable coverage for people with chronic medical problems without creating perverse incentives for insurers to avoid selling to those same individuals. So, the evolutionary argument goes, let states continue to experiment with different approaches to insurance regulation; we can then weight the results of these approaches, and the good will drive out the bad. To be sure, the Federal government can set national objectives. It can insist on protections for vulnerable individuals. And it can help the free flow of dispassionate information on the strengths and weaknesses of alternative approaches. But the evolution school sees the trial-and-error process of state-led competitive federalism as the key to achieving continuous innovation.

It’s true that central direction and planning can sometimes be what’s needed to reach a specific, identifiable goal, like winning a war or landing a man on the Moon. But spurring innovation in the organization and delivery of health care is like constantly improving our education system or ensuring that American industry is always continuously creative. It is about a process and not about achieving a specific, known result. And it cannot be centrally directed because it is about encouraging creativity without knowing what the best future should or will look like.