A version of this story appeared in Newsweek on October 31, 2013. This is my submitted draft.

Healthcare.gov wasn’t supposed to be this way.

Before it was the topic of Congressional inquiries and citizen frustration, the site was supposed to be the Affordable Care Act’s crown jewel. Built to facilitate the ACCA’s health care exchanges, the site’s existence and much of its structure were mandated by law. It has to do many things, none of them trivial, especially at scale: verify a person’s identity, legal residence, and income; record his personal information; match him with health insurance plans for which he is eligible; calculates whether or not he’s eligible for a subsidy (and how much); help him compare those plans by costs and features; and finally enroll him in a plan with the chosen provider.

None of these things are trivial, especially at scale. Even getting enrolled in an insurance plan period can be tricky; actually comparison-shopping for one in a state that may offer hundreds of plans almost demands eschewing pen and paper for the internet, even before the federal government comes in. It requires working with all fifty states — even though the exchange may be federal, the insurers aren’t —  as well as dozens of insurers. What’s more, since the ACA was designed to support state exchanges, parts of the site’s spine, specifically the data hub that pings the IRS, Social Security, Homeland Security, and credit bureaus, would have to work with each of those state websites, federal departments, and credit agencies too. Or more accurately, it would have to work with their software and data centers, which are often themselves equally if not more byzantine.

So, like every modern website that actually has to make something happen rather than just display text, Healthcare.gov separated the site into a frontend that interacts with the customer and a backend that (among other things) fetches and matches data and actually moves your information through the system. But unlike most enormous modern websites that actually do make things happen — the Googles, the Facebooks, the Amazons — Healthcare.gov divided this work among a variety of companies, each of which had experience with their piece of the project. (For obvious reasons, no company had experience building a federal health insurance exchange.)

The most important of these companies running the backend were CGI Federal, which would build the insurance marketplaces, as it had for state Medicaid programs, and QSSI, which would build the data hub to verify citizens’ eligibility for both the federal and state exchanges. The homepage itself would eventually be built by Development Seed, a small, young DC-based company that came in as a subcontractor under Aquilent, who navigated the bureaucracy and bid for the contract. “If I were to bid on the whole project,” Development Seed’s Eric Gundersen told Slate’s David Auerbach, “I would need more lawyers and more proposal writers than actual engineers to build the project. Why would I make a company like that?”

The entire project would be overseen by CMS, the Centers for Medicare and Medicaid Services, part of the Department of Health and Human Services. CMS wasn’t a software company, and had no experience running a project this large or visible, but responsibility had to be at the government level. But CMS would also be busy monitoring and coordinating the development of each of the state exchanges.

It’s important to remember that when the law was passed and even when the government contracts were doled out, Healthcare.gov was a relatively tiny piece of the overall health exchange pie. The real plums were the state exchanges: there were more of them, and most of them paid more money. CGI’s contracts to run exchanges in Colorado, Massachusetts, Vermont, and Hawaii taken together were worth more than twice as much as its contract for Healthcare.gov ($71 million); New York’s exchange alone was worth $183.6 million for Computer Sciences Corp., and California’s $359 million for Accenture. Healthcare.gov wasn’t supposed to be the primary end-to-end portal for citizens seeking health care. It was supposed to be no big deal. It was supposed to be a lot of things.

***

To understand what has gone wrong with Healthcare.gov, you first have to understand that just like the law that spawned it, the site is fundamentally a compromise between two increasingly polarized groups. This compromise is all the harder as these groups increasingly lack a common language with which to speak to each other. In an attempt at moderation and to foster unity, the Obama administration tried to bring the two groups together into a grand synthesis, blending ideas from both sides. The administration’s plan tried at all costs to preserve elements of the old system rather than start from scratch, even as experienced bystanders warned that the two systems were practically as well as ideologically incompatible. What we have left is a complicated mess, only capable of forward movement by lurching from crisis to crisis. The only thing worse than the reality is the rhetoric, as both sides point fingers at the other, seeking to deflect blame. At this point, even moderates in both camps find they have little to say to each other.

The electoral conflict may be between the Democrats and the Republicans, but the technological conflict is between an emerging set of practices in software development that are at odds with how traditional government IT has tended to work. Most web applications built today in Silicon Valley’s orbit are agile, open, iterative. You test a minimum viable version of your application and refine your approach as you add more features and users. You move fast, break things, and stitch them back together again.

The federal government is not built to work that way. It is built to accomodate and coordinate monoliths and their schedulers. When the government shut down on October 1, the most lasting effect was on the sheer number of meetings and hearings and reviews that would now need to be rescheduled. The law or guideline takes effect on a single day and date and it effects everyone everywhere. And so it has gravitated towards IT companies that speak that language: IBM, Deloitte, CGI Federal, Aquilent. Companies with enormous legal teams that architect and plan their projects well in advance to deliver full products on budget and on schedule. (This hardly ever happens, but let’s indulge the fiction.)

That approach may no longer be best suited for delivering state-of-the-art services over the web. It’s certainly no longer best suited to work with a government which can no longer maintain its part of the bargain: delivering fixed guidelines and requirements that contractors can design around, with the assurance that the government will remain open for business.

So no one, neither the agile developers or the old government IT guard, is happy. They’re not happy with the government, and they’re positively furious with each other. What’s worse, it’s not even politics keeping the two groups apart. Or maybe it is. What counts as politics in 2013, when the signature legislative achievement of the century (depending on how you feel about the Patriot Act) finds itself tethered to JSON requests, debates over IT procurement procedures, and a data center’s uptime?

The only hidden workings more alarming, disgusting, yet somehow necessary than those of government may be those of a large and complex web application. There are things buried there that it would be better if one had never seen. Both governments and software are products of impossible compromise, unbearable paperwork, and human misery that would be immeasurable if it were not someone’s job to carefully measure and document it. And when the government and software development come together, as with Healthcare.gov, you don’t get the sum of their problems, but the product. This may not be most contentious software review in human history, but it will probably be the first to be powered by Congressional subpoenas.

***

The problems started early, but for the most part stayed hidden. In June, the GAO publicly warned that much of the state and federal exchanges’ work remained to be done, that many deadlines had been missed and guidelines changed, and that exchanges’ smooth operation by October 1 could not be guaranteed. But the contractors publicly affirmed that the site would be ready. It wasn’t just Healthcare.gov that separated into a frontend and backend. The companies building the site did, too. They understood the political pressure the administration was under, and that they too would soon be under.

The guidelines kept changing. There wasn’t enough time to test. Every company was confident in the piece it was building itself, but some of companies had stopped communicating with each other. Even CGI couldn’t access the tools they needed to run proper diagnostics on its own marketplace. Those were operated by another company, URS. In a monthly report filed in September, CGI begged CMS to give it access to URS’s central logs and monitoring tools; “we have repeatedly asked CMS and URS but have not been granted this access.” CGI had first logged this problem in June, but it had persisted through September.

URS is an enormous Fortune 500 company that provides a huge array of services to the government, military and civilian, from construction to IT. In 2012, it did $4.3 billion of work for the federal government, making it the 13^th largest contractor by dollars. It deals in petroleum, cement, electricity. It builds bridges and railways. It settled lawsuits in 2010 charging negligence for its role as safety inspector for the I-35 bridge that collapsed, without admitting liability. A URS spokesperson confirmed to Newsweek that the company was a vendor to a CMS contractor, but to date was “not aware of any performance issues” regarding its work.

It’s unclear whether CGI Federal ever received access the tools that it asked for, or the series of meetings it wanted with CMS leadership. Meanwhile, days after filing this report, CGI’s Cheryl Campbell testified to the House Energy and Commerce Committee that it had completed an operational readiness review and that its part of the exchanges would be ready on time. QSSI, Serco (who would manage paper applications), and credit agency Equifax all told the committee that the exchanges would be ready.

Development Seed proudly showed how its website would be built from state-of-the-art open-source technology that it would enhance and release back to the developer community via GitHub; testing with users would continue beyond the launch, and the site would continue to improve. HHS was also pleased with its new site: Chief Technology Officer Bryan Sivak told The Atlantic’s Alex Howard that the new site was “fast, built in static HTML, completely scalable and secure.” It was hoped the site’s design and speed would save money and attract younger users, on multiple devices, in multiple languages. The hope was to create a shining new standard for how innovative government IT projects could be done.

Meanwhile, behind the scenes, contractors and subcontractors began placing worried calls to each other. Everyone was frustrated with CMS’s lack of ownership of the project, its distraction, its inability to corral the many companies it had to coordinate. Each had its own story of bad data sets, broken APIs, information or access it needed from other companies. “This is a shitshow,” some said to each other privately. No one was certain whether the site would work, which parts would break first, or even whether they would quickly be able to find and fix what was broken.

At least one contractor, QSSI, warned the Obama administration directly that there were serious problems just weeks before the launch. But the law mandated that the exchanges be opened on October 1. Everyone was determined to have the web exchanges be open too.

As the shutdown loomed, that was the administration’s bet. The ACA had been passed through Congress and the executive branch would put it into effect. Congress had failed to fund the rest of the federal government, and so the government would be shut down. The executive branch would carry out the law, and let the gap between the resources and the needs be self-evident.

***

From a user perspective, one of the troubling things about Healthcare.gov is that when the site’s backend can’t find the data it needs, the site fails silently. The user isn’t sure whether it’s an internet connection problem or whether he’s accidentally keyed in the wrong information. The sites maintain their beautiful, unerring masks, while everything behind them bangs together and falls to pieces.

This isn’t limited to Healthcare.gov. This week, when one of the data centers powering QSSI’s vaunted data hubs failed, all of the exchange sites, state and federal, stopped working. The only thing the sites’ operators can do, once they realize there’s been a failure somewhere up the stack, is to close the entire site. You can’t shop for a plan without logging in, you can’t enter in your login information and then not shop. For a site designed to fulfill many functions and join together many separate pieces, it is unusually all or nothing. There’s simply nowhere else to go.

And despite the Obama administration’s promise of a “tech surge” (an unfortunate but sadly appropriately resonant metaphor) and appointment of former OMB official Jeffrey Zients to get the system fully functional by the end of November, there simply is no cavalry on the horizon, no armies of engineers who can wade in to fix what’s been broken.

When it seemed that CGI Federal had been the source of the problems, the administration turned over management to QSSI. Then the data centers QSSI were using failed. When the administration worried that the open-source code for the site’s frontend might be causing problems, it shut down the GitHub repository, affecting precisely nothing. It is a cascading problem with no clear villains or clear solutions.

Three things are likely to happen by the end of the next month. The site will be gradually begin to mostly work. There will be serious calls to do as a matter of practicality what the administration absolutely did not want to do as a matter of politics: extend the deadline for the individual mandate to account for the fact that it has been difficult for many people to sign up for plans. And the Energy Committee hearings, which now just as in September are purely for show, will give way to the House Oversight Committee, who will subpoena everyone involved to find out exactly what happened, or at least what was put on paper or electronic documents. The American people will learn more about IT procurement, web development, and client-and-server requests than it ever wanted to know.

The US government built these websites in its own image: a potent and destructive stew of idealism and cynicism, the past and the future, technical prowess and bureaucratic bungling, lies and reality. Healthcare.gov shows us what we are, and the view is disturbing. We are a people of great wealth and great need. We are a surveillance state and a welfare state. We are slowly and painfully trying to build something that can reach beyond our squabbles and our inertia and take us somewhere new. We are preoccupied with ourselves and our pasts. When we enter our data into the system, and the system responds that it cannot verify our identity, we do not know where the fault lies — in the flashing electronic pulses light-years away, or in ourselves.

Healthcare.gov shows us the uncomfortable truth that we are becoming strangers to ourselves and each other. And perhaps we always were.