A few weeks ago, when I wrote that, "forced to choose, the average FOSS-based business is going to choose business interests over FOSS [free and open source software] every time," many people, including Mathew Aslett and Matt Assay, politely accused me of being too cynical. Unhappily, you only have to look at the relations between Red Hat and Fedora, the distribution Red Hat sponsors, during the recent security crisis for evidence that I might be all too accurate.
That this evidence should come from Red Hat and Fedora is particularly dismaying. Until last month, most observers would have described the Red Hat-Fedora relationship as a model of how corporate and community interests could work together for mutual benefit.
Although Fedora was initially dismissed as Red Hat's beta release when it was first founded in 2003, in the last few years, it had developed laudatory open processes and become increasingly independent of Red Hat. As Max Spevack, the former chair of the Fedora Board, said in 2006, the Red Hat-Fedora relationship seemed a "good example of how to have a project that serves the interests of a company that also is valuable and gives value to community members."
Yet it seems that, faced with a problem, Red Hat moved to protect its corporate interests at the expense of Fedora's interests and expectations as a community -- and that Fedora leaders were as surprised by the response as the general community.
Outline of a crisis
What happened last month is still unclear. My request a couple of weeks ago to discuss events with Paul W. Frields, the current Fedora Chair, was answered by a Red Hat publicist, who told me that the official statements on the crisis were all that any one at Red Hat or Fedora was prepared to say in public -- a response so stereotypically corporate in its caution that it only emphasizes the conflict of interests.
However, the Fedora announcements mailing list gave the essentials. On August 14, Frields sent out a notice that Fedora was "currently investigating an issue in the infrastructure systems." He warned that the entire Fedora site might become temporarily unavailable and warned that users should "not download or update any additional packages on your Fedora systems." As might be expected, the cryptic nature of this corporate-sounding announcement caused considerable curiosity, both within and without Fedora, with most people wanting to know more.
A day later, Frield's name was on another notice, saying that the situation was continuing, and pleading for Fedora users to be patient. A third notice followed on August 19, announcing that some Fedora services were now available, and providing the first real clue to what was happening when a new SSH fingerprint was released.
It was only on August 22 that Frields was permitted to announce that, "Last week we discovered that some Fedora servers were illegally accessed. The intrusion into the servers was quickly discovered, and the servers were taken offline . . . .One of the compromised Fedora servers was a system used for signing Fedora packages. However, based on our efforts, we have high confidence that the intruder was not able to capture the passphrase used to secure the Fedora package signing key."
Since then, plans for changing security keys have been announced. However, as of September 8, the crisis continues, with Fedora users still unable to get security updates or bug-fixes. Three weeks without these services might seem trivial to Windows users, but for Fedora users, like those of other GNU/Linux distribution, many of whom are used to daily updates to their system, the crisis amounts to a major disruption of service.
A conflict of cultures
From a corporate viewpoint, Red Hat's close-lipped reaction to the crisis is understandable. Like any company based on free and open source software, Red Hat derives its income from delivering services to customers, and obviously its ability to deliver services is handicapped (if not completely curtailed) when its servers are compromised. Under these circumstances, the company's wish to proceed cautiously and with as little publicity as possible is perfectly natural.
The problem is that, in moving to defend its own credibility, Red Hat has neglected Fedora's. While secrecy about the crisis may be second nature to Red Hat's legal counsel, the FOSS community expects openness.
In this respect, Red Hat's handling of the crisis could not contrast more strongly with the reaction of the community-based Debian distribution when a major security flaw was discovered in its openssl package last May. In keeping with Debian's policy of openness, the first public announcement followed hard on the discovery, and included an explanation of the scope, what users could do, and the sites where users could find tools and instructions for protecting themselves.