Free For All - How Linux and the Free Software Movement Undercut the High Tech Titans, Peter Wayner

17. Money

Everyone who's made it past high school knows that money changes everything. Jobs disappear, love crumbles, and wars begin when money gets tight. Of course, a good number of free source believers aren't out of high school, but they'll figure this out soon enough. Money is just the way that we pay for things we need like food, clothing, housing, and of course newer, bigger, and faster computers.

The concept of money has always been the Achilles heel of the free software world. Everyone quickly realizes the advantages of sharing the source code with everyone else. As they say in the software business, “It's a no-brainer.” But figuring out a way to keep the fridge stocked with Jolt Cola confounds some of the best advocates for free software.

Stallman carefully tried to spell out his solution in the GNU Manifesto. He wrote, "There's nothing wrong with wanting pay for work, or seeking to maximize one's income, as long as one does not use means that are destructive. But the means customary in the field of software today are based on destruction.

“Extracting money from users of a program by restricting their use of it is destructive because the restrictions reduce the amount and the way that the program can be used. This reduces the amount of wealth that humanity derives from the program. When there is a deliberate choice to restrict, the harmful consequences are deliberate destruction.”

At first glance, Richard Stallman doesn't have to worry too much about making ends meet. MIT gave him an office. He got a genius grant from the MacArthur Foundation. Companies pay him to help port his free software to their platforms. His golden reputation combined with a frugal lifestyle means that he can support himself with two months of paid work a year. The rest of the time he donates to the Free Software Foundation. It's not in the same league as running Microsoft, but he gets by.

Still, Stallman's existence is far from certain. He had to work hard to develop the funding lines he has. In order to avoid any conflicts of interest, the Free Software Foundation doesn't pay Stallman a salary or cover his travel expenses. He says that getting paid by corporations to port software helped make ends meet, but it didn't help create new software. Stallman works hard to raise new funds for the FSF, and the money goes right out the door to pay programmers on new projects. This daily struggle for some form of income is one of the greatest challenges in the free source world today.

Many other free software folks are following Stallman's tack by selling the services, not the software. Many of the members of the Apache Webserver Core, for instance, make their money by running websites. They get paid because their customers are able to type in www.website.com and see something pop up. The customer doesn't care whether it is free software or something from Microsoft that is juggling the requests. They just want the graphics and text to keep moving.

Some consultants are following in the same footsteps. Several now offer discounts of something like 25 percent if the customer agrees to release the source code from the project as free software. If there's no great proprietary information in the project, then customers often take the deal. At first glance, the consultant looks like he's cutting his rates by 25 percent, but at second glance, he might be just making things a bit more efficient for all of his customers. He can reuse the software his clients release, and no one knows it better than he does. In time, all of his clients share code and enjoy lower development costs.

The model of selling services instead of source code works well for many people, but it is still far from perfect. Software that is sold as part of a shrink-wrapped license is easy for people to understand and budget. If you pay the price, you get the software. Services are often billed by the hour and they're often very open-ended. Managing these relationships can be just as difficult as raising some capital to write the software and then marketing it as shrink-wrapped code.

17.1. Cygnus--One Company that Grew Rich on Free Software

There have been a number of different success stories of companies built around selling free software. One of the better-known examples is Cygnus, a company that specializes in maintaining and porting the GNU C Compiler. The company originally began by selling support contracts for the free software before realizing that there was a great demand for compiler development.

The philosophy in the beginning was simple. John Gilmore, one of the founders, said, “We make free software affordable.” They felt that free software offered many great tools that people needed and wanted, but realized that the software did not come with guaranteed support. Cygnus would sell people contracts that would pay for an engineer who would learn the source code inside and out while waiting to answer questions. The engineer could also rewrite code and help out.

David Henkel-Wallace, one of the other founders, says, “We started in 1989 technically, 1990 really. Our first offices were in my house on University Avenue [in Palo Alto]. We didn't have a garage, we had a carport. It was an apartment complex. We got another apartment and etherneted them together. By the time we left, we had six apartments.”

While the Bay Area was very technically sophisticated, the Internet was mainly used at that time by universities and research labs. Commercial hookups were rare and only found in special corners like the corporate research playpen, Xerox PARC. In order to get Net service, Cygnus came up with a novel plan to wire the apartment complex and sell off some of the extra bandwidth to their neighbors. HenkelWallace says, “We started our own ISP [Internet Service Provider] as a cooperative because there weren't those things in those days. Then people moved into those apartments because they were on the Internet.”

At the beginning, the company hoped that the free software would allow them to offer something the major manufacturers didn't: cross-platform consistency. The GNU software would perform the same on a DEC Alpha, a Sun SPARC, and even a Microsoft box. The manufacturers, on the other hand, were locked up in their proprietary worlds where there was little cross-pollination. Each company developed its own editors, compilers, and source code tools, and each took slightly different approaches.

One of the other founders, Michael Tiemann, writes of the time: “When it came to tools for programmers in 1989, proprietary software was in a dismal state. First, the tools were primitive in the features they offered. Second, the features, when available, often had built-in limitations that tended to break when projects started to get complicated. Third, support from proprietary vendors was terrible . . . finally, every vendor implemented their own proprietary extensions, so that when you did use the meager features of one platform, you became, imperceptibly at first, then more obviously later, inextricably tied to that platform.”

The solution was to clean up the GNU tools, add some features, and sell the package to people who had shops filled with different machines. Henkel-Wallace said, “We were going to have two products: compiler tools and shell tools. Open systems people will buy a bunch of SGIs, a bunch of HPs, a bunch of Unix machines. Well, we thought people who have the same environment would want to have the same tools.”

This vision didn't work out. They sold no contracts that offered that kind of support. They did find, however, that people wanted them to move the compiler to other platforms. “The compilers people got from the vendors weren't as good and the compiler side of the business was making money from day one,” says Henkel-Wallace.

The company began to specialize in porting GCC, the GNU compiler written first by Richard Stallman, to new chips that came along. While much of the visible world of computers was frantically standardizing on Intel chips running Microsoft operating systems, an invisible world was fragmenting as competition for the embedded systems blossomed. Everyone was making different chips to run the guts of microwave ovens, cell phones, laser printers, network routers, and other devices. These manufacturers didn't care whether a chip ran the latest MS software, they just wanted it to run. The appliance makers would set up the chip makers to compete against each other to provide the best solution with the cheapest price, and the chip manufacturers responded by churning out a stream of new, smaller, faster, and cheaper chips.

Cygnus began porting the GCC to each of these new chips, usually after being paid by the manufacturer. In the past, the chip companies would write or license their own proprietary compilers in the hope of generating something unique that would attract sales. Cygnus undercut this idea by offering something standard and significantly cheaper. The chip companies would save themselves the trouble of coming up with their own compiler tools and also get something that was fairly familiar to their customers. Folks who used GCC on Motorola's chip last year were open to trying out National Semiconductor's new chip if it also ran GCC. Supporting free software may not have found many takers, but Cygnus found more than enough people who wanted standard systems for their embedded processors.

Selling processor manufacturers on the conversion contracts was also a bit easier. Businesses wondered what they were doing paying good money for free software. It just didn't compute. The chip manufacturers stopped worrying about this when they realized that the free compilers were just incentives to get people to use their chips. The companies spent millions buying pens, T-shirts, and other doodads that they gave away to market the chips. What was different about buying software? If it made the customers happy, great. The chip companies didn't worry as much about losing a competitive advantage by giving away their work. It was just lagniappe.

Cygnus, of course, had to worry about competition. There was usually some guy who worked at the chip company or knew someone who worked at the chip company who would say, “Hey, I know compilers as well as those guys at Cygnus. I can download GCC too and underbid them.”

Henkel-Wallace says, “Cygnus was rarely the lowest bidder. People who cared about price more than anyone else were often the hardest customers anyway. We did deals on a fair price and I think people were happy with the result. We rarely competed on price. What really matters to you? Getting a working tool set or a cheap price?”

17.2. How the GPL Built Cygnus's Monopoly

The GNU General Public License was also a bit of a secret weapon for Cygnus. When their competitors won a contract, they had to release the source code for their version when they were done with it. All of the new features and insights developed by competitors would flow directly back to Cygnus.

Michael Tiemann sounds surprisingly like Bill Gates when he speaks about this power: “Fortunately, the open source model comes to the rescue again. Unless and until a competitor can match the one hundred-plus engineers we have on staff today, most of whom are primary authors or maintainers of the software we support, they cannot displace us from our position as the 'true GNU' source. The best they can hope to do is add incremental features that their customers might pay them to add. But because the software is open source, whatever value they add comes back to Cygnus. . . .”

Seeing these effects is something that only a truely devoted fan of free software can do. Most people rarely get beyond identifying the problems with giving up the source code to a project. They don't realize that the GPL affects all users and also hobbles the potential competitors. It's like a mutual disarmament or mutual armament treaty that fixes the rules for all comers and disarmament treaties are often favored by the most powerful.

The money Cygnus makes by selling this support has been quite outstanding. The company continues to grow every year, and it has been listed as one of the largest and fastest-growing private software companies. The operation was also a bootstrap business where the company used the funds from existing contracts to fund the research and development of new tools. They didn't take funding from outside venture capital firms until 1995. This let the founders and the workers keep a large portion of the company, one of the dreams of every Silicon Valley start-up. In 1999, Red Hot merged with Cygnus to “create an open source powerhouse.”

The success of Cygnus doesn't mean that others have found ways of duplicating the model. While Cygnus has found some success and venture capital, Gilmore says, “The free software business gives many MBAs the willies.”Many programmers have found that free software is just a free gift for others. They haven't found an easy way to charge for their work.

17.3. Snitchware

Larry McVoy is one programmer who looks at the free source world and cringes. He's an old hand from the UNIX world who is now trying to build a new system for storing the source code. To him, giving away source code is a one-way train to no money. Sure, companies like Cygnus and Red Hat can make money by adding some extra service, but the competition means that the price of this value will steadily go to zero. There are no regulatory or large capital costs to restrain entry, so he feels that the free software world will eventually push out all but the independently wealthy and the precollege teens who can live at home. “We need to find a sustainable method. People need to write code and raise families, pay mortgages, and all of that stuff,” he says.

McVoy's solution is a strange license that some call “snitchware.” He's developing a product known as BitKeeper and he's giving it away, with several very different hooks attached. He approached this philosophically. He says, “In order to make money, I need to find something that the free software guys don't value that the businesspeople do value. Then I take it away from the free software guys. The thing I found is your privacy.”

BitKeeper is an interesting type of product that became essential as software projects grew larger and more unwieldy. In the beginning, programmers wrote a program that was just one coherent file with a beginning, a middle, some digressions, and then an end. These were very self-contained and easily managed by one person.

When more than one programmer started working on a project together, however, everyone needed to work on coordinating their work with each other. One person couldn't start tearing apart the menus because another might be trying to hook up the menus to a new file system. If both started working on the same part, the changes would be difficult if not impossible to sort out when both were done. Once a team of programmers digs out from a major mess like that, they look for some software like BitKeeper to keep the source code organized.

BitKeeper is sophisticated and well-integrated with the Internet. Teams of programmers can be spread out throughout the world. At particular times, programmers can call each other up and synchronize their projects. Both tightly controlled, large corporate teams and loose and uncoordinated open source development teams can use the tool.

The synchronization creates change logs that summarize the differences between two versions of the project. These change logs are optimized to move the least amount of information. If two programmers don't do too much work, then synchronizing them doesn't take too long. The change logs build up a complete history of the project and make it possible to roll back the project to earlier points if it turns out that development took the wrong path.

McVoy's snitchware solution is to post the change logs of the people who don't buy a professional license. These logs include detailed information on how two programs are synchronized, and he figures that this information should be valuable enough for a commercial company to keep secret. They might say, “Moved auction control structure to Bob's version from Caroline's version. Moved new PostScript graphics engine to Caroline's version from Bob's.”

McVoy says, “If you're Sun or Boeing, you don't want the Internet to be posting a message like 'I just added the bomb bay.' But for the free software guys, not only is that acceptable, but it's desirable. If you're doing open source, what do you have to hide?”

BitKeeper is free for anyone to use, revise, and extend as long as they don't mess with the part that tattles. If you don't care about the world reading your change logs, then it's not much different from the traditional open source license. The user has the same rights to extend, revise, and modify BitKeeper as they do GNU Emacs, with one small exception: you can't disable the snitch feature.

McVoy thinks this is an understandable trade-off. “From the business guys you can extract money. You can hope that they'll pay you. This is an important point I learned consulting at Schwab and Morgan Stanley. They insist that they pay for the software they get. They don't want to pay nothing. I used to think that they were idiots. Now I think they're very smart,” he says.

The matter is simple economics, he explains. “They believe that if enough money is going to their supplier, it won't be a total disaster. I call this an insurance model of software.”

Companies that pay for the privacy with BitKeeper will also be funding further development. The work won't be done in someone's spare time between exams and the homecoming game. It won't be done between keeping the network running and helping the new secretary learn Microsoft Word. It will be developed by folks who get paid to do the work.

“There's enough money going back to the corporation so it can be supported,” McVoy says. “This is the crux of the problem with the open source model. It's possible to abuse the proprietary model, too. They get you in there, they lock you in, and then they rape you. This business of hoping that it will be okay is unacceptable. You need to have a lock. The MIS directors insist you have a lock.”

He has a point. Linux is a lot of fun to play with and it is now a very stable OS, but it took a fair number of years to get to this point. Many folks in the free source world like to say things like, “It used to be that the most fun in Linux was just getting it to work.” Companies like Morgan Stanley, Schwab, American Airlines, and most others live and die on the quality of their computer systems. They're quite willing to pay money if it helps ensure that things don't go wrong.

McVoy's solution hasn't rubbed everyone the right way. The Open Source Initiative doesn't include his snitchware license in a list of acceptable solutions. “The consensus of the license police is that my license is NOT open source,” he says. “The consensus of my lawyer is that it is. But I don't call it open source anymore.”

He's going his own way. “I made my own determination of what people value in the OS community: they have to be able to get the source, modify the source, and redistribute the source for no fee. All of the other crap is yeah, yeah whatever,” he says.

“The problem with the GPL is the GPL has an ax to grind, and in order to grind that ax it takes away all of the rights of the person who wrote the code. It serves the need of everyone in the community except the person who wrote it.”

McVoy has also considered a number of other alternatives. Instead of taking away something that the free software folks don't value, he considered putting in something that the businesses would pay to get rid of. The product could show ads it downloaded from a central location. This solution is already well known on the Internet, where companies give away e-mail, searching solutions, directories, and tons of information in order to sell ads. This solution, however, tends to wreck the usability of the software. Eudora, the popular e-mail program, is distributed with this option.

McVoy also considered finding a way to charge for changes and support to BitKeeper. “The Cygnus model isn't working well because it turns them into a contracting shop. That means you actually have to do something for every hour of work.”

To him, writing software and charging for each version can generate money without work--that is, without doing further work. The support house has to have someone answering the phone every moment. A company that is selling shrink-wrapped software can collect money as people buy new copies. McVoy doesn't want this cash to spend tipping bartenders on cruise ships, although he doesn't rule it out. He wants the capital to reinvest in other neat ideas. He wants to have some cash coming in so he can start up development teams looking at new and bigger projects.

The Cygnus model is too constraining for him. He argues that a company relying on support contracts must look for a customer to fund each project. Cygnus, for instance, had to convince Intel that they could do a good job porting the GCC to the i960. They found few people interested in general support of GNU, so they ended up concentrating on GCC.

McVoy argues that it's the engineers who come up with the dreams first. The customers are often more conservative and less able to see how some new tool or piece of software could be really useful. Someone needs to hole up in a garage for a bit to create a convincing demonstration of the idea. Funding a dream takes capital.

To him, the absence of money in the free software world can be a real limitation because money is a way to store value. It's not just about affording a new Range Rover and balsamic vinegars that cost more than cocaine by weight. Money can be a nice way to store up effort and transport it across time. Someone can work like a dog for a six months, turn out a great product, and sell it for a pile of cash. Ten years later, the cash can be spent on something else. The work is effectively stored for the future.

Of course, this vision isn't exactly true. Cygnus has managed to charge enough for their contracts to fund the development of extra tools. Adding new features and rolling them out into the general distribution of some GNU tool is part of the job that the Cygnus team took on for themselves. These new features also mean that the users need more support. On one level, it's not much different from a traditional software development cycle. Cygnus is doing its work by subscription while a traditional house is creating its new features on spec.

In fact, Cygnus did so well over such a long period of time that it found it could raise capital. “Once Cygnus had a track record of making money and delivering on time, investors wanted a piece of it,” says Gilmore.

Red Hat has managed to sell enough CD-ROM disks to fund the development of new projects. They've created a good selection of installation tools that make it relatively easy for people to use Linux. They also help pay salaries for people like Alan Cox who contribute a great deal to the evolution of the kernel. They do all of this while others are free to copy their distribution disks verbatim.

McVoy doesn't argue with these facts, but feels that they're just a temporary occurrence. The huge growth of interest in Linux means that many new folks are exploring the operating system. There's a great demand for the hand-holding and packaging that Red Hat offers. In time, though, everyone will figure out how to use the product and the revenue stream should disappear as competition drives out the ability to charge $50 for each disk.

Of course, the folks at Cygnus or Red Hat might not disagree with McVoy either. They know it's a competitive world and they figure that their only choice is to remain competitive by finding something that people will want to pay for. They've done it in the past and they should probably be able to do it in the future. There are always new features.

17.4. Bounties for Quicker Typer-Uppers

Some developers are starting to explore a third way of blending capital with open source development by trying to let companies and people put bounties out on source code. The concept is pretty simple and tuned to the open software world. Let's say you have an annoying habit of placing French bon mots in the middle of sentences. Although this looks stupide to your friends, you think it's quite chic. The problem is that your old word processor's spell checker isn't quite à la mode and it only operates avec une seule langue. The problem is that you've spent too much time studying français and drinking de café and not enough time studying Java, the programming language. You're très désolé by your word processor's inability to grok just how BCBG you can be and spell-check in deux languages.

The bounty system could be your savior. You would post a message saying, “Attention! I will reward with a check for $100 anyone who creates a two-language spell-checker.” If you're lucky, someone who knows something about the spell-checker's source code will add the feature in a few minutes. One hundred dollars for a few minutes' work isn't too shabby.

It is entirely possible that another person out there is having the same problem getting their word processor to verstehen their needs. They might chip in $50 to the pool. If the problem is truly grande, then the pot could grow quite large.

This solution is blessed with the wide-open, free-market sensibility that many people in the open software community like. The bounties are posted in the open and anyone is free to try to claim the bounties by going to work. Ideally, the most knowledgeable will be the first to complete the job and nab the payoff.

Several developers are trying to create a firm infrastructure for the plan. Brian Behlendorf, one of the founding members of the Apache web server development team, is working with Tim O'Reilly's company to build a website known as SourceXchange. Another group known as CoSource is led by Bernie Thompson and his wife, Laurie. Both will work to create more software that is released with free source.

Of course, these projects are more than websites. They're really a process, and how the process will work is still unclear right now. While it is easy to circulate a notice that some guy will pay some money for some software, it is another thing to actually make it work. Writing software is a frustrating process and there are many chances for disagreement. The biggest question on every developer's mind is “How can I be sure I'll be paid?” and the biggest question on every sugar daddy's mind is “How can I be sure that the software works?”

These questions are part of any software development experience. There is often a large gap between the expectations of the person commissioning the software and the person writing the code. In this shadow are confusion, betrayal, and turmoil.

The normal solution is to break the project up into milestones and require payment after each milestone passes. If the coder is doing something unsatisfactory, the message is transmitted when payment doesn't arrive. Both SourceXchange and CoSource plan on carrying over the same structure to the world of bounty-hunting programmers. Each project might be broken into a number of different steps and a price for each step might be posted in advance.

Both systems try to alleviate the danger of nonpayment by requiring that someone step in and referee the end of the project. A peer reviewer must be able to look over the specs of the project and the final code and then determine whether money should be paid. Ideally, this person should be someone both sides respect.

A neutral party with the ability to make respectable decisions is something many programmers and consultants would welcome. In many normal situations, the contractors can only turn to the courts to solve disagreements, and the legal system is not really schooled in making these kinds of decisions. The company with the money is often able to dangle payment in front of the programmers and use this as a lever to extract more work. Many programmers have at least one horror story to tell about overly ambitious expectations.

Of course, the existence of a wise neutral party who can see deeply into the problems and provide a fair solution is close to a myth. Judging takes time. SourceXchange promises that these peer reviewers will be paid, and this money will probably have to come from the people offering the bounty. They're the only ones putting money into the system in the long run. Plus, the system must make the people offering bounties happy in the long run or it will fail.

The CoSource project suggests that the developers must come up with their own authority who will judge the end of the job and present this person with their bid. The sponsors then decide whether to trust the peer reviewer when they okay the job. The authorities will be judged like the developers, and summaries of their reputation will be posted on the site. While it isn't clear how the reviewers will be paid, it is not too much to expect that there will be some people out there who will do it just for the pleasure of having their finger in the stew. They might, for instance, want to offer the bounty themselves but be unable to put up much money. Acting as a reviewer would give them the chance to make sure the software did what they wanted without putting up much cash.

One of the most difficult questions is how to run the marketplace. A wide-open solution would let the sponsors pay when the job was done satisfactorily. The first person to the door with running code that met the specs would be the one to be paid. Any other team that showed up later would get nothing.

This approach would offer the greatest guarantees of creating well-running code as quickly as possible. The programmers would have a strong incentive to meet the specs quickly in order to win the cash. The downside is that the price would be driven up because the programmers would be taking on more risk. They would need to capitalize their own development and take the chance that someone might beat them to the door. Anxious sponsors who need some code quickly should be willing to pay the price.

Another solution is to award contracts before any work is done. Developers would essentially bid on the project and the sponsor would choose one to start work. The process would be fairly formal and favor the seasoned, connected programmers. A couple of kids working in their spare time might be able to win an open bounty, but they would be at a great disadvantage in this system. Both CoSource and SourceXchange say that they'll favor this sort of preliminary negotiation.

If the contracts are awarded before work begins, the bounty system looks less like a wild free-for-all and more like just a neutral marketplace for contract programmers to make their deals. Companies like Cygnus already bid to be paid for jobs that produce open source. These market-places for bounties will need to provide some structure and efficiencies to make it worth people's time to use them.

One possible benefit of the bounty system is to aggregate the desires of many small groups. While some bounties will only serve the person who asks for them, many have the potential to help people who are willing to pay. An efficient system should be able to join these people together into one group and put their money into one pot.

CoSource says that it will try to put together the bounties of many small groups and allow people to pay them with credit cards. It uses the example of a group of Linux developers who would gather together to fund the creation of an open source version of their favorite game. They would each chip in $10, $20, or $50 and when the pot got big enough, someone would step forward. Creating a cohesive political group that could effectively offer a large bounty is a great job for these sites.

Of course, there are deeper questions about the flow of capital and the nature of risks in these bounty-based approaches. In traditional software development, one group pays for the creation of the software in the hope that they'll be able to sell it for more than it cost to create. Here, the programmer would be guaranteed a fixed payment if he accomplished the job. The developer's risk is not completely eliminated because the job might take longer than they expected, but there is little of the traditional risk of a start-up firm. It may not be a good idea to separate the risk-taking from the people doing the work. That is often the best way to keep people focused and devoted.

Each of these three systems shows how hard the free software industry is working at finding a way for people to pay their bills and share information successfully. Companies like Cygnus or BitKeeper are real efforts built by serious people who can't live off the largesse of a university or a steady stream of government grants. Their success shows that it is quite possible to make money and give the source code away for free, but it isn't easy.

Still, there is no way to know how well these companies will survive the brutal competition that comes from the free flow of the source code. There are no barriers to entry, so each corporation must be constantly on its toes. The business becomes one of service, not manufacturing, and that changes everything. There are no grand slam home runs in that world. There are no billion-dollar explosions. Service businesses grow by careful attention to detail and plenty of focused effort.