The Modern Memex
A Short History of the World Wide Web
In 1945, Vannevar Bush wrote an essay titled “As We May Think”, which was an expanded version of a prior essay of his titled “Mechanization and the Record” from 1939. This was, quite possibly, the single most influential essay written within the world of computing despite the essay dealing with computers only briefly throughout its text. What Vannevar is addressing in this essay is the inability of any individual to make use of the collected knowledge of mankind.
The difficulty seems to be, not so much that we publish unduly in view of the extent and variety of present day interests, but rather that publication has been extended far beyond our present ability to make real use of the record.
Vannevar imagined slight elaborations on then existing technologies to photograph life’s events (especially those of scientists) with wearable cameras, to record human speech as text via a nearby machine, to automate mathematical calculation, to automate logic operations, and further to correlate these with time stamps. These methods of data collection only made the problem of useful search and retrieval worse. Vannevar stated the issue quite well:
The prime action of use is selection, and here we are halting indeed. There may be millions of fine thoughts, and the account of the experience on which they are based, all encased within stone walls of acceptable architectural form; but if the scholar can get at only one a week by diligent search, his syntheses are not likely to keep up with the current scene.
Vannevar correctly observed that a key problem in finding data is that human brains do not sort data into clean hierarchical forms but rather via association. These associations can be myriad and somewhat random. From my own observations, the easiest example of this is with the human relation to time. The vast majority of us do not recall memories in clearly chronological order. If I try to remember being ten years of age, I struggle. If I think of my fifth grade year of compulsory education instead, many memories come to mind. Among those memories specific events, specific smells, specific songs, certain people, certain automobiles, or even more specifically for my brain a certain computer can then help to arrange memories of that year. This realization that the mind works by associations pushed him to envision the memex.
The memex is a private library of books, records, pictures, newspapers, and communications housed in a desk. The data is stored on microfilm, and can then be projected onto screens on the desk’s top where a keyboard, buttons, and levers for the operation of the memex are likewise found. The memex also has a spot on its top where any physical media can be placed; a lever then pulled will capture a photo of the media and store it on microfilm. All of this data is entered into the memex with mnemonic codes to identify it, and all data can be searched and retrieved via those codes. In the case of a book, pages within can be rapidly projected and scanned until the desired page is found, and even skipped by tens or hundreds. Any individual projection can be annotated by keyboard or stylus, the data stored, and later recalled. Due to having multiple screens, multiple microfilm projections can be viewed at any given time. The key advantage of the memex, however, is the association of one microfilm to any other via associative indexing. An association can be created by the user via a new unique mnemonic code, and thereby multiple items can be recalled simultaneously via the single mnemonic. There’s then the ability to have a series of mnemonics connected together by the data they represent, and then to rapidly cycle through them following their trail.
Vannevar felt that the memex could then have ready made content created and sold explicitly for it. One example he mentions is an encyclopedia where entries that relate to each other would have associative indexes. More interestingly, he thought about physicians being able to rapidly consult former cases of a given affliction to gain insights into a current patient.
While the memex was not built in anyway that would be directly recognizable to Bush, it influenced two men rather signicantly, Ted Nelson and Doug Engelbart. Ted Nelson worked on a hypertext system called Project Xanadu, and Douglas Engelbart created the oN-Line System which was shown with a mouse, a keyboard, video conferencing, hypertext, word processing, hypermedia, dynamic file linking, bootstrapping, and real time collaborative editing in 1968.
Sir Timothy John Berners-Lee was born on the 8th of June in 1955. At a young age, he attended a school between two train tracks. This provided him ample time to engage in trainspotting (watching trains and writing down the numbers of each engine). The young Berners-Lee found this interesting enough to begin model railroading, and this hobby helped to teach him a bit about electronics as he built gadgets to control the movement of his model locomotives. His parents both worked on the Ferranti Mark-I, and they encouraged his interest in electronics. Over dinner in the Berners-Lee household, discussions about electronics and computers weren’t uncommon. He then attenended The Queen’s College in Oxford from 1973 to 1976 earning his Bachelor’s in physics.
Berners-Lee found employment at Plessey following his graduation where he worked as an engineer, and then moved on to D. G. Nash in 1978 where he helped make typesetting software for printers as well as a multitasking operating system. The next move was in the latter half of 1980. He worked as a contractor for CERN. He was fascinated by the structure and relationships among people and projects there, and he decided to write a program that would allow him to trace implications of that structure. This was called Enquire, named after an old book he had found in his parents’ personal library: Enquire Within Upon Everything. The tool he created was a hypertext program that ran on an 80x24 terminal, and the links it made would imply a relationship such as person made X, or X was made by person, or this uses that, or that is described by this. These links were bidirectional, so if a link were added in one place the reverse link would automatically be made. Enquire went with Berners-Lee and was ported from the Nosk Data Nord 10, to the PC, and then to VAX/VMS. He found the tool useful just in making the links between data, and he didn’t get too far with his planned tracing and analysis tools.
At this point, Berners-Lee wasn’t aware of Nelson’s work or Engelbart’s work, but as he continued to work on Enquire he became aware of both. According to him, their ideas helped to refine his own and ultimately helped him later. He does note, however, that by the time he started working on hypertext the idea was known and already in use elsewhere.
From 1981 to 1984, Berners-Lee worked at Image Computer Systems Ltd. There he developer real time control firmware, graphics software, communications software, and a macro language. After this, he became a fellow at CERN working on distributed real-time systems for scientific data acquisition and for systems control.
While working at CERN for the second time, Berners-Lee was still using Enquire to keep track of software, users, documents, and other things he needed to keep information on for his projects. He began to realize that most of the coordination of the project was keeping his documentation in Enquire up-to-date, and then making others aware of changes. He also realized that a large amount of time was spent trying to get similar information from others.. He knew that he needed something like Enquire, but it would need to be accessible by all, scalable, and easily linkable. He made a proposal to his boss, Mike Sendall, who allowed him to proceed with it.
The next step was evaluating a development machine for this new hypertext project. He knew that he didn’t want to deal with writing characters to a screen, didn’t want to worry about fonts, didn’t want to deal with building most of his tooling. NeXT was selling NeXT cubes which shipped with NeXTStep. The programming environment shipped with multifont editable text objects which Berners-Lee felt would be easily turned into hypertext objects. He made the request to purchase a NeXT cube, and Sendall again said okay. The prototype took a month to build. The editor took another month. By Christmas of 1990, the World Wide Web was working.
In the Summer of 1991, Tim Berners-Lee made the server, the NeXTStep client, and the line-mode client available on the internet via FTP, and posted information to that effect on usenet (alt.hypertext). Reactions were mixed, but as Berners-Lee himself stated: the people of the internet built the web.
There was pressure from users to create a web client for X Windows, but by this point he was out of resources (early 1992). As a result, he began looking for volunteers. Students at Helsinki University of Technology stepped up to create Erwise. Around the same time, Pei Wei created ViolaWWW at Berkeley. Wei had first created the Viola programming language, which was far ahead of its time. The language is object oriented and it is implemented through a visual interface. Pei Wei specifically designed Viola to cope with complicated behavioral simulations, and he built concurrency into the language. To get ViolaWWW running, one first had to install the Viola language, and then install WWW as a Viola application. Browsers were also written for tk, curses, motif, and Macintosh by other people and teams. Also around this time, the Lynx browser was released. Lynx is a text mode browser that supports many different systems, and is still around today.
ViolaWWW was the first browser to gain wide acceptance. At CERN, the folks running the computing department would point the interested to ViolaWWW, and word of mouth did quite a bit from there. At the National Center for Supercomputing Applications (NCSA) at University of Illinois at Urbana Champlain, David Thompson began using ViolaWWW. When he showed this to the Software Design Group, one student in particular was very interested. Marc Andreessen chose to build a WWW client for X Windows as his project starting in late 1992. Andreessen was a driven man. He apparently drank quite a bit of espresso, turned out new versions hourly, and delivered Mosaic to the world. It was easy to install, and it provided a point-and-click interface to the World Wide Web on the 23rd of January in 1993.
Mosaic was written in C, supported HTTP, FTP, NNTP, telnet, WAIS, Archie, and Gopher. It displayed images in-line, could have multiple windows open at one time, supported document printing, could support multiple fonts, had keyword search, had a hotlist (essentially bookmarks in a sidebar), and it was ported to all of the major systems of the day: Amiga, Macintosh, UNIX, VMS, OS/2, and Windows. Contrary to popular belief, Mosaic was not the first graphical browser, but it was the first browser to support the submission of forms to a server.
In January of 1994, Jim Clark was leaving Silicon Graphics. He had downloaded Mosaic, and he’d spent an entire day exploring both it and the web. He wanted to meet the creators of the browser, and an email of his reached Marc Andreessen. They met at Cafe Verona in Palo Alto at 7AM. Andreessen recalls this as the first time he’d woken at 7AM instead of still having been awake at 7AM in over four years. As a result, he was quite tired. Clark told him, “I want to start a new thing. I don't know what it is yet. But I want to figure it out, and I'm looking for people to do it with me.” The two had several more meetings to discuss ideas. They spoke at Clark’s home, as well as at restaurants, and often with others in attendance.
Clark and Andreessen eventually wrote a business plan for an online gaming company, and they were talking to Nintendo. The plan was to have this available for the upcoming Nintendo 64, and Nintendo had agreed to provide funding. Unfortunately, as negotiations went on, Nintendo wanted to have full ownership, and Clark and Andreessen backed out.
Sitting in Clark’s living room, the two were out of ideas. They decided to hire the team from NCSA who had worked with Andreessen on Mosaic, and to then create a “Mosaic killer.” Clark had no idea how they were going to make money doing it, and said something to that effect. Despite that, he chose to fund it and told Andreessen to hire the team. They founded Mosaic Communications Corporation on the 4th of April in 1994. John Giannandrea recalls:
I was employee No. 18. I joined very early on with a bunch of people from Silicon Graphics. Jim had hired what we called the NCSA kids, so basically all the people who ever worked on a web browser were hired in like one week. Then he hired an equal number of seasoned SGI engineers. They hired all these young kids who had never shipped a product before. And then they hired a bunch of people who worked at SGI and had shipped very complicated products to FORTUNE 500 companies and defense contractors many times over. And that was the first 30 employees or so of Netscape.
The buzz and the hype started early. In May of 1994, the New York Times ran a headline “New Venture in Cyberspace by Silicon Graphics Founder” which mentioned both the Mosaic browser and its quick growth, as well as the new company. The article also had to explain what the World Wide Web is as most readers wouldn’t have known at the time.
Clark wanted to be the first to market with a commercial web browser, and this put pressure on the team to develop quickly. One would assume that quick development wasn’t much of an issue for Andreessen. At the same time, Clark was starting to fear for himself financially. He was funding this out of his of his own pocket. Initially, Clark approached VCs who’d backed SGI, namely New Enterprise Associates (NEA). Clark was asking for a valuation of $18 million (unheard of at the time), and this was far more than NEA was comfortable with spending. NEA then invited Kleiner Perkins to talk to Clark in the hopes that they’d be able to convince Clark to lower the valuation. Instead, Kleiner Perkins invested $5 million into Mosaic CC at a $21 million valuation.
The University of Illinois owned Mosaic so the team had had to start from scratch. They also needed to deliver a better product than NCSA Mosaic. Unfortunately, the University sued them early in their company’s life, and they were forced to change the name and pay the school quite a sum of money. The name became Netscape Communications Corporation on the 14th of November.
The first beta was released on the 13th of October. The release of Netscape Navigator 1.0 was on the 15th of December in 1994. The 1.0 release was made available for download at midnight. The server had been set to play the sound of a cannon being fired every time a download occurred. The first downloads were from Japan and Australia (they were awake), and while things started slowly, the pace quickened. By morning, tens of thousands of copies had been downloaded. That cannon sound was disabled rather quickly. By the 1st of January in 1995, downloads were well into the millions.
At this point, the company was nine months old, had spent much of their $9 million in funding, and they’d not made a single dollar in sales. This changed with corporate licensing for browsers. Sales for the first quarter of 1995 were around $5 million, and this jumped to $12 million for the second quarter. Netscape filed to go public on the 23rd of June 1995 with the IPO taking place on the 9th of August, just sixteen months after formation. The initial price was $28/share with a high of $75/share and a closing price on the day of $58.25/share. This gave Netscape a market value of $2.9 billion. Netscape went public without being profitable, but it had shown a doubling in revenue (minimum), for each quarter. Greg Sands recalls:
We had doughnuts and bagels laid out that morning, and we all knew that stocks started trading at 6:30 in the morning California time, and so a bunch of people showed up early. We sat around for an hour or an hour and a half, and nothing was happening. We didn't know what to make of it. So we all just went back to work. Someone grabbed me during the day and said, "Dial Charles Schwab, 1-800-SCHWAB," and a voice said, "Welcome to Charles Schwab. If you're interested in the Netscape IPO, press one." That was when it was clear to me that this was really a big deal.
Netscape was also selling the server software that enabled many of the advanced features found on the web at the time. This meant things like SSL, and form submissions. Both products (server and client) served to make Netscape a good deal of money. The next releases would see Java integrated into the browser, the introduction of a mail client, the addition of JavaScript, frames, cookies, and more. The problem with this kind of explosive growth is the message it sends to the wider market. The message was received.
Microsoft wasn’t idle. In August of 1995, Microsoft released Internet Explorer 1.0, which was built on licensed software from Spyglass, who’d themselves licensed NCSA Mosaic. This first version of IE was included in the Windows 95 Plus! package in August.
Still Netscape’s browser (now called Navigator as they also shipped a mail client, contact manager, and a WYSIWYG HTML editor that were together a suite) was strong. Revenue in 1995 was $85 million, the stock price hit $171/share, and market share for Navigator was at 90%. Revenues went to $346 million in 1996, and the total number of downloads of Netscape Navigator hit 45 million by August of 1996.
Microsoft’s next release, Internet Explorer 2.0, was made available to all Windows users at no monetary cost. This lack of cost included enterprise customers, a sign of things to come. Internet Explorer 3.0 was the first truly serious competitor to Netscape’s suite of software. IE3 in August 1996 brought the first implementation of CSS to market, and it supported ActiveX, Java, inline multimedia, and it was bundled with Internet Mail and News, NetMeeting, and Address Book. This version was itself bundled with Windows 95 OSR2, and had both 16 bit and 32 bit versions. Microsoft hadn’t just responded, they’d responded well. Market share for IE would grow to 30% during the time IE3 was current.
IE4 was released in September of 1997, and was included in Windows 98. The browser was also made available for NT, older Windows versions, Mac OS 7+, Solaris, and HP-UX. This version introduced support for Dynamic HTML, PNGs, favicons, group policy controls, site subscriptions (allowing for notifications of site updates), and better integration with other software such as: the desktop (via active desktop), Frontpage, NetMeeting, NetShow, Chat, RealPlayer, Outlook Express (which replaced Internet Mail and News), and Channels (which was similar to RSS). A key change with IE4 was Microsoft’s own browser rendering engine (Trident), which allowed developers to bring HTML and web content into their own applications.
At around 1AM on the 1st of October in 1997, some engineers from Microsoft placed a large letter ‘e’ (the IE logo) on the front lawn of Netscape’s headquarters, along with a greeting card that read: It's just not fair. Good people shouldn't have to feel bad. Best wishes, the IE team. There was also a balloon that said: We love you. Netscape employees responded by knocking down the IE logo and placing their mascot on top of it.
In the fourth quarter of 1997, Netscape posted an operating loss of $132 million. In January of 1998, Netscape made Navigator free to the general public, began a large series of layoffs, and started the open source software oriented Mozilla Foundation. The starting code base for Mozilla was version 5 of Netscape’s suite. On the 17th of March in 1999, Netscape was acquired by AOL. On the 15th of July in 2003, Time Warner (AOL and Time Warner merged on the 11th of January in 2001 and dropped AOL from their name around this same time) disbanded Netscape.
Internet Explorer began to dominate the market quickly, peaking at 95% in 2004. Many claim that this dominance was due to Microsoft having included the browser with Windows, but I doubt this to be the true cause. Netscape didn’t immediately fall out of favor. IE4 was simply the superior product or at least the cheaper one, and people chose to use it. These were still the early days of the web, and people were using what worked best. This was proven later when Mozilla released the slimmed down version of Navigator named Phoenix.
Phoenix had a bit of a name issue, and was renamed to Firebird. This was then changed to Firefox. Firefox 1.0 was released on the 9th of November in 2004, and began quickly gaining market share, peaking in 2010 at 32%.
Around the time that Firefox was released, Google hired several Firefox developers who quickly put together a new browser. News of this leaked, and the first reports of a new browser surfaced in September of 2004. More serious development began in 2006 under the leadership of Sundar Pichai. The first beta was released on the 2nd of September in 2008, and the first stable release was made on the 11th of December in 2008. In 2011, Chrome overtook Firefox’s spot as number 2 in market share, and in 2012 Chrome overtook IE as number 1. It has remained in that place ever since.
In the world wide web and in the tools that humans have built to view it, the vision of Vannevar Bush’s memex has been made real. It works quite differently from how he imagined it, but the memex of today fits in a person’s pocket.