Musings on the growth of the Web

How many pages are there on the World Wide Web? At the date I write this (early May 1996), the Lycos catalog claims to be indexing over 40 million pages, and the growth rate still appears to be an exponential one.

I suppose that anyone who is reading this on the Web knows already knows something of this, to say the least, but I have to confess that I am still personally astonished by the Web, both in its potential and in the actuality. It seems to me to be a quite new and unique phenomenon, still groping its way towards what it might become (it would remind me of the lines from Yeats contemplating "what rough beast, its hour come round at last, slouches towards Bethlehem to be born" if I didn't in general think the Web a positive rather than a sinister phenomenon).

The Web is increasingly becoming a kind of universal publishing medium, with enormous advantages to anyone who has anything to say (as well as to those who think they have, but really don't).

• it is very cheap to publish on
• there are essentially no distribution costs
• it is available everywhere in the world
• it is instantly updatable
• it doesn't have the environmentally damaging problems of paper publishing

...and most importantly of all:

• anyone at all, with cheap equipment and very modest skills, can publish whatever they want.

I don't have a crystal ball; I don't know exactly where all of this is leading (if I did, I suppose I could make myself rich). But it does seem to me that the Web is destined ultimately to take over most newspaper and magazine publishing, as well as scientific and specialist publishing, and almost all forms of amateur publishing. Books will, however, I suspect remain with us for quite a lot longer.

The sheer volume of material available on the web, and the thousands of pages being added every day, mean that exploring the web can be an intimidating and bewildering experience. There is something out there on every subject under the sun, if only you could find it.

And it is only going to get worse.

It is already apparent that a large percentage of web pages are essentially nothing more than indexes or guides to other web pages. Increasingly there are indexes to the indexes; and indexes to the indexes to the indexes, and so on ad infinitum. This is all as it should be: one of the great miracles of the web is that is a kind of self-organising entity.

The other fascinating thing about this self-organisation is to watch or to be a participant in the snowball effect that is shaping its development. The Web has been a great place for enthusiasts of every kind to publish something about their enthusiasm. What is fascinating is how simple pages put up by individuals about some interest, if they appear early enough, become by default the main source of information on that topic, and the snowball effect starts to work. More people find that page through indexes like Lycos; they contact the information provider; send more information; more links are added to similar pages; the page becomes more and more important as a source of information, and so on it goes.

Exactly this happened to me with my very modest pages about Wilkie Collins, a Victorian writer who was important in the early development of mystery and horror writing. I now find my pages listed all over the place in references to Victorian literature; people send me e-mail and more information; I link to other sites; I get more e-mail...; and now the daily accesses to this page about a long-dead writer who is reasonably obscure are growing every month (and are twice what I get to these pages under my Grilled Pterodactyl heading).

To me, the most interesting parts of the Web are these pages that spring from an individual's personal enthusiasms. You can keep your Time Magazine pages, or your Coca-Cola pages: the material from these sources is typically corporate and self-serving. It is the quirky and the personal that I love on the Web, and it is there that you will find the greatest creative inspiration.

Is there a topic under the sun that doesn't have one or more Web pages dedicated to it? It hardly seems possible any more. Go to any of the search engines and type in any word at random, restricting the search to those pages where your chosen word appears in the title of the page. Unless your topic is truly obscure, you will get a list of scores of pages. What is important and relevant, and what is not? Ah, there's the rub. But such a search can be entertaining even if the found pages aren't quite what you were after.

Just for fun, a month or so ago I tried this kind of search for a random word: ROCKETS. I thought I would share with you what I found.

	Deuce's Houston Rockets Page The Houston Rockets, it seems, are a highly regarded basketball team in the United States. My web search found a good half dozen web sites dedicated to the team, listing their players, their performance, their up-coming matches. Sport is a heavy presence on the web, but it doesn't interest me much.
Rocket From the Crypt Rocket from the Crypt are a Canadian rock group. This site reviews their work and lists forthcoming gigs where you can go to hear them. There's also a site dedicated to the disgustingly named Snot Rockets, but we won't go into that.
	BayNar Model Rocket Club This is more like it! A model rocket club based in Silicon Valley. Wonder if they control them with microchips...?
Andøya Rocket Range This is fascinating (to me, at least). A Norwegian site spelling out the benefits of using their space centre. Guess space launching is becoming both commercial and international. Of course, being an Australian, I should be selling you the benefits of using our own Woomera rocket range, but...
	Alex's Rocket Page This is my favourite kind of web site: very personal and touching. Alex Stokes is 6 years old. Here he is with one of his favourite model rockets (I think his dad helped him with the web page...). Can't you just see his pride, can't you just see him and his family getting a buzz out of the fact that people all over the world can potentially have a look at his page...?
And a link to my own article about my misspent youth: Rockets in Eltham Lower

With all of this variety; with all of this personal enthusiasm; with all of this power for the individual to publish their own thoughts and interests, how can the Web do anything except go from strength to strength? And how futile and misguided must be the attempts by governments of all complexions in countries ranging from China to the United States to control the freedom of expression they see there? Good luck guys, you're going to need it.

---

The title of this piece is from Tennyson's "The Lady of Shallot":

"Out flew the web and floated wide;
The mirror crack'd from side to side"

Some other more relevant references:

• Lycos - one of the best search engines on the Web
• The Book of Sand - my idiosyncratic view on first encountering the World Wide Web