itemaps are probably the most crucial pages in all Web sites. They might not be most helpful to human visitors, but they greatly assist crawlers, thereby attracting, promoting and inviting more traffic from search engines. Site maps can be perceived as ‘crawling maps’, regardless of how shallow or deep they are. These maps not just the spine, but possibly the entire skeleton of complex Web sites where crawling will take a huge number of distinct routes through pages.

Google have recently introduced RSS sitemaps. This means that new site content will be appended to the map whose form is a long aggregated feed, i.e. links with minimal content and without unimportant media and layout detail. This move by Google encouraged many Webmasters to go on the RSS wagon and XML their Web sites. This benefits Google in a variety of ways. First of all, there is a clear pairing between content and dates. If the site delivers timely news, this will become a significant factor. Secondly, as any site is described fully by its map, there is no need for repeated crawling. There are significant savings in terms of bandwidth, which appear to allow search engines to crawl a sparser portion of the Web, as well as reduce the burden on Web servers, whose load involves a great deal of pages being served to crawlers or robots.

There are hidden dangers in moving towards RSS sitemaps. Typical HTML/XHTML/other site maps get neglected as they are laborious to maintain and as time goes by they may appear redundant, much like an older generation of pages that have gone completely out of date. One of the worrying implications is that opponents — in this particular case MSN and Yahoo predominantly — get ‘robbed’ of the true, old-styled site maps, which are conceded altogether or simply neglected in favour of RSS sitemaps. Therefore, they must follow suit and take advantage of the public RSS site maps, just as Google did. They might need to woo Webmasters and get those sitemap submitted to them as well. Is it going to be an easy task? Probably not, especially while Google’s impact is by far superior.

The phenomenon above is the introduction and absorbance of new technologies by force. Google are forcing, not malevolently though, trends of crawling and push for methods to change. Microsoft exhibited some similar behaviour by introducing new elements to HTML without consent from the community. They incorporated these element into Internet Explorer; they considered the Internet to be a source to serve Internet Explorer rather than the reverse, whereby the Web is open and accessible to all. By doing so, Microsoft encouraged Web developers to construct Web pages that work exclusively under Internet Explorer and slowly killed their main opponent: Netscape Navigator. But that is all history as Internet Explorer (version 6) lags behind Opera, Firefox and arguably behind Safari too.

In practical terms, do RSS sitemaps lead to any gains? We have discussed this issue to death at the primary SEO-related newsgroup. Borek expressed his skepticism about Google sitemaps:

So far Google just fetches my sitemaps 4 times a day. One site is PR3 5 months old, second is PR2 several years old, redesigned in June. No signs of crawl on either (and there are not spidered pages on both sites).

The bottom line, from my point-of-view, is that for news-delivering sites, RSS sitemaps provide a good opportunity to conquer valuable SERP‘s very quickly. For most standard sites with a decent amount of bandwidth to spare, RSS sitemaps appear like an overkill, even when HTML to XML implementation are virtually available ‘off-the-shelf’. RSS sitemaps may also be valuable for blogs where the nature of publication is linear rather than hierarchical or lateral.

Other related threads on the topic:

• Re: Google sitemaps
• Re: Google Site-map and logs

Related News: Google to Patent Ads in Feeds

Let us think of nature for a moment. Think of rabbits whose reproduction rate is potentially very high. Once you have too many rabbits in the wild, not enough resources exist in nature in order to feeds them all. What’s more, predators that feed on rabbit will prosper and rapidly bring the number of rabbit down, thereby quickly reproducing themselves — the predators. It is a cycle in nature that has been explained by biologists many times in the past.

The same principles apply to the Web. When demand for Web sites is high (e.g. the birth in the early nineties), smashing success hits the few trailblazers and other envious business follow suit by joining. Yet, at some stage, the bubble must burst and there will be no more place for new sites to be accommodated. Is Internet reaching an anti-climax? I doubt so. According to one source, we are now observing a second dot-com boom develop. According to insightful predictions at Wired Magazine, we are yet to see the best of the Web:

2015

The Web continues to evolve from a world ruled by mass media and mass audiences to one ruled by messy media and messy participation. How far can this frenzy of creativity go? Encouraged by Web-enabled sales, 175,000 books were published and more than 30,000 music albums were released in the US last year. At the same time, 14 million blogs launched worldwide. All these numbers are escalating. A simple extrapolation suggests that in the near future, everyone alive will (on average) write a song, author a book, make a video, craft a weblog, and code a program. This idea is less outrageous than the notion 150 years ago that someday everyone would write a letter or take a photograph.

…

By 2015, desktop operating systems will be largely irrelevant. The Web will be the only OS worth coding for. It won’t matter what device you use, as long as it runs on the Web OS. You will reach the same distributed computer whether you log on via phone, PDA, laptop, or HDTV.

I have made merely identical predictions in the past:

• Blogs Recession
• Towards On-Line Operating Systems

According to Bloomberg, innovation definitely required motivation:

Microsoft has had the ability to develop a satellite map service for MSN since 1998. Chairman Bill Gates decided in April, the same month Google released its version, to rush the project. He gave his engineers a deadline of 100 days. Stephen Lawler, who runs the Virtual Earth group, and his team figured they would need about a year to get the satellite mapping technology ready.

Also on the same topic, it is rather frustrating that it took several years until Microsoft decided to fix the bugs inherent in Internet Explorer 6 and add valuable yet fundamental features like support for feeds. This move was a result of Firefox, of course, but more sadly, Internet Explorer 7 imitates Firefox, much like Virtual Earth intends to match Google Maps and show that development in the Microsoft compus is not dead, yet. Note the subtle use of the word development, not innovation.

Getting data more quickly using a CLI

ome time ago I came across YubNub – YubNub.org which is a command-line interface for the Web. It allows you to operate in and on the Web much more efficiently than by using the traditional GUI‘s. You can, for example, query Google for ‘saturn’ by typing in

g saturn

YubNub provides a very quick way of navigation the Web as chaining of commands is possible too and nearly 100 commands exist. The site was a great find, but I do not believe it can replace a keyboard-navigable portal (see example). Maybe I can embed the YubNub command-line in portals at some stage, which would make it ever more flexible. While on the issue of portals, Google have extended features offered by their Portal (personalised homepage) service. Among the new features:

• Adding your own bookmarks
• Selecting from more news feeds
• Adding your own RSS news feeds

This news item comes from MOSNEWS.COM:

Kushnir, 35, headed the English learning centers the Center for American English, the New York English Centre and the Centre for Spoken English, all known to have aggressive Internet advertising policies in which millions of e-mails were sent every day.

For harassment purposes, their telephone numbers were disclosed in Wikipedia.

Figures have recently come out, which break down search engine usage in the States. According to one report, Google has been losing some of its impact

Google’s market share of US searches for June 2005 was at 36.9% compared with 37.5% in May 2005.

Then again, according a different study which was cited on the very same site, Google’s usage has been rising lately.

Google’s share of U.S. searches hit 52% in June, up from 45% a year ago, according to Web analytics firm WebSideStory Inc.

A careful read would reveal the obvious. The total volume of queries declines throughout the summer and likewise Web traffic in general. Google is probably more prolific outside the United States and among IT-savvy users.

June 2005 referrals to this domain

Example image from the local Gallery installation

Gallery is one of the most popular photo management packages on the Web. As a matter of fact, this domain has a Gallery installation (feed: ) as well. Gallery was built in PHP and is absolutely free. The number of features it makes available to site visitors and the site administrator is endless.

A fourth beta version of Gallery 2 has just been released and a demonstration purposes site was set up to boast the features of this next generation of Gallery: interface language, tree structure navigation and nice flexible page layouts are only a few enhancement that catch the eye.

I am not too keen on the beta name though: “Flippin’ Sweet!“. Not something you can recommend to your boss…