Introduction About Site Map

XML
RSS 2 Feed RSS 2 Feed
Navigation

Main Page | Blog Index

Saturday, October 22nd, 2005, 7:18 am

Collaborative Effort to Crawl the Web

Iuron

ONLY a few days ago, somebody had me aware of the Majestic-12 distributed search engine. The idea behind the engine is persistent use of other people’s computer power and bandwidth. The goal is mind is to crawl and potetially index the Web reasonably well.

This sudden ‘enlightement’, to me at least, provided somewhat of an insight. It affected the matter of practicability in my Open Source Iuron, which is in its early stages and more of a porposal at this stage. As explained before, Iuron does not index pages; it aspires to gain actual knowledge from the Internet instead. This can potentially make PageRank (or equivalents) obsolete, I believe, thereby reducing spam and search engine cheats.

Within a few days, I will be meeting the person who is arguably the father of the Semantic Web. My project will be difficult to lift off the ground without some support. Nonetheless, this now appears to be a hindrance with a simple solution. It is, after all, the kind of project where the vast requirement for bandwidth and computer power can be obtained in more or less the same way as Majestic-12. Since it is Open Source, willingness on the public’s behalf should not be a considerable peril.

On an unrelated topic which is paranoia, I recently noticed a referral reduction from Google. It became conspicuosuly significant in recent days so I thought it was an attempt to silence me. It finally turns out to have been merely a side-effect of a large-scale update at Google’s end. Many Web sites were in fact affected by this and distress became apparent in a few newsgroups. It was even pointed out that msn.com was assigned PageRank 2!

Comments are closed.

Back to top

Retrieval statistics: 21 queries taking a total of 0.124 seconds • Please report low bandwidth using the feedback form
Original styles created by Ian Main (all acknowledgements) • PHP scripts and styles later modified by Roy Schestowitz • Help yourself to a GPL'd copy
|— Proudly powered by W o r d P r e s s — based on a heavily-hacked version 1.2.1 (Mingus) installation —|