The Typo Trap, 4 days after launch : how did this internal linkage mayhem get crawled by search engines ? Every page there generates about 120 random internal links from 3000 keywords, plus an infinite number of misspelled links (with tag rel="nofollow"), I was wondering if it would catch some bots.
- MSN Bot came by about 8 hours after the stuff went live. And indexed 8 times the index.php page since.
- Googlebot first came about 9 hours after public launch. Then, 10 hours later, it came back, and fetched 96 pages in less than 30 seconds. How nice on server load.
- Yahoo's Slurp showed up about 3 hours after the site was launched. It has been indexing more than 8000 pages since its first visit, and counting as of writing, following even links with rel="nofollow". I guess this is part of Yahoo's "we are bigger than Google" plan.
Want to share or tweet this post? Please use this short URL: http://ozh.in/9i