web crawler

Popular Terms
Search engine for locating information on WWW, it indexes all the words in a document, adds them to a database, then follows all hyperlinks and indexes and adds that information also to the database. Also called web spider, spider.

Use 'web crawler' in a Sentence

Altavista was the Google of its day, as it was often the first destination of dialup users looking for the accurately indexed content the web crawler was then known for.
20 people found this helpful
You need to know how to fully optimize your site to show up on any web crawler and be seen by many people.
18 people found this helpful
I really liked the web crawler and it made me think of a spider when I imagined it being used online.
16 people found this helpful

Email Print Embed