Journal article
Indexing aids at corporate Web sites: The use of robots.txt and META tags
Information processing & management, Vol.38(2), p209
01 Mar 2002
Abstract
Sixty corporate Web sites selected from the Fortune Global 500 companies were examined in 2000 and again in 2001 to see if they provided support for automatic indexing. In particular, use of the robots.txt and Meta tags for keywords and description was examined. Slightly fewer than half of the sites provided one or both of these aids. Among sites providing indexing aids there was a clear under-representation of Asian sites. Nearly 80% of all the sites used Java, suggesting a reasonable level of technical sophistication among Web site creators. About one-third of the sites used cookies, raising the possibility that repeat visitors might find the navigation of the site customized to their needs. Overall an increase in the use of indexing aids, especially Meta tags, represents one way in which Web robots could index sites more quickly and thus improve overall index coverage of the Web.
Metrics
8 Record Views
Details
- Title
- Indexing aids at corporate Web sites: The use of robots.txt and META tags
- Creators
- M DrottM Carl Drott - Information Science (Informatics)
- Publication Details
- Information processing & management, Vol.38(2), p209
- Publisher
- Elsevier Science Ltd
- Resource Type
- Journal article
- Language
- English
- Academic Unit
- Information Science (Informatics)
- Identifiers
- 991019170600604721