Web Server robots.txt Information Disclosure

This script is Copyright (C) 2005-2014 Tenable Network Security, Inc.


Synopsis :

The remote web server contains a 'robots.txt' file.

Description :

The remote host contains a file named 'robots.txt' that is intended to
prevent web 'robots' from visiting certain directories in a website for
maintenance or indexing purposes. A malicious user may also be able to
use the contents of this file to learn of sensitive documents or
directories on the affected site and either retrieve them directly or
target them for other attacks.

See also :

http://www.robotstxt.org/wc/exclusion.html

Solution :

Review the contents of the site's robots.txt file, use Robots META tags
instead of entries in the robots.txt file, and/or adjust the web
server's access controls to limit access to sensitive material.

Risk factor :

None

Family: Web Servers

Nessus Plugin ID: 10302 ()

Bugtraq ID:

CVE ID: