Web Server robots.txt Information Disclosure

info Nessus Plugin ID 10302
New! Plugin Severity Now Using CVSS v3

The calculated severity for Plugins has been updated to use CVSS v3 by default. Plugins that do not have a CVSS v3 score will fall back to CVSS v2 for calculating severity. Severity display preferences can be toggled in the settings dropdown.

Synopsis

The remote web server contains a 'robots.txt' file.

Description

The remote host contains a file named 'robots.txt' that is intended to prevent web 'robots' from visiting certain directories in a website for maintenance or indexing purposes. A malicious user may also be able to use the contents of this file to learn of sensitive documents or directories on the affected site and either retrieve them directly or target them for other attacks.

Solution

Review the contents of the site's robots.txt file, use Robots META tags instead of entries in the robots.txt file, and/or adjust the web server's access controls to limit access to sensitive material.

See Also

http://www.robotstxt.org/orig.html

Plugin Details

Severity: Info

ID: 10302

File Name: webserver_robot.nasl

Version: 1.41

Type: remote

Family: Web Servers

Published: 10/12/1999

Updated: 11/15/2018

Dependencies: http_version.nasl