Detections
Analytics

Robots.txt File Detected

info Web App Scanning Plugin ID 98705

Language:

Synopsis

Robots.txt File Detected

Description

The remote host contains a file named 'robots.txt' that is intended to prevent web 'robots' from visiting certain directories in a website for maintenance or indexing purposes. A malicious user may also be able to use the contents of this file to learn of sensitive documents or directories on the affected site and either retrieve them directly or target them for other attacks.

Solution

Review the contents of the site's robots.txt file, use Robots META tags instead of entries in the robots.txt file, and/or adjust the web server's access controls to limit access to sensitive material.

See Also

https://www.robotstxt.org

Plugin Details

Severity: Info

ID: 98705

Type: remote

Family: Web Servers

Published: 9/19/2019

Updated: 12/17/2020

Scan Template: api, basic, full, pci, scan