Scrapy is a high-level web crawling and scraping framework for Python. If you use `HttpAuthMiddleware` (i.e. the `http_user` and `http_pass` spider attributes) for HTTP authentication, all requests will expose your credentials to the request target. This includes requests generated by Scrapy components, such as `robots.txt` requests sent by Scrapy when the `ROBOTSTXT_OBEY` setting is set to `True`, or as requests reached through redirects. Upgrade to Scrapy 2.5.1 and use the new `http_auth_domain` spider attribute to control which domains are allowed to receive the configured HTTP authentication credentials. If you are using Scrapy 1.8 or a lower version, and upgrading to Scrapy 2.5.1 is not an option, you may upgrade to Scrapy 1.8.1 instead. If you cannot upgrade, set your HTTP authentication credentials on a per-request basis, using for example the `w3lib.http.basic_auth_header` function to convert your credentials into a value that you can assign to the `Authorization` header of your request, instead of defining your credentials globally using `HttpAuthMiddleware`.

The remote Ubuntu 18.04 LTS / 20.04 LTS / 22.04 LTS / 24.04 LTS host has packages installed that are affected by multiple vulnerabilities as referenced in the USN-7476-1 advisory.

    It was discovered that Scrapy improperly exposed HTTP authentication

    credentials to request targets, including during redirects. An attacker

    could use this issue to gain unauthorized access to user accounts. This issue only affected Ubuntu 18.04     LTS and Ubuntu 20.04 LTS. (CVE-2021-41125)

    It was discovered that Scrapy did not remove the cookie header during cross-domain redirects. An attacker     could possibly use this issue to gain unauthorized access to user accounts. This issue only affected     Ubuntu 18.04 LTS, Ubuntu 20.04 LTS, and Ubuntu 22.04 LTS. (CVE-2022-0577)

    It was discovered that Scrapy inefficiently parsed XML content. An attacker could use this issue to cause     a denial of service by sending a crafted XML response. This issue only affected Ubuntu 18.04 LTS, Ubuntu     20.04 LTS, and Ubuntu 22.04 LTS. (CVE-2024-1892)

    It was discovered that Scrapy did not properly check response size during decompression. An attacker could     send a crafted response that would exhaust memory and cause a denial of service. This issue only affected     Ubuntu 18.04 LTS, Ubuntu 20.04 LTS, and Ubuntu 22.04 LTS. (CVE-2024-3572)

    It was discovered that Scrapy did not remove the authorization header during cross-domain redirects. An     attacker could possibly use this issue to gain unauthorized access to user accounts. This issue only     affected

    Ubuntu 18.04 LTS, Ubuntu 20.04 LTS, and Ubuntu 22.04 LTS. (CVE-2024-3574)

    It was discovered that Scrapy did not remove the authorization header during redirects that change scheme     but remain in the same domain. This issue could possibly be used by an attacker to expose sensitive

    information or to gain unauthorized access to user accounts. (CVE-2024-1968)

Tenable has extracted the preceding description block directly from the Ubuntu security advisory.

Note that Nessus has not tested for these issues but has instead relied only on the application's self-reported version number.

The remote Debian 9 host has packages installed that are affected by multiple vulnerabilities as referenced in the dla-2950 advisory.

    It was found that Scrapy, a framework for extracting data from websites, could send HTTP Authorization as     well as cookies to other domains in case of redirections, possibly leaking user credentials. For Debian 9     stretch, these problems have been fixed in version 1.0.3-2+deb9u1. We recommend that you upgrade your     python-scrapy packages. For the detailed security status of python-scrapy please refer to its security     tracker page at: https://security-tracker.debian.org/tracker/python-scrapy Further information about     Debian LTS security advisories, how to apply these updates to your system and frequently asked questions     can be found at: https://wiki.debian.org/LTS

Tenable has extracted the preceding description block directly from the Debian security advisory.

Note that Nessus has not tested for these issues but has instead relied only on the application's self-reported version number.

ID	Name	Product	Family	Severity
235158	Ubuntu 18.04 LTS / 20.04 LTS / 22.04 LTS / 24.04 LTS : Scrapy vulnerabilities (USN-7476-1)	Nessus	Ubuntu Local Security Checks	medium
158996	Debian DLA-2950-1 : python-scrapy - LTS security update	Nessus	Debian Local Security Checks	medium

Detections

Analytics

CVE-2021-41125

medium

Tenable Plugins

medium

medium