Scrapy is a high-level web crawling and scraping framework for Python. If you use `HttpAuthMiddleware` (i.e. the `http_user` and `http_pass` spider attributes) for HTTP authentication, all requests will expose your credentials to the request target. This includes requests generated by Scrapy components, such as `robots.txt` requests sent by Scrapy when the `ROBOTSTXT_OBEY` setting is set to `True`, or as requests reached through redirects. Upgrade to Scrapy 2.5.1 and use the new `http_auth_domain` spider attribute to control which domains are allowed to receive the configured HTTP authentication credentials. If you are using Scrapy 1.8 or a lower version, and upgrading to Scrapy 2.5.1 is not an option, you may upgrade to Scrapy 1.8.1 instead. If you cannot upgrade, set your HTTP authentication credentials on a per-request basis, using for example the `w3lib.http.basic_auth_header` function to convert your credentials into a value that you can assign to the `Authorization` header of your request, instead of defining your credentials globally using `HttpAuthMiddleware`.

The Linux/Unix host has one or more packages installed that are impacted by a vulnerability without a vendor supplied patch available.

  - Scrapy is a high-level web crawling and scraping framework for Python. If you use `HttpAuthMiddleware`     (i.e. the `http_user` and `http_pass` spider attributes) for HTTP authentication, all requests will expose     your credentials to the request target. This includes requests generated by Scrapy components, such as     `robots.txt` requests sent by Scrapy when the `ROBOTSTXT_OBEY` setting is set to `True`, or as requests     reached through redirects. Upgrade to Scrapy 2.5.1 and use the new `http_auth_domain` spider attribute to     control which domains are allowed to receive the configured HTTP authentication credentials. If you are     using Scrapy 1.8 or a lower version, and upgrading to Scrapy 2.5.1 is not an option, you may upgrade to     Scrapy 1.8.1 instead. If you cannot upgrade, set your HTTP authentication credentials on a per-request     basis, using for example the `w3lib.http.basic_auth_header` function to convert your credentials into a     value that you can assign to the `Authorization` header of your request, instead of defining your     credentials globally using `HttpAuthMiddleware`. (CVE-2021-41125)

Note that Nessus relies on the presence of the package as reported by the vendor.

The remote Ubuntu 18.04 LTS / 20.04 LTS / 22.04 LTS / 24.04 LTS host has packages installed that are affected by multiple vulnerabilities as referenced in the USN-7476-1 advisory.

    It was discovered that Scrapy improperly exposed HTTP authentication

    credentials to request targets, including during redirects. An attacker

    could use this issue to gain unauthorized access to user accounts. This issue only affected Ubuntu 18.04     LTS and Ubuntu 20.04 LTS. (CVE-2021-41125)

    It was discovered that Scrapy did not remove the cookie header during cross-domain redirects. An attacker     could possibly use this issue to gain unauthorized access to user accounts. This issue only affected     Ubuntu 18.04 LTS, Ubuntu 20.04 LTS, and Ubuntu 22.04 LTS. (CVE-2022-0577)

    It was discovered that Scrapy inefficiently parsed XML content. An attacker could use this issue to cause     a denial of service by sending a crafted XML response. This issue only affected Ubuntu 18.04 LTS, Ubuntu     20.04 LTS, and Ubuntu 22.04 LTS. (CVE-2024-1892)

    It was discovered that Scrapy did not properly check response size during decompression. An attacker could     send a crafted response that would exhaust memory and cause a denial of service. This issue only affected     Ubuntu 18.04 LTS, Ubuntu 20.04 LTS, and Ubuntu 22.04 LTS. (CVE-2024-3572)

    It was discovered that Scrapy did not remove the authorization header during cross-domain redirects. An     attacker could possibly use this issue to gain unauthorized access to user accounts. This issue only     affected

    Ubuntu 18.04 LTS, Ubuntu 20.04 LTS, and Ubuntu 22.04 LTS. (CVE-2024-3574)

    It was discovered that Scrapy did not remove the authorization header during redirects that change scheme     but remain in the same domain. This issue could possibly be used by an attacker to expose sensitive

    information or to gain unauthorized access to user accounts. (CVE-2024-1968)

Tenable has extracted the preceding description block directly from the Ubuntu security advisory.

Note that Nessus has not tested for these issues but has instead relied only on the application's self-reported version number.

There are packages installed that are affected by a vulnerability referenced in the following CVE:

  - Scrapy is a high-level web crawling and scraping framework for Python. If you use `HttpAuthMiddleware`
    (i.e. the `http_user` and `http_pass` spider attributes) for HTTP authentication, all requests will expose
    your credentials to the request target. This includes requests generated by Scrapy components, such as
    `robots.txt` requests sent by Scrapy when the `ROBOTSTXT_OBEY` setting is set to `True`, or as requests
    reached through redirects. Upgrade to Scrapy 2.5.1 and use the new `http_auth_domain` spider attribute to
    control which domains are allowed to receive the configured HTTP authentication credentials. If you are
    using Scrapy 1.8 or a lower version, and upgrading to Scrapy 2.5.1 is not an option, you may upgrade to
    Scrapy 1.8.1 instead. If you cannot upgrade, set your HTTP authentication credentials on a per-request
    basis, using for example the `w3lib.http.basic_auth_header` function to convert your credentials into a
    value that you can assign to the `Authorization` header of your request, instead of defining your
    credentials globally using `HttpAuthMiddleware`. (CVE-2021-41125)

The remote Debian 9 host has packages installed that are affected by multiple vulnerabilities as referenced in the dla-2950 advisory.

    It was found that Scrapy, a framework for extracting data from websites, could send HTTP Authorization as     well as cookies to other domains in case of redirections, possibly leaking user credentials. For Debian 9     stretch, these problems have been fixed in version 1.0.3-2+deb9u1. We recommend that you upgrade your     python-scrapy packages. For the detailed security status of python-scrapy please refer to its security     tracker page at: https://security-tracker.debian.org/tracker/python-scrapy Further information about     Debian LTS security advisories, how to apply these updates to your system and frequently asked questions     can be found at: https://wiki.debian.org/LTS

Tenable has extracted the preceding description block directly from the Debian security advisory.

Note that Nessus has not tested for these issues but has instead relied only on the application's self-reported version number.

ID	Name	Product	Family	Severity
250486	Linux Distros Unpatched Vulnerability : CVE-2021-41125	Nessus	Misc.	medium
235158	Ubuntu 18.04 LTS / 20.04 LTS / 22.04 LTS / 24.04 LTS : Scrapy vulnerabilities (USN-7476-1)	Nessus	Ubuntu Local Security Checks	medium
416609	SCA: security update for Scrapy (GHSA-jwqp-28gf-p498)	Tenable Cloud Security	SCA Checks	medium
416609	SCA: security update for Scrapy (GHSA-jwqp-28gf-p498)	Tenable Self-Hosted Container Security	SCA Checks	medium
158996	Debian DLA-2950-1 : python-scrapy - LTS security update	Nessus	Debian Local Security Checks	medium

Detections

Analytics

CVE-2021-41125

medium

Tenable Plugins

medium

medium

medium

medium

medium