Testing the Effectiveness of your Patch Management System

If you've invested a lot of money into a commercial patch management system or perhaps you've grown your own, how do you know how effective it is? With Nessus's agent-less host based patch audits, it can effectively be used to understand how effective your patching solution has been. It can also be used to identify hosts which are not being patched or are under management. Lastly, we will discuss how to determine how effective your patch management system has been. This post considers both users who just use Nessus as well as Security Center customers.

Why do patches fail?

There are many reasons that patches fail. Here are just a few and some corresponding examples:

• Too Secure - a UNIX or Windows server can be configured such that the remote user account or local user agent that is pushing the patch doesn't have enough rights.
• Bad Network Settings - a server with out of date network settings like a stale DNS server or stale local router may look like it is alive, but a patch could fail to install because of poor network access. Firewall rule changes can effect some systems as well.
• Patch Dependencies - a patch management system which did not take into account the required dependencies of patch might not realize that the patch didn't get installed.
• Lack of disk space - if patches are all sent to a certain partition or drive and that drive is out of space, the patch might not run. Self extracting patches might run into this issue as well.
• No Bandwidth - if your network can't handle pushing 1000s of 10 megabyte files to your systems at the same time, this may impact a patch being delivered and ultimately being installed.

Testing for Network Vulnerabilities

A basic Nessus scan without credentials will discover vulnerabilities in systems that run a service. Many clients, such as iTunes, run services which can be identified the same way an Apache or Exchange server can be found.

At the network layer, Nessus will identify the underlying application and their vulnerabilities, but not necessarily an operating system level patch. For example, a vulnerable version of Apache may be found, but Nessus may not know that the underlying operating system is Windows, RedHat, SuSE or OS X. Nessus will recommend that the vulnerable service be upgraded, but won't specify the exact patch for the underlying operating system.

Testing for Bad Credentials

Nessus scans can be configured to use a variety of UNIX and Windows credentials. Nessus plugin #21745 will alert when a system was attempted to be access with a set of credentials and when they failed. This can help identify systems with the wrong security settings.

If users have the Security Center deployed, each separate asset group can be configured with their own set of credentials. This can help automate patch testing, but also find when a host credentials within a given asset list were changed or are no longer valid.

Testing for Unmanaged Systems

Nessus Direct Feed users can perform custom checks for a wide variety of files, file content and registry values. These values can be used to identify a corporate managed asset, as well as evidence that a system is running a certain type of patch agent. For example, the following Nessus 3 .audit content will look for the presence of a certain version of Adobe:

<custom_item>
type: REG_CHECK
description: "Check the key HKLM\SOFTWARE\Adobe\Acrobat Reader\7.0\AdobeViewer"
value_type: POLICY_TEXT
value_data: "HKLM\SOFTWARE\Adobe\Acrobat Reader\7.0\AdobeViewer"
reg_option: MUST_EXIST
key_item: "EULA"
</item>

If your managed systems ran an agent, had certain registry settings or files that indicated it was managed, you could use something similar to scan those systems to ensure they had the proper settings. These checks do require credentials and work for UNIX and Windows systems.

Testing for Missing Patches

All Nessus users can audit Windows and UNIX systems for missing patches. To scan for missing patches, your Nessus scanner needs to be configured with the credentials of the networks you are auditing. If using Nessus to perform these sorts of scans is a new concept to you, please read the paper "Nessus Credential Checks for UNIX and Windows". This explains how to configure Nessus and the target hosts to perform these audits.

Unlike a patch management system, Nessus is simply logging into the audited device and pulling a list of patches. Although this process does require network access and proper security settings, it is ultimately less complex than installing a patch. Since there are less things that can go wrong, perform this sort of audit with Nessus will likely help identify systems that are missing patches.

How effective is your patch management system?

Lastly, from a security and network management system, there are several questions worth asking to help judge the effectives of the patch management system.

• Are all security patches being applied? Your organization may or may not require 100% coverage of all security patches. If they do, using Nessus and the Security Center can help prove that the patch system is or isn't working. If less than 100% coverage is acceptable, doing an external audit like this can help identify security risks that have not been accounted for by the patch process.
• How quickly are patches applied? Your organization may require all patches to be installed in a given amount of time. Nessus and the Security Center can help test for discrepancies with this policy and report the overall progress.
• Are new hosts under the patch management program? As new servers or even desktops are added to the infrastructure, being able to detect that they are falling further and further behind in their patch cycle is something accomplished with the Security Center.
• What about embedded devices? Both Nessus and the Security Center can help find patch issues in embedded devices like routers, switches and printers. Tenable has seen many organizations deploy a very comprehensive desktop patching program only to ignore vital devices like their firewalls and printers.

Regardless of what you are tracking, the Security Center can be used to show just about any type of vulnerability or configuration issue mapped over time. The above image shows a trend for the past 90 days (today being on the left) where there has been a significant reduction in measured vulnerabilities. This graph can easily be produced with the click of the button for specific ports, Nessus families or various asset groups. The Security Center also has a 3D tool (click here for a video) which can be used to show where various issues are at in the network.

Mapping Patching into Network Management

Tenable Network Security is a strong believer in network controls. Besides Nessus and the Security Center, we also offer other products to perform log analysis and passive network monitoring. All of our products can be used to identify managed and unmanaged devices, unauthorized change and conformity to corporate configuration guidelines.  All of this has direct impact on the effectives of any patch management infrastructure. For more information about network controls, consider downloading the "Network Security Implications of 'Visible Ops'" paper or request a copy of the "Realtime Compliance Monitoring" paper by emailing [email protected].