NSS Labs Announces 2017 Breach Detection Systems Group Test Results

4 Products Receive Recommended Rating; 1 Receives Neutral Rating; 1 Receives Security Recommended Rating; 1 Receives Caution Rating

AUSTIN, Texas, Oct. 19, 2017 -- NSS Labs, Inc., a global leader and trusted source for independent fact-based cybersecurity guidance, today announced the results of its Breach Detection Systems (BDS) Group Test. For the 2017 BDS Group Test, NSS Labs expanded test techniques to include new evasions. Evasion techniques are a means of disguising and modifying attacks in order to avoid detection and blocking by security products. These methods are increasingly selected by threat actors to circumvent security controls. Test results indicate that most breach detection systems are unable to stop all evasions, which is a major concern.

According to the Verizon 2017 Data Breach Investigations Report, 75% of data breaches are perpetrated by outsiders. BDS products are now more widely adopted in enterprise security deployments to catch attacks that bypass other perimeter defenses such as next generation firewalls. The NSS Labs 2017 Enterprise Security Architecture Study indicates that 44.1% of US enterprises deploy BDS products. These products provide enhanced detection of advanced malware, zero-day attacks, and targeted attacks. Data obtained by BDS threat analysis helps to accelerate investigation and remediation, thereby reducing the business impact of threats.

The BDS Group Test reports offer insights into which infection vectors were effective and how long it took for products to detect them. When evaluating breach detection systems, time-to-detect metrics are critical as they directly impact ROI for enterprises.

Seven products from six market-leading security vendors were tested for security effectiveness, performance, stability and reliability, and total cost of ownership (TCO). Of the seven products tested, four received a Recommended rating and two scored 100% for security effectiveness.

Key findings from the test:

• Five out of the seven products tested missed evasions.
• Overall Security Effectiveness ranged between 80.2% and 100.0%.
• The average Security Effectiveness rating was 93.2%; five products received a Security Effectiveness rating above the average, and two received a Security Effectiveness rating below the average.
• False positive rates ranged from 0% to 0.36%.
• TCO per Protected Mbps ranged between US$16 and US$128, with most tested products costing less than US$44 per protected Mbps.
• The average TCO per Protected Mbps (Value) was US$48.82; five products demonstrated value above the average, and two demonstrated value below the average.

“The 2017 BDS Group Test revealed that most products were able to detect the majority of breaches within 60 minutes, but some products took several hours to detect the same breaches,” said Jason Brvenik, Chief Technology Officer at NSS Labs. “To reduce the frequency and impact of security incidents, enterprises must consider the detection rate and the time a product takes to detect attacks.”

The following products were tested:

• Check Point Software Technologies 15600 Next Generation Threat Prevention & SandBlast™ (NGTX) Appliance R77.30
• Cisco FirePower 8120 v.6 & Cisco AMP v.5.1.9.10430
• FireEye Network Security NX 10450 v7.9.2 & EX 8400 v7.9.0
• FireEye Network Security 6500NXES-VA v7.9.2
• Fortinet FortiSandbox-2000E v.FSA 2.4.1 & FortiClient (APT Agent) v.5.6.0.1075
• Lastline Enterprise v7.25
• Trend Micro Deep Discovery Inspector Model 4000 v3.8 SP5 & OfficeScan (OSCE) v.12.0.1807

NSS Labs is committed to providing empirical data and objective group test results that enable organizations to make educated decisions about purchasing and optimizing security infrastructure products and services. As with all NSS Labs group tests, there is no fee for participation, and the test methodology is available in the public domain to provide transparency and to help enterprises understand the factors behind test results. Click here for more information about our group test policies.

Click here for more information on this test, and the test methodology used, or to purchase the individual Test Reports. Click here to download the Security Value Map^™, which provides a graphic comparison of Security Effectiveness and TCO across the tested products.

Additional Resources: 

• Purchase the NSS Labs Breach Detection Systems Group Test Reports
• Read our blog titled “Breach Detection Systems: A look at U.S. Deployments”
• View the test methodology
• Learn more about NSS Labs’ Security Product Testing
• Learn more about NSS Labs’ Test Policies
• Learn more about NSS Labs’ CAWS Continuous Security Validation Platform
    • Follow NSS Labs on Twitter 
    • Follow NSS Labs on LinkedIn 
    • Browse the NSS Labs Research Library
  

About NSS Labs, Inc.
NSS Labs, Inc. is recognized globally as the most trusted source for independent, fact-based cybersecurity guidance. Our mission is to advance transparency and accountability within the cybersecurity industry. We empower enterprises by providing them with timely, relevant information on which to base their decisions. Our unmatched foundation in security testing, along with our extensive research and global threat analysis capabilities, provide the basis for our CAWS Continuous Security Validation Platform. CAWS measures the ongoing effectiveness of security controls, providing a real-time score card to help business leaders substantiate their security investments. Businesses can also leverage the threat data delivered by CAWS to strengthen their cyber risk posture and mitigate threats to their operating systems and applications. CISOs, Chief Security Architects, SOC and Threat Analysts, and information security professionals from many of the world's largest and most demanding enterprises rely on trusted insights from NSS Labs. For more information, visit www.nsslabs.com.

Contact:
Tom Resau
W2 Communications
Phone: +1 703-877-8103
[email protected]