Share with your network!
Cloudmark recently conducted a study of the effectiveness of different antivirus (AV) products in detecting malware distributed via email spam. We found that even the best performing AV product in our study, Sophos, only detected 48% of all the samples that were eventually flagged as malicious, and that many big names in the AV industry did substantially worse.
Cloudmark receives several million spam reports a day from spam traps and trusted users. Between May 28th and August 12th of 2015 we evaluated all suspicious file attachments from this data. We defined suspicious as executable or html files, or compressed files containing executables or html. We did not look at PDFs, or MS Office documents. Though these sometimes contain malicious content they have too many legitimate uses to be automatically regarded as suspicious. We calculated a hash from the file and checked to see if we had already seen it. If not then we submitted it to VirusTotal.com (VT) for analysis. VT is an invaluable service that runs submitted samples against fifty or more different AV programs and tells you which of them flag each sample as malicious. We used the VT API, which queues requests and may take an hour or more before the scans are performed. We recorded the results from VT, and if none of the AV engines flagged it as malicious, we requested a rescan twelve hours later.
We submitted a total of 33,688 suspicious files and 93% of those were flagged as malicious on either the first or second scan. However, not all of the AV engines caught all of the malware. The chart below shows the percentage of confirmed malware detected on the first scan by each of the AV engines in VT's system. Click on the image for an enlarged version.
- Malware samples were tested when first received by Cloudmark. We would expect detection rates to be significantly better once the AV companies have had time to produce signatures for each new variant.
- Static file analysis is only the first step in protection against malware. A good AV program will also detect malicious behavior such as updating registry keys or system DLLs which provides a second line of defense. This was not tested in our study.
- Detection rates may change based on the particular attacks that are current at any given time. We will continue to collect this data and may update this study in future.
- Malware may also be delivered via other means: watering hole attacks (malicious or compromised web sites), malvertising (malicious web advertising), etc. These were not addressed in our study.