Consumers Union, the well-respected consumer product testing organization and publisher of Consumer Reports, decided it was pointless to test antivirus programs against already known viruses.
"If signatures were updated instantaneously, you would never need prediction. You'd always be protected. But it doesn't always work that way, obviously," said Evon Beckford, senior director of electronics operations for the Consumer's Union.
The aim of the tests was to see if the program is capable of recognizing a variance of known viruses. The company licensed a third-party lab to create 5,500 "test" viruses; the vast majority are variants of known viruses for testing in its report.
Consumer Reports managed to do something no one else has done. It got antivirus vendors to all agree on one thing: They hated the idea.
"The AV community has always been very strongly opposed to the creation of new malware for any purpose," said John Hawes of Virus Bulletin, in a blog entry. "There's just no need for it - plenty of new viruses are being written all the time, why would anyone in a responsible position want to add to the glut?"
"Creating new viruses for the purpose of testing and education is generally not considered a good idea - viruses can leak and cause real trouble," said Igor Muttik of McAfee (Quote, Chart) in his own blog.
"This is a really unwise thing to do. There are plenty of 'real' viruses, worms and Trojans around without well-meaning organizations generating more of them, for whatever reason," said David Emm, senior technology consultant at Kaspersky Labs.
You would think CR had been playing with Ebola strains in a buffet line of the Bellagio Hotel, judging by the industry's reaction.
Peter Firstbrook, research director for information security and privacy at Gartner, is not very sympathetic. "The AV guys are being ridiculous," he said. "The biggest problem with the AV vendors is they are totally reactive to new viruses. They all do well on the known virus list. Big deal, so you can catch a known virus."
The reason for creating variants of known viruses is because most viruses are just modifications of existing viruses. "If you're a virus writer, particularly an inexperienced one, that's what you do, modify an existing one. You don't try to create a new exotic virus," said Beckford.