My Top 10 Criteria for Evaluating Technology

The last time I looked, there were over 1000 cybersecurity companies in business! As a service provider, I’m constantly being solicited with partnering requests to introduce the latest and greatest widget to my customers.

I thought I would share some of factors I consider when evaluating technology:

• Trust – In this industry, trust is everything. If I have previously worked well with a vendor, technology, or person, I’m likely going to be more receptive.
• Functionality – The tool in question has to work well, but doesn’t have to be perfect, particularly if we’re dealing with a startup Does the tool perform a function that is currently lacking? Does the tool complement or consolidate existing functions?
• Ease of Use – The user experience and interface can make or break a tool. It may be the best thing since sliced bread but if the UI is too confusing or inefficient, I’m not going to want to use it. If I can manage the tool on top of my current workload, that demonstrates a reasonable user experience.
• Reporting – I very rarely compromise with this aspect of a tool. In most cases, I expect to be able to schedule daily, weekly, and monthy reports for delivery to internal and customer teams. There have been situations where a tool is so good, that we work around the reporting limitations.
• MSP Capable – Most cybersecurity technology is built with the enterprise customer in mind. As we work with many SMBs and SMEs, a tool being able to function in an MSP context is ideal. This has nothing to do with being able to process monthly licensing. More often than not, we need to be able to split a large license into many smaller ones for multiple customers.
• Scale – This is related to the previous point. There is some amazing technology out there but quite often, the vendor won’t even talk to you unless you’re over 2,000 endpoints or the deal is worth more than $3M. I completely understand the desire to go after the whales, but those blinders can be detrimental to sustaining long term business.
• Integration – Does it play well with others? Can the tool work, automatically or manually, with other tools in our arsenal? For example, an email gateway would ideally propagate a threat’s telemetry e.g. file hash, URL, domain to a firewall, or IPS for blocking and containment. If an automatic response is not possible, being able to copy and paste the data into other tools is an acceptable compromise. Admittedly we run pretty quiet and well-tuned environments that are not inundated with hundreds of threat events each day.
• Support – What do we do if we need help? If our only recourse is a forum, I’m already half-way out the door. Having been in the industry for 25+ years, I am quite capable of searching knowledge base and some Google-fu, however, sooner or later I want to talk to a person. I don’t mind opening cases by email but would prefer a chat option with escalation capabilities.
• Ease of doing business – Ultimately it doesn’t matter if we have a direct or indirect (distributor) relationship. What does matter is that the effort of doing business must be lower than the reward. For example, if we have to work with a new distributor that has outrageous partnering terms, we’re most likely walking away from the table.
• Cost – Yes, cost is the last on this list. Assuming all the previous criteria have been met, short of an outrageous price, we can normally justify the value of a particular technology.

On a parting note, with so many vendors in each domain of cybersecurity, one behavior that needs to change is the expectation that customers rip and replace technology due to a better price or shinier tool. Let’s be honest, if we could afford every tool on the market or even every tool on a trade-show floor, would we be better off? Can we be expected to become trained with every combination and iteration? I think not!