Here is a scenario that plays out every week in companies of all sizes. A CISO gets 30 minutes with a vendor rep. The demo is slick. The pricing deck is surprisingly reasonable. Someone mentions a competitor is already using this tool. A purchase order is raised three weeks later. Six months on, the tool sits half-configured, the vendor's support team takes four days to respond to critical tickets, and the ISO 27001 auditor is asking pointed questions about a control gap the tool was supposed to close.
The problem is almost never the product itself. It is that the evaluation process was designed around the vendor's sales cycle rather than your actual security needs.
โMost organisations evaluate security vendors the same way they'd buy software, based on features and price. That's the wrong framework entirely. You're not buying a feature set. You're entering a long-term risk relationship with a company that will have deep access to your most sensitive infrastructure.โ
โ Aditya Hadke, Project Delivery Lead, SecComply
The Questions You Must Ask Before Any Demo
Before you ever sit through a demo, you should have a structured set of questions ready, not the questions vendors expect, but the ones that actually surface risk.
What Certifications Actually Mean
Security certifications have become a bit like hygiene badges, everyone has them, but they do not all mean the same thing.
| Certification | What It Means | What It Doesn't Mean | Require? |
|---|---|---|---|
| ISO 27001 | Documented ISMS reviewed by an accredited auditor | That every control is watertight or continuously monitored | โ Yes |
| SOC 2 Type II | Controls tested over 6โ12 months, not a point in time | That scope covers everything you care about | โ Yes, prefer Type II |
| SOC 2 Type I | Controls existed at a single point in time | That they work consistently under real conditions | โก Acceptable if recent |
| Pen Test Report | A third party actively tried to break them | That findings were fully remediated | โ Ask for remediation summary |
| GDPR/HIPAA 'Compliant' | Usually just a self-assessment | Formal certification (neither framework has one) | โ Ask for DPA/BAA instead |
Always ask for the SOC 2 report's scope section first. Vendors sometimes get SOC 2 certified for a narrow slice of their infrastructure, not the systems that actually handle your data. Scope gaps are one of the most common ways vendors technically hold a certification that does not apply to how you use them.
Red Flags That Are Easy to Miss
- They refuse to share their SOC 2 report without a custom NDA that takes weeks to negotiate
- Their certifications are more than 18 months old and they cannot explain why
- They have no written sub-processor list, or it has not been updated since last year
- Support SLAs are buried in appendices and worded to give them maximum wiggle room
- They cannot name a specific person responsible for your account's security incidents
- The security questionnaire goes to their sales team, not their security team
- They promise compliance with your framework but cannot map their controls to it
- Their breach notification clause gives them 72 hours or more to notify you
The Evaluation Scorecard
Use this framework to score vendors consistently across the dimensions that actually matter. Apply it across every vendor to create a true apples-to-apples comparison.
โ Vendor Evaluation Dimensions
Getting the Contract Right
Even if a vendor checks every box in the evaluation, the contract is where things can quietly go wrong. These are the clauses that matter most from a security and compliance standpoint.
Limitation of liability clauses that cap vendor exposure to one month of fees are extremely common and extremely dangerous. If a vendor breach results in a regulatory fine under GDPR, that fine can reach 4% of your global annual turnover. One month of SaaS fees will not touch it. Push for reasonable liability coverage tied to actual harm, or seek cyber insurance that explicitly covers third-party vendor incidents.
The most important contractual element for compliance-focused organisations is the Data Processing Agreement (DPA) or Business Associate Agreement (BAA) under HIPAA. Without a valid DPA, you are technically in breach of GDPR every time personal data flows to that vendor.
Breach notification within 48 hours ยท Explicit right to audit ยท Sub-processor change notification with 30+ days notice ยท Data deletion on contract termination within 30 days with written certification ยท Liability coverage that reflects actual risk exposure rather than one month of fees
Not sure if your vendors are actually secure?
SecComply's third-party risk assessments help you build a vendor program that satisfies ISO 27001 Annex A and SOC 2, and actually protects your organisation.
Book a Free Consultation โ