Multi-Model Validation

Multi-model validation is Compass's approach to ensuring AI-generated reports are accurate and reliable. Instead of relying on a single AI model's output, Compass cross-validates findings across multiple models to catch errors, hallucinations, and blind spots. Available on Pro and Enterprise plans.

Why It Matters

AI models can make mistakes — they can misinterpret data, overstate risks, or miss important patterns. In an IAM context, an incorrect finding could lead to:

• Wasted effort fixing a non-existent problem
• A real security gap being overlooked
• Incorrect compliance scores misleading auditors

Multi-model validation reduces these risks by having multiple AI models independently analyse your data and flagging discrepancies.

How It Works

1. Primary analysis — The primary model generates the full report from your connector data
2. Validation pass — A second model reviews the primary model's findings against the raw data
3. Discrepancy resolution — Where the models disagree, the system flags the finding and provides both perspectives
4. Confidence scoring — Each finding receives a confidence score based on model agreement

What Gets Validated

• Metric calculations — Are the numbers correctly derived from connector data?
• Finding severity — Is the risk rating appropriate given the evidence?
• Recommendations — Are the suggested actions practical and correctly prioritised?
• Compliance mapping — Are the right controls mapped to the right frameworks?
• Benchmark comparisons — Are the industry comparisons fair and current?

Confidence Indicators

Reports show confidence levels throughout:

• High confidence — Both models agree on the finding and its severity
• Medium confidence — Models agree on the finding but differ on severity or detail
• Low confidence — Models disagree, or insufficient data to validate — flagged for human review

Bring Your Own Keys (BYOK)

On Pro and Enterprise plans, you can use your own API keys for the validation models. This gives you:

• Control over which models are used for validation
• Usage visibility through your own API dashboard
• Ability to use models hosted in your own infrastructure (Enterprise)