Case Study: Riskified Scales Penetration Testing Depth While Preserving Audit Assurance

Industry: E-commerce Fraud Prevention and Risk Intelligence
Organization: Riskified
Security Leadership: Yossi Yeshua, Chief Information Security Officer
Product: Terra Security Continuous Pentesting

Background

Riskified is a global technology company providing AI-driven fraud prevention, chargeback management, and risk intelligence solutions for online merchants. Operating at the core of digital commerce, Riskified’s platform supports high-volume, real-time decisioning across complex web applications and integrations.

As a public company serving large merchants, Riskified operates in an environment where application security must balance speed, scale, and rigor, while also meeting governance, safety, and audit expectations.

The Challenge: Depth, Continuity, and Assurance

For modern SaaS platforms, traditional penetration testing faces a structural limitation. Human-led testing alone cannot continuously explore large and rapidly evolving web attack surfaces with sufficient depth.

At the same time, fully automated or fully autonomous testing approaches introduce their own constraints. In regulated and audited environments, penetration testing programs often require:

• Human-reviewed and human-signed reports
• Demonstrable oversight and safety controls
• Outputs that auditors and stakeholders recognize as credible assurance

This creates a practical tension for security leaders:

• Automation is necessary to achieve depth and continuity at scale
• Human involvement is still required for governance, compliance, and audit acceptance

A Pragmatic Hybrid Approach

Riskified adopted an approach that combines Terra’s agentic AI-driven penetration testing system with human oversight focused on safety, validation, and compliance rather than manual depth alone.

As Yossi Yeshua, Chief Information Security Officer at Riskified, explains, “Terra’s approach, combining their Agentic AI system with human oversight, gives the depth and scale a modern security organization needs in their pentest program while increasing accuracy and validating exploitability specific to your web attack surface.”

In this model, agentic AI enables continuous and deep exploration of the web attack surface, while human oversight ensures results are controlled, reviewed, and suitable for use within formal security and audit processes.

Operational Impact

By aligning automation and human assurance, Riskified’s security program can:

• Achieve broader coverage and higher frequency of testing than traditional pen testing allows.
• Maintain audit-ready penetration testing outputs with human verification.
• Improve both MTTD (Mean Time to Detect) and MTTR (Mean Time to Remediation)

This balance allows security teams to scale their penetration testing programs without introducing risk from unsupervised automation or friction from purely manual processes.

Takeaway

Riskified’s experience reflects a broader reality for modern security organizations. The future of penetration testing is not fully human or fully autonomous, but a carefully designed combination of agentic automation for depth and human oversight for assurance. By embracing this hybrid model, organizations can meet the demands of scale, accuracy, and compliance simultaneously.

Terra Security platform is hosted on AWS and leverages Amazon Bedrock as the foundation for its agentic, generative AI capabilities. Amazon Bedrock provides managed access to foundation models with enterprise-grade security, isolation, and governance controls, allowing Terra to run AI-driven security workflows without managing model infrastructure or exposing customer data to external model providers.

By using Amazon Bedrock, Terra ensures that generative AI reasoning and decision-making occur within a controlled AWS environment, aligned with the security and compliance expectations of regulated customers like Evinova. Self-hosted models and third-party hosted LLM APIs were evaluated but rejected due to operational complexity, scaling challenges, and data governance risks.