Amazon Bedrock AgentCore Evaluations integrates Jev and Strands Decider 2B decision models

AWS said Amazon Bedrock AgentCore Evaluations scores AI agents on demand during development, in batch jobs against stored sessions, and online against production traffic. The capability includes 15 built-in evaluators and supports custom evaluators defined as LLM-as-a-judge or code-based functions running in AWS Lambda. Code-based evaluators can call external systems, models, or APIs, including decision models. The blog post explains how to connect two decision model backends through code-based evaluators: Jev, a hosted decision model from TypeSafe, and Strands Decider 2B, an open-source model users run themselves. Decision models accept a structured state and a fixed set of allowed answers, then return a calibrated probability distribution instead of free-form text. AWS said they fit questions with fixed answers, such as whether an agent refused a request or stayed on topic, and can work as a screening layer that passes or fails turns before built-in evaluators assess the ones that need deeper review. Code samples for Jev and Strands Decider 2B are available in the AgentCore sample repository.