Question

A healthcare company runs a patient-facing chatbot built on a self-hosted open-source LLM deployed on Amazon EKS (not an Amazon Bedrock foundation model). Compliance requires the same PII masking, denied-topic blocking, and prompt-attack filtering that the company already configured in an Amazon Bedrock Guardrail for its Bedrock-based internal tools. The team wants to reuse that exact guardrail configuration for the EKS-hosted model without migrating the model to Bedrock. What is the recommended approach?

Accepted Answer

Call the Bedrock ApplyGuardrail API directly from the EKS application to evaluate inputs and outputs against the existing guardrail, independent of any Bedrock model invocation.

Answer

Guardrails can only be applied through InvokeModel/Converse on Bedrock foundation models, so the team must reimplement the filtering logic manually in the EKS application.

Answer

Register the EKS model endpoint as a Bedrock custom model import so that guardrails are automatically applied to all inference requests.

Answer

Place an Amazon API Gateway with a Lambda authorizer in front of the EKS model; the authorizer natively enforces Bedrock guardrail policies on request bodies.