Question

A financial services company built a document-processing pipeline that ingests batches of loan applications from S3. A Step Functions Standard workflow must extract structured data from each PDF by invoking a Bedrock foundation model. Batches vary from 5 to 3,000 documents. During large batches, the workflow fails intermittently with ThrottlingException from Bedrock, and the team also hits the maximum request payload limit when passing all document contents into a single state. The team wants a resilient design that processes each document independently while respecting Bedrock's account-level invocation quotas. Which approach best meets these requirements?

Accepted Answer

Use a Distributed Map state that reads the S3 manifest, sets MaxConcurrency to bound parallelism, and invokes a Lambda-backed child workflow per document with retry and exponential backoff on ThrottlingException.

Answer

Load all document contents into a single Pass state array and invoke one Bedrock InvokeModel call that processes the entire batch in one request.

Answer

Replace Step Functions with a single long-running Lambda function that loops over every document sequentially, relying on the Lambda 15-minute timeout to complete large batches.

Answer

Use an Inline Map state with no concurrency limit so all documents are processed simultaneously for the fastest possible completion time.