Question

A software team has a CI/CD pipeline running on Cloud Build. Their integration test suite frequently produces intermittent failures ('flaky tests') that are not caused by real code defects but by timing issues and shared test data collisions when tests run in parallel. Developers have started re-running the pipeline until it passes, which erodes trust in the test results and slows down deployments. As the architect, what should you recommend to improve the reliability and value of the test process?

Accepted Answer

Isolate test data per test run (e.g., ephemeral datasets/namespaces), quarantine identified flaky tests into a separate non-blocking track, and fix their root causes before returning them to the gating suite.

Answer

Configure the pipeline to automatically retry the entire test suite up to three times and mark the build green if any attempt passes.

Answer

Remove the integration tests from the pipeline entirely and rely on manual QA verification before each production release.

Answer

Reduce test parallelism to run all integration tests sequentially so no data collisions can occur, accepting the longer build times.