Question

A data engineering team at a media company wants to build a content recommendation feature. They have 5 million article titles and descriptions stored in a BigQuery table. They need to compute semantic similarity between articles so that editors can find related content using SQL, without exporting data or managing separate infrastructure. Which approach best prepares this text data for the similarity use case?

Accepted Answer

Use ML.GENERATE_EMBEDDING with a remote model referencing a Vertex AI text embedding endpoint, store the resulting vectors in BigQuery, then compute similarity with ML.DISTANCE or a VECTOR_SEARCH index.

Answer

Train a BigQuery ML k-means clustering model on the raw title strings and use the cluster assignments as similarity scores.

Answer

Concatenate title and description into a single column and apply ML.NGRAMS, then join rows on matching n-grams to identify related articles.

Answer

Export the text to Cloud Storage, run a Dataflow job with a custom embedding library, and re-import the vectors as a federated external table.