Question

You are fine-tuning a GPT-4o model in Azure OpenAI to classify customer support tickets into 12 categories. Your training file contains 3,000 curated JSONL examples. After the first fine-tuning job, the training loss drops steadily but you have no way to tell whether the model will generalize to unseen tickets before you deploy it. You want to detect overfitting during the job itself and compare candidate hyperparameter configurations objectively. What should you do?

Accepted Answer

Supply a separate validation file to the fine-tuning job so Azure OpenAI reports validation loss and validation token accuracy alongside training metrics.

Answer

Increase the number of epochs so the training loss converges to zero, which guarantees the model has learned the categories.

Answer

Add all 3,000 examples to a single training file and rely on the final training loss value to judge generalization.

Answer

Deploy the fine-tuned model to a serverless endpoint and manually inspect a handful of live predictions to decide if it generalizes.