Question

Your team is training a deep neural network for tabular regression on Vertex AI. You want to tune the optimizer type (either 'adam' or 'sgd'). When 'sgd' is chosen, you also want to tune a 'momentum' value, but this parameter is meaningless when 'adam' is selected. You want to avoid wasting trial budget exploring 'momentum' values that have no effect. What is the most appropriate way to configure the Vertex AI hyperparameter tuning job?

Accepted Answer

Define 'momentum' as a conditional parameter that is only active when the categorical 'optimizer' parameter equals 'sgd'.

Answer

Run two entirely separate tuning jobs—one hardcoded with 'adam' and one with 'sgd'—and manually compare the best trials.

Answer

Include 'momentum' as an unconditional continuous parameter and ignore it inside the training code when 'adam' is selected.

Answer

Use grid search over all combinations of 'optimizer' and 'momentum' so that invalid pairings are naturally skipped.