You have created a Vertex AI pipeline that includes two steps. The first step preprocesses 10 TB data completes in about 1 hour, and saves the result in a Cloud Storage bucket. The second step uses the processed data to train a model. You need to update the model’s code to allow you to test different algorithms. You want to reduce pipeline execution time and cost while also minimizing pipeline changes. What should you do?
- A. Add a pipeline parameter and an additional pipeline step. Depending on the parameter value, the pipeline step conducts or skips data preprocessing, and starts model training.
- B. Create another pipeline without the preprocessing step, and hardcode the preprocessed Cloud Storage file location for model training.
- C. Configure a machine with more CPU and RAM from the compute-optimized machine family for the data preprocessing step.
- D. Enable caching for the pipeline job, and disable caching for the model training step.
Suggested Answer:
D
🗳️