Page Reader AI | Exam Professional Machine Learning Engineer topic 1 question 299 discussion

Exam Professional Machine Learning Engineer topic 1 question 299 discussion - ExamTopics

AI Summary Hide AI Generated Summary

Problem: Evaluating Distilled LLMs

The scenario involves evaluating multiple versions of distilled LLMs after batch inference, with results stored in Cloud Storage. The goal is to create an efficient evaluation workflow integrated with a Vertex AI pipeline to assess performance and track artifacts.

Options

A. Custom Python component: Reads outputs, calculates metrics, and writes to BigQuery.
B. Dataflow component: Distributed processing of outputs, metric calculation, and writing to BigQuery.
C. Custom Vertex AI Pipelines component: Reads outputs, calculates metrics, and writes to BigQuery.
D. AutoSxS pipeline component: Processes outputs, aggregates metrics, and writes to BigQuery.

Solution

The suggested answer is C: Creating a custom Vertex AI Pipelines component. This option is chosen because it directly integrates with the existing Vertex AI pipeline, providing a streamlined and efficient solution for managing the evaluation workflow within the same environment.

Your team is experimenting with developing smaller, distilled LLMs for a specific domain. You have performed batch inference on a dataset by using several variations of your distilled LLMs and stored the batch inference outputs in Cloud Storage. You need to create an evaluation workflow that integrates with your existing Vertex AI pipeline to assess the performance of the LLM versions while also tracking artifacts. What should you do?

A. Develop a custom Python component that reads the batch inference outputs from Cloud Storage, calculates evaluation metrics, and writes the results to a BigQuery table.
B. Use a Dataflow component that processes the batch inference outputs from Cloud Storage, calculates evaluation metrics in a distributed manner, and writes the results to a BigQuery table.
C. Create a custom Vertex AI Pipelines component that reads the batch inference outputs from Cloud Storage, calculates evaluation metrics, and writes the results to a BigQuery table.
D. Use the Automatic side-by-side (AutoSxS) pipeline component that processes the batch inference outputs from Cloud Storage, aggregates evaluation metrics, and writes the results to a BigQuery table.

Show Suggested Answer Hide Answer

Was this article displayed correctly? Not happy with what you see?

See Archived Versions