Exam Professional Data Engineer topic 1 question 284 discussion - ExamTopics


A data engineer needs to store and efficiently access time-series data from 1000 sensors, requiring both low-latency retrieval and daily complex analytics, leading to the optimal solution of using Bigtable for storage with daily exports to BigQuery.
AI Summary available β€” skim the key points instantly. Show AI Generated Summary
Show AI Generated Summary

You have a network of 1000 sensors. The sensors generate time series data: one metric per sensor per second, along with a timestamp. You already have 1 TB of data, and expect the data to grow by 1 GB every day. You need to access this data in two ways. The first access pattern requires retrieving the metric from one specific sensor stored at a specific timestamp, with a median single-digit millisecond latency. The second access pattern requires running complex analytic queries on the data, including joins, once a day. How should you store this data?

  • A. Store your data in BigQuery. Concatenate the sensor ID and timestamp, and use it as the primary key.
  • B. Store your data in Bigtable. Concatenate the sensor ID and timestamp and use it as the row key. Perform an export to BigQuery every day.
  • C. Store your data in Bigtable. Concatenate the sensor ID and metric, and use it as the row key. Perform an export to BigQuery every day.
  • D. Store your data in BigQuery. Use the metric as a primary key.
Show Suggested Answer Hide Answer
Suggested Answer: B πŸ—³οΈ

Was this article displayed correctly? Not happy with what you see?

Tabs Reminder: Tabs piling up in your browser? Set a reminder for them, close them and get notified at the right time.

Try our Chrome extension today!


Share this article with your
friends and colleagues.
Earn points from views and
referrals who sign up.
Learn more

Facebook

Save articles to reading lists
and access them on any device


Share this article with your
friends and colleagues.
Earn points from views and
referrals who sign up.
Learn more

Facebook

Save articles to reading lists
and access them on any device