The Avant-garde Guide To Professional-Data-Engineer Free Demo

NEW QUESTION 1

When running a pipeline that has a BigQuery source, on your local machine, you continue to get permission denied errors. What could be the reason for that?

• A. Your gcloud does not have access to the BigQuery resources
• B. BigQuery cannot be accessed from local machines
• C. You are missing gcloud on your machine
• D. Pipelines cannot be run locally

Answer: A

Explanation:
When reading from a Dataflow source or writing to a Dataflow sink using DirectPipelineRunner, the Cloud Platform account that you configured with the gcloud executable will need access to the corresponding source/sink
Reference:
https://cloud.google.com/dataflow/java-sdk/JavaDoc/com/google/cloud/dataflow/sdk/runners/DirectPipelineRun

NEW QUESTION 2

As your organization expands its usage of GCP, many teams have started to create their own projects. Projects are further multiplied to accommodate different stages of deployments and target audiences. Each project requires unique access control configurations. The central IT team needs to have access to all projects. Furthermore, data from Cloud Storage buckets and BigQuery datasets must be shared for use in other projects in an ad hoc way. You want to simplify access control management by minimizing the number of policies. Which two steps should you take? Choose 2 answers.

• A. Use Cloud Deployment Manager to automate access provision.
• B. Introduce resource hierarchy to leverage access control policy inheritance.
• C. Create distinct groups for various teams, and specify groups in Cloud IAM policies.
• D. Only use service accounts when sharing data for Cloud Storage buckets and BigQuery datasets.
• E. For each Cloud Storage bucket or BigQuery dataset, decide which projects need acces
• F. Find all the active members who have access to these projects, and create a Cloud IAM policy to grant access to all these users.

Answer: AC

NEW QUESTION 3

You have developed three data processing jobs. One executes a Cloud Dataflow pipeline that transforms data uploaded to Cloud Storage and writes results to BigQuery. The second ingests data from on-premises servers and uploads it to Cloud Storage. The third is a Cloud Dataflow pipeline that gets information from third-party data providers and uploads the information to Cloud Storage. You need to be able to schedule and monitor the execution of these three workflows and manually execute them when needed. What should you do?

• A. Create a Direct Acyclic Graph in Cloud Composer to schedule and monitor the jobs.
• B. Use Stackdriver Monitoring and set up an alert with a Webhook notification to trigger the jobs.
• C. Develop an App Engine application to schedule and request the status of the jobs using GCP API calls.
• D. Set up cron jobs in a Compute Engine instance to schedule and monitor the pipelines using GCP API calls.

Answer: D

NEW QUESTION 4

What is the general recommendation when designing your row keys for a Cloud Bigtable schema?

• A. Include multiple time series values within the row key
• B. Keep the row keep as an 8 bit integer
• C. Keep your row key reasonably short
• D. Keep your row key as long as the field permits

Answer: C

Explanation:
A general guide is to, keep your row keys reasonably short. Long row keys take up additional memory and storage and increase the time it takes to get responses from the Cloud Bigtable server.
Reference: https://cloud.google.com/bigtable/docs/schema-design#row-keys

NEW QUESTION 5

To run a TensorFlow training job on your own computer using Cloud Machine Learning Engine, what would your command start with?

• A. gcloud ml-engine local train
• B. gcloud ml-engine jobs submit training
• C. gcloud ml-engine jobs submit training local
• D. You can't run a TensorFlow program on your own computer using Cloud ML Engine .

Answer: A

Explanation:
gcloud ml-engine local train - run a Cloud ML Engine training job locally
This command runs the specified module in an environment similar to that of a live Cloud ML Engine Training Job.
This is especially useful in the case of testing distributed models, as it allows you to validate that you are
properly interacting with the Cloud ML Engine cluster configuration. Reference: https://cloud.google.com/sdk/gcloud/reference/ml-engine/local/train

NEW QUESTION 6

Which action can a Cloud Dataproc Viewer perform?

• A. Submit a job.
• B. Create a cluster.
• C. Delete a cluster.
• D. List the jobs.

Answer: D

Explanation:
A Cloud Dataproc Viewer is limited in its actions based on its role. A viewer can only list clusters, get cluster details, list jobs, get job details, list operations, and get operation details.
Reference: https://cloud.google.com/dataproc/docs/concepts/iam#iam_roles_and_cloud_dataproc_operations_summary

NEW QUESTION 7

Your company is currently setting up data pipelines for their campaign. For all the Google Cloud Pub/Sub streaming data, one of the important business requirements is to be able to periodically identify the inputs and their timings during their campaign. Engineers have decided to use windowing and transformation in Google Cloud Dataflow for this purpose. However, when testing this feature, they find that the Cloud Dataflow job fails for the all streaming insert. What is the most likely cause of this problem?

• A. They have not assigned the timestamp, which causes the job to fail
• B. They have not set the triggers to accommodate the data coming in late, which causes the job to fail
• C. They have not applied a global windowing function, which causes the job to fail when the pipeline is created
• D. They have not applied a non-global windowing function, which causes the job to fail when the pipeline is created

Answer: C

NEW QUESTION 8

You have a data stored in BigQuery. The data in the BigQuery dataset must be highly available. You need to define a storage, backup, and recovery strategy of this data that minimizes cost. How should you configure the BigQuery table?

• A. Set the BigQuery dataset to be regiona
• B. In the event of an emergency, use a point-in-time snapshot to recover the data.
• C. Set the BigQuery dataset to be regiona
• D. Create a scheduled query to make copies of the data to tables suffixed with the time of the backu
• E. In the event of an emergency, use the backup copy of the table.
• F. Set the BigQuery dataset to be multi-regiona
• G. In the event of an emergency, use a point-in-time snapshot to recover the data.
• H. Set the BigQuery dataset to be multi-regiona
• I. Create a scheduled query to make copies of the data to tables suffixed with the time of the backu
• J. In the event of an emergency, use the backup copy of the table.

Answer: B

NEW QUESTION 9

Your company handles data processing for a number of different clients. Each client prefers to use their own suite of analytics tools, with some allowing direct query access via Google BigQuery. You need to secure the data so that clients cannot see each other’s data. You want to ensure appropriate access to the data. Which three steps should you take? (Choose three.)

• A. Load data into different partitions.
• B. Load data into a different dataset for each client.
• C. Put each client’s BigQuery dataset into a different table.
• D. Restrict a client’s dataset to approved users.
• E. Only allow a service account to access the datasets.
• F. Use the appropriate identity and access management (IAM) roles for each client’s users.

Answer: BDF

NEW QUESTION 10

You have spent a few days loading data from comma-separated values (CSV) files into the Google BigQuery table CLICK_STREAM. The column DT stores the epoch time of click events. For convenience, you chose a simple schema where every field is treated as the STRING type. Now, you want to compute web session durations of users who visit your site, and you want to change its data type to the TIMESTAMP. You want to minimize the migration effort without making future queries computationally expensive. What should you do?

• A. Delete the table CLICK_STREAM, and then re-create it such that the column DT is of the TIMESTAMP typ
• B. Reload the data.
• C. Add a column TS of the TIMESTAMP type to the table CLICK_STREAM, and populate the numericvalues from the column TS for each ro
• D. Reference the column TS instead of the column DT from now on.
• E. Create a view CLICK_STREAM_V, where strings from the column DT are cast into TIMESTAMP value
• F. Reference the view CLICK_STREAM_V instead of the table CLICK_STREAM from now on.
• G. Add two columns to the table CLICK STREAM: TS of the TIMESTAMP type and IS_NEW of the BOOLEAN typ
• H. Reload all data in append mod
• I. For each appended row, set the value of IS_NEW to tru
• J. For future queries, reference the column TS instead of the column DT, with the WHERE clause ensuring that the value of IS_NEW must be true.
• K. Construct a query to return every row of the table CLICK_STREAM, while using the built-in function to cast strings from the column DT into TIMESTAMP value
• L. Run the query into a destination table NEW_CLICK_STREAM, in which the column TS is the TIMESTAMP typ
• M. Reference the table NEW_CLICK_STREAM instead of the table CLICK_STREAM from now o
• N. In the future, new data is loaded into the table NEW_CLICK_STREAM.

Answer: D

NEW QUESTION 11

You architect a system to analyze seismic data. Your extract, transform, and load (ETL) process runs as a series of MapReduce jobs on an Apache Hadoop cluster. The ETL process takes days to process a data set because some steps are computationally expensive. Then you discover that a sensor calibration step has been omitted. How should you change your ETL process to carry out sensor calibration systematically in the future?

• A. Modify the transformMapReduce jobs to apply sensor calibration before they do anything else.
• B. Introduce a new MapReduce job to apply sensor calibration to raw data, and ensure all other MapReduce jobs are chained after this.
• C. Add sensor calibration data to the output of the ETL process, and document that all users need to apply sensor calibration themselves.
• D. Develop an algorithm through simulation to predict variance of data output from the last MapReduce job based on calibration factors, and apply the correction to all data.

Answer: A

NEW QUESTION 12

You work for a car manufacturer and have set up a data pipeline using Google Cloud Pub/Sub to capture anomalous sensor events. You are using a push subscription in Cloud Pub/Sub that calls a custom HTTPS endpoint that you have created to take action of these anomalous events as they occur. Your custom HTTPS endpoint keeps getting an inordinate amount of duplicate messages. What is the most likely cause of these duplicate messages?

• A. The message body for the sensor event is too large.
• B. Your custom endpoint has an out-of-date SSL certificate.
• C. The Cloud Pub/Sub topic has too many messages published to it.
• D. Your custom endpoint is not acknowledging messages within the acknowledgement deadline.

Answer: B

NEW QUESTION 13

What are two of the benefits of using denormalized data structures in BigQuery?

• A. Reduces the amount of data processed, reduces the amount of storage required
• B. Increases query speed, makes queries simpler
• C. Reduces the amount of storage required, increases query speed
• D. Reduces the amount of data processed, increases query speed

Answer: B

Explanation:
Denormalization increases query speed for tables with billions of rows because BigQuery's performance degrades when doing JOINs on large tables, but with a denormalized data
structure, you don't have to use JOINs, since all of the data has been combined into one table. Denormalization also makes queries simpler because you do not have to use JOIN clauses.
Denormalization increases the amount of data processed and the amount of storage required because it creates redundant data.
Reference:
https://cloud.google.com/solutions/bigquery-data-warehouse#denormalizing_data

NEW QUESTION 14

You are selecting services to write and transform JSON messages from Cloud Pub/Sub to BigQuery for a data pipeline on Google Cloud. You want to minimize service costs. You also want to monitor and accommodate input data volume that will vary in size with minimal manual intervention. What should you do?

• A. Use Cloud Dataproc to run your transformation
• B. Monitor CPU utilization for the cluste
• C. Resize the number of worker nodes in your cluster via the command line.
• D. Use Cloud Dataproc to run your transformation
• E. Use the diagnose command to generate an operational output archiv
• F. Locate the bottleneck and adjust cluster resources.
• G. Use Cloud Dataflow to run your transformation
• H. Monitor the job system lag with Stackdrive
• I. Use the default autoscaling setting for worker instances.
• J. Use Cloud Dataflow to run your transformation
• K. Monitor the total execution time for a sampling of job
• L. Configure the job to use non-default Compute Engine machine types when needed.

Answer: B

NEW QUESTION 15

You’ve migrated a Hadoop job from an on-prem cluster to dataproc and GCS. Your Spark job is a complicated analytical workload that consists of many shuffing operations and initial data are parquet files (on average
200-400 MB size each). You see some degradation in performance after the migration to Dataproc, so you’d like to optimize for it. You need to keep in mind that your organization is very cost-sensitive, so you’d like to continue using Dataproc on preemptibles (with 2 non-preemptible workers only) for this workload.
What should you do?

• A. Increase the size of your parquet files to ensure them to be 1 GB minimum.
• B. Switch to TFRecords formats (app
• C. 200MB per file) instead of parquet files.
• D. Switch from HDDs to SSDs, copy initial data from GCS to HDFS, run the Spark job and copy results back to GCS.
• E. Switch from HDDs to SSDs, override the preemptible VMs configuration to increase the boot disk size.

Answer: C

NEW QUESTION 16

Your company receives both batch- and stream-based event data. You want to process the data using Google Cloud Dataflow over a predictable time period. However, you realize that in some instances data can arrive late or out of order. How should you design your Cloud Dataflow pipeline to handle data that is late or out of order?

• A. Set a single global window to capture all the data.
• B. Set sliding windows to capture all the lagged data.
• C. Use watermarks and timestamps to capture the lagged data.
• D. Ensure every datasource type (stream or batch) has a timestamp, and use the timestamps to define the logic for lagged data.

Answer: B

NEW QUESTION 17

Which of the following is NOT a valid use case to select HDD (hard disk drives) as the storage for Google Cloud Bigtable?

• A. You expect to store at least 10 TB of data.
• B. You will mostly run batch workloads with scans and writes, rather than frequently executing random reads of a small number of rows.
• C. You need to integrate with Google BigQuery.
• D. You will not use the data to back a user-facing or latency-sensitive application.

Answer: C

Explanation:
For example, if you plan to store extensive historical data for a large number of remote-sensing devices and then use the data to generate daily reports, the cost savings for HDD storage may justify the performance tradeoff. On the other hand, if you plan to use the data to display a real-time dashboard, it probably would not make sense to use HDD storage—reads would be much more frequent in this case, and reads are much slower with HDD storage.
Reference: https://cloud.google.com/bigtable/docs/choosing-ssd-hdd

NEW QUESTION 18

You are developing a software application using Google's Dataflow SDK, and want to use conditional, for loops and other complex programming structures to create a branching pipeline. Which component will be used for the data processing operation?

• A. PCollection
• B. Transform
• C. Pipeline
• D. Sink API

Answer: B

Explanation:
In Google Cloud, the Dataflow SDK provides a transform component. It is responsible for the data processing operation. You can use conditional, for loops, and other complex programming structure to create a branching pipeline.
Reference: https://cloud.google.com/dataflow/model/programming-model

NEW QUESTION 19

If you want to create a machine learning model that predicts the price of a particular stock based on its recent price history, what type of estimator should you use?

• A. Unsupervised learning
• B. Regressor
• C. Classifier
• D. Clustering estimator

Answer: B

Explanation:
Regression is the supervised learning task for modeling and predicting continuous, numeric variables. Examples include predicting real-estate prices, stock price movements, or student test scores.
Classification is the supervised learning task for modeling and predicting categorical variables. Examples include predicting employee churn, email spam, financial fraud, or student letter grades.
Clustering is an unsupervised learning task for finding natural groupings of observations (i.e. clusters) based on the inherent structure within your dataset. Examples include customer segmentation, grouping similar items in e-commerce, and social network analysis.
Reference: https://elitedatascience.com/machine-learning-algorithms

NEW QUESTION 20

You work for a bank. You have a labelled dataset that contains information on already granted loan application and whether these applications have been defaulted. You have been asked to train a model to predict default rates for credit applicants.
What should you do?

• A. Increase the size of the dataset by collecting additional data.
• B. Train a linear regression to predict a credit default risk score.
• C. Remove the bias from the data and collect applications that have been declined loans.
• D. Match loan applicants with their social profiles to enable feature engineering.

Answer: B

NEW QUESTION 21

You have an Apache Kafka Cluster on-prem with topics containing web application logs. You need to replicate the data to Google Cloud for analysis in BigQuery and Cloud Storage. The preferred replication method is mirroring to avoid deployment of Kafka Connect plugins.
What should you do?

• A. Deploy a Kafka cluster on GCE VM Instance
• B. Configure your on-prem cluster to mirror your topics tothe cluster running in GC
• C. Use a Dataproc cluster or Dataflow job to read from Kafka and write to GCS.
• D. Deploy a Kafka cluster on GCE VM Instances with the PubSub Kafka connector configured as a Sink connecto
• E. Use a Dataproc cluster or Dataflow job to read from Kafka and write to GCS.
• F. Deploy the PubSub Kafka connector to your on-prem Kafka cluster and configure PubSub as a Source connecto
• G. Use a Dataflow job to read fron PubSub and write to GCS.
• H. Deploy the PubSub Kafka connector to your on-prem Kafka cluster and configure PubSub as a Sink connecto
• I. Use a Dataflow job to read fron PubSub and write to GCS.

Answer: A

NEW QUESTION 22
......