Google Cloud Dataflow

 

• Cloud Dataflow is a fully managed data processing service for executing a wide variety of data processing patterns.

Features

• Dataflow templates allow you to easily share your pipelines with team members and across your organization.
• You can also take advantage of Google-provided templates to implement useful but simple data processing tasks.
• Autoscaling lets the Dataflow automatically choose the appropriate number of worker instances required to run your job.
• You can build a batch or streaming pipeline protected with customer-managed encryption key (CMEK) or access CMEK-protected data in sources and sinks.
• Dataflow is integrated with VPC Service Controls to provide additional security on data processing environments by improving the ability to mitigate the risk of data exfiltration.

Pricing

• Dataflow jobs are billed per second, based on the actual use of Dataflow batch or streaming workers. Additional resources, such as Cloud Storage or Pub/Sub, are each billed per that service’s pricing.