What is Dataflow?

What is Dataflow, and how can you use it for your data processing needs? In this episode of Google Cloud Drawing Board, Priyanka Vergadia walks you through Dataflow, a serverless system for processing and enriching data, supporting both streaming and batch models.
Chapters:
0:00 - 0:14 Video snapshot
0:15 - 0:57 What is Dataflow
0:58 - 2:23 How does Dataflow work
2:24 - 3:34 How to use Dataflow
3:35 - 3:58 Dataflow security
3:58 - 4:24 What does it cost to use Dataflow
4:25 - 4:55 Dataflow use cases
Check out other episodes of Google Cloud Drawing Board → goo.gle/GoogleCloudDrawingBoard
Subscribe to get all the episodes as they come out → goo.gle/GCP
Follow Priyanka on Twitter → goo.gle/2V19Zs7
product: Dataflow; fullname: Priyanka Vergadia;
#GoogleCloudDrawingBoard
Credits for some icons icons8.com/

[@MartinCapodici]

Thanks, this is ELI5. This is what I needed (the text docs assume I want to use the thing, but this helps me understand if it is even useful).

[@mayuryadav8010]

made simple to understand, great job

[@Sunil-ib7ez]

Really great presentation and I loved the way you have explained it. Only one request / suggestion, background music is distracting, at-least for me. So I would like to view more videos but without background music.

[@mahadevhallale5029]

simple and easy to understand ..Thanks Google cloud team

[@googlecloudtech]

You are welcome!

[@jitrammaharjan9506]

Really helpful thanks

[@diptyprakashswain1121]

Kickass explainer video

[@punitpalial]

Ok so basically Dataflow is about streamlining all the data that is collected from various sources and compiling at one place and then effectively process them to find suitable solutions.

[@divukman]

Thank you for saving me 5 minutes of annoying video...

[@juniafindlay5310]

Thank you!

[@divertechnology]

great video is useful

[@abhinavo5873]

Great video...

[@kiranshenvi2626]

Hi, I have a query how to read data of a table from another project which is in separate region. GCP doesn't allow to use query involving 2 tables from different regions. Please advise.

[@brahma0115]

Hi Google cloud team, could you please make a video on difference between data flow and data fusion?

[@DataMan90210]

Great video explaining Dataflow. What is used to create this visuals? Asking because I want to make something similar for my certification prep.

[@ioptia]

Priyanka!!!! You r amazing!!!!!

[@divertechnology]

me fan too

[@lionelshaneyfelt5607]

Google Cloud iz Awezome! ;)

[@Filip-ci3ng]

Please make a video to compare dataflow with Data Fusion. It's pretty confusing for new GCP teams we need to make decision, one or another. I personally HATE UI click ETL tools because I do not want to have to click on tons of buttons in order to understand what is a block doing so for now I feel like dataflow is a better choice. So data Fusion is not per my liking but I need more information in order to argue against it.
We have developers highly skilled at SQL and willing to learn Python for accessing buckets/ listing files and whatever pre-code needed before a pipeline kicks off.

[@pvergadia]

We realize that we made a typo at 4:14 it should be FlexRS instead of FelxRS.

[@TradeWithCodeOfficial]

Can we create dataflow jobs in javascript language?

[@rajashekhar433]

Looking for Machine Learning video in this way

[@pvergadia]

Noted! thanks for the ask Raj!

[@diptyprakashswain1121]

👍

[@nameunknown007]

Yes please, music is annoying and distracting, at least lower the background music. Thanks a lot for uploading these videos!

[@abdullahalhanip3318]

thank you dear... i am from Bangladesh

[@sinaashrafi218]

Good information but sound sucks!

[@mikelewis495]

You took the time to break your 5 minute video down into chapters?