Spark Streaming - Apache Spark Tutorial For Beginners

You will learn spark streaming in this session and how to process data in real time using spark streaming. Topics covered include what is spark streaming, how spark streaming works, spark streaming architecture, microbatches in spark streaming, batch interval, spark streaming context, dstream, sliding window, window duration and window size, stateful transformation in spark streaming, stateless transformation in spark streaming, transformations and actions in spark streaming, spark streaming sources, spark with kafka, files and socket streaming and various other details.
Limeguru Website:
www.limeguru.com
LimeGuru CZcams Channel
/ limeguru
Limeguru Facebook Page
/ limeguru

[@ankitajindal1539]

Excellent explanation. Keep up the good work and thanks !!

[@Redstarcrew]

this explanation was very good, thank you sir

[@Jay-vh3gx]

Thanks.. for crystal clear explanation 👍

[@ramum4684]

Excellent job you doing Lime Guru. Thanks for the effort . Expect more on Apache Spark

[@bnsagar90]

Good explanation. I didn’t find batch streaming and sliding windows concepts explained like this.

[@medotop330]

And I am

[@yogeshkawde8926]

Job well done. Nicely explained. This video deserves more views and likes. Kudos.👍👍👍

[@medotop330]

Thanks too much

[@kuntalchowdhury5336]

Good explanation.

[@lavanyamukka5597]

Good explanation 😍

[@sujitunim]

Nicely explain

[@bealegopro]

Good job

[@debapriyamaity1202]

Good explanation. Can I use Spark streaming to read from hive table with millions of records.
Say my usecase is like I have a source java program which uses Spark stream to read data from hive table and keep on posting data packets to some kafka connector.
I am mainly concerned with memory usage. What is your opinion on this ?

[@Remesh143]

Hello, As you mentioned on streaming source-File. Is it possible to stream a file as source and as and when new lines append to a file(csv file), is it possible to do streaming on top of that? if yes.. which streaming source method i need to call...please help

[@akashhudge5735]

in Dstream in one batch you get only one RDD, check below link
stackoverflow.com/questions/35164634/how-many-rdds-does-dstream-generate-for-a-batch-interval