Spring Tips: Spring Batch and Apache Kafka
Vložit
- čas přidán 27. 07. 2024
- Hi Spring fans! Welcome to another installment of [_Spring Tips_ (@SpringTipsLive)]( / springtipslive ! In this installment we look at the just-landed community contribution in Spring Batch adding support for Apache Kafka. This new support offers some tantalizing opportunities to bridge batch and streaming workloads. Stage large writes to backend warehouses with Kafka and drain the stream with Spring Batch. Load large amounts of data from existing datastores and funnel it into stream processing pipelines. And the possibilities become even more interesting when you consider Spring Integration, Spring Cloud Stream, Spring Cloud Stream Kafka Streams, and Spring Cloud Data Flow. If you want a quick primer on Spring Batch-basics you might like the Spring Tip installment on Spring Batch from - _gasp_! - two years ago!
speaker: Josh Long (@starbuxman)
twitter: / starbuxman - Věda a technologie
Excellent I was thinking in a way to include kafka in my ETL process and this is exactly that!
Wow.. Its a very detailed explanation. Thanks for the video and really appreciate this.
If possible coold you please provide a github link for the test program you built....
Nyan cat loading bar ✔
I don't get it why you would want a batch job to read from Kafka, why not just have Kafka consumer reading from the Topic directly ?
did you understand this one? I am also not able to understand.I dont see any examples showing error handling and all in this.
He could do that too but he's showing how to do via job that's the main moto of this video.
does the consumer job ends ever ?? because it is listening to the kafka ?
it did terminate at 43:47, the default polling timeout is 30 sec
could u send the example code to github?
GitHub有啊，一般他都会将代码推到GitHub的
@@zhaowentao6036 呵呵，地址？
@@user-vq0mo1011a github.com/spring-tips/kafka-and-spring-batch it's there, along with all the other Spring Tips code. Thanks for watching
@@JoshLong To create main() use shortcut type `psvm` in IntelliJ.
I don't understand people who use dark theme in translations
Why do you annotate every method with bean....? It is kinda non-sense.
kretynek1 If you're asking this question then honestly this talk isn't for you...
@@ITech2005 do you know why though? if that returned data type is only needed in the called method, why add @Bean everywhere? just curious
@@blasttrash I dont know what part of the vid youre referring to, but keep in mind that some beans are needed by the Spring Framework integration. So spring may use them to instantiate the Kafka client, but you wouldnt need to directly use those beans in your code. Spring MVC and Security are the same way in that some beans need to ve created to configure the framework but you never directly use them.
@@blasttrash And just to back up a bit, if the method is annotated with @Bean then it provides a class that will be injected elsewhere. You wont see where its being used by looking at the method alone. IDEs like Intellij can help you find where classes are injected or used.
@@ITech2005 oh that makes sense. Thanks a lot for your reply even after 2 years.