What Is A Message Queue + RabbitMQ and MassTransit Integration
Vložit
- čas přidán 26. 07. 2024
- ☄️ Master the Modular Monolith Architecture: bit.ly/3SXlzSt
📌 Accelerate your Clean Architecture skills: bit.ly/3PupkOJ
🚀 Support me on Patreon to access the source code: / milanjovanovic
Message queues are required in most modern systems for the great benefits they bring. They're very useful for designing decoupled communication between components by passing messages to the queue. In this video, I'll show you how to use a message queue with MassTransit and connect it to RabbitMQ to publish messages.
Join my weekly .NET newsletter:
www.milanjovanovic.tech
Read my Blog here:
www.milanjovanovic.tech/blog
Subscribe for more:
/ @milanjovanovictech
Chapters
0:00 Running RabbitMQ in Docker
0:46 Configuring message broker settings
2:12 MassTransit configuration
4:30 Why is a message queue useful?
5:40 Publishing messages to the queue
8:21 Implementing EventBus with MassTransit
10:46 Consuming messages from the queue - Věda a technologie
Want to master Clean Architecture? Go here: bit.ly/3PupkOJ
Want to unlock Modular Monoliths? Go here: bit.ly/3SXlzSt
Great video. Do you have a blog post for this video? Couldn't find it. (I like reading instead of watching videos 😊)
joined it, thanks for putting all that content out there! amazing stuff
Great video, glad you're having fun exploring MassTransit!
I'm humbled, Chris! I've been using MassTransit on an internal project for the past 2 years, and I love it. Planning to make a lot more content around what it can do :)
The creator of MassTransit 🚌 himself!
Very good video, but I think Mass Transit + RabbitMQ would deserve and need more content. I hope you will do some follow up videos for this. Show the difference between Orchestration and Choreography. Maybe do a seperate little project and debug through 3-4 microservices involved in a transaction etc.
For sure, I have a lot more content planned around distributed systems!
@@MilanJovanovicTech Looking forward for that))
@Milan awesome video on RabbitMQ.
Looking forward to see more video on this playlist like Mediator to RabbitMQ, Mass Transit - Publish vs. Send, Commands etc.
They're coming
Impeccable timing, I'm just trying to get started with rabbitmq 🙂
I have a sixth sense for this 😁
Milan, great video as always!
Glad you enjoyed it!
Great video, I Learn a lot from u thank u
Sure thing! Check this one also for the internals: czcams.com/video/zKppEQIzvcU/video.html
Great as always
Thanks again!
It's very interesting video. Thank you a lot for all.
Glad you liked it!
Great One Milan as usual
I was searching for a related video on the channel to comment about new video idea
It is about how to create a consumer and producer console template so that when i need new consumer or producer i just go and create a new project of that template 😅
I'll record a video today about Pub-Sub with Redis
Good video and useful to explain the subject matter. I used to agree but honestly if you want to go at scale you need to move to the 2.0 (ish) of queue tech which are the cloud native queues (rabbit types) and logs (think kafka) in gcp, azure and aws. These systems provide much deeper serverless options that can scale up with little hassle. (chief arch, fortune 5)
Good thing that with MassTransit, the messaging is abstracted. You can easily plug in Azure Service Bus. I used AWS Rabbit MQ for example.
@@MilanJovanovicTech you mean that this example code run perfectly in an Azure bus? Could be good to see a video where you explain the connection with Azure bus. Thanks so much for explaining this topic.
I usually add event to the object name as well, it helps me to show explicit intent modeling IMO
Exactly, for the same reason I also append Command/Query to class names
Great video. But its more interesting to see how to interact with non-.net consumers. Who do not know anything about masstransit and its envelope and other things.
Passing JSON contracts is always an option
Excellent video Milan as always! I just started to learn MassTransit this week and I was need one of your videos about MassTransit, so you read my mind!
2 questions: Can you share the source code of the video? And, MassTransit have a Mediator pattern implementation, can we replace the MediatR library for MassTransit Mediator? Greetings from Argentina!!!
Send me an email for the source code :)
I haven't used MassTransit's Mediator, to be honest, so I can't really comment on it.
Pressed like before actually watching the video.
I appreciate the love 😊
Let me know what you thought of the video.
@@MilanJovanovicTech It was great. Thank you for sharing your knowledge
Great video!
So RabbitMQ is the lightweight database/queue persisting the events.
You hook up Mass Transit to connect to this 'queue' and whenever an event is persisted to the queue the appropriate mass transit event handler will consume it.
Is this an alternative to the transactional outbox pattern where you raise events to the DB and process them with a background service? They are similar. I can see the benefit of both for sure.
This one feels more lightweight than the TransactionalOutbox -> Background Service reading jobs. But maybe that's because I am considering that with the raising of domain events and the aggregate root unit of work commits to db.
You can think of a queue as a pipe. Something comes in on one end, and comes out on the other end.
It's different from an outbox since it's another external system to your database. It increases the surface area for things to fail.
I'm planning another video soon about Sagas and it'll cover queues a little bit more
Thanks for another great video.
should it not be extending your CLEAN architecture video where the Outbox pattern was used to persist events in the database and Quartz/Hangfire as background service using MassTransit to push those events from the database to Queuing service?
I'm saving that for another video 😁
@@MilanJovanovicTech and i m waiting for it :) Thanks
Great video, but could you explain how you've set up your project to run in a docker container. Apologies if I've missed a video where you explained this.
I don't think I covered a Docker-compose set up from scratch. Will make a video abut that!
Really nice explanation/demo. I recommend you to upgrade your mic and then you'll sounds like a real pro content creator. Knowledge wise, seems like you already got what it takes :)
Oh, really. You think the mic doesn't cut it? 🤔
@@MilanJovanovicTech the content is great, maybe I just grew accustomed to Jeffery Way voice
Thank you, I have one question. When you publish the message how the RabbitMq stores the data? I mean does it make some kind of serialization ( json, byte array) ?
I think you need to check out the docs for the best answer.
All your videos are great. Can you share the code that you are using in your videos.
Yes, here: www.patreon.com/milanjovanovic
And on some of them, check the pinned comment in case the source code is public :)
Hi Milan! Can I ask you something? When you use records in C# is it much simple and less code with only the constructor and pass the values through the constructor instead of properties? or this is just a clean-written code? After all, records are read-only I know. Maybe I'm wrong. Can't be modified additionally in another place with accessing some property. Thanks for the videos 🙂. Поздрав од Македонија. 😉
I find it simpler, that's why. Plus I get immutability, etc. By no means a convention, but an approach I found useful. MassTransit also recommends using interfaces for message contracts. But this is annoying in that you'll have to make implementations on sending side, so double code. Pozdrav iz Niša :)
@@MilanJovanovicTech Thanks a lot for answering me 🙂
What is the level of support for Mass Transit for the Outbox pattern? Is this going to be presented later or we rely on the video already shared in the Clean Architecture series?
I haven't explored it, honestly! I'm sure there's good support however, but I'll do some research and make a video.
I checked it quickly in the docs they do have something cooked up, I would love to see this in a video :))
Is this a good replacement for outbox pattern? Still experimenting and all those patterns are a tiny bit hard to swallow without practical experience.
Thanks for the great video
Not the same thing
Is OK to connect to the Message Queue / RabbitMQ from a client running javascript? Or would it be a design failure, an overkill, (too much load/ package size, etc.) and should translate to REST apis / websocket instead for the small clients?
Isn't that a security concern? 🤔
@@MilanJovanovicTech That's is also why I am asking, There most be reasons it is not a widespread way of doing it.
I don't know , but you're right, I think, to protect the REST APIs + Websocket / SignalR, etc. with TLS + token auth, etc is much simpler and usual then to have the equivalent with MQTT messages on the client I don't know if such a solution exists at all.
However on even smaller client, like a microcontroller, a direct MQTT still make sense.
Great Video, but one question.
what is difference between event bus and domain event from previous videos?
One is in-memory, one is a separate process
@@MilanJovanovicTech can we implement domain event with event bus? Can you tell a little more. Thanks
Hi , pls would request you to make more detailed videos involving 2 microservice + az function using Mass Transit + RabbitMq.
Will do, just takes some time 😁
@@MilanJovanovicTech Thanks
What exchange is mass transit using? Topic? Fanout? Direct? I'm assuming it setups these with the bindings to the queue as well.
It's different based on the underlying transport, so for RabbitMQ these are the docs: masstransit.io/documentation/transports/rabbitmq
I put a breakpoint inside of Consume() method but apparently I can't get there. I registered my Consumer inside of MassTransit configurator. I notice activity in RabbitMQ, but the consumer method isn't executed.
Why?🤔
I just recently started learning RabbitMQ. Shouldn't I have a queue created in this case?
Probably missing a call to ConfigureEndpoints inside the call the UsingRabbitMq (I forgot to mention that in the video, my bad)
@@MilanJovanovicTech You're right, I added it and now it's ok. Thanks for feedback
What's the reason for abstracting the IPublishEndpoint behind another interface IEventBus?
You may want to introduce some logic around the publish action, and making an abstraction for that is a good idea. But you absolutely don't need to do it, and can use IPublishEndpoint directly
I don't quite understand why the extra abstraction in the way of the EventBus class
Enforcing some constraints
Great Video
I just Noticed that with Core 6 and MassTransit 8.1.2
you need to add rabbitMqConfigurator.ReceiveEndpoint In the Configuration for UsingRabbitMq in the Bus configurator Or am i missing somthing ?
just for Reference
busConfigurator.UsingRabbitMq((context, rabbitMqConfigurator) =>
{
var messageBrokerSettings = context.GetRequiredService();
rabbitMqConfigurator.Host(new Uri(messageBrokerSettings.Host),
h =>
{
h.Username(messageBrokerSettings.Username);
h.Password(messageBrokerSettings.Password);
});
rabbitMqConfigurator.ReceiveEndpoint("EndpointListenerName", endpointConfigurator =>
{
endpointConfigurator.ConfigureConsumer(context);
});
});
Yeah, must've omitted that part
Great, why not fire the Publish method inside a background job using Quartz or Hangfire like you did in TransactionalOutbox video?
Didn't want to complicate it for an intro video, but that's a viable option. And what I generally do in production
@@MilanJovanovicTech Got it. Hope to see more videos on this subject ..
What exchange and queue this message was published? Because I can't see any configuration for exchange or queue
Good question, I guess the library provides some default values?
MassTransit creates the required queues/topics for you in the background. It's abstracted away because it can vary depending on the actual message transport you're using.
@@MilanJovanovicTech OK and where is configuration for message transport? Because I see just publish message by default
@@jacobjacob8062 At thr beginning, the call to UsingRabbitMq
@@MilanJovanovicTech there is only configuration for host... But what about message transport?
I am new to the Message Queue World, I am trying to learn about it and this video helped me a lot with that.
I have implemented everything as you did. Though, in the RabbitMQ management page, the exchange does not show any consumers and the messages just disappear on their own. No logs, no emails, no hitting breakpoints, nothing. What may cause this?
I think you're missing some configuration code. Best refer to the MassTransit docs
@@MilanJovanovicTech Yeah, I have missed something. Apparently, at the very end of the lambda function of UsingRabbitMq, there should be a "configurator.ConfigureEndpoints(context);" which I was missing. It solved the problem!
@@anilkarasahh I believe I also missed it in the video, which added to the confusion :/
@@MilanJovanovicTech It sometimes feels magical that something seems to be working on a tutorial and does not on me because something is missing. Never understood why and how :)
Milan forgot to configure the endpoints in the video on UsingRabbitMq. serviceBusBusFactoryConfigurator.ConfigureEndpoints(busRegistrationContext);
Hi, is there a Github source for this demo solution?
No, I share the code on my Patreon: www.patreon.com/milanjovanovic
Can you make a video for Masstransit saga?
I just recorded one for Rebus Saga, but the concepts will be similar to MassTransit
@@MilanJovanovicTech can't wait to watch
Why register the IEventBus as Transient and not as Scope?
No particular reason. The IPublishEndpoint is scoped, so scoped would've been more correct there.
Question, would it be wise to implement multiple IConsumer of different types to the same concrete class? Assuming the events it consumes are related?
Good question! In theory that should work without a problem. On the other hand, I never tried it out so I have no idea how if it will work.
Rabbitmq is good, but Kafka scales better I think
Do you need to scale?
do you guys memorize the code or what... i have never seen him using documentation in any video! so he memorize the syntax before or he opens the doc in the second screen i don't get it... and he never shares the original links. Please guys go and read the documentation you'll never learn like this
Or he simply worked with the library for a long time that he knows the code by heart? 😅
That's how I make most of my videos to be honest, I talk about what I know.
And of course I do my research beforehand, to brush up my memory. But you can't expect me to teach you everything in a 10-20min video. My vision is more to explore a concept and give you some direction, and it's up to you to go and do your own research.
I have a shirt like your one 😂. It is clone 😂
Pure class 😁👌
Everything is cool, thanks! But! It's all in the sandbox category. What if we have a producer that is an external service (not in the same namespace as the consumer), and we need to beat off messages that accidentally entered our consumer in such a way as to not crash the application. To make everything beautiful and asynchronous. Here you would show such a complex example. how great it would be! but these are all "Hello, World!" The examples don't really help, to be honest. All those examples from the documentation are too primitive and have nothing to do with the combat environment. Thank you.
That's a pretty fair critique, to be honest. Now that I'm doing this full-time, I actually have the time to start working on some "real world" use cases for the videos
@@MilanJovanovicTech This is not criticism, this is a cry from the heart)) I have just such a task, I am intensively looking for combat options for Rabbitmq, and, alas, I can’t find it)) SOS))