NodeJs Microservices using RabbitMQ (Message Queueing)
Vložit
- čas přidán 5. 07. 2024
- We are going to use RabbitMQ in order to communicate between different microservices in NodeJs . This example uses the Direct exchange type in a logging system.
Source Code : github.com/charbelh3/RabbitMQ...
RabbitMQ Basics : • RabbitMQ basics (Inclu...
RabbitMQ RPC model (request-reply) : • RabbitMQ - RPC with No...
Timestamps
---------------------
0:00- Introduction
0:20- Explaining the system to develop
1:50- Creating the Logger Microservice (Producer/ Publisher)
2:44- Steps to create a Producer
3:08- Creating the Producer class
11:33- Creating the API
15:27- Testing the API and the producer with Postman
17:30- RabbitMQ Management UI
19:05- Creating the Info Microservice (First Consumer)
19:12- Steps to create a Consumer
20:34- Creating the consumeMessages function
27:26- Testing the Info Consumer
29:50- Analysing the changes in rabbitMQ Management
30:33- Creating the WarningAndError Microservice (Second Consumer)
32:34- Testing the WarningAndError Consumer
32:55- Testing the whole application
36:10- Analyzing the changes in rabbitMQ Management
36:48- Taking a look at our Initial Diagram
37:25- Explaining the important of ACK
39:17- Final Recap
Watched both RabbitMQ videos back to back. Thanks. 🐰
Great explaination.
Great explanation! Thanks!
Thank you very much, so clear
Great tutorial! Nice and simple!
Wow, nice video! I get an task to implement both consumer and subscriber, that helped me a lot to understand the library and I did some unit tests here.
I'm glad it was useful to you!
This is superb
Excelent! !
Nice one and great tutorial
Thanks!
great tutorial
M late to see this video but it's very well explained.👏
thank sir... , helpfull👍
my lord thank you!
thsnks its perfect course
You're welcome!
RPC model in RabbitMQ (request-reply pattern) : czcams.com/video/APfWkfkjRj8/video.html
very good teacher, thank you @Computerix, is there any chance to create a tutorial where the consumer is in a browser web application? like a log viewer web page for live monitoring?
Thank you for the suggestion!
How does the consumeMessages() in infoMS log all the published messages, despite calling consumeMessages() function only once in app.js ? Does calling this function once force app.js to keep listening to queue?
channel.consume() keeps on listening to the queue to consume new messages. It's like when you use app.listen() in your server to keep on listening to incoming requests on your server :)
@@Computerix That was super helpful, thank you! Your videos have been really helpful to me
@@parthpawar7837 I'm very glad to hear that! Thanks for your feedback
Thank you so much 👍🙏👋👌👏🏻🤝🫶🏻✌️🫡😎
Hey i need your help in this when i am starting the server 15:35 in my pc its showing cannot
const Producer=new Producer();
^
ReferenceError: Cannot access 'Producer' before initialization
Hello, make sure to import the variable as "Producer" (Capital P), and then the new variable (which is used as a class instance) should be different from that one, which is why I used "producer" with a lower "p". Point is, they should be different names :)
Hope this helps !
Thank you for video. Is it possible to use nitro.js instead of express.js
Hello bro.
thanks for the video.
I do have some questions regarding the channels.
I want to create a simple chat application using RabbitMq, does that mean that for every user on the app, there will be a new channel?
Hello! You can either make 1 channel and then handle the routing of the messages using the direct exchange (you can use the username or userID as a unique key for example) and then you create a queue for every new user that joins the chat app with the username/userId as a binding key
or you can create a separate channel for every user (but that would cause performance issues if you have many users)
So it would be better to stick to the first option
@@Computerix Thanks a lot for the quick reply.
I will go ahead to implement the first option.
However, I do have another question.
Can you also explain how I can possibly implement a group chat?
What I have in mind is a scenario whereby, a group created by a user will have a unique queue name which will serve as an identifier and the moment a user joins a group, he will begin to listen to that queue. I will also ensure that all these queues are all on the same channel.
@@japan9169 Before answering, I wanna note that my previous answer assumes that you can only use RabbitMQ to create a chat app. In a real world application, it would be much better to use web sockets (sockets.io for example) for chat apps, instead of a message broker. So if the app you're making is for production, it would be better to either use Web Sockets, or a mix of RabbitMQ with sockets.io
In that case, you could make 1 queue only for the whole chatting system (e.g : messaging-queue), and the message payload would contain the userID or groupID. Whenever the consumer receives a new message, it looks at the userID / groupID and you could emit a message to that ID. The client would be listening for new messages (using socket.on('new message', (payload)=>{ // code }) for example...)
However, if you're only using rabbitMQ for your chat app for practicing reasons, then you should continue using the way proposed earlier (1 queue per user, although keep in mind, queues have metadata, so having 50k users would mean 50k queues, each having their own metadata even if they are idle..)
Regarding your question about the group chats (and assuming you're not using the Web Sockets options). Using 1 queue per group is NOT the right way to do it, since rabbitMQ uses the round robin algorithm, meaning when a queue has multiple consumers, it would send the message to the consumers in a round robin fashion (not all users in the group chat would receive the message, BUT only 1).. So, a way to achieve what you need, would be using an exchange of type FANOUT (meaning it broadcasts the messages it receives to ALL connected queues), so you would need to create a different exchange for different group chats, and then bind a new queue for every new user that joins the group chat(you can reuse the previous created user queues)
@@Computerix Yh that is absolutely correct, I want to use RabbitMq for the chat app for learning purposes.
It seems like RabbitMq will not be the best match for creating a group.
I will simply create a basic chat application where two people can communicate using rabbitMq
@@japan9169 good luck !
Hello this is a great tutorial, but i have a question, you create a async function called consumeMessages(). but why you didn't use await when calling the function? thank you
Hello ! Normally when you have an async function and call it with await, it means you're waiting for it to complete by blocking the execution of the rest of the program. In this case, we don't care about blocking / waiting for a specific response before excuting the next lines :)
@@Computerix hooo I get itt. Thank youu for your responsee. Really appreciate that! 👍
@bernardusyoga5936 you're welcome! Further more, in the case of this video, we're calling the consumeMessages() function from the global scope, which means that we can't use await either way (since we're in the global scope)
I want to know whether we should acknowledge the message consumed by us before processing the message or after all the processing is done ??
It depends on your use case. If you need the message to be removed from the queue AFTER all the processing is done and not miss any potential steps, then you should acknowledge the message after finishing all the processing you require. That way, if an error occurs during processing, you do not lose the message, since it will be requeued since it wouldn't be acknowledged up until the processing succeeds..
but when I change in key it still consume the message
very well explained, Thanks
you're welcome :)
Nice one
Niece Video
God bless you
Can we have the source code
Thanks for the feedback !
Here you go : github.com/charbelh3/RabbitMQ-Logger-Example
Do we need exchange? I think only queue would do.
There is always an exchange when sending messages, even when you use channel.sendToQueue(queueName, data) instead of channel.publish(exhangeName, key, data). The difference is when using sendToQueue, you're using the default exchange which is of type direct, and the queueName would be used as key name. In this example, since the exchange we're using is of type direct, you could use sendToQueue directly. I've made a video that uses what you've mentioned: czcams.com/video/APfWkfkjRj8/video.html
How to clear message/ channel once its delivered successfully
26:07 if the message is consumed, the consumer service must call the ack() function to acknowledge the message and clear from the queue/channel.
Sorry for the late reply. We used channel.ack(msg) in the code after consuming the message, in order to inform RabbitMQ that the message has been successfully processed and can be removed from the queue (that way it would not be requeued and consumed more than once)
TypeError: Producer is not a constructor ,getting this err
when i'm starting the server
You probably made a small mistake, feel free to check the code: github.com/charbelh3/RabbitMQ-Logger
Isn't that same as socket io?
They are different. RabbitMQ is a message broker that allows complex message routing based on different options (exhange type/routing keys) and you have the capability to control the message durability, message routing and message acknowledgement (delivery guarentee).
Socket.io is an event-driven library for real-time web applications, meaning it helps implement real-time communication solutions in applications, and does not provide message queuing or delivery guarentees.
For example, if you need to show live updates on a browser you would use socket.io. If you need to process jobs and in a controlled manner, you would use a message broker (in our case rabbitmq)
@@Computerix Thanks for the replay but I think socket io has acknowledgement system too.
@@muhammadfahad3483I wasn't aware of that. But the main difference between socket.io and rabbitMQ is the use case. Socket.io is used in real-time web application. RabbitMQ is used as a message broker and you have more control over what happens and how jobs are processed, divided etc.. Hope his helps :)
@@Computerix Thanks again, I will research on my own too.
Requesting your help :
I followed along the video up to 16:41
On visiting local host 3000 I get Cannot GET / before using postman.
during the process of sending POST request in postman to the local host my program fails and I get this Error
node:internal/process/promises:288
triggerUncaughtException(err, true /* fromPromise */);
^
Error: connect ECONNREFUSED ::1:5672
at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1487:16) {
errno: -61,
code: 'ECONNREFUSED',
syscall: 'connect',
address: '::1',
port: 5672
}
it seems you are unable to connect your app with the rabbitMQ server, can you try opening "localhost:15672/" on the browser ? It should open up the rabbitMQ management tool. If it doesn't, there is a chance you didn't install rabbitMQ correctly
Make sure rabbitMQ is installed correctly on your machine www.rabbitmq.com/download.html
@@Computerix Hi, installed rabbitmq, I am able to open localhost:15672/ but still having the same error as mentioned above when hitting api from postman. Kindly help in resolving this issue!