Fast and efficient data serialisation with Protocol buffers (protobuf) in .NET
Vložit
- čas přidán 22. 11. 2020
- Subscribe: bit.ly/ChapsasSub
Become a Patreon and get source code access: / nickchapsas
Hello everybody I'm Nick and in this video I will show you what are the advantages of using Protocol buffer (or protobuf) in order to serialize and deserialize your data in .NET. Not only you will be able to run faster operations because of it but you will also be able to store a significantly smaller object while gaining performance. In this video I will use Redis as an example of a use case.
Don't forget to comment, like and subscribe :)
Social Media:
Follow me on GitHub: bit.ly/ChapsasGitHub
Follow me on Twitter: bit.ly/ChapsasTwitter
Connect on LinkedIn: bit.ly/ChapsasLinkedIn
#dotnet #csharp #protobuf
It's a great video and a great use case of proto serialization, really out of the box, it open my mind for a lot places where I can use it. Thanks a lot!!!!!
Moving our internal microservices call to gRPC, speed matters! Thanx for Redis case, that's a great improvement also!
Fantastic video, you are very good at explaining things in plain English.
Thanks Nick for sharing with us. You are saviour. God bless you ❤️.
You mentioned a docker-compose you were going to add to the description, but I'm not seeing it. It IS a helpful video!
The memory gains in the Redis cache are a great benefit, but if the aim was optimising response time would it make more sense to cache the response body verbatim? I.e. if the response body was the json string then if you had a cache hit, you wouldn't even need to deserialize (and then serialize into your dto again) as you can just pass on the cached value instead (which is already a json string of your dto).
Hi Nick, I know this is an older video, but could you maybe try out different data serialization libraries, compare them and share you insight about them. Json, MessagePack, protobuf and MemoryPack?
awesome man, thanks for on point explanation
Thanks a lot! Very useful video!
Really want to see grpc video! Protobuf looks very efficient too
Very good technology ill have to take a look. how does it handle adding a new property to the contract?
What do think using protobuf to pull tracking history which is in our case about 2MB atleast ..takes about 15 sec?is this production stable? I'm thinking of trying it.
It is protobuf make more memory allocation cause intensive MemoryStream usage?
Is there any way to automatically create contracts without all the messing around with the attributes..? I'd rather just use JSON at this point, less stuff to get it to work
Hey Nick, Great Video thanks a lot for posting. One question for you. can we use protobuf to do serialization and deserialization for the rest calls ? Like I wanted to communicate between my angular app and aspcore api with protobuf instead of JSON objects. Please advice
You actually can. There is also a nuget package that adds a header that you can use to enable it. The problem is that the recipient needs to be aware of the protobuf contract. If you get to this point then you might as well use gRPC
Hi, How to use buffers ring in microservices with ocelot?. Lookl forward to avideos.
you scratched the surface, now theres an itch, need more input!! :P
Ok.. So basically this is only to demo the local serialiazation/deserialization for your in mem cache right? Coz i see youre still sending the json format over the wire! if so then where's the real benefit of using protobuf in the first place.
The benefit is that Redis will store less data which means that more data can be cached and the data will be deserialized and serialized faster and with less memory from and to Redis. The only place where JSON is returned is on the API layer, but that's the other end of the app and it's irrelevant to this demo. If you want protobuf based responses you'd need to implement the same serializer for the API layer and expose it using the application/protobuf content type, but at that point you might as well migrate to gRPC. Btw in case it wasn't clear, protobuf is sent over the wire to Redis. It's not an in-process cache.
@@nickchapsas Yep I got that one. Just making sure the wire activity is still on the json format. Thanks man! Good demo.
Protocol buffers don't allow types to be nullable, which is a huge drawback for this use case.
First, not supporting null is a benefit not a drawback. Besides that, you can use "optional" if some value can be null.
Oh, hi, Nick! Thanks for sharing all the great content you do. I disagree on this one, though. The optional keyword doesn't let the value be "unset". So if your JSON schema has optional fields, your protocol buffer can't represent that schema without adding boolean fields and mapping logic. And in general, once you pollute the data with default values, you can't change the default any more. Nulls serve a real need in that regard.
@@lkarabeg If your contract is so fluid that it can have so many nullable values then you probably have to look into that first.
What is it? "A language-neutral, platform neutral, extensible mechanism for serializing structure data".
No, what is it really? "A good binary serializer".
I don't think they describe it well for adoption.