How To Stream Large Files Over TCP In Golang
Vložit
- čas přidán 6. 06. 2024
- ► Join my Discord community for FREE education 👉 / discord
► Become a Patreon for exclusive tutorials 👉 / anthonygg_
► Follow me on Twitter 👉 / anthdm
► Follow me on GitHub 👉 github.com/anthdm
In this Golang tutorial, I will teach you how you can stream large files over a TCP network in Golang.
#golang
► Join my Discord community for free education 👉 discord.com/invite/bDy8t4b3Rz
► Become a Patreon for exclusive tutorials👉 www.patreon.com/anthonygg_
► Buy me a coffee 👉 donate.stripe.com/aEU2a6ayH2uCa3u4gg
Thanks for watching
"just use little-endian and drink margaritas at the beach"
"not too big because my pc is pepega"
these videos are just perfect, perfect pace, perfect content, perfect tone, perfect accent
this video is really great and very educational with the different steps/tests. Thanks for your work, the days are too short to watch all your videos :)
Thank you!
man... i cant express how much this will do for my cdn in golang... Dear Anthony, bless you😃
Your welcome my man!
скинь ссылочку на cdn
Your videos are very helpful, no doubt this is the best golang content creator
Before watching this video I just listen what streaming is but today I have seen how actually it works. This streaming concept clear but buffer is and how stream it. Great we are expecting more exciting and interesting videos.
Like your work! Thank you! Learned a lot through your videos
Awesome , Thnx for the amazing Go content.
i need to see ur video over and over! Perfectly fine.
Great video. I saw a few problems with the final solution. In the sendFile you append the size of file to the conn. In readLoop you read that size in the loop which is problem. And even if the size is in out of the loop, this will try to copy the entire byte data from the conn into buf in one go.
I'm new to programming and I think this is cool, gonna save the video, I think it's useful :)
nice video anthony
Thank you for sharing
You are the best!
Thank you for your video.
How will this transmission work if the sender don't know the file size? Is it possible just send the EOF in the end of transmission? Does conn send EOF when the connection closed?
the best
MR Anthony I really appreciate your content. Please I have a small question I want to know why I opened my activity monitor and my RAM is getting too large even though It's streaming
Awesome
I just don't get one thing, what condition will stop the for loop ? Cause we want to stop it at some point right ?
Time to build my own OTT platform startup
Why are you reading the file size within the for loop?
Yes, it leads to neverending stream of random bytes.
Get it above the for loop and it will do what's intended.
Great as always
But your first implementation is also a streaming one: you are not reading whole file into memory, you have a fixed buffer which is updated on each Read.
I think io.Copy just does the same thing
I think io copy is bit more effiecient under the hood.
Anthony is a Go Master ultimatum
The undisputed.
I am very new in go. Can you specify the advantages of streaming file data comparing non-streaming file data in transfer? Thank you sir.
Non streaming is saving the complete file in memory. Hence 30 gb files will not work on a simple server
@@anthonygg_ even with the second appraoch if you send 30 gb file it will still allocate 30 gb of memory `binary.Read(con, binary.LittleEndian, &size)` since size is 30 gb
Awesome video. Big thanks.
I have a question, if I set size: 400000 program exit with code 1.
Error message:
recieves 400000 bytes over the network
2022/12/16 20:50:03 EOF
Process finished with the exit code 1
Could be a read or write timeout. You can configure that
@@anthonygg_ Thanks
the for loop on the read with the end solution is a bit redundant i think. Wouldn't this solution just endlessly (for huge files) read into memory anyway? the Buffer will just fill up continuously, instead of allowing you to process x number of bytes?
I need to check this, you could be right
Can I download twitch stream and stream it to youtube upload API directly, using this method?
That will involve some more work my man
How does the method "binary.Read" know where the size data is and where the file data is?
Server sends, filesize. Binary reads filesize. Server sends filedata. Binary read file data.
Would be amazing if you could share github snippets with your videos
Im getting paid way to less for that. This is free content made in my free time. Leeched by many. Returned by a few.
@@anthonygg_ I feel you! Your hard work shows even in your free stuff.
brother, this is the second time i see you sending stuff over the network without taking into account that the operating system can not always send N-bytes just becuase you wish to send N-bytes.
it may send N/2.. or much less.
if you wish to stream, you have to keep track of a pointer to how much has been send.
if N/2 has been send, then you send from N/2 until N..
you can not declare the chunk to be sent, if only a piece of it may be sent.
the issue does not occur locally, it occurs when you do this over the net!
What are you talking about my man. Read net.Conn.
I have no idea what the dude is saying either. RTFM @@tanko.reactions17
You need read file size before loop 😅
rand.Reader is missing