The Virgin Media ISP outage - What happened?
VloĆŸit
- Äas pĆidĂĄn 7. 06. 2024
- BGP (Border gateway protocol) withdrawals caused the Virgin media ISP customers to lose their Internet connection. I go into details on this video.
0:00 Intro
2:00 What happened?
4:11 How BGP works?
11:50 Version media withdrawals
15:00 Deep dive
Fundamentals of Backend Engineering Design patterns udemy course (link redirects to udemy with coupon)
backend.husseinnasser.com
Fundamentals of Networking for Effective Backends udemy course (link redirects to udemy with coupon)
network.husseinnasser.com
Fundamentals of Database Engineering udemy course (link redirects to udemy with coupon)
database.husseinnasser.com
Follow me on Medium
/ membership
Introduction to NGINX (link redirects to udemy with coupon)
nginx.husseinnasser.com
Python on the Backend (link redirects to udemy with coupon)
python.husseinnasser.com
Become a Member on CZcams
/ @hnasr
Buy me a coffee if you liked this
www.buymeacoffee.com/hnasr
Arabic Software Engineering Channel
/ @husseinnasser
đ„ Members Only Content
âą Members-only videos
đ Backend Engineering Videos in Order
backend.husseinnasser.com
đŸ Database Engineering Videos
âą Database Engineering
đïžListen to the Backend Engineering Podcast
husseinnasser.com/podcast
Gears and tools used on the Channel (affiliates)
đŒïž Slides and Thumbnail Design
Canva
partner.canva.com/c/2766475/6...
Stay Awesome,
Hussein - VÄda a technologie
Hi Hussein, I am a SW Engineer and I love your videos. I am a Virgin Media customer and the outage has actually lasted a lot longer than the first day and continued for another 2-3 days afterwards. Today has been the first day without an interruption. Yesterday we were seeing it go up and down every 30 minutes. If Virgin Media releases a statement going into more detail about what caused the outage from their side, it would be great if you could do a follow up video which goes into more detail (i.e. why they sent the BGP withdrawals in the first place). Many thanks again!
This is fascinating, definitely my fav video on this channel to date!
This was fun! Thank you for making this video, Hussein!
Sending love to the Engineers who worked hard to bring Virgin Media back online!
Love how you always clearly paraphrase tech blogs! I often watch your channel and when there's a notification of you uploading a new content I'll surely immediately jump in and watch!!
I was wondering lately, how do you get these informative blogs? Are you subscribed on their newsletter of sort? or do you dig them on the internet? Sorry for such a silly question.
Nice video, I have Virgin Media and during the outage some sites worked as normal, CZcams being one of them. It would be great to have a deeper explanation for why this was, your videos are great for gaining a better understanding!
thank you for sharing, my guess is your ip prefix was not among the ones that got withdrawn and as a result ip packets can find their way to your router.
I canât explain why some websites wouldnât work, but I can guess, one theory it could be that virgin couldnât accept BGP updates some ASes where those websites belong to,
the other is CZcams ISP (google) still had the old BGP entry for virgin media and held on to it longer and didnât accept the withdrawal compared to other BGP routers. the other bgp routers lost those entries causing the failure to route the ip packets back to virgin.
best thing to understand what happened is traceroute and tcpdump during the outage
Please make a dedicated video on BGP! Ty :)
nicely explained.
@hnasr I remember facebook running into problems in 2021 and you made video about it.
Curious to know why outage occurred like what happened behind the scene which lead ISP to withdraw connections ?
This looks similar to that Roger comms outage in Canada last year
Omg no wonder I was troubleshooting WiFi during the night.. I thought it was due to the apparent cloudflare maintenance in manchester or something to do with pihole DHCP server... But changing DNS didn't do anything too :)
We donât know the actual cause though. So far all explanations are around the symptoms of whatâs been seen. And explaining this is good for people who donât know how BGP works etc but all the BGP withdrawals are the symptoms. No one from VM has come out and said who fat fingered a change or what config stuff went wrong to actually cause the issue. đ€
Apparently it was an internal ddos on their core routers which caused issues with there fpc cards
You know I think it's time to start to disintermediate ISPs at the layer 3 level. Just use them for layer two. In the commercial space I've set up autonomous systems for high net worth individuals or folks who just want to have complete independence. BGP in 2023 is not a heavy protocol to run. There are even implementations on raspberry pi. Yes you have to peer and you also need an ASN. But that's all able to be arranged, especially in cities with colocation hotels that peer with just about everybody. Hurricane offers later 2 and peering. Independent ISPs are also many times layer two providers. They're easy to work with. And in New York City you're looking at under $800 a month for fully symmetrical 1 gig independent wavelength stand with a 4-hour service contract. It's not as labyrinth or black boxe as most people would think. No it's not going to be $39 a month. But it offers so much security privacy and you control your own reliability. DIY layer 3
best youtuber
business as usual for vm customers, sadly. so glad i'm not one anymore. and it's even worse for their business customers. i rented 5 static ip's from them, and regularly experienced quite bad packet fragmentation, mismanagement of their gre tunnelling, and generally poor qos. and a cursory search just now shows their current business customers still experience these issues.
Although i managed to understand this, some visuals if possible would be easier to go through
"Virgin media got f***ked" sorry I couldn't control myself lol
Virgin Media turned on torrenting