The Docker HOST and MACVLAN Networks - Docker Networks part 2
Vložit
- čas přidán 7. 06. 2024
- MANY THANKS TO ALL MY PATRONS on / onemarcfifty !!!
Part 2 of the Docker Networking series. The docker bridge network is fine for most tasks with docker containers. In this episode we will use Portainer again to define docker containers using the docker host network and the docker MacVLAN Network.
Breakdown of the episode:
01:26 the docker host network
03:30 Performance considerations: Host vs. bridge network
08:10 the Docker MacVlan Network
Portainer Documentation is here: documentation.portainer.io/v2...
sudo apt install docker.io
(on a Pi it may be called docker-ce)
One liner to start portainer:
docker run -d -p 8000:8000 -p 9000:9000 --name=portainer --restart=always -v /var/run/docker.sock:/var/run/docker.sock portainer/portainer-ce
(In portainer, click on "Networks" on the left, you'll see the networks)
(on the command line)
docker network ls
Commands that I use in the video:
(Pulling the image and running it)
docker run -d nginxdemos/hello
with the host network:
docker run --network host -d nginxdemos/hello
creating the macvlan network in a one-liner:
docker network create -d macvlan -o parent=eth0 myMacVlan
In order to execute shell (/bin/bash) in the container on the command line
docker exec -it CONTAINERNAME /bin/bash
Commands I type INSIDE the containers:
ip addr
ping www.google.com
apt update && apt install iperf3
running iperf3 in server mode
iperf3 -s
running iperf3 in client mode
iperf3 -c (serverIP) -p (portNumber)
apt update && apt install iproute2 dhcpcd5 iputils-ping
Please support me on patreon: / onemarcfifty
My youtube channel: / onemarcfifty
Marc on Twitter: / onemarcfifty
Marc on Facebook: / onemarcfifty
Marc on Reddit: / onemarcfifty
Chat with me on Discord: / discord
Licence-free music on / Lizenzfreie Musik von www.terrasound.de/lizenzfreie... music on / Lizenzfreie Musik von www.terrasound.de/lizenzfreie...
The docker logo from Wikimedia Commons By dotCloud, Inc. - File:Docker (container engine) logo.png, Apache License 2.0, commons.wikimedia.org/w/index...
The Thumbnail of this video is under the Apache 2.0 License: www.apache.org/licenses/LICEN... - Věda a technologie
![Eminem - Houdini [Official Music Video]](http://i.ytimg.com/vi/22tVWwmTie8/mqdefault.jpg)
Thank you for making these! I used your macvlan suggestion in another video’s comment section to host two instances of forked-daapd on one server but this is above and beyond. Very informative and I’m looking forward to what you’re going to do next.
Thanks a million for your nice feed-back! Stay tuned ;-)
Fantastic, it will be great to know more about the docker-compose features as it seems like a way to automate the deployment of the docker images. I also agree with creating a tutorial using two different physical hosts, docker hosts, and take it to the next level by simulating a larger project of containers. Thank you very much Marc! all the very best and until the next one.
Hi again, and thanks for your feedback ! Yes, docker compose is a great way to bring up the whole stack (Containers, network, Volumes...) at once - Especially great if you get a ready-made compose yaml file for the purpose. Kind of a docker version of turnkey really ;-)
This 2 part tutorial are the first of your videos I watch. I have been self-training in docker for the last few weeks and these are the best introduction to the topic of docker networking I have found by far. Love the hands on approach with exercises that really help get the notions clarified. Bravo! subscribed now.
Hi Cédric, thank you very much.
Really excellent videos - I finally think I'm getting to understand docker networking! I find your explanations and demos very clear (even if I have to watch them a few times). There were a couple of comments about you going too fast, I have to say I didn't find that: I stop try what you've suggested and replay - that works for me.
Many thanks for this feedback David - searching the right speed and depth is an eternal journey for me ;-)
Of course yes! We are all interested into your next docker networking video!!
Thanks for the feed back ! I'll keep working on these then ;-)
I've been fighting with this for 2 full days. Thank you, Thank you. Now I have the application that "HAS" to be on the same network as the host, there on a macvlan, and the other containers on a local bridge network they share with that app.
Love your videos! Keep up the excellent work!
Thank you very much!
Another great video! Thank you for the useful content and keep up the good work. I would love to see more docker networking as well as security. Would be great to see how to segregate services and and access control.
Many thanks Daniel. I will do more on those subjects, but rather every now and then because obviously I am a bit late to the Docker party on youtube ;-)
Nice. Thank you & team for the efforts
Many thanks Mehedi, I'll pass it on to the team (Me, myself and I) ;-)
thanks for the video! very useful and easy to follow for someone just starting out
Thank you very much - I am glad that you liked it!
I would like to thank you for this very useful video and wish you good luck in your future work.
Hi Omar, thank you very much.
You have discussed a very useful capability of a docker host/container in MACVLAN... I have not found this searching many tutorials in which I am sure there should be... Imagine a docker firewall inside a host, with all packets passing thru and processed by the docker firewall; even the host should use the guest docker firewall as its gateway to the outside world.
The security possibilities will be endless in this arrangement, as all services can be made to only have a bridge internal IP, all using the docker firewall. ALL as in all packets inisde or outside should be made to pass only thru the firewall. Since this firewall is a docker instance, everything will be very flexible, easy and secure.
That is why I have searched for this docker capability, MACVLAN will make this possible.
Thank you for your good and clear overview. God bless.
Hi Mario, that is actually an awesome idea - I might pick it up ;-)
Great job. great content that you can't find maybe anywhere else; because he's mixing something enthusiastic with something scientific.
Thank you very much ;-)
keep it going my dude, networking is one of the most difficult things for me to grasp when it comes to docker/portainer
Thank you very much ;-)
I love your tutorials, thanks a lot from Enschede NL !!!
dank jij wel en groetjes !!
I would like to see comparison of macvlan vs. ipvlan networks in docker, and some practical examples of each. Thanks again for your terrific videos!
Many thanks for the feedback - much appreciated ;-)
Usualy I don't hit the subscribe button very fast, but in this case 2 out of 2 videos where just amazingliy good. Thanks for this easy understandable video with with comprehensible examples!
Awesome, many thanks ;-)
Thank You. Your way of explanation make me docker networking more interesting.When could we expect more videos about docker swarm with two ore more hosts and overlay and ingress networks?
Great content that we cannot find anywhere else.
Hi Vlado, many thanks for your feedback. Glad you liked the video. At the moment, time is a bit of a constraint to me but I do have Docker on my list for future videos. Just can't give a precise date for the time being ;-)
Interesting video. Thanks for sharing this.
You are more than welcome ;) Thanks for watching !
thanks for the video! very useful and super well thought out didactically. I already have material to play for the weekend. thanks again! (sorry for my bad English ;))
Awesome, many thanks!
Great video, thank you!
Many thanks Aleksandr.
Thank you very much , a lot of info , please more videos on Firewall for docker networking and docker swarm
Oh - Firewall for Docker - great idea! Thanks a lot for this. I’ll think it through. I am not sure if I will do anything on swarm though as everyone seems to go for K8 these days…
I’m loving this series. Thank you for putting it together. I’m very interested in learning about the macvlan, this was very helpful. The one thing that still unclear to me though is how do I take that macvlan attach it to vlan that I have configured in my router. For example, I have a VLAN that I use for IOT devices and now I want to associate a docker container with that IOT VLAN
Hello Marc. Nice two videos for newbies on docker. Keep it up. When possible do OpenWRT on docker with adblock and setup as a gateway pc.
Many thanks for the feedback - I'll give it a thought ;-)
You have saved me, I have tried all the google tutorials online. But terminal did not work to enable access from other non docker machines on my network to my macvlan container. Thank you so much 🙏
Hey cool - glad it helped ;-)
Very useful, please do how to make firewall containers
Hi Ismail, its noted. Many thanks for the feedback!
great 👍 thank you 😊
Thank you very much - glad you liked the video ;-)
This video material should be on the official docker documentation website
Hi Aleksej, thank you very much - very kind of you ;-)
Hi Marc,
Thank you for your wonderful tutorials, you have a great way of explaning things clearly!
I have also been following the OpenWRT VLAN tutorials and, as I am running Docker on OpenWRT (on a NanoPi R5S), I wondered whether you would cosider doing a tutorial about that?
Specifically how the networking in Docker relates to the VLANs in OpenWRT. At the moment, my Docker containers are accessible from all VLANs and I don't fully understand why and how to make Docker 'VLAN aware'?
Anyway, thanks again for making these and sharing with us
👍
Hi Stephen, many thanks for the friendly feedback! I might actually consider doing something along that line.
awesome !!
Thank you - glad you like the videos!
will you make videos about differences between ipvlan and macvlan in docker networking?
Thanks for the great video! Considering all of the lately topics, have you considered a video on how to configure two routers for high availability? I think it would be useful when having your main router as а VM in Proxmox configured with a backup router when updating the Proxmox host or during debugging.
Awesome idea! I need to give this a bit of thought - not sure how many people would use a VM as their main router though.
@@OneMarcFifty, thanks for your time. Even if they don't I think a backup router is a very good idea. I've been trying to do so with keepalived but couldn't succeed.
WOW!! Looked for promox and looked for hours!! Very good job, thank you!!
Many questions but have to get new hardware first to install proxmox.
First to ask is: can you convert a running metal machine and convert it into a virtual machine? Very curious in this item!
Yes, I still use VMWare where 9 servers are running for long time. Also use Parallels on MAC and Virtual box for testing. I was Novell guy since 3.11 (1988) and grew along till 2008. Then changed job and country and got never really back into networks. Too bad, miss it! Time to retire soon and get back into networks and pick up some programming!
Hello Henk, Yes - I have made a video on that matter : czcams.com/video/X3q5nLNBHig/video.html
Awesome Tutorials, learned a lot in Part 1 and had my head explode in Part 2 :'D. Could you possibly create a tutorial, how to create and configure the docker networks correctly, so you could expose services run in docker to the outside world (ideally with a domain) - so basicly including portforwarding and a reverse proxy?
I am mainly asking because I have a slight Network related issue or I believe it is. I have 2 physical devices behind my router, that I both want to expose to the internet. One is a Synology-NAS and the other being a NUC-Server with Ubuntu, Docker, Webmin installed (from your other Tutorial). What I'm currently trying is to use nginx or traefik to forward the incoming requests for services run on the NUC-Server correctly. However I might have understood something wrong how to configure the ports in my router or in the docker networks. Right now I have port 80/443 opened which point to my NUC, where I also run nginx reverse proxy and I'm trying to point to various services and the NAS as well. However I really never got everything to work and maybe a tutorial would help.
Appreciate your tutorials, because they are well structured and very thought through! Thanks
Many many thanks for your kind feedback ! The network security aspects of port forwarding, reverse proxa and client certificates etc. are on my list for so long - I will definitely do something in this space. However, time is a big constraint for me at the moment ;-(
Did you end up making a video showing how to import a disk image in a docker container?
Hi - actually no. At the moment I am more intrigued on how to move a rootfs between Docker, WSL2 and Proxmox ;-) But I might have another look into containerization of VM/ISO etc. at another point in time ;-)
Great video! Have any command to notify the router about the macvlan created with specific ip?
You mean something like "reverse DHCP", i.e. the router getting a notification that the IP address is taken ? No, none that I am aware of...
@@OneMarcFifty there a few bizarre options, but works like a relay, I realize a better design for my network. Curious fact, I pissed off with you after buyed a mu 4a but after i discover that my wifi connections (drop) was because the wrong route to my dns. When configured correctly things works great! Hahaha
Hi Marc. Pretty late to the party but ran into a couple of problems with some of the steps in the video whose content I would say is excellent! I am running Docker in VirtualBox on Windows 10 with the adapter in bridged mode. I never was able to resolve the address of client on the Docker container in bridged mode when doing iperf3 -c . I also never was able to get Docker container to be able to ping an address on the macvlan even after successfully getting a DHCP address from my DHCP server even though the VirtualBox adapter was bridged and set to promiscuous mode. Any thoughts or pointers to where I might be able to resolve these two problems?
Hi Matt, even though I am not using the described scenario myself, here are a couple of ideas to help you troubleshoot: First, check outgoing traffic from the docker container to the outside world. (Ping, traceroute and the like) to verify that network connection is OK. Next try incoming connections from the docker host and then from the outside world. If those fail then presumably it could be the Windows Firewall blocking. From what you describe it seems like network connectivity is OK - did you get an IP address from your LAN ? Just trying to exclude that it's actually the docker host serving DHCP.
I would have liked to see how to connect to the containers from the host using macvlan, but I can understand that it was getting off topic.
Hi Franco - in essence as a host on a MacVlan would behave like any other host in the network, you should be able to connect to it from any workstation - actually including the host itself.
Nice
Thank you ;-)
Hey, would like to know what OS you're using for this demo?
Hi, I think I used Ubuntu 18 at the time - it turned out that the behavior of dhclient has changed since then. Your container would presumably need more privileges
I just want to be able to have static ips in my lan by mapping mac to ip once router dhcp gives ip... but of course portainer makes this wonky for me...thanks for great video i will need to figure out a subnet solution.... UGH, why is this so hard i thought having docker container as static IP on LAN would be easy since many used that... but nope, so close, but no cigar...
Yeah - it's a headache ;-)
Marc, I really like your videos. But! I have a problem when it comes to get an IP with DHCPCD every time and I went through that section over and over again but with the same results:
dhcpcd eth0
eth0: if_init: Read-only file system
eth0: interface not found or invalid
dhcpcd exited
I can't understand what I'm missing here. Please advice if you can.
That had worked with Ubuntu 18 but stopped working with later versions. You might need to give the container more privileges than the video describes
Macvlan is supported on cloud instances ex: aws?
Not sure - you would have to have a physical network available I'd say.
I would love to know the reason portainer requires these 'config' macvlan networks. Does it provide any benefit or is it just to work around some issue integrating with Docker?
Btw, when setting up the macvlan network you said that you configured an ip range that has nothing to do with your LAN but it was within the same subnet so I did I miss something? Honestly I don't understand what the purpose of the IP Range is since it doesn't seem to be possible to define an arbitrary range and I couldn't subsequently create the actual macvlan network unless the gateway address was within the 'IP range', even though the gateway address was within the subnet.
Oh, I get the comment regarding addresses outside of your LAN now,, I was confused because I thought you were referring specifically to the chosen IP range but in fact I guess the IP range could have been left empty/172.21.0.0/24
Hi Robert, yes - I could have left it empty - but I just wanted to make sure (and also wanted to show) that the DHCP request goes to a different range.
Hi, ich habe gesehen, dass du da openwrt als Container hast. Kannst du dazu ein Video machen? Ich bekomme das leider einfach nicht zum laufen...
Hallo MingKwan, schau Dir mal das Projekt von Jordan Sokolic (oofnikj) an: github.com/oofnikj/docker-openwrt
A testbed of OpenWrt on docker would be great...
It‘s noted;-)
Hi,thanks for thé video.
When I do the dhcpcd interface name
It says interface not found or invalid dhcpcd exited
My interface is enp0s31f6 and it is the same interface on my host machine with ubuntu 18.04.
What could be the problem please
I will definitely have to take another look at this dhcp issue. I believe Ubuntu have switched to NetworkManager.
"Ózóne"... Marc... let's go to the import of a vhd(x) to inside a container... ;-)
I am actually thinking of running kvm inside a Docker container and run a Windows on it ;-)
that's my Marc...👌waiting anxiously!!!
what if i want to use a socks5 proxy to route certain machines from a lan on one subnet through a different gateway on another subnet that only my router-server is connected to via its two nics? how would i go about doing that with docker?
never mind i managed to do it with macvlan adapter combined with bridge adapter
It depends if your router-server is the docker host or not. If it is not then you would presumably need to use MacVLAN. You could then however create a different route into the second LAN from inside the container using ip route.
I liked the video, very informative. But in the video you mentioned about setting a defined mac address for the container, and you don't show how to do that... Such a missed opportunity. You should at least post the commands in the video description.
Hi Dennis, many thanks for pointing this out - I actually found this blog entry which nicely describes how to get this done: marcelwiget.blog/2018/10/01/macvlan-docker-interface-with-explicit-endpoint-mac-address/
macvlan did not work for me.
Hi - it seems that things have changed in Ubuntu:latest with regards to dhcpcd - it's running in the background by default now. Maybe try with another Ubuntu container version (18 or 20). I haven't figured out yet what is needed to make it work on 21.
Really like the way you described it. But frankly while demonstrating, your pace is too fast. Please keep your pace slow, and explain it a little further in-depth.
Great feed-back, many thanks!!! How much slower do you think it should be - could you maybe give me a time marker, how long it does take and how long it should take? That would be awesome 🙄
@OneMarcFifty I just wanted to say that slower your video speed a little bit while you showing or demonstrating things so that eyes, mind can correlate with what you are saying. Overall I like your pauses and speed, just in this one you go little faster than usual, that's what I felt. :)
Oh OK I see - I will have to watch it again myself in a calm hour. Many many thanks for letting me know again!!!
Hi, after im leaving the bridge and joinin the macvlan network.
im gettin a ip from dhcpcd. i still cant ping google.
i get "From 169.254.117.17 icmp_seq=1 Destination Host Unreachable"
please help :)