TrueNAS Scale: Setting up Sandboxes with Jailmaker
Vložit
- čas přidán 19. 06. 2024
- How to setup Sandboxes with Jailmaker for Docker, Dockge, Jellyfin and any other docker compose stacks on TrueNAS Scale Dragonfish (24.04)
‼️ IMPORTANT: Jailmaker 2.0.0 has now been released. When I use `./jlmkr.py install` in the video, you should instead setup an alias as per github.com/Jip-Hop/jailmaker#... ‼️
Sandboxes allow installing software alongside the TrueNAS Scale operating system, such as Docker, Dockge and Jellyfin. Sandboxes have negotiated native speed access to the CPU, Memory and Filesystem, which is far more efficient that either a Virtual Machine, or Kubernetes/K3S based application clusters.
Please like 👍 and subscribe 🛎️ if you'd like to see more videos like this
#truenas #jailmaker #tutorial #docker #dockercompose #dockge #jellyfin #dragonfish #truecharts
Chapters:
0:00 Intro
0:57 Overview
2:14 Creating datasets
2:58 Installing Jailmaker
4:36 Creating a Docker jail
10:45 Installing Dockge
12:48 Installing Jellyfin
TrueNAS Sandbox Docs:
www.truenas.com/docs/scale/sc...
Jailmaker Github:
github.com/Jip-Hop/jailmaker
Dockge Homepage:
dockge.kuma.pet
Jellyfin Docker Installation:
jellyfin.org/docs/general/ins...
Jailmaker Bridge Static IP Config:
github.com/Jip-Hop/jailmaker/...
Jailmaker MAC-VLAN Static IP Config:
github.com/Jip-Hop/jailmaker/...
TrueNAS Dragonfish Relese Notes: www.truenas.com/docs/scale/24...
My Guide to Setting up a Static IP and Network Bridge on TrueNAS:
• TrueNAS Scale: Setting...
Jailmaker thread on TrueNAS forums:
forums.truenas.com/t/linux-ja...
This Video's thread on TrueNAS forums:
forums.truenas.com/t/video-se... - Věda a technologie
![Eminem - Houdini [Official Music Video]](http://i.ytimg.com/vi/22tVWwmTie8/mqdefault.jpg)
whoever this dude is on the bottom right corner is the greatest man who ever lived
LOVE docker in jail! For the exact reasons you described , Bind Mount! Low overhead! Simplicity!
This might well be THE BEST tech tutorial I have ever used. Well done, and thank you so much!
Just setting the pool on apps made my consumption jump from 30 to 40watt. Curious to see how sandbox jails go.
Old jails on core where great, but limited. Now we have best of both worlds, performance en app diversity!
Thank you. TrueNAS needs more such tuts for n00bs like me
This is so amazing, helped reduce the load on my machine and improve performance:)
Awesome! I would love to see how you backup these datasets, how you restore them and potential things to watch out for. I like this format. Subscribed
I'm working on a video for that :)
My favorite part is 7:49 into it when he sets up the bridge by snapping his fingers!
Excellent tutorial. But there isn't an "install" argument to jlmkr so something must have changed. But the readme in the repo is correct.
`install` command was removed in jailmaker 2.0, I mention this in the description of the video and provided a link to the alias section in the manual, which you should follow instead
👍
Nice timely concise explanation! Thanks. I'll give it a go soon.
Absolutely fantastic tutorial, very clear and you touch on potential issues that may arise. Thanks for a great video 👍
Many thanks, worked for me.
This is awesome, really appreciate the guide.
Glad it helped!
Thank you.
Hei Stux :D It's moogle from the forum. nice video i'm watching. much appreciated.
I first heard about jailmaker from you on the truenas forum. I don't think there was a video about this till yours. so this will help others. Really.... i came from QTS container station for deploying dockers. Jailmaker is WAY better 😍
Really glad to hear it, I think Sandboxes and Jailmaker is a game change for TrueNAS.
@@CaptStux yeah just looking at power consumption and flexibility to upgrade/downgrade, limit certain functions already makes it better than the alternatives
Hmmmm. As it turns out I wanna set-up a new install, and probably test going to Scale from Core. Could be a good time. I like how you seem to explain the whole process. I'll be sure to circle back then. Look forward to more content.
Hi Stux, thanks for your video. I'm glad that i watched and learned from your tutorial for my first setup/attempt in learning & doing jail-docker stuffs..felt good to see the things i typed in the shell turned out all good instead of error here and there..all thanks to your video!
I have a question though, in term of "best" practice for users perspective, should we be using 1 jail for 1 container/app? or 1 jail with multiple container/apps? Process of setting up the networking for multiple jails is it the same?
Not even sure I'm asking the question with the correct term...Hope you give your advise on this..
subbed. Well done sir 😶🌫
my layout is like docker > data, stacks, old-archive. data is where the docker container persistent data go to, into each of their own folder e..g jellyfin, immich and so on. For stacks this is used by dockge to place docker compose.yaml and env file. The third folder was simply docker containers from my old QNAP QTS which i am still going through what i want to recover and what not to (it's inert, may delete it once i am done).
^^;
Anyway you can demonstrate the gpu passthrough to the jail, nvidia if possible.
Thank you for taking the time to provide this tutorial. I have very recently switched from Ubuntu Server with docker running all apps/services including dockge to TrueNas and wish I had this info when I switched. The way you word it is that this replaces the current app system? So will this mess with currently installed apps or render them useless in any way or can this work side by side?
When setup with macvlan or bridge networking, the sandbox is effectively a separate “machine” to the host.
Ie it’s side by side.
Hey Stux, another question.
Would this also allow gpu transcoding for plex/jellyfin? Seems like your example shows cpu transcodes (unless im misunderstanding) which is great. Just hoping a cheap gpu can help alleviate my cpu during multiple streams if i move plex from truenas app to a jail.
Thanks!
Yes. AMD, Nvidia and Intel all work, possibly depending on TrueNAS drivers etc. Refer Jailmaker GitHub website for more info.
whooooooooooo!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Just a tip for next time you make a video. Scroll down a bit so what you are writing isn't at the bottom of the page as the youtube play bar always gets in the way and I can't actually see what you are typing. Because when you pause the settings bar just stays there and you can't see what you've typed.
Great video, thank you!
does this support passing usb devices to the docker container?
I plan to experiment with zigbee2mqtt that needs usb zigbee stick - any advice / pointers?
You can bind individual devices, not sure if this is what you need, but it might be
@@CaptStux Would I do that in the same section I use to bind mounts in the config? I'm having the hardest time getting my Coral USB device to show up in a docker container. It shows as available if I shell into Docker jail but it doesn't seem to assign to any specific container I try to add it to in my compose.
@@cliph6068 yes, same section. Once bound into the jail, I would expect you'd then have to bind it into the container too.
If you ask on the TrueNAS Jailmaker thread, you'll get much better visibility:
forums.truenas.com/t/linux-jails-sandboxes-containers-with-jailmaker/417
@@CaptStux So I was able to get it to work by not mapping it in my docker compose for the container or docker from host. I saw that it was being passed through in docker without editing the config but it always failed to load in a contain when I specified the mapping in compose. I tried omitting it from compose and finally loaded in the container. Not sure what that's about but I'm happy to finally get it working.
Could you possibly do a tutorial using jailmaker to create traefik or caddy? Reverse proxy is the one thing keeping me from moving all to docker.
Thinking of doing one.
Will probably demonstrate traefik as it uses textual configuration on the compose files. Nginx Proxy Manager would be easier to get immediate results via a gui.
I actually use a virtualized pfsense as my router/firewall and use its dns, dhcp, acme and haproxy features instead.
Been fiddling with setting up servarr apps on TrueNAS Scale and just hate creating apps with kubernetes and not understanding whats going on under the hood (skill issue on my behalf...). All I wanted was something like this. Spin up and monitor docker containers and thats it. Easy to understand.
Would there be an issue mounting transcodes/docker image data on a separate pool? I have my app data and transcodes on an nvme drive but my media stored on HDD.
In terms of backup, I've been using Backblaze setup with the built in TrueNAS Scale system for backing up data. I can't see any reason why I couldn't keep doing that with this setup correct? You're still mounting to datasets so backing up would be fine? I've never had to recover so I'm a bit worried about how that would look but willing to document and test myself. Seems like you already responded to someone about recovery and plan to do a video (awesome!).
This video was extremely helpful, thank you so much for taking the time to make it!
>Would there be an issue mounting transcodes/docker image data on a separate pool?
Should be no issue.
>In terms of backup, I've been using Backblaze setup with the built in TrueNAS Scale system for backing up data. I can't see any reason why I couldn't keep doing that with this setup correct?
Correct.
>You're still mounting to datasets so backing up would be fine? I've never had to recover so I'm a bit worried about how that would look but willing to document and test myself. Seems like you already responded to someone about recovery and plan to do a video (awesome!).
I do. I'm currently delayed due to a bug in 24.04.0, waiting for 24.04.1
>This video was extremely helpful, thank you so much for taking the time to make it!
Thankyou
Does it support amd passthrough if I added amd to the config file?
AMD Passthrough does work, but is not fully integrated yet, you can find instructions on how to enable here:
github.com/Jip-Hop/jailmaker/issues/109
And if you test it, let jiphop et al know in that thread.
@@CaptStux legend, thanks again
So if I udnerstand it correctly this approach compared to VM is that all host resources are dynamically managed based on sandbox needs and I don't need to use NFS for datasets access inside correct? What about nvidia GPU passhtrough and nvidia container toolkit?
Yes, correct, also, there is simply less overhead as an entire machine does not need to be virtualized.
Nvidia, AMD and Intel GPU passthrough is supported, but you should look at the jailmaker github discussions for exact instructions on how to configure.
@@CaptStux Will definitely check that later. Great tutorial btw. For now I just try Dockge. Seems nice for what I need and I'm having hard time with portainer so maybe Dockge is the solution. Thanks again.
Hello Sydney,
can you explain to me what to enter in the nano shell at network.
Been struggling all day to get that one right.
Hopefully you can give me some guidance.
Thanks in advance for the clear explanation.
My truenas network interface settings are:
Greetings Koen from de Netherlands.
Name: bond1
Description: LAGG
DHCP: no
IPv6: no
Merge protocol link: LOADBALANCE
Send hash policy: LAYER2+3
Merge interface mapping: eno1,eno2
MTU: 1500
Hi Koen, Greetings from Sydney :)
I'd suggest asking on the Jailmaker thread at the TrueNAS forums, and I'm sure someone could help you out
forums.truenas.com/t/linux-jails-sandboxes-containers-with-jailmaker/417/47
I have two pools, one flash, one disks. Can I install jail maker on the flash pool and it be able to talk to the disk pool where my media is?
yes. You bind your media using the full path, which includes the pool name, ie '/mnt/pool_name/media'
startup=0 likely means start this process first. He glossed over that in the explanation... he said you want this, but didn't explain why. I'm guessing it's the startup sequence order.
startup=0 means do not start this jail at startup
startup=1 means to start this jail at startup
`jlmkr startup` starts all jails with startup=1
the order is undefined.
When I am done installing Dockge, my port at 5001 says it cannot connect to the server, removing the 5001 gets me back into my truenas. Any clue where to hash this one out?
I appreciate your time and help
If you followed the tutorial, you would've used the docker template, and then your jail should be using either a DHCP assigned or static IP. Either way, if you are accessing your TrueNAS gui it means you are not using the jail's IP.
Inside your jail type "ip a" to check its IP Address. If you have one, then great, otherwise, type 'watch ip a' to wait for one... ctrl-c to exit.
Once you have an IP, try using that IP to connect to dockge (with :5001)
if you still can't access the dockge instance, ensure that you can "ping www.google.com" successfully.
does the sandbox persisit within each scale update ?
It should.
This is a quote from the TrueNAS docs: "These containers persist across upgrades in 24.04 (Dragonfish) and later SCALE major versions"
So this is the advantage over using it as an app?
@@hamurabidios one advantage. there are others like utilization of full hardware as needed if I'm understanding correctly so no fooling with setting limits like with apps/VM's, also for a lot of us long time multi-linux users like myself that just transitioned from running headless debian based server with docker running all applications/services to TrueNas with jails/apps I think that the method shown here could have made that transition a bit easier imho. I almost decided to pay for unraid but decided on TrueNas instead and love it so far. I am just hoping that when I decide to implement this method that it doesn't screw up existing apps installed the usual way....
why was the jellyfin user ID 3001? That shows "mwriter" as 3001
It’s not actually shown in the video. It’s visible on the users list, but it is 3001. The number is picked by TrueNAS, so it depends on how many users have been created.
The reason the users and groups ids are out of sync on this instance of TrueNAS is because I had already created an “mwriter” group for “media writers”, as part of setting up the “media” dataset
Works for the most part for me, however since i use macvlan i cannot access the apps running on the host.
And adding a bridge is hard, because apps binds the entire /24 i am using.
Tried adding another subnet and binding to it instead, using macvlan but then the jail vm has net, but for some reason the containers do not .. :(
I have a video demonstrating how to solve this by setting up a bridge, also, in the description I mention how to solve the apps binding issue
czcams.com/video/uPkoeWUfiHU/video.html
@@CaptStux I did manage to get a bridge going, but now i have to disable and reenable the gpu every restart, because it can't find the bridge on boot!
@@ssunde4698 this may be a bug in 24.04.0, I would suggest trying again when 24.04.1 is released, and that still doesn't work for you, report the bug to TrueNAS.
@@CaptStux NAS-127870 Why do i hit every single bug ;_;
I'm getting "Failed to get shell PTY: There is no system bus in container docker." Not sure what i did wrong :(
This has been resolved in a post on the Jailmaker thread
It’s due to a kernel limit that needs to be increased. See the thread.
forums.truenas.com/t/linux-jails-sandboxes-containers-with-jailmaker/417/293
@@CaptStux thanks, that’s actually me asking on the forums. I figured it might be the proper way. Thanks for taking the time!
@@CaptStux I'd like to know as well. Can't enter Shell, get error on no system bus in container. Thanks!
hey yall
first one
Thanks for the video! I followed steps but once I saved the docker template, I got some errors:
Press Enter to open the text editor.1
Traceback (most recent call last):
File "/mnt/NAS10T/jailmaker/jlmkr.py", line 2170, in
main()
File "/mnt/NAS10T/jailmaker/jlmkr.py", line 2165, in main
sys.exit(func(**args))
^^^^^^^^^^^^
File "/mnt/NAS10T/jailmaker/jlmkr.py", line 1335, in create_jail
jail_name, config, start_now = interactive_config()
^^^^^^^^^^^^^^^^^^^^
File "/mnt/NAS10T/jailmaker/jlmkr.py", line 1084, in interactive_config
config.read_file(f)
File "/usr/lib/python3.11/configparser.py", line 734, in read_file
self._read(f, source)
File "/mnt/NAS10T/jailmaker/jlmkr.py", line 203, in _read
return super()._read(lines, fpname)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/configparser.py", line 1112, in _read
raise DuplicateOptionError(sectname, optname,
configparser.DuplicateOptionError: While reading from '/tmp/tmp_hzqqkim' [line 83]: option 'gpu_passthrough_intel' in section 'a' already exists
Any help would be highly appreciated!
Hi Sean,
Its reporting a duplicate option error with 'gpu_passthrough_intel'. Double check the config.
If you still have trouble, I'd suggest asking on the Jailmaker thread at the TrueNAS forums, and I'm sure someone could help you out
forums.truenas.com/t/linux-jails-sandboxes-containers-with-jailmaker/417