How to Use the rsync Command to Transfer Files (Linux Crash Course Series)
Vložit
- čas přidán 5. 07. 2023
- The Linux Crash Course series teaches you a valuable Linux-related skill, in one video at a time. In this video, the rsync command is shown.
Check out the dedicated tutorial on apt here ➜ • Linux Crash Course - T... *🎓 FULL LINUX COURSES FROM LEARN LINUX TV*
• Linux Crash Course ➜ linux.video/cc
• Learn tmux ➜ linux.video/tmux
• Learn vim ➜ linux.video/vim
• Bash Scripting Series ➜ linux.video/bash
• Proxmox VE ➜ linux.video/pve
• Getting Started with Ansible (Udemy) ➜ learnlinux.link/ansible
• LPI Linux Essentials Workshop (Udemy) ➜ learnlinux.link/linux-essentials
🌐 LEARN LINUX TV ON THE WEB
• Main site ➜ www.learnlinux.tv
• Community ➜ community.learnlinux.tv
• Official Github Account ➜ github.com/LearnLinuxTV
• Enterprise Linux Security Podcast ➜ enterpriselinuxsecurity.show
• The Homelab Show Podcast ➜ thehomelab.show
• Jay on Udemy ➜ www.udemy.com/user/jay-lacroix-3
• Jay on Twitter ➜ x.com/JayTheLinuxGuy
• Content Ethics ➜ www.learnlinux.tv/content-ethics
• Request Assistance ➜ www.learnlinux.tv/request-ass...
⚠️ DISCLAIMER
Learn Linux TV provides technical content that will hopefully be helpful to you and teach you something new. However, this content is provided without any warranty (expressed or implied). Learn Linux TV is not responsible for any damages that may arise from any use of this content. Always make sure you have written permission before working with any infrastructure and that you are compliant with all company rules, change control procedures, and local laws.
#Shorts #Linux #Howto - Věda a technologie
i would like to let you know something very useful to your guide. When you using the -a archive option you must be aware that using this include and preserve all those options (-r recursively, -p permissions, -t modifcation time, -l symlinks, -g group, -D devices) so you don't need to use the -r option as in your example when using the -a option. Hope this make the rysnc -a option more clear... thank you
I have a server in LA, and when I am at school working on my coding, I can upload my work, go home download it, work on and upload it and have fresh code available all the time. Also prevents me from having to use the schools horrifically slow vpn. Also while the other kids are struggling to get there homework open in eclipse, I am already rocking my code in Vim on a remote server in LA!
Wondering why are not already using git.
Thanks so much for your linux series. I especially like the way you provide links in the ...more under the video so that we can have a TOC to help us find the desired content. Blessings on you and yours from the banks of the Suwannee River in North Florida.
Man Jay i am forever grateful and thanful for all the knlowledge and tuts you are dropping every week, i learned so much from you when it comes to Linux, i am now looking forward to buy your books :)🙏💯
Easy to understand and very helpful. Thx!
rsync is an excellent tool and worth your while spending some time learning it's options.
I always use:
Rsync -Pavh --dry-run blah blah
Progress
Archive
Verbose
Human readable
Super useful
Loving the content here!! Thx!!!
Thanks for this tutorial, it will become very useful I think ... I am ditching Wordpress and going back to static websites, and have decided I will simply update the files locally and rsync them to the website. Though having said that there are a few people that argue the ssh vs ftp and reliability issues, so I am still working some things out
Really loving your videos. I would like to see you do one on rclone as well.
Awesome video, thanks man!
Very nice explanations. Thank you Jay. Do you use a program in particular for your markdown notes, or just a text editor ?
Excellent explanation. Thanks
Rsync will resume tranfers if the link drops, with scp you'll have to start the transfer over in that case. Also, the OpenSSH folks declared scp deprecated a few years ago. Rsync -avP is how I use it.
Thanks for the tutorial. I am working with linux for a long time now, but still I watch your basic tutorials there is always some little bit that surprise me. Little comment on the tutorial - I think 'command' is bash specific. Better use 'which' in my opinion.
Thanks for another great one
GREAT VIDEO! YOUR THE BEST! FROM DETROIT MI!
Awesome video! I'm going to try and use this to back up my Ubuntu Minecraft server to my WSL
Gentoo's Portage package manager uses rsync to synchronize your package list to the current in the repo. It also uses it when issuing commands with emerge to install packages. Rsync is an intragal part of the initial tool kit required for the installation of that distro. Rsync is really cool.
Another Gentoo enjoyer I see.
In a practical sense RSYNC is sometimes over-used by some sysadmins - I remember it being used for large-group transfers (entire database filesets across 100mbit, we're talking instances of 40gig) and they were creating a huge overhead on the disk packs on both sides in one chunk. The database files were a mixture of some which changed heavily a a lot which didn't but every time you ran RSYNC with the checksum calculation the overhead on the disk was kinda wild. A later admin saw this and changed to a process where he created an MD5 checksum separately and worked out the files which had actually changed that needed to be replicated to the target; he then used RSYNC on only those files. To boot he had a checksum file which he could also transfer to the target to do a validation offline without involving the source system. This was on a shared SAN so disk utilisation spikes had to be considered, this method reduced peak demand quite a bit; the way they were using RSYNC before it was initially creating its internal block checksums on every single file every time (I think they didn't use the change date/time metric).
This guy got me into checksum files as a standard practice and I've done them for everything - I archived data to several thousand DVD-R's and produced a checksum for each file set (burnt with the target files to disc) so when retrieved we could work out if there'd been any degradation or loss of files on the optical media. It's a bit of due diligence which can really save your arse especially if your processes are audited.
great video
Thanks for the video. it is very informative. One question, how do you add a password to the destination address? Many of my systems have a password for access and don't have ssh keys. Thanks.
Hello, can you also compare rsync with unison for file sync? Thank you.
I have studied a lot about rsync recently!
My simple use case is to maintain backups of files to an external attached drive.
I was disappointed to see that in backing up from macOS to exFat, the metadata/resource fork stuff created by Mac (or some other cause) causes MANY unchanged files to be transferred again unnecessarily.
I just saw a Reddit post that said, now, in macOS Ventura, inclusion of the standard “--modify-window=1” flag fails to be useful due to a kernel flaw.
This topic is probably not substantial enough for a separate video, but I look forward to your comment.
Cheers!
¡Gracias!
I used rsync to clone my Arch-install from one SSD to another (faster). It was easy, just looking up the exact parameters on the wiki (I needed an extra one for VM's) and everything went well. All I had to do after that is install the bootloader again and I could boot on it.
Isn't the -a option (archive) the same as doing -rlptgoD ? if so the -r (recursive) option is not needed?
Thanks.
16:50 Hey Jay, and how we can do a real sync? Is there a command for that?
Rclone is so much better than rsync in any way... So many times I've been having problems with rsync's low transfer rates due to unknown reasons. It's an old software, guys. Rsync on the other hand does the same, is breeze to use, and didn't fail me even once!
I imagine people still use rsync due to the habit mostly (like I did).
An example of functional superiority of rclone: it has a buil-in mechanism for hashing ssh passwords of the hosts you want to sync with. Very handy.
Can you share your .md note files? Or is there somewhere you've already published them so we can look at them?
How and where do you set that ssh banner message for every ssh connection to the server?
One limitation with (-a) archive option is that it won't preserve ownership/group unless you're syncing to target as user "root".
or super user (SU/SUDO). -a is a group of flags that has the -o option that means: preserve owner (super-user only)
If you do PXE Diskless-Boot, you have to transfer the entire Root-Directory to a remote system.
What's the best way, to do this? So that no files go lost, because of access rights and other stuff.
The only reliable way I found, is: Create a complete partitionimage "rootpartition.img" , and then mount it on the remote system.
That's the only way that works for me.
I've a question related with linux (not this vid): How to install fingerprint reader: U.ARE.U 4500 fingerprint reader Digital Persona, inc (crossmatch patented) on Linux mint? 😢
Any way that works and is safe?
I learned that I can *ls* the subdirectory using: ls/backup/notes. Never knew that.
Thanks for the video! I gotta ask, where'd you get that shirt! I want one for my daughter and I! :) Much appreciated!
Oh cool- I'm glad you covered rsync. I'm an intermediate user, but you touched on an 'issue' that I just experienced on 16TB of backup data. I was using the --delete flag (and -av) for backing up my NAS drives... at some point the NAS went down, but my backup server kept diligently running its daily rsync. GASP, when I found all backup folders were EMPTY. --delete did what it does, and... poof. Any solution here? My local NAS mounts weren't present and I lost backup data. Is there a solution you might suggest while still using rsync?
Second; I've been looking at borg-backup since that issue... I don't love it. Any other suggestions?
Thanks!!
You have to be careful with the design, it sounds like what you were doing wasn't really a backup at all. Any accidentally deleted file would have had at most one day of protection. There's no way around keeping multiple copies.
For example you could rsync to a different destination directory each time. If bandwidth is limited, first make a copy of the destination, then rsync. Those are the safest ways, consuming a lot of redundant storage. Unfortunately doing anything too clever to save space is a risk. Filesystem-level deduplication or snapshotting are probably the lowest risk, or you could triage the data and reduce the retention accordingly. The risks are a bug in the filesystem or putting valuable data in the wrong retention level. Define some requirements and how you're going to evaluate them, choose your level of risk, keep some offline copies and test thoroughly
@@shanent5793 I don't understand what yer saying.
I got my RSYNC working. I am thinking. the cache file. I dont know if deleting it is good. But when RSYNC is done. Can it leave the cache alone.
waiting for an episode about kill command 😁
Is there a way to transfer files from an open media vault server to a Synology server?
Another rsync problem with multi hosts: I CANNOT use auto completion when I type the full path of destination! Each time I have to ssh to the destination host and pwd the full path, then copy and paste.
Weird, everything I've read about backing up says to use rsync. But Syncthing seems to work so much better.
I use it as a live backup or clone of my work files and do an off site backup periodically. It's mostly to protect against system or drive failures. If something goes wrong, a full up to date copy is always ready.
What do you think?
It's because in reality rsync has old architecture, operates on a signle thread etc. It's a very popular tool and it's been there since... Always? When it comes down to syncing large amount of files or big ones, syncthing is going to be better in a lot of cases. There is a modernized version of rsync called rclone that is much more efficient and ready for nowadays workloads. PS: I use both syncthing and rclone (and rsync in the past).
People tend to not write all over the Internet about their problems with rsync. They rather move to something else instead, like rclone which is very similar. This is how the legend of mighty rsync goes on and on :-)
Is that a rhel or Debian server?
sorry, I didn’t catch that the first 20 times…😂
Great tool, great video.
$ command -v rsync -->> what is command in this??
$ which rsync --->> usually this was used to check if it is installed.
the -a option includes the -r so when you added it, you can remove the -r as no longer necessary. The -avh --delete is so ingrained in my fingers (muscle memory) I have to pause when I want anything else and think about it.
rsyncd.conf should be covered next time you return to this topic
8:54 What do you mean with "and DNS is often a problem" ???
谢谢！Thank you for your tutorial. Now I use Rsync to do replication between two NAS. The only pity is that Rsync has no windows version.
maybe you could use Windows Subsystem for Linux (WSL) but it might be a pain
@@ironfist7789wsl has extremely poor disk performance iirc
@@-aexc- ah ok, that sucks
Wow, a dry run, I can't even imagine what a "wet" would do, but it certainly gonna be a liquidfall?
May you share what Linux distro you’re using and the theme? Maybe not
I'm guessing Debian.
The link for Akamai is missing
I assume SAMBA is installed on both the source and target machines?
smb client always is - at most distros while the server part has to be installed, that is correct.
I hate it whenever I forget to put a slash "/" at the end of the folder name.
I use scp command for the same including the -r flag. Works same way. Don't see any difference Jay
Are your notes published to the public?
Eyy Im fast af boi.
are you sharing that notes folder somewhere? interesting
Concerning the single-directional nature of rsync (without ‘delete’ flag), the *best* wording I saw to describe this is, “Rsync is ADDITIVE”
Yes, seems an important distinction if one is looking for a .photoslibrary backup, for instance.
For backup, yeah. But if you are migrating data from OS version 6 to OS version 8, you may want to preserve some of the new files generated on version 8.
😊
😊
BB❤ VP❤❤b❤❤@@jameswhite1910
I haven't seen the difference between rsync and scp.
Kisuka dil jeet ka tum tumra hak adaya karna joga❤❤
First viewer
😅😅9m
😅9😅😅
Nn
KN
nHistory HiKstory
oion0lpm9
9nnn
899mm
Nn
Nnnnn😅.lmko
Tumhumra pas sa kaya sikhsa
Mila tumko etni jara bat ka liya
Tum gio sisa ka umra gana sunna abhisa etna lachar aga baro dakho kaya hota hai❤❤
Why do you need command lines in linux. That is so old days. Like using dos when you have windows
Because command lines are super leet
If you are allergic to using the terminal there are GUI front-ends. It's linux - so you can choose!
Also... even Windows have a tool like this one! (robocopy)
Thanks thanks for explaining
Also for the purpose of explaining rsync, using it on the terminal it becomes completely independent on what distribution you are running, the commands are always the same, and the output looks the same.
Which emulator do you use?
Hey this is a great video. rsync is a great tool and with argument -a
-a, --archive
This is equivalent to -rlptgoD. It is a quick way of saying you want recursion and want to preserve almost everything. :)