Never bothered to get into #snapper for #Linux, but now I am just amazed how easy the #snapshots for #BTRFS are automatically managed. In this case for #Debian, there is even hooks for #apt that are installed with snapper.
A remaining question is, if the pre and post snapshots are also cleaned automatically?
as my volume was full, and i wanted to keep my several years old snapshots i needed to solve the issue diffrently. Disappontingly i did not find any usable solution so i made it myself:
may it be of great use for other btrfs folks out there..
#linux #admin #filesystem #server #floss #gpl #freeculture #ilovefs
So my server ran full, and it has years of snapshots, i actually do not want to delete, however i spotted (with the amazing ndcu) some 500GB of Backups i do not need. So i want to delete them.
apparently to do this you need to set each snapshot to writable and delete the file/folder you want. (see https://www.suse.com/support/kb/doc/?id=000019594)
they have also a script there doing this in a handle able manner. however it is only for Files and not for Folders. But i need to delete Folders. Ideally i want to be ably to provide the path to it so i do not delete some identically named folders elsewhere.
I tried to adapt the script but i do not quiet understand it.
original script
```
file=$1
while read a
do snapshot=$(echo ${a%%/$file})
btrfs property set $snapshot ro false
rm -f $a
btrfs property set $snapshot ro true
done < <(find /.snapshots/ -name $file)
**question 1:**
what does this line exactly do?
do snapshot=$(echo ${a%%/$file})
```
question 2
in what way/order does this "< <" syle of conding feed the paths to the loop?
because this (my for folders adapted test script)
file=$1
while read a
do snapshot=$(echo ${a%%/$file})
echo "$snapshot snpsho"
# btrfs property set $snapshot ro false
#rm -rf $a ## delete command adapted for folders
echo "foundet $a"
#btrfs property set $snapshot ro true
done < <(find /home/ -type d -path $file) ## find command adapted for folders and paths
this does not give me any output, altough the find command on its own totally does
also why, do we still not have tools that do make sense with btrfs filesystems. like:
- gui option in filemanager to delete a file/folder though all spanshots
- gui option in filemanager to easily compare/resore older versions of a file. (like windows)
this should be quiet easy and would be heavily helpfull...
According to the lead author the calling it Butter-FS originated because it comes from CoW which stands for copy-on-write, but many call it Better-FS, and it actually stands for B-tree file system. It’s been around since 2007, so is relatively new versus other file systems, but the feeling seems to be now that it is finally quite stable. The exception being for new features still being developed. I’m a novice still at BTRFS but these are some of my reasons for wanting to fully move across to it.
Potential downsides (yes it is no magic bullet) are that it can be slightly slower due to compression and validations, and this supposedly is especially not best for large databases. The noatime option can be used to disable the Linux ‘access time’ write to every file read. Also it will use a bit more space as it copies updated data to new sectors instead of overwriting data like ext4 and others do. Although there is RAID functionality, BTRFS is not actually doing backups (it mirrors). You still need to backup off-site and to other media. There is no in-built filesystem encryption (it is planned though), but you can use other standards, but these could potentially affect some advantages of BTRFS eg. using raw block devices.
But the advantages may well outweigh the disadvantages:
1. Copy-on-Write: Any existing data being edited or updated, is left untouched on the drive, which means potentially less loss, and way easier and quicker to reliably roll back to a previous state.
2. Snapshots: Manual or automated, are extremely quick as they do not recopy all previously existing data. It is essentially a snapshot of the metadata of the status of files. Where this is done for the boot drive, such snapshots can be configured to appear automatically in the GRUB boot menu, for quickly reverting back to a previous version. No manual booting from LiveCD, chrooting, Clonezilla restores needed. Suse produced an excellent Snapper app for managing snapshots, but Timeshift also supports them.
3. Software RAID: What stands out is that drives need not be matched sizes at all. You can also add to a running system, and just rebalance BTRFS. BTRFS rebuilds involve only the blocks actively used by the file system, so rebuilds are much quicker than most other systems.
4. Self-healing: Checksums for data and metadata, automatic detection of silent data corruptions. Checksums are verified each time a data block is read from disk.
5. Three different compression options: ZLIB, LZO, and ZSTD differ in terms of speed and amount of compression. You can compress only new files, or process the whole file system, or just do specific individual files if you wish. Compression is supported on a per mount basis. If compression makes the file size any bigger than the original, then the Btrfs filesystem will, by default, not compress that file.
6. Utilities: Scrub for validating checksums, defragmenting while subvolumes are mounted. Check (unmounted drives) is similar to fsck. Balancing for adding new drives to a RAID or other changes made to BTRFS.
7. Send/Receive of subvolume changes: Very efficient way of mirroring to a remote system via various options over a LAN or the Internet.
8. Disk Partitioning: In theory you could use no partitions at all, but it is recommended you create at least one (GRUB prefers it). Rest of the drive though can be BTRFS subvolumes that you can resize on the fly without unmounting or using a LiveCD.
9. Very large VM files: You can add them to separate subvolumes you create, and have them act as independent files without copy (remember you are still backing up aren’t you).
10. Conversion from other filesystems (ext2, ext3, ext4, reiserfs) to btrfs: Copy on write algorithms allow BTRFS to preserve an unmodified copy of the original FS, and allow administrator to undo the conversion, even after making changes in the resulting BTRFS filesystem.
11. Linux kernel includes BTRFS support so no need to install drivers, just the software utility apps to manage it.
So doing my /home partition was quite easy to just do a conversion. I’ve not yet quite decided how to do my boot drive as I need to think about what I want to do with subvolume creation and what best practices I need to consider with inclusion of GRUB etc. A Clonezilla copy of my boot drive means I can experiment and quickly restore without worries, though.
I'm working on a #BASH script that prepares #Fedora (which uses #BtrFS) to be able to use #Snapper properly (for taking #snapshots and being able to roll them back). With a complete separation between the system's side and the user's side.
Keep in mind, it's not finished yet, so please do not use it on main systems.
If anyone would like to contribute, you're more than welcome to do so.
https://gitlab.com/ds.shell-scripts/ds.buttery-snaps
#Btrfs Vs. #Ext4 : Essentials, Benefits, And Disadvantages • Tux Machines ⇨ http://www.tuxmachines.org/node/161597 #GNU #Linux #TuxMachines
" #System76 reveals a new tool to make Pop's desktop faster than the rest, and we break down that recent #Btrfs defrag infinite loop bug." https://www.jupiterbroadcasting.com/147562/linux-action-news-226/ #gnu #linux
#Linux stable and longterm #kernel 5.16.5, 5.15.19, 5.10.96, and 5.4.176 are out.
They contain a variety of fixes and improvements.
5.16.y for example among others fixes issues that can lead to high CPU load with (auto)defrag:
“FOSS means that effort is shared across organizations and lowers maintenance costs significantly” (src: comment by JohnFOSS on itsfoss.com)
“The whole purpose behind ZFS was to provide a next-gen filesystem for UNIX and UNIX-like operating systems.” (src: comment by JohnK3 on itsfoss.com)
“The performance is good, the reliability and protection of data is unparalleled, and the flexibility is great, allowing you to configure pools and their caches as you see fit. The fact that it is independent of RAID hardware is another bonus, because you can rescue pools on any system, if a server goes down. No looking around for a compatible RAID controller or storage device.”
“after what they did to all of SUN’s open source projects after acquiring them. Oracle is best considered an evil corporation, and anti-open source.”
“it is sad – however – that licensing issues often get in the way of the best solutions being used” (src: comment by mattlach on itsfoss.com)
“Zfs is greatly needed in Linux by anyone having to deal with very large amounts of data. This need is growing larger and larger every year that passes.” (src: comment by Tman7 on itsfoss.com)
“I need ZFS, because In the country were I live, we have 2-12 power-fails/week. I had many music files (ext4) corrupted during the last 10 years.” (src: comment by Bert Nijhof on itsfoss.com)
“some functionalities in ZFS does not have parallels in other filesystems. It’s not only about performance but also stability and recovery flexibility that drives most to choose ZFS.” (src: comment by Rubens on itsfoss.com)
“Some BtrFS features outperform ZFS, to the point where I would not consider wasting my time installing ZFS on anything. I love what BtrFS is doing for me, and I won’t downgrade to ext4 or any other fs. So at this point BtrFS is the only fs for me.” (src: comment by Russell W Behne on itsfoss.com)
“Btrfs Storage Technology: The copy-on-write (COW) file system, natively supported by the Linux kernel, implements features such as snapshots, built-in RAID, and self-healing via checksumming for data and metadata. It allows taking subvolume snapshots and supports offline storage migration while keeping snapshots. For users of enterprise storage systems, Btrfs provides file system integrity after unexpected power loss, helps prevent bitrot, and is designed for high-capacity and high-performance storage servers.” (src: storagereview.com)
BTRFS is GPL 3.0 licenced btw.
bachelor projects are written about btrfs vs zfs (2015)
ext4 is good for notebooks & desktops & workstations (that do regular backups on a separate, external, then disconnected medium)
is zfs “better” on/for servers? (this user says: even on single disk systems, zfs is “better” as it prevents bit-rot-file-corruption)
with server-hardware one means:
zfs has awesome features such as:
many more featuers:
(src)
look at WRITTEN, not at USED.
https://papers.freebsd.org/2019/bsdcan/ahrens-how_zfs_snapshots_really_work/
so on a single-drive system, performance wise ext4 is what the user wants.
on multi-drive systems, the opposite might be true, zfs outperforming ext4.
“is not necessary nor recommended to partition the drives before creating the zfs filesystem” (src, src of src)
http://perftuner.blogspot.com/2017/02/zfs-zettabyte-file-system.html
there is no raid10 in zfs, only raid5, which means: at least one disk is used for checksums
more RAM is better than an dedicated SSD/NVMe cache, BUT zfs can do both! which is remarkable.
(the optimum probably being RAM + SSD/NVMe caching)
“our ZFS support with ZSys is still experimental.”
https://ubuntu.com/blog/zfs-focus-on-ubuntu-20-04-lts-whats-new
Linus: “And honestly, there is no way I can merge any of the ZFS efforts until I get an official letter from Oracle that is signed by their main legal counsel or preferably by Larry Ellison himself that says that yes, it’s ok to do so and treat the end result as GPL’d.” (itsfoss.com)
comment by vagrantprodigy: “Another sad example of Linus letting very limited exposure to something (and very out of date, and frankly, incorrect information about it’s licensing) impact the Linux world as a whole. There are no licensing issues, OPENZFS is maintained, and the performance and reliability is better than the alternatives.” (itsfoss.com)
it is Open Source, but not GPL licenced: for Linus, that’s a no go and quiet frankly, yes it is a problem.
“this article missed the fact that CDDL was DESIGNED to be incompatible with the GPL” (comment by S O on itsfoss.com)
“There is always a thing called “in roads”, where it can also be called “bait”.
“The article says a lot in this respect.
“That Microsoft founder Bill Gate comment a long time ago was that “nothing should be for free.”
That too rings out loud, especially in today’s American/European/World of “corporate business practices” where they want what they consider to be their share of things created by others.
Just to be able to take, with not doing any of the real work.
That the basis of the GNU Gnu Pub. License (GPL) 2.0 basically says here it is, free, and the Com. Dev. & Dist.
License (CDDL) 1.0 says use it for free, find our bugs, but we still have options on its use, later on downstream.
..
And nothing really is for free, when it is offered by some businesses, but initial free use is one way to find all the bugs, and then begin charging costs.
And it it has been incorporated into a linux distribution, then the linux distribution could later come to a legal halt, a legal gotcha in a court of law.
In this respect, the article is a good caution to bear in mind, that the differences in licensing can have consequences, later in time.Good article to encourage linux users to also bear in mind, that using any programs that are not GNU Gen. Pub. License (GPL) 2.0 can later on have consequences for use having affect on a lot of people, big time.
That businesses (corportions have long life spans) want to dominate markets with their products, and competition is not wanted.
So, how do you eliminate or hinder the competition?
… Keep Linux free as well as free from legal downstream entanglements.”
(comment by Bruce Lockert on itsfoss.com)
Imagine this: just as with Java, Oracle might decide to change the licence on any day Oracle seems fit to “cash in” on the ZFS users and demand purchasing a licence… #wtf Oracle
Guess one is not alone with that thinking: “Linus has nailed the coffin of ZFS! It adds no value to open source and freedom. It rather restricts it. It is a waste of effort. Another attack at open source. Very clever disguised under an obscure license to trap the ordinary user in a payed environment in the future.” (comment by Tuxedo on itsfoss.com)
GNU Linux Debian warns during installation:
“Licenses of OpenZFS and Linux are incompatible”
“You cannot change the license when forking (only the copyright owners can), and with the same license the legal concerns remain the same. So forking is not a solution.” (comment by MestreLion on itsfoss.com)
“This effort is fast-forwarding delivery of advances like dataset encryption, major performance improvements, and compatibility with Linux ZFS pools.” (src: truenas.com)
tricky.
of course users can say “haha” 
“accidentally deleted millions of files” “no backups” “now snapshots would be great”
or come up with a smart file system, tha can do snapshots.
https://openzfs.github.io/openzfs-docs/Getting%20Started/Debian/index.html
with ext4 it was recommended to put GNU Linux /root and /swap on a dedicated SSD/NVMe (that then regularly backs up to the larger raid10)
but than the user would miss out on the zfs awesome restore snapshot features, which would mean:
https://openzfs.org/wiki/Main_Page
#linux #gnu #gnulinux #opensource #administration #sysops #zfs #openzfs #filesystem #filesystems #ubuntu #btrfs #ext4 #gnu-linux #oracle #licence
Originally posted at: https://dwaves.de/2022/01/20/the-most-controversial-filesytem-in-the-known-universe-zfs-so-ext4-is-faster-on-single-disk-systems-btrfs-with-snapshots-but-without-the-zfs-licensing-problems/
Links 11/1/2022: #Btrfs Improvements and DXVK 1.9.3 • Techrights ⚓ http://techrights.org/2022/01/11/dxvk-1-9-3/ ䷉ #Techrights #GNU #Linux #FreeSW | ♾ Gemini address: gemini://gemini.techrights.org/2022/01/11/dxvk-1-9-3/
Arch-Based #EndeavourOS 21.4 Released With FSTRIM, #Btrfs Zstd, #PipeWire By Default http://www.tuxmachines.org/node/158687#comment-31957 #archlinux
Extent-tree-v2: Global Roots and Block Group Root • Tux Machines ⇨ http://www.tuxmachines.org/node/157828 #GPL #btrfs #Linux #TuxMachines
On-Disk Format Changes Ahead To Improve "Painful" Parts Of #Btrfs Design - Phoronix ⚓ https://www.phoronix.com/scan.php?page=news_item&px=Btrfs-Improving-Painful-Parts ䷉ #Phoronix #Kernel #Linux 🐧
Examining #btrfs , Linux’s perpetually half-finished filesystem • 𝖳𝗎𝗑 𝖬𝖺𝖼𝗁𝗂𝗇𝖾𝗌 ⇨ http://www.tuxmachines.org/node/156058 #GPL #Kernerl #Linux #TuxMachines
Having spent two days exploring the wonderful features of the BTRFS file system (copy-on-write, fast auto snapshots, rollbacks, RAID across different sized drives, etc) and having converted my /home partition to BTRFS, I was exploring possibly doing my / boot partition too, but I see so many people asking how it can be done and is there no distro already doing it. Yes you can set it up yourself, but it is quite a learning curve. Well, Suse does BTRFS by default, but on the Arch side, Manjaro does not.
Now I found Garuda Linux which is essentially as friendly as Manjaro Linux (same installer), with desktop choices for KDE, Xfce, GNOME, LXQt-kwin, Wayfire, Qtile, BSPWM, i3wm and Sway. So that makes it a rolling Arch distro with BTRFS with zstd compression already in place by default, with Timeshift for doing the snapshotting, and it is even configured to have GRUB ready for quick and easy rollbacks if needed.
See Garuda Linux Official website
#technology #linux #opensource #BTRFS #GarudaLinux
Garuda Linux is an appealing Arch Linux based Distro with BTRFS (modern filesystem), Linux-zen kernel, auto snapshots, gaming edition and a lot more bleeding-edge features.
#Btrfs Adds Degenerate RAID Support, Performance Improvements With Linux 5.15 - Phoronix ⚓ https://www.phoronix.com/scan.php?page=news_item&px=Linux-5.15-Btrfs ䷉ #Phoronix #Kernel #Linux 🐧 | more in http://schestowitz.com/2021/08/31/#latest
#Btrfs Set To Land Support For IDMAPPED Mounts In Linux 5.15 • 𝗧𝘂𝘅 𝗠𝗮𝗰𝗵𝗶𝗻𝗲𝘀 ⇨ http://www.tuxmachines.org/node/154430 #GNU #Linux #TuxMachines
