DefederateLemmyMl

  • Gen𝕏
  • Engineer ⚙
  • Techie 💻
  • Self hoster 🖧
  • Linux user 🐧
  • Ukraine supporter 🇺🇦
  • Pro science 💉
  • Dutch speaker
  • 0 Posts
  • 4 Comments
Joined 2 years ago
cake
Cake day: August 8th, 2023

help-circle
  • You are misunderstanding.

    The file cache is never written out to the swapfile, because files are already on disk, like you say. The file cache is kept in memory and the kernel may decide it’s more advantageous to swap out unused anonymous memory pages to disk than flushing a file from the cache. You can use the vm.swappiness parameter to finetune this behavior to your liking btw. Lower values favor keeping more anonymous memory pages in memory, higher values favor file backed pages.

    To give an extreme example of where this is useful: I have a use case where I process a number of large video files (each 2GiB-10Gib in size). The job I’m doing involves doing several passes over the same file sequentially. You can bet your ass that caching them in memory speeds things up dramatically: the first pass, where it has to read the file on disk is at 200x speed (relative to the video’s duration), the second pass at 15000x speed.

    Even in less extreme circumstances it also helps by keeping frequently accessed files in your home directory in memory, for example your browser profile. Your browser and desktop environment would be much more sluggish if it had to reach out to disk every time for every file it touched.

    And you are free to disagree, but it’s not my opinion but the opinion of kernel developers of just about every operating system built the past 4 decades. So I’d say: take up the argument with them and see how far you get.


  • I have 20GB in my current setup and it was never full. If anything gets swapped in this situation it means it needlessly slows me down.

    Not necessarily. Your memory also contains file backed pages (i.e. “file system cache”). These pages are typically not counted when determining “memory usage”, because they can always be discarded.

    It is often advantageous to keep frequently use files in cache in favor of unfrequently used memory pages.


  • I’ve never understood why GNU/Linux actually needs swap

    It doesn’t. It’s just good to have in most circumstances.

    Also, sidenote: “GNU” doesn’t apply here. Swapping is purely kernel business, no GNU involvement here.

    Okay, I created a 4G partition for it, having 32G of RAM. I never used all that RAM, but even so, stuff regularly ends up in swap. Why does the OS waste write cycles on my SSD if it doesn’t have to?

    Physical memory does not just contain program data, it also contains the filesystem cache, which is also important for performance and responsiveness. The idea is that some of the least recently used memory pages are sometimes evicted to swap in favor of more file caching.

    You can tweak this behavior by setting the vm.swappiness kernel parameter with sysctl. Basically higher values mean higher preference for keeping file backed pages in memory, lower values mean higher preference for keeping regular memory pages in memory.

    By default vm.swappiness = 60. If you have an abundance of memory, like a desktop system with 32G, it can be advantageous to lower the value of this parameter. If you set it to something low like 10 or 1, you will rarely see any of this paradoxical swap usage, but the system will still swap if absolutely necessary. I remember reading somewhere that it’s not a good idea to set it to 0, but I don’t remember the reason for that.

    Alternatively, there is no rule that says you can’t disable swap entirely. I’ve run a 32G desktop system without any swap for years. The downside is that if your 32G does run out, there will be no warning signs and the OOM killer will unceremoniously kill whatever is using the most memory.

    tl;dr just do this:

    sysctl vm.swappiness=10
    echo "vm.swappiness=10" > /etc/sysctl.d/99-swappiness.conf
    

  • 0 swap: which was pretty awful with constant unexpected system freezes/crashes

    I’ve run Arch without swap for many years without issues. The key of course is that you need enough RAM for what you are trying to do with your computer.

    There’s no reason why a 32GB RAM + 0GB swap system should have more problems than a 16GB RAM + 16GB swap system with the same workload. If anything, the former is going to run much better.

    swap file: finicky but doable

    What is finicky about a swap file?

    It’s just this:

    mkswap -U clear --size 4G --file /swapfile
    swapon /swapfile
    

    Done

    If anything it’s way easier to create a file in your filesystem than having to (re-)partition your drive to have a swap partition. Much more flexible too if you want to change your swap configuration in the future.