I have been setting up Zram, Swap, Swappiness and EasyOOM daemon on 16gb ram boxes, or lower. Someone asked me about 32gb of ram, or more, and I’m unsure. Wondering if others have experimented with this!
been testing this quite a lot for star citizen which regularly sees 20-40 GB use. zram is the easiest solution to avoiding OOM Killer by a long shot. zswap was swapping over 100 GB to disk cumulative throughout the day and still oom killing the game. zswap is also a pain in the ass to configure for the lay user and harder to get human readable stats. I don’t want lay users going anywhere near their bootloader but zswap is all kernel command line to make permanent changes
i don’t see a lot of swap use on my systems, but i do have zswap backed by a swap partition on a couple of them; otherwise i generally i just use a basic swap partition on most everything.
basic systems (browser, email, light docs/media use) i set up for others that are using ext4 might use a swapfile instead if it was the default for whatever distribution i used for it.
Thanks!
It’s shit info. zram is actually better, more so with high ram size+high usage situations.
Anything you’d like to dispute specifically or we should just take your “it’s shit” over a detailed explanation?
In my testing, zram has much, much better compression than zswap.
The points about LRU inversion, cgroups, and so on are valid, but at the end of the day, I don’t really care. I was able to open as many firefox tabs as I wanted with zram, but I could not do so with zswap, and that’s what matters to me.
The author of a blogpost is a facebook engineer. Millions of ultra high performance Linux servers are a very different usecase than a single desktop. It’s perfectly reasonable for a solution for one to not be appropriate for the other.
Copied from my previous comment about this where ISO also gave a similar reply and was met with a similar response lmao.
This testing compares apples to oranges. Differently sized swap and quite obviously different workloads. Given how very much compress ratios depend on the specific data that is compressed, this experimental setup cannot produce valid results.
This is exacerbated by your swap being full. Zswap is more of a cache in front of your actual swap; it requires physical swap to function. If the physical swap is full, it cannot receive more data! Zswap not doing very much when the swap is full is totally expected behaviour because it simply doesn’t. The solution to that is to size your swap sensibly. (Admittedly, this does not appear to be documented clearly.)
zswap uses the exact same allocator as zram these days (zsmalloc). It’d be very surprising if it had different space efficiency characteristics. It’s not impossible (could be a bug) but claiming so would require quite certain evidence IMHO.
RE: LRU inversion: the problem with not caring about it is that it’s not a visible problem until it very suddenly is. Your system will not gradually degrade but very suddenly and unpredictably hit a wall that it cannot get itself over.
It’s not the opinion itself, it’s just the attitude. Your comment is a perfect example of what I consider a good reply as you brought both hard data and some nuance in expressing how you formed your opinion
Shit info from a kernel dev who works on the memory management subsystem?
Alright, I will only reply to you, since you raised a fair question.
First of all, I must admit that I thought what was linked was an earlier similar writing, but the general theme is still the same.
The problem with the writing is that it focuses on use-cases like Android and some servers, but doesn’t take into account other use-cases. It also seems to come with the assumption that setup is done by the distributor only, or if it’s done by the user, it’s a configure-and-forget situation.
What he represents is:
- Limited RAM space
- Swap will always/often happen (outside of (z)ram)
- Single tier of non-RAM swap
- Non-ram swap is significantly slower
- OOM can be preferable over (outside of ram) swapping
- Swapped out pages stay where they are until they are required by their process (important).
Now let’s look at a possible modern workstation setup:
- Large RAM size
- Swap is rarely hit, especially if set up with zram.
- Multiple swap tiers beyond zram/zswap
- Intel Optane disk used as a super-fast zram write-back device, or a high-priority swap
- Fast NVME disk used as a second tier swap disk
- Large HDD swap partition used as a third tier swap disk
- The biggest consideration is avoiding worst case latency, i.e. hitting HDD swap.
- Killing processes MUST be avoided, unless exceptional circumstances are hit where the kernel’s OOM would kick in anyway. This holds true even when HDD swap starts getting used.
- When unusual loads are observed, swapped pages can be moved around by the user (or a tool), by turning swap devices off and on. This is how you can empty the HDD swap partition for example.
This last point in particular should make it clear why his “imagination” was rather limited in his LRU inversion section.
Swap is rarely hit, especially if set up with zram.
This is not a good thing btw. Any unused anonymous page takes up space that could instead be used for file-backed pages that make your system faster.
Multiple swap tiers beyond zram/zswap
Swap is not tiered storage!
Priorities control order of preference, not tiers. If you run out of space on a higher priority, it will not move that swap’s data to a lower priority swap. It will keep all of it exactly where it is and new data will hit the lower prio swap instead, no matter how hot it is.
Intel Optane
Cool tech but it’s dead and was quite niche even when it was alive.
zram write-back device
Not a thing you actually want to use for swap. It’s not an automatic writeback that is integrated into the Linux MM in any way. (Probably has some use-case for non-swap zram purposes though.)
Large HDD swap partition used as a third tier swap disk
This makes no sense at all unless you are extremely space-constrained on the NVMe and absolutely must not OOM – even if progress stalls to an absolute crawl.
swapped pages can be moved around by the user (or a tool), by turning swap devices off and on.
This is neither feasible nor desirable. You don’t have enough granularity to do anything useful by doing so.
Even if you had, it’d work against the MM because it resurrects pages as “hot” that have been cold for a long time.
In any situation where swap is important, making the kernel think cold pages are hot is the very last thing you want.
I too wish it were but tiered/transcedental memory is not a thing in Linux and these hacks do not change that fact; they merely look similar if you don’t look close enough.
I cannot think of a single use-case where this would be preferable to a decently sized physical swap with zswap XOR just zram swap (if physical swap is infeasible).
Intel optane? is there even any advantage left for optane compared with a fast, modern nvme disk?
It was just an example of a “smaller+faster than your average SSD”.
and I was mentioning something similar to my setup instead of an imaginary use-case.
deleted by creator
zram makes sense if you do not have swap.
zswap is probably enabled by default in most distros. It compresses cold pages on the fly so that they’re ready to quickly get swapped in and out.
I do hit the swap partition occasionally on my 32GB systems.
It doesn’t really kick in until you have proper pressure. I want my swap partition for hibernation, anyway.
zram makes sense if you do not have swap.
wrong.
zswap is probably enabled by default in most distros.
wrong
My workstation has 48 GB RAM with 50% allocation allowed to zram, no disk swapping. It works just fine. Once I use up the majority of my RAM, it kicks in the same way it would on any other system with less RAM.
Shouldn’t total ram amount always be powers of 2 (forgot the reason exactly) ? It sounds like 32 +16 wouldn’t be ideal ? I remember that from my early days, but maybe it has changed ?
That is correct, IIRC, the mismatch does limit how much of it can run in dual-channel. Even if a single stick is natively 24 or 48 GB, there is additional strain on the memory controller. It is the way it is on my setup since I had planned an upgrade to a full 64 GB and was holding off until a good deal on the remaining 32 GB kit, which will never come unless the AI bubble bursts.
There are some 24GB sticks around, same with 48GB sticks
Not according to the chrisdown blog post above.
What about it? I see it kicking in at least 10 GB before my RAM is full and I haven’t noticed any fundamental differences between how zram works on my 48 GB workstation and my 8 GB devices. Maybe I’ve never had a workload that filled all 48 GB + extra zram capacity, but it’s never given me an issue.
Use zswap not zram.
Genuinely curious: what are you doing to be needing this?
I cannot think of any modern usecase for swap a part from hybernation
Local AI can chew it up. Wasn’t able to run certain jobs on 64Gb until I switched to zswap.
What kind of token per second are you getting with your model partially on disk?
I’ve been using lower ram machines lately, so made me curious about if people are using things like zram with 32gb+
Compiling Librewollf with a sufficient number of jobs is a great way to eat up 32GB of RAM, and the some.
Swapping will make it slower not faster, reduce jobs?
No. Most of the build jobs are fast and small. A couple take up gigabytes at a time. Swap is on SSD anyway. Its fast enough.
I also use NixOS and GuixSD. There are cases where binary caches haven’t caught up with the package definition. Situations like Librewolf, LibreOffice and kernel all compiling at the same time.
Sure I could mess around with limiting jobs and build runners… Or I could just have swap, and never worry about it again.
Just a BTRFS swapfile. I’m not worried too much about performance. I rarely hit it and most of the time i don’t even need it. Until I do.
Ok, you do you. In my experience that is not how that works, but there is an argument to not worry about it if it works well enough.
Why do you think 32GiB is special compared to 16GiB?
And wtf is EasyOOM?You maximize the usefulness of
zramby actually increasing sappiness, and givingzramdevices high priority. e.g.sysctl vm.swappiness=100 for i in {1..8}; do swapon /dev/zram${i} -p 32767 doneThen you enable other swap devices with lower priority.
This is the way regardless of how much RAM you have. I mean, it may be pointless if you never ever exceed, let’s say 10/32GiB (including caching). But it still wouldn’t be harmful in any way.
That isn’t how swappiness works.
Changing the sysctl for
swappinessonly adjusts the ratio of anonymous and file pages, it doesn’t set a “threshold” or “aggressivity” in swapping pages, nor does it dictate how much or how little to swap.It’s generally ill-advised to touch swappiness at all unless you know what you’re doing. You can start here.
If you’re going to hand out free advice, at least make sure the advice is worth the price of admission.
If you’re going to hand out free advice, at least make sure the advice is worth the price of admission.
This is very ironic, considering your comment is a mix of straw man and wrong.
“a strawman is when people disagree with me!”
I simply haven’t had the chance to try it yet, so asking
Oh, you wrote “easy” not “early” in OP. In any case, this looks stupid. But to each their own, I guess.
