I did the following:
mkdir /sys/fs/cgroup/memory/test
echo 32212254720 > /sys/fs/cgroup/memory/test/memory.limit_in_bytes
cgexec -g memory:test …
But when the cgroup's memory is almost fully used, then processes start to degrade but no OOM happens:
❯ cat memory.limit_in_bytes
32212254720
❯ cat memory.usage_in_bytes
32197885952
❯ cat memory.oom_control
oom_kill_disable 0
under_oom 0
oom_kill 0
In htop all the processes are shown in uninterruptable state (D sign).
How to properly setup cgroup to let the OOM killer do it's job and kill processes?