5

I want to understand the memory usage metrics shown in htop in more detail.

My current understanding is this:

  • VIRT (virtual memory) - This value shows the quantity of virtual memory used. Since all memory is virtualized, I would assume this is the total memory usage of a process, calculated from the number of allocated virtual pages. These pages might be resident in physical memory, or might be swapped out.
  • RES (resident virtual memory) - This value is the quantity of virtual memory used, but only counting the pages which are physically present in RAM. It is always less than or equal to VIRT.
  • SHR - This is where I get a bit confused. There are two possibilities for a virtual page. It is either present in physical memory, or not (because it has been swapped out/paged out). SHR seems to be unrelated. I am somewhat guessing here - I have read the documentation, but the interpretation isn't totally obvious to me. This is some kind of shared memory metric. I would guess that it is the quantity of VIRT which is shared with other processes.

The reason I find this a bit confusing is that RES and SHR seem to be two different kinds of subtypes of VIRT. RES cannot be greater than VIRT. SHR cannot be greater than VIRT.

The interpretation of RES seems sensible - it tells us something about physical memory usage.

SHR doesn't have such a simple interpretation. It just means "shared". It seems like we don't know or can't know if this memory is physically paged in RAM or swapped out. So it seems like a bit of a weird or difficult to understand metric.

Improve this question
4
  • SHR is separate, because it gives you an idea: if you kill the process, you are not probably freeing the shared part (because it is used also by other processes). Commented yesterday
  • @GiacomoCatenazzi Ah, ok I see that's the use case/context for it. RES = you will get this much physical memory back. VIRT = says nothing about how much memory (physical or swap space) you will get back, SHR = says something about the amount you definitely won't get back from VIRT Commented yesterday
  • 1
    @user3728501 you might get some of SHR back (if the process is the only user of a shared memory mapping, it will be freed when the process exits); but SHR is similar to VIRT, it reflects mappings rather than actual physical use. Commented 11 hours ago
  • I have come to appreciate USS (effectively "nonshared resident") and PSS (effectively "nonshared resident, plus shared divided by number of sharing processes"). However, these apparently rely on something called "smaps", which is expensive to compute and requires root to see stats across users. And while my top has them, I don't know what htop has. (There is a corresponding RSS that is "RES computed from smaps", but given the limitations it seems much less useful than RES.) Commented 9 hours ago

2 Answers 2

Reset to default
7

VIRT only shows the virtual memory allocation; it doesn’t necessarily correspond to pages at all. Processes can allocate memory without ever using it.

RES as you say is resident memory, that is to say, pages present in physical memory allocated for the process.

SHR counts memory that’s shared between processes. This happens for example with shared libraries: their read-only pages are mapped in memory once, and every process relying on them gets the same pages. See How to know shared memory between two processes?

This is documented in more detail in the htop manual.

Improve this answer
0

VIRT (virtual memory) - [...] total memory usage of a process, calculated from the number of allocated virtual pages.

Correct

These pages might be resident in physical memory, or might be swapped out.

... or backed by disk files, or allocated but never touched and thus not backed by anything, or backed by video RAM or other PCIe device memory (e.g. an X server mmapping /dev/mem).

Probably also counts mmap(PROT_NONE) guard regions that things like V8 Javascript JIT will allocate after a large array, so it can catch SIGSEGV instead of JITing extra instructions to do bounds checks on accesses. Especially sequential accesses so the size of the guard region only needs to be 1 page. These regions will fault if touched, but are mappings that exist so other allocations can't pick them, and do show in /proc/<PID>/smaps (I assume).

With lazy allocation, the first read or write of a new anonymous page from the kernel (e.g. stack, BSS, brk / mmap(MAP_ANONYMOUS)) causes a soft page-fault to wire up that virtual address to a physical page, if you don't use MAP_POPULATE or mlock to make that happen before accessing the page. (I don't think madvise(MADV_WILLNEED) will pre-fault a range of anonymous pages; I think it's intended for mappings backed by disk files. Which are also lazy, even if the pages were already hot in the pagecache, but then it's just a soft page-fault to wire them up (create page-table-entries for them), not a hard page-fault to do I/O.)

If the first access is a read, you'll get a copy-on-write mapping of a shared physical page of zeroes. (So you can get TLB misses but L1d cache hits, if you're only reading a calloced region.) On the first write, a new physical page is allocated and zeroed. (Linux zeroes on demand, so CPU caches are hot for this page; some other kernels like BSD do or did zero pages ahead of time while the CPU was otherwise idle, which costs extra bookkeeping and isn't always needed if it instead decides to use the page as part of the pagecache, as a destination for an I/O read or to copy from user-space for a write system-call.)

Improve this answer

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.