| Main |Site Index |Download |Disclaimer |Privacy |
Windows’s Virtual Address Space: A Short MSDN Story
What do we have in this Module?
-
Introduction
-
Virtual Address Space and Physical Storage
-
Page State
-
Scope of Allocated Memory
-
Memory Protection
-
Copy-on-Write Protection
-
Loading Applications and DLLs
-
Virtual Address Space – 64 bits
Introduction
The virtual addresses used by a process do not represent the actual physical location of an object in memory. Instead, the system maintains a page map for each process, which is an internal data structure used to translate virtual addresses into corresponding physical addresses. Each time a thread references an address, the system translates the virtual address to a physical address. The virtual address space is divided into partitions as follows: The 2 GB partition in low memory (0x00000000 through 0x7FFFFFFF) is available to the process, and the 2 GB partition in high memory (0x80000000 through 0xFFFFFFFF) is reserved for the system. Windows Server 2003 Enterprise Edition, Windows 2000 Advanced Server, and Windows NT 4.0 SP3 Server Enterprise Edition: Optionally, the 3 GB partition in low memory (0x00000000 through 0xBFFFFFFF) is available to the process, and the 1 GB partition in high memory (0xC0000000 through 0xFFFFFFFF) is reserved for the system.
Windows Me/98/95: The following table describes the partitions.
Range | Usage |
0K - ~64K (0xFFFF) | Not writable. This boundary is approximate due to the way the system loads some features of Microsoft® MS-DOS®. This memory is private to the process. |
~64K (0x10000) -4 MB (0x3FFFFF) | Reserved for MS-DOS compatibility. This memory is fully readable and writable by the process. However, this range of memory may have some MS-DOS–related structures or code in it, so processes should not arbitrarily read from or write to it. This memory is private to the process. |
4MB (0x400000) -2GB (0x7FFFFFFF) | Available for code and user data. User data is readable and writable by the process. Code is execute-only. This memory is private to the process. |
2GB (0x80000000) -3GB (0xBFFFFFFF) | Shared area, readable and writable by all processes. A number of system DLLs and other data are loaded into this space. |
3GB (0xC0000000) -4GB (0xFFFFFFFF) | System memory, readable or writable by any process. However, this is where low-level system code resides, so writing to this region may corrupt the system, with potentially catastrophic consequences. |
Table 1. | |
Virtual Address Space and Physical Storage
The virtual address space of each process is much larger than the total physical memory available to all processes. To increase the size of physical storage, the system uses the disk for additional storage. The total amount of storage available to all executing processes is the sum of the physical memory and the free space on disk available to the paging file, a disk file used to increase the amount of physical storage. Physical storage and the virtual address space of each process are organized into pages, units of memory, whose size depends on the host computer. For example, on x86 computers the host page size is 4 kilobytes.
To maximize its flexibility in managing memory, the system can move pages of physical memory to and from a paging file on disk. When a page is moved in physical memory, the system updates the page maps of the affected processes. When the system needs space in physical memory, it moves the least recently used pages of physical memory to the paging file. Manipulation of physical memory by the system is completely transparent to applications, which operate only in their virtual address spaces.
Page State
The pages of a process's virtual address space can be in one of the following states.
State | Description |
Free | The page is neither committed nor reserved. The page is not accessible to the process. It is available to be committed, reserved, or simultaneously reserved and committed. Attempting to read from or write to a free page results in an access violation exception. A process can use the VirtualFree() or VirtualFreeEx() function to release reserved or committed pages of its address space, returning them to the free state. |
Reserved | The page has been reserved for future use. The range of addresses cannot be used by other allocation functions. The page is not accessible and has no physical storage associated with it. It is available to be committed. A process can use the VirtualAlloc() or VirtualAllocEx() function to reserve pages of its address space and later to commit the reserved pages. It can use VirtualFree() or VirtualFreeEx() to decommit committed pages and return them to the reserved state. |
Committed | Physical storage is allocated for the page, and access is controlled by a memory protection option. The system initializes and loads each committed page into physical memory only during the first attempt to read or write to that page. When the process terminates, the system releases the storage for committed pages. A process can useVirtualAlloc() or VirtualAllocEx() to allocate committed pages. These functions can commit pages that are already committed. The GlobalAlloc() and LocalAlloc() functions allocate committed pages with read/write access. |
Table 2. | |
Scope of Allocated Memory
All memory a process allocates by using the memory allocation functions (HeapAlloc(),VirtualAlloc(), GlobalAlloc(), or LocalAlloc()) is accessible only to the process. However, memory allocated by a DLL is allocated in the address space of the process that called the DLL and is not accessible to other processes using the same DLL. To create shared memory, you must use file mapping. Named file mapping provides an easy way to create a block of shared memory. A process can specify a name when it uses the CreateFileMapping() function to create a file-mapping object. Other processes can specify the same name to either the CreateFileMapping() or OpenFileMapping() function to obtain a handle to the mapping object. Each process specifies its handle to the file-mapping object in the MapViewOfFile() function to map a view of the file into its own address space. The views of all processes for a single file-mapping object are mapped into the same sharable pages of physical storage. However, the virtual addresses of the mapped views can vary from one process to another, unless the MapViewOfFileEx() function is used to map the view at a specified address. Although sharable, the pages of physical storage used for a mapped file view are not global; they are not accessible to processes that have not mapped a view of the file. Any pages committed by mapping a view of a file are released when the last process with a view of the mapping object either terminates or unmaps its view by calling the UnmapViewOfFile() function. At this time, the specified file (if any) associated with the mapping object is updated. A specified file can also be forced to update by calling the FlushViewOfFile() function. If multiple processes have write access to shared memory, you must synchronize access to the memory. |
Memory Protection
Memory that belongs to a process is implicitly protected by its private virtual address space. In addition, Windows provides memory protection using the virtual memory hardware. The implementation of this protection varies with the processor. For example, code pages in the address space of a process can be marked read-only and protected from modification by user-mode threads. The following table lists the memory-protection options provided by Windows. You must specify one of the following values when allocating or protecting a page in memory.
Value | Meaning |
PAGE_EXECUTE | Enables execute access to the committed region of pages. An attempt to read or write to the committed region results in an access violation. This flag is not supported by CreateFileMapping(). |
PAGE_EXECUTE_READ | Enables execute and read access to the committed region of pages. An attempt to write to the committed region results in an access violation. This flag is not supported by CreateFileMapping(). |
PAGE_EXECUTE_READWRITE | Enables execute, read, and write access to the committed region of pages. This flag is not supported by CreateFileMapping(). |
PAGE_EXECUTE_WRITECOPY | Enables execute, read, and write access to the committed region of image file code pages. The pages are shared read-on-write and copy-on-write. This flag is not supported by VirtualAlloc(),VirtualAllocEx(), or CreateFileMapping(). |
PAGE_NOACCESS | Disables all access to the committed region of pages. An attempt to read from, write to, or execute the committed region results in an access violation exception, called a general protection (GP) fault. This flag is not supported by CreateFileMapping(). |
PAGE_READONLY | Enables read access to the committed region of pages. An attempt to write to the committed region results in an access violation. If the system differentiates between read-only access and execute access, an attempt to execute code in the committed region results in an access violation. |
PAGE_READWRITE | Enables both read and write access to the committed region of pages. |
PAGE_WRITECOPY | Gives copy-on-write protection to the committed region of pages. This flag is not supported by VirtualAlloc() or VirtualAllocEx(). Windows Me/98/95: This flag is not supported. |
Table 3. | |
The following are modifiers that can be used in addition to the options provided in the previous table, except as noted.
Value | Meaning |
PAGE_GUARD | Pages in the region become guard pages. Any attempt to access a guard page causes the system to raise a STATUS_GUARD_PAGE_VIOLATION exception and turn off the guard page status. Guard pages thus act as a one-time access alarm. When an access attempt leads the system to turn off guard page status, the underlying page protection takes over. If a guard page exception occurs during a system service, the service typically returns a failure status indicator. This value cannot be used with PAGE_NOACCESS. This flag is not supported by CreateFileMapping(). Windows Me/98/95: This flag is not supported. To simulate this behavior, usePAGE_NOACCESS. |
PAGE_NOCACHE | Does not allow caching of the committed regions of pages in the CPU cache. The hardware attributes for the physical memory should be specified as "no cache." This is not recommended for general usage. It is useful for device drivers; for example, mapping a video frame buffer with no caching. This value cannot be used with PAGE_NOACCESS. This flag is not supported by CreateFileMapping(). |
PAGE_WRITECOMBINE | Enables write-combined memory accesses. When enabled, the processor caches memory write requests to optimize performance. Thus, if two requests are made to write to the same memory address, only the more recent write may occur. Note that thePAGE_GUARD and PAGE_NOCACHE flags cannot be specified with PAGE_WRITECOMBINE. If an attempt is made to do so, the SYSTEM_INVALID_PAGE_PROTECTION NT error code is returned by the function. This flag is not supported by CreateFileMapping(). |
Table 4. | |
Copy-on-Write Protection
Copy-on-write protection is an optimization that allows multiple processes to map their virtual address spaces such that they share a physical page until one of the processes modifies the page. This is part of a technique called lazy evaluation, which allows the system to conserve physical memory and time by not performing an operation until absolutely necessary. For example, suppose two processes load pages from the same DLL into their virtual memory spaces. These virtual memory pages are mapped to the same physical memory pages for both processes. As long as neither process writes to these pages, they can map to and share, the same physical pages, as shown in the following diagram.
Figure 1.
If Process 1 writes to one of these pages, the contents of the physical page are copied to another physical page and the virtual memory map is updated for Process 1. Both processes now have their own instance of the page in physical memory. Therefore, it is not possible for one process to write to a shared physical page and for the other process to see the changes.
Figure 2.
Loading Applications and DLLs
When multiple instances of the same Windows-based application are loaded, each instance is run in its own protected virtual address space. However, their instance handles (hInstance) typically have the same value. This value represents the base address of the application in its virtual address space. If each instance can be loaded into its default base address, it can map to and share the same physical pages with the other instances, using copy-on-write protection. The system allows these instances to share the same physical pages until one of them modifies a page. If for some reason one of these instances cannot be loaded in the desired base address, it receives its own physical pages.
DLLs are created with a default base address. Every process that uses a DLL will try to load the DLL within its own address space at the default virtual address for the DLL. If multiple applications can load a DLL at its default virtual address, they can share the same physical pages for the DLL. If for some reason a process cannot load the DLL at the default address, it loads the DLL elsewhere. Copy-on-write protection forces some of the DLL's pages to be copied into different physical pages for this process, because the fixups for jump instructions are written within the DLL's pages, and they will be different for this process. If the code section contains many references to the data section, this can cause the entire code section to be copied to new physical pages.
Virtual Address Space – 64 bits
By default, 64-bit Microsoft® Windows®-based applications have a user-mode address space of 8 terabytes. However, you can specify that the system should allocate all memory for the application below 2 gigabytes. This feature is beneficial for 64-bit applications if the following conditions are true:
- A 2 GB address space is sufficient.
- The code has many pointer truncation warnings.
- Pointers and integers are freely mixed.
- The code has polymorphism using 32-bit data types.
All pointers are still 64-bit pointers, but the system insures that every memory allocation occurs below the 2G, so that if the application truncates a pointer, no significant data is lost. Pointers can be truncated to 32-bit values, and then extended to 64-bit values by either sign extension or zero extension. To specify this memory limitation, use the /LARGEADDRESSAWARE:NO linker option. However, be aware that problems can occur when using this option. If you build a DLL that uses this option and the DLL is called by an application that does not use this option, the DLL could truncate a 64-bit pointer whose upper 32 bits are significant. This can cause application failure without any warning.
---------------End--------------
Further reading and digging:
Microsoft C references, online MSDN.
Microsoft Visual C++, online MSDN.
ReactOS - Windows binary compatible OS - C/C++ source code repository, Doxygen.
- Structure, enum, union and typedef story can be foundC & C++ struct, enum, typedef etc..
- For Multibytes, Unicode characters and Localization please refer toUnicode & Multibyte 1 (Story) andUnicode & Multibyte 2 (Implementation).
Check the best selling C / C++ and Windows books at Amazon.com.