HOWTO: Map Adapter RAM into Process Address Space
ID: Q189327
|
The information in this article applies to:
-
Microsoft Win32 Device Driver Kit (DDK) for Windows NT, version 4.0
SUMMARY
You might run into one of the following common problems while mapping a
large amount of adapter memory:
- Unavailability of system PTEs (NO_MORE_SYSTEM_PTES).
- Not enough contiguous process address space.
This article explains why these problems occur and how to circumvent them.
It also explains two ways to map adapter memory into a process address
space.
MORE INFORMATION
Introduction to Page Table Entries (PTE)
Every process that runs on a Windows NT system has a 4 GB virtual address
ranging from 0x00000000 to 0xFFFFFFFF for its use. Of this, the upper 2 GB
address ranging from 0x8000000 to 0xFFFFFFFF is common to all processes
running in the system, and it is called kernel or system address space. The
lower region ranging from 0x00000000 to 0x7FFFFFFF is called user address
space.
From the process perspective, each element of virtual address conceptually
refers to a byte of physical memory. It is the responsibility of the
Virtual Memory Manager (VMM) in conjunction with processor memory manager
unit (MMU) to translate or map each virtual address into a corresponding
physical address. The VMM performs the mapping by dividing the RAM into
fixed-size page frames, creating page tables to store information about
these page frames, and mapping them. Each PTE represents a page frame and
contains information necessary for the VMM to locate a page.
On an x86-based system that uses a 4 KB page size, the maximum number of
PTEs required to map 2 GB of address space is 524,288 (2 GB/4 KB). On a
typical system, this space is used as follows: a maximum of 50,000 PTEs
(approximately 195 MB address space) are reserved for general use and the
rest is used in mapping system cache, hyperspace, paged pool, nonpaged
pool, crash dump area, and so on. This PTE pool size is automatically
determined at system startup based on the amount of physical memory in the
system. This pool is squeezed in between paged pool and nonpaged pool,
which also grows with the amount of physical memory in the system as well.
The system uses these PTEs to create kernel thread stacks, load device
drivers (and their DLLs), to map system virtual address space for I/O
transfers or callers of MmMapIoSpace/MmMapLockedPages/
MmGetSystemAddressForMdl/MmAllocateNonCachedMemory, and for other
miscellaneous purposes. This system PTE pool can become heavily used and
heavily fragmented. This means that your driver might not be able to get
enough contiguous virtual address space from the system PTE pool at any
given time, even though the total address space still remaining in it might
be large enough. It also means that if your driver uses system PTE pool all
up, other parts of the system will degrade, even resulting in threads not
being created, system stalls, and outright bug checks (because some drivers
call allocate-system-memory with the MustSucceed parameter set).
The bottom line is that you must be extremely careful when using this pool-
only map. You should use the portions of the adapter RAM that you really
need system access to from any process context, and only map the amount you
need. Do not map the whole adapter range if you do not really need to
access it all from system mode.
IMPORTANT: Unmap the memory as soon as you are done with it in the right
process context. Otherwise, the system will run out of PTEs, and it will
bug check. You can increase the default number of PTEs-calculated based on
the total system memory-up to a maximum value by adding a number (equal to
the number of pages to be increased) to the registry at:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager
\Memory Management\SystemPages
You can also monitor currently available "Free System Page Table Entries"
using Performance Monitor.
Another important point to note is that the adapter memory pages are not
used for page replacement. They remain mapped into the process's address
space even if the process is idle or entirely swapped out.
Following are the two most common ways to map adapter memory. The first
method maps the memory directly into the process's user-space, and the
second method maps into process's system-space and optionally into user-
space. Both these methods require the physical memory to be contiguous.
Because of the PTE limitation, you might not be able to map a large amount
of memory in system-space. However, you might be able to map into the user-
space if it's not fragmented or used up.
Section Object Method
There is a sample (MAPMEM) in the Windows NT 4.0 DDK (available through
MSDN Professional membership) that shows how to perform this mapping. Here
is an outline of the technique:
- Get the translated physical address of the adapter memory
(HalTranslateBusAddress).
- Open a section handle (ZwOpenSection) to the physical memory device
\Device\PhysicalMemory.
- Reference the object handle (ObReferenceObjectByHandle) to prevent it
from being deleted.
- Map the memory (ZwMapViewOfSection).
The virtual address you get from the ZwMapViewOfSection is only valid in
the context of the process it is mapped. If you want to access the memory
in your driver's Deferred Procedure Call (DPC) or Interrupt Service Routine
(ISR), which runs in an arbitrary process context, you should also map the
memory in the system address space (use the next method). You should unmap
(ZwUnMapViewOfSection) the memory in the same process context that you
mapped before it exits.
MmMapIoSpace Method
This method shows how to map memory in the process's system and user
address space.
- Get the translated physical address of the adapter memory
(HalTranslateBusAddress).
- Map the memory into nonpaged system address space as follows:
SystemVirtualAddress = MmMapIoSpace(PhysicalAddress, SizeofMemory,
CacheEnable);
- Allocate an Mdl:
Mdl = IoAllocateMdl(SystemVirtualAddress, SizeOfMemory, FALSE, FALSE,
NULL);
- Build the Mdl to describe the memory pages:
MmBuildMdlForNonPagedPool(Mdl);
- Map the memory into the process's user-space using MmMapLockedPages.
Because there is an inconsistency in the return value of this function
between pre-SP4 and post-SP4 releases of Windows NT, the following
statement should be used to make this code compatible with all versions
of Windows NT:
UserVirtualAddress = (PVOID)(((ULONG)PAGE_ALIGN(MmMapLockedPages(Mdl,
Mode))) + MmGetMdlByteOffset(Mdl));
The advantage of this method is that you get SystemVirtualAddress, which
can be used in any process context (such as DPCs and ISR), and an
UserVirtualAddress that can be used by the user-mode application in whose
context it is mapped. If you map into a process user-space, you should
unlock and unmap the memory as shown below:
- MmUnlockPages(Mdl);
- MmUnmapIoSpace(UserVirtualAddress, SizeofMemory); This statement should
be called only while running in the context of the process in which you
mapped.
REFERENCES
Windows NT 4.0 DDK Documentation
"Inside Windows NT" by Helen Custer
Additional query words:
MmMapIoSpace MmMapLockedPages ZwMapViewOfSection NO_MORE_SYSTEM_PTES IoFreeMdl(Mdl);
Keywords : kbKMode kbNTOS400
Version :
Platform :
Issue type : kbhowto
Last Reviewed: March 5, 1999