Okay, because I still see this question over and over, I have decided to post my FAQ regarding memory stuff and Windows.
The following is a FAQ for some common questions regarding the memory options and support by various Windows SKU’s and x86 CPU architecture variants.
Q: I have 4GB of memory in my new Super Duper computer, why do I only see 2.75GB?
A: 32bit Client OS’s (XP SP2, Vista) no matter what memory remap options are available in hardware, will always be 4GB minus memory reservations. Memory reservations are portions of the physical address map that are masked out so that memory mapped IO (memio) devices (such as video hardware) can be accessed by the CPU for IO operations using those memory addresses. This means, that if you have 4GB of memory, with a hardware platform that supports 4GB of RAM, and you have 1GB of memory mapped IO device reservations, then you will only see 3GB.
Q: But I have a server OS system that when I enabled PAE on it, showed all 4GB of RAM, why don’t 32bit clients work this way also?
A: Server operating systems (all SKU’s) support remap with PAE on hardware with remap support so all 4GB or more, depending on SKU, will be visible. Memory remap support is a function of the computers bios in conjunction with the CPU’s PAE 36bit physically extended address space functionality, wherein the physical memory, located at regions of the physical memory map that are masked out by memio reservations, is remapped into address space above 32bit boundary. Client 32bit SKU’s, due to problems with drivers and, in some cases the device firmware of consumer grade hardware, do not support memory remapping (even though PAE is enabled for NX support). 32bit client OS’s (XP SP2, Vista) no matter what memory remap options are available in hardware, will always be 4GB minus memory reservations.
Q: I have 4GB of RAM in my new Lenovo T60P and I am running Vista x64, why do I still see only 3GB or RAM!?
A: 64bit client and server OS’s all support remap on hardware with remap support and, as such all physical memory will be visible up to the limits of the SKU and hardware. Unfortunately, while nearly all desktop and server x64 systems built in the last 3 years support PAE/x64 remap, prior to the new Santa Rosa chipset (which shipping in June-July 2007), no Intel chipset based notebooks, even if equipped with x64 capable Core2 based CPU’s, support memory remap. As a result most Intel based notebooks purchased in the last year will never see all 4GB of installed RAM.
Q: I have 8GB of RAM, why should I run a 64bit OS when I could just run 32bit Server SKU with PAE?
A: Even if you have a 32bit server OS with PAE enabled, the actual max virtual address space is still 4GB (2^32). 2GB (or 1 GB with /3GB switch) for shared kernel virtual address space and 2GB (or 3GB) for user mode virtual address space for each individual process. PAE simply allows multiple processes to have larger working sets (combined working sets ≥ 4GB) by extending the physical address space of the machine (to 36bits) and then enabling an additional (3rd) layer of page translation to allow mapping of pages into said memory. In cases where a single process needs to access more memory than is possible in the 32bit address space, AWE can be used. AWE maps views of the extended 36bit physical address space into a single processes 32bit virtual address space and switches between those views to allow the application to “see” the memory in a direct (but non-linear) fashion (think EMS bank switching, as it’s somewhat similar in concept) .
Regardless, while these tricks may allow applications to use more physical memory, they don’t remove the fundamental limitations of 32bit virtual addressing – the linear virtual address space is still capped at 4GB of which no more than 2GB can ever be allocated to the OS’s kernel mode components which imposes severe limitations on resources available for allocating kernel data structures. The 64bit virtual address space, in contrast, is 16 Exabyte’s in size, of with 16TB is currently mapped out (8TB user starting at the bottom of the 16EB address space, 8TB kernel starting at the top of the address space). As a result of the huge address space, kernel limits are practically non-existent and applications can directly address as much memory as they need, up to the maximums of the hardware.
Q: But I am running 32bit apps, what good does 64bit Vista do?
A: As stated above, all the extra address space allows the OS itself to make use of the physical memory that is present and also removes some of the (now) severe limits that a 32bit address space imposes on the OS. This will allow you to run more processes concurrently (and in the case of a multi-user TS environment) far more sessions. Additionally, 32bit applications compiled with the large address space aware option can use the entire 32bit address space as user mode, so 32bit apps are not limited to 2 or 3GB, but can in fact allocate all 4GB of address space (and if you have enough physical memory) nearly all 4GB can be resident in memory (working set).
Q: I moved to 64bit on my new 4GB machine, but now my most memory hungry applications are crashing left and right with memory allocation errors, what gives?
A: This is likely because the app is attempting larger than 2GB process address space allocations without having the IMAGE_FILE_LARGE_ADDRESS_AWARE bit set in the image header. In order to set this bit, one has to use the /LARGEADDRESSAWARE linker flag when compiling a > 2GB user mode 32 bit application.
So how can one fix this? The good news is there is a utility included with Visual Studio named EditBin that can modify the image header and set the IMAGE_FILE_LARGE_ADDRESS_AWARE bit. If you have VS2005, simply copy the problem executable and from the command prompt run “editbin /LARGEADDRESSAWARE nameofexe.EXE”, replacing nameofexe.EXE with the problem exe you need to fix. Then simply copy your fixed exe over the original provided by the ISV. However, when you have verified that this fixes the problem, please contact the publisher as the ISV needs to fix the application.
If you don’t have VS2005, you can download Visual C++ 2005 Express Edition from here for free.
Comment by naphtali011 — August 9, 2007 @ 7:17 am
OK, I have just upped the RAM on my Vista PC from 1GB to 5GB physical, reported in msinfo32 as 3.25GB, being max 4GB addressable minus reservations.
So if Vista needs to reserve 0.75GB for various purposes, how come when I only had 1GB of physical RAM was it not limiting me to 0.25GB usable!
Comment by Dennis Wright — July 23, 2008 @ 10:08 pm
In my typical fashion, it has taken me 2 months to reply, but here it goes:
Dennis, memory IO reservations have nothing to do with Vista’s requirements and everything to do with hardware. These memory reservations are required by devices so that memory mapped IO can undertaken with the device in question. As such, the device is mapped into a region of the 32bit linear address space. Once the device is mapped into that space, nothing else, including physical memory, can be mapped at those same address locations.
So, if you have, say, a video card with a 256MB framebuffer, in order for the system to communicate with that framebuffer efficiently, it is mapped into the systems physical linear address space and subsequently maps out the physical memory that would otherwise be available at that location. Now, if no physical memory is actually present at that location, there is no issue, which is why on sub 4GB 32bit configurations, this wasn’t an issue – there was no collision between devices needing address space and physical memory.
Now, the way 64bit systems work around this is that they have a huge 40-44bit (depending on CPU type) physical address space currently available, so physical memory and memIO reservations need not overlap (though if you actually were to max out the 40-44 bits of physical memory support of a current generation of x64 CPU, you would indeed see that about 768MB of your 1TB of expected memory would be missing).
On 32bit server systems, we are able to enable something called PAE with remap support, which are 32bit physical address extensions, wherein we enable 3-level page translation to a 36bit physical address space. As with 64bit systems, once the physical address space has been expanded, we no longer need have the overlap situation, which is why you don’t see the problem on Server 2003/2008 32bit. Again, we don’t allow address space remap in the client 32bit SKU’s due to a history of problematic device drivers and physical hardware that didn’t always play nice in a remap scenario (e.g. older consumer hardware with firmware or drivers that expected an unmodified 32bit physical address map).
Comment by makfu — September 25, 2008 @ 3:42 am