Win32 Programming

Win32 Programming Lesson 18: More Memory Mapped Files and the HEAP (Finally, cool stuff!)

Where are we? • We’ve gotten pretty familiar with the idea of memory-mapped files but there are some important concepts we haven’t looked at • Finish up today, and then look at the heap

Sharing Data • There are lots of different ways to share data between processes • But the “lowest level” way is really in memory, via a memory-mapped file • Two or more processes map the same base address – hence they are sharing the same physical memory

Avoiding the “real” File system • Very inconvenient if every memory-mapped file actually had to exist on disk • Imagine that you simply wanted to use the mechanism to pass data, not keep it • Can call CreateFileMapping with INVALID_HANDLE_VALUE as the hFile parameter, and the memory-mapped file is backed by the page file

30 Second Quiz • What’s wrong with this code? • HANDLE hFile = CreateFile(...); HANDLE hMap = CreateFileMapping(hFile, ...); if (hMap == NULL) return(GetLastError());

Answer… • You’ll get back INVALID_HANDLE_VALUE from the first call • Which means…

MMFExample • Simple program • When it maps the view of the file, it transfers data between the two programs • Nice method of sharing between two processes!

Sparsely Committing… • Remember we talked about how to commit memory for files? • Same discussion for Memory-mapped files – that is, we don’t need to commit all our memory at once

Consider • CELLDATA CellData[200][256]; • If sizeof(CELLDATA) is 128 that’s about 6MB. • Better to share as a sparsely-committed file mapping object • If we’re sharing via the paging file, can use SEC_RESERVE or SEC_COMMIT

SEC_RESERVE • When you pass in SEC_RESERVE you don’t actually commit the space • Just returns a HANDLE to the file mapping object • Any attempts to access the memory cause a memory violation

VirtualAlloc (again) • Solution: Call VirtualAlloc to allocate the memory as we use it!

The HEAP • Heap is a fantastic tool for allocating small blocks of memory • Perfect for linked lists and trees • Advantage: can ignore allocation granularity • Disadvantage: slow, with no direct control of physical allocation • Better yet, internals not entirely documented

Default • Each process gets a default heap of 1MB • Can specify at link time (/HEAP:) • Used by many Windows/C RTL functions • Access to the HEAP is serialized (why, and what does this mean?) • Can obtain a handle via: • HANDLE GetProcessHeap();

More than 1 Heap is a… • Five reasons you might want to do this: • Component protection • More efficient memory management • Local access • Avoiding thread sync overhead • Quick free

1: Component Protection • Imagine you have two structures: a linked list and a binary tree • If you share one heap, and one has a bug, the problem may show up in the other structure • If we have different heaps, problems tend to be localized (unless you *really* mess up!)

2: Memory Management • Heaps work best when all the objects in them are the same size • Imagine mixing two different sizes; when you free object 1 object 2 may not be a perfect fit • Better to allocate all objects of the same size in the same place

3: Local Access • Swapping to disk is really expensive • Best to keep things you use together close together

4: Avoiding thread-sync issues • Heaps are serialized by default • CPU overhead involved in keeping heap access thread safe • If you tell the system a heap is single-threaded, you can lose this overhead • DANGER WILL ROBINSON: YOU ARE NOW RESPONSIBLE FOR THREAD SAFETY!!!

5: Quick free • Instead of freeing things up block by block you can choose to free the entire heap in one go • That’s *really* quick

So… how? • HANDLE HeapCreate( DWORD fdwOptions, SIZE_T dwInitialSize, SIZE_T dwMaximumSize); • Options: 0, HEAP_NO_SERIALIZE, HEAP_GENERATE_EXCEPTIONS • Serialize turns off checking for Alloc and Free • Can manage this yourself via Critical Sections if you want to

HEAP_GENERATE_EXCEPTIONS • Raise an exception whenever an allocation request fails • Basically, it’s just about whether you want to catch exceptions or check return values – depends on the application • Oh… if dwMaximumSize is 0 the size is unlimited…

Allocating Memory from the Heap • PVOID HeapAlloc( HANDLE hHeap, DWORD fdwFlags, SIZE_T dwBytes); • Flags: HEAP_ZERO_MEMORY, HEAP_GENERATE_EXCEPTIONS, HEAP_NO_SERIALIZE • Exceptions: STATUS_NO_MEMORY, STATUS_ACCESS_VIOLATION

Changing the size… • PVOID HeapReAlloc( HANDLE hHeap, DWORD fdwFlags, PVOID pvMem, SIZE_T dwBytes); • New flag: HEAP_REALLOC_IN_PLACE_ONLY • Means that the location won’t change

Obtaining the Size • SIZE_T HeapSize( HANDLE hHeap, DWORD fdwFlags, LPCVOID pvMem); • Flags: 0 or HEAP_NO_SERIALIZE

Freeing a block • BOOL HeapFree( HANDLE hHeap, DWORD fdwFlags, PVOID pvMem); • Flags? You tell me…

Destroying a Heap • BOOL HeapDestroy(HANDLE hHeap); • TRUE on success • You can’t destroy the default heap!

Heaps with C++ • Under C you would use malloc • In C++ can use new/delete • CSomeClass *pSomeClass = new CSomeClass; • delete pSomeClass; • Now the clever bit: overload new/delete…

Prototype • class CSomeClass { private: static HANDLE s_hHeap; static UINT s_uNumAllocsInHeap; // Other private data and member functions public: void* operator new (size_t size); void operator delete (void* p); // Other public data and member functions };

Code… HANDLE CSomeClass::s_hHeap = NULL; UINT CSomeClass::s_uNumAllocsInHeap = 0; void* CSomeClass::operator new (size_t size) { if (s_hHeap == NULL) { // Heap does not exist; create it. s_hHeap = HeapCreate(HEAP_NO_SERIALIZE, 0, 0); if (s_hHeap == NULL) return(NULL); } // The heap exists for CSomeClass objects. void* p = HeapAlloc(s_hHeap, 0, size); if (p != NULL) { // Memory was allocated successfully; increment // the count of CSomeClass objects in the heap. s_uNumAllocsInHeap++; } // Return the address of the allocated CSomeClass object. return(p); }

And delete… void CSomeClass::operator delete (void* p) { if (HeapFree(s_hHeap, 0, p)) { // Object was deleted successfully. s_uNumAllocsInHeap--; } if (s_uNumAllocsInHeap == 0) { // If there are no more objects in the heap, // destroy the heap. if (HeapDestroy(s_hHeap)) { // Set the heap handle to NULL // so that the new operator // will know to create a new heap if a // new CSomeClass // object is created. s_hHeap = NULL; } } }

Misc Functions… • DWORD GetProcessHeaps – returns handles to all heaps in the process • BOOL HeapValidate – check that a heap is a-okay… • UINT HeapCompact – coalesce free blocks • HeapLock and HeapUnlock – used for thread sync • HeapWalk – for debugging; lets you enumerate sections/blocks in the heap

Next • Next, it gets difficult • DLLs

Win32 Programming