Win32 Programming

Win32 Programming Lesson 17: Memory Mapped Files (Finally, cool stuff again, all this work is getting tedious!)

Where are we? • We’ve gotten pretty familiar with memory • But now it’s time to look at some of the clever tricks you can play with memory

Memory mapped files • Three primary purposes • Loading and executing .exe and DLL files • Accessing files on disk quickly without need to buffer • Allow multiple processes to share the same data in memory

CreateProcess Revisited • Five steps: • Find the .exe file on disk • Create a new Process kernel object • Create a private address space for the process • Reserve space in memory at 0x00400000 (by default) and load the .exe image • Physical storage is from the .exe image, not the paging file

Next… • System uses LoadLibrary to load the needed DLLs from disk • Once again, physical storage underlying the DLLs is on disk, not the system paging file (clever eh?) • If everything can’t be mapped CreateProcess fails and returns FALSE

Multiple Instances • Very clever – remember COPY_ON_WRITE? • DLLs share the same image in memory – but get new pages when they modify their memory image • However, what if we want to share variables that change?

Background • To work out how to do this, we need to understand a little more of the foundations • Every .exe or DLL has multiple sections • Each section has one or more of the following attributes: • READ • WRITE • EXECUTE • SHARED (effectively turns off COPY_ON_WRITE) • Can see this using DumpBin

Using a Shared Data segment • See example

Memory mapped data files • How to reverse the order of all bytes in a file? Four ways… • One file, one buffer (load it all!) • Two files, one buffer (read it in in chunks, reverse each chunk) • One file, two buffers (read in 1024 bytes from the start and the end, and swap them) • One file, no buffers!

Whassat? • Yep, one file, zero buffers • Map the file into memory • Work on the memory image directly, letting the OS handle the buffering

Six step process • You must: • Create or open the kernel object that references the file • Create a file-mapping kernel object • Tell the system to map all or part of the file • Tell the system when mapping is no longer needed • Close the file-mapping object • Close the file

Step 1: Opening the file • Easy: Createfile • HANDLE CreateFile( PCSTR pszFileName, DWORD dwDesiredAccess, DWORD dwShareMode, PSECURITY_ATTRIBUTES psa, DWORD dwCreationDisposition, DWORD dwFlagsAndAttributes, HANDLE hTemplateFile);

Parms • dwDesiredAccess • 0 – just get the attributes • GENERIC_READ • GENERIC_WRITE • GENERIC_READ | GENERIC_WRITE • dwShareMode • 0 – not shared • FILE_SHARE_READ • FILE_SHARE_WRITE • FILE_SHARE_READ | FILE_SHARE_WRITE

Step 2: Creating the file-mapping • HANDLE CreateFileMapping( HANDLE hFile, PSECURITY_ATTRIBUTES psa, DWORD fdwProtect, DWORD dwMaximumSizeHigh, DWORD dwMaximumSizeLow, PCTSTR pszName); • See MSDN

Parms • dwMaximumSizeHigh/Low are important • 64-bit representation of the file length so there is enough disk space if you want to make writes • See example for what happens as we step through…

Step 3: Map the file in memory • Using: • PVOID MapViewOfFile( HANDLE hFileMappingObject, DWORD dwDesiredAccess, DWORD dwFileOffsetHigh, DWORD dwFileOffsetLow, SIZE_T dwNumberOfBytesToMap);

Parms • FILE_MAP_WRITE: You can read and write file data. CreateFileMapping had to be called by passing PAGE_READWRITE. • FILE_MAP_READ: You can read file data. CreateFileMapping could be called with any of the protection attributes: PAGE_READONLY, PAGE_READWRITE, or PAGE_WRITECOPY. • FILE_MAP_ALL_ACCESS: Same as FILE_MAP_WRITE. • FILE_MAP_COPY: You can read and write file data. Writing causes a private copy of the page to be created.

Step 4: Unmap the file • BOOL UnmapViewOfFile(PVOID pvBaseAddress); • And flush, if you need to… • BOOL FlushViewOfFile( PVOID pvAddress, SIZE_T dwNumberOfBytesToFlush); • pvAddress is what you want to flush…

Step 5/6 Cleaning up • CloseHandle(hFileMapping); • CloseHandle(hFile); • Easy  • But remember kernel usage counts – means that we can close the file much earlier for neater code if we want to

Example • Let’s look at the one in the book – file reverse…

Processing a really big file • __int64 qwFileOffset = 0, qwNumOf0s = 0;while (qwFileSize > 0) { // Determine the number of bytes to be mapped in this view DWORD dwBytesInBlock = sinf.dwAllocationGranularity; if (qwFileSize < sinf.dwAllocationGranularity) dwBytesInBlock = (DWORD) qwFileSize; PBYTE pbFile = (PBYTE) MapViewOfFile( hFileMapping, FILE_MAP_READ, (DWORD) (qwFileOffset >> 32), // Starting byte (DWORD) (qwFileOffset & 0xFFFFFFFF), // in file dwBytesInBlock); // # of bytes to map // Count the number of Js in this block. for (DWORD dwByte = 0; dwByte < dwBytesInBlock; dwByte++) { if (pbFile[dwByte] == 0) qwNumOf0s++; } // Unmap the view; we don't want multiple views // in our address space. UnmapViewOfFile(pbFile); // Skip to the next set of bytes in the file. qwFileOffset += dwBytesInBlock; qwFileSize -= dwBytesInBlock; }

Specifying the Base Address • Say you want to share a memory-mapped file which contains a linked-list • Need to have the same base address in both processes • Can call MapViewOfFileEx

Assignment • Recreate my little shared-memory form program  • EASY!

Win32 Programming