read the transcript!
what’s mmap for?
person 1: I want to work with a VERY LARGE FILE but it won’t fit in memory
person 2: You could try mmap!
(mmap = “memory map”)
load files lazily with mmap
When you mmap a file, it gets mapped into your program’s memory.
2 TB file: 2 TB of virtual memory
but nothing is ACTUALLY read into RAM until you try to access the memory.
(how it works: page faults!)
how to mmap in Python
import mmap f= open("HUGE.txt") mm= mmap.mmap (f. filenol), 0)
(this won’t read the file from disk! Finishes ~instantly.)
print (mm C-1000:7)
this will read only the last 1000 bytes!
sharing big files with mmap
three processes: we all want to read the same file!
mmap: no problem!
Even if 10 processes mmap a file, it will only. be read into memory once
dynamic linking uses mmap
program: I need to use libc.so.6 (standard library)
ld dynamic linker: you too eh? no problem. I always mmap, so that file is probably loaded into memory already.
anonymous memory maps
- not from a file (memory set to by default)
MAP.SHARED, you can use them to share memory with a subprocess!