Large Memory Management issues: Performance, Fragmentation, Movable objects and Huge Page overhead.

Presented by Christopher Lameter
Monday 11:35 a.m.–12:25 p.m. in Great Hall CB01.05.009
Target audience: Developer

Abstract

Memory management with larger memory sizes of around a couple of terabyte leads to a number of issues that make life hard. This talk is a look at the issues and at the work in progress on these issues. First, huge pages are not supported for the page cache which leads to numerous performance issues. It begins with the long time it takes to just copy a file of a few gigs in size (minor compared to the size of the memory in the system of a few terabytes after all...) to the performance problems when accessing large amounts of 4k pages from a memory mapped file. Since huge pages are not supported TLB trashing often becomes an issue. Devices have issues writing large blocks of data to disk due to large scatter gather lists having to be processed in memory. Reclaim becomes slow since the system is literally churning through billions of 4k page to figure out which ones to evict. Second, huge pages cause a memory overhead that surprises users. The kernel manages 4k chunks of the huge page in many cases and needs to have metadata structures for each 4k page. Other metadata also exists. This results in gigabytes of memory just not being present. The user bought a system with a lot of memory but the Linux just cannot use it. Third, fragmentation becomes an issue in particular if huge pages and 4k pages are frequently allocated and freed (like we have to because the page cache does not support 4k after all). So over time systems have to be rebooted to allow the allocation of huge pages. Huge pages are actually only a small problem because devices on large systems often want to allocate large contiguous buffers for I/O (like specialized networking devices using RDMA).

Presented by

Christopher Lameter

Christoph Lameter is working as a lead in research and development for Jump Trading LLC (an algorithmic trading company) in Chicago and maintains the slab allocators and the per cpu subsystems in the Linux Kernel. He contributed to a number of Linux projects since the initial kernel releases in the early 90s. As a kernel developer at SGI he helped pioneer the use of Linux for Supercomputing and developed the necessary kernel capabilities for HPC applications. Currently he is working on improving Linux through the use of new faster APIs to a variety of high performance devices and is evaluating new technologies that allow faster processing.