Posted Mon Aug 1 10:32:50 2011

Steps to enable Oracle's Transcendent Memory with frontswap, cleancache, self-ballooning and frontswap-selfshrinking on Xen guests to increase performance on low-memory systems.

Frontswap acts like a proxy for pages being swapped, allowing other systems to handle the page before it gets saved to disk. With tmem this means that these pages can be saved in unused host memory. If there is no host memory available, the page is saved to disk as normal

Cleancache allows the kernel to move clean (unmodified) cache pages which are about to be dropped to another system. Similar to frontswap, with tmem, this means that these cache pages can be saved to unused host memory. If there is no host memory available the page is dropped as normal.

Tmem utilizes unused memory on the host temporarily assigning it for guest use. This is done by "pools" which have 2 flags describing their type. Pools can be persistent or ephemeral and private or shared. Frontswap uses a private, persistent pool while cleancache uses a private, ephemeral pool (cleancache pages that are lost are simply reread from disk).

Selfballooning controls the in-kernel Xen balloon driver, targeting a goal value (vm_committed_as), thus pushing less frequently used clean page cache pages (through the cleancache code) into Xen tmem where Xen can balance needs across all VMs residing on the physical machine.

Frontswap-selfshrinking controls the number of pages in frontswap, driving it towards zero (effectively doing a partial swapoff) when in-kernel memory pressure subsides, freeing up RAM for other VMs.

Xen 4.1 or better is required to support this.

1. mkdir linux
2. cd linux
3. git init
4. git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-2.6-stable.git
5. git remote add tmem git://git.kernel.org/pub/scm/linux/kernel/git/djm/tmem.git
6. git fetch -a
7. git checkout -b master v2.6.39.3
8. git merge remotes/tmem/stable/cleancache-v9-plus-frontswap-v5-with-tmem
9. git cherry-pick a50777c791031d7345ce95785ea6220f67339d90 #You'll need to merge drivers/xen/Makefile
10. make menuconfig #Select cleancache and frontswap under Processor types and features and self-ballooning from Drivers->Xen
11. Build/install the kernel in whatever fashion you normally do
12. Add 'tmem' to the kernel commandline (GRUB_CMDLINE_LINUX="tmem" in /etc/default/grub on Debian)