OS-7061: bhyve: set MAP_NORESERVE when mmap'ing guest memory


Issue Type:Improvement
Priority:4 - Normal
Created at:2018-07-03T15:36:16.929Z
Updated at:2018-07-12T13:21:06.800Z


Created by:Hans Rosenfeld [X]
Reported by:Hans Rosenfeld [X]
Assigned to:Hans Rosenfeld [X]


Fixed: A fix for this issue is checked into the tree and tested.
(Resolution Date: 2018-07-12T13:21:06.771Z)

Fix Versions

2018-07-19 Wall Market (Release Date: 2018-07-19)

Related Links




When setting up memory for a VM, bhyve will first create an anonymous mapping with mmap() for all of guest memory, then allocate the guest physical memory and map it into that previously mmap'ed address range. I noticed that the mmap() can fail for large VMs before any real memory is allocated.

As we allocate all guest physical memory in a separate step we can safely set the MAP_NORESERVE flag in mmap to get a mapping that doesn't reserve swap space.


Comment by Hans Rosenfeld [X]
Created at 2018-07-03T16:17:45.724Z


On my test machine with 128G RAM, all VMs still run fine with this change. I can now easily run 24 4G VMs without problems and the system remains usable: with 96G of RAM assigned to VMs, I can still build illumos in an OS zone, which previously would fail due to resource shortages even with only 20 such VMs running. Creating a 92G VM also worked fine, and creating a 96G VM failed due to exceeding max_page_get instead of failing in mmap(). Pushing the system to its limits caused a hard hang once I tried allocating more than 108G to VMs.

Comment by Jira Bot
Created at 2018-07-12T13:20:34.640Z

illumos-joyent commit 52b23dd769a5fb97e9d1ae1c891b7ba6e15532a1 (branch master, by Hans Rosenfeld)

OS-7061 bhyve: set MAP_NORESERVE when mmap'ing guest memory
Reviewed by: Jerry Jelinek <jerry.jelinek@joyent.com>
Reviewed by: Patrick Mooney <patrick.mooney@joyent.com>
Approved by: Patrick Mooney <patrick.mooney@joyent.com>