|Priority:||4 - Normal|
|Created by:||Hans Rosenfeld [X]|
|Reported by:||Hans Rosenfeld [X]|
|Assigned to:||Hans Rosenfeld [X]|
Fixed: A fix for this issue is checked into the tree and tested.
(Resolution Date: 2018-07-12T13:21:06.771Z)
2018-07-19 Wall Market (Release Date: 2018-07-19)
When setting up memory for a VM, bhyve will first create an anonymous mapping with mmap() for all of guest memory, then allocate the guest physical memory and map it into that previously mmap'ed address range. I noticed that the mmap() can fail for large VMs before any real memory is allocated.
As we allocate all guest physical memory in a separate step we can safely set the MAP_NORESERVE flag in mmap to get a mapping that doesn't reserve swap space.
On my test machine with 128G RAM, all VMs still run fine with this change. I can now easily run 24 4G VMs without problems and the system remains usable: with 96G of RAM assigned to VMs, I can still build illumos in an OS zone, which previously would fail due to resource shortages even with only 20 such VMs running. Creating a 92G VM also worked fine, and creating a 96G VM failed due to exceeding max_page_get instead of failing in mmap(). Pushing the system to its limits caused a hard hang once I tried allocating more than 108G to VMs.
illumos-joyent commit 52b23dd769a5fb97e9d1ae1c891b7ba6e15532a1 (branch master, by Hans Rosenfeld)
OS-7061 bhyve: set MAP_NORESERVE when mmap'ing guest memory
Reviewed by: Jerry Jelinek <email@example.com>
Reviewed by: Patrick Mooney <firstname.lastname@example.org>
Approved by: Patrick Mooney <email@example.com>