OS-7397: reduce lock contention in bhyve instr emul


Issue Type:Improvement
Priority:4 - Normal
Created at:2018-11-27T22:34:50.688Z
Updated at:2018-12-06T22:25:06.711Z


Created by:Patrick Mooney [X]
Reported by:Patrick Mooney [X]
Assigned to:Patrick Mooney [X]


Fixed: A fix for this issue is checked into the tree and tested.
(Resolution Date: 2018-12-05T19:32:02.176Z)

Fix Versions

2018-12-06 Grizz (Release Date: 2018-12-06)

Related Links




On hosts that lack APICv capabilities, guests which access the APIC from multiple vCPUs simultaneously can lead to contention on vms_lock in the VM shim via vm_fault_quick_hold_pages(). This lock protects the list of valid mappings and is used to synchronize modifications to resources in the vmspace. As it stands today, the resources accessed by vm_fault_quick_hold_pages() do not strictly require the protection of vms_lock. External invariants in bhyve prevent modification of the vmspace layout while any vCPU is running. By removing lock acquisition from this specific codepath, we can free instruction emulation to proceed in parallel.


Comment by Patrick Mooney [X]
Created at 2018-12-04T23:23:27.027Z

The primary method of testing this change was to boot guests on a patched PI. The code path in question can be very "hot" for certain workloads. Windows, for example, seems to use the HPET which it accesses with a moderate frequency. Cycling the guests through startup and shutdown exercising the edges of the vm_object_t reference counting to ensure proper resource clean-up.

Comment by Jira Bot
Created at 2018-12-05T19:29:36.936Z

illumos-joyent commit 5b87e03ada692fb585a6a15a0cccfff5eb39e8ea (branch master, by Patrick Mooney)

OS-7397 reduce lock contention in bhyve instr emul
Reviewed by: John Levon <john.levon@joyent.com>
Reviewed by: Hans Rosenfeld <hans.rosenfeld@joyent.com>
Approved by: Hans Rosenfeld <hans.rosenfeld@joyent.com>