OS-6930: bhyve PIR can still miss PIR wake-ups

Details

Issue Type:Bug
Priority:4 - Normal
Status:Resolved
Created at:2018-05-04T04:47:09.122Z
Updated at:2018-06-01T16:21:10.823Z

People

Created by:Patrick Mooney [X]
Reported by:Patrick Mooney [X]
Assigned to:Patrick Mooney [X]

Resolution

Fixed: A fix for this issue is checked into the tree and tested.
(Resolution Date: 2018-06-01T16:21:02.400Z)

Fix Versions

2018-06-07 Train Graveyard (Release Date: 2018-06-07)

Related Links

Labels

bhyve

Description

After closely re-inspecting OS-6829 as part of the effort to upstream it, I found one series of events which would still cause a missed wake-up when using VMX PIR functionality.

By storing the interrupt priorities a bit field in the reserved bits of pending in the PIR descriptor, the OS-6829 fix counts on stale priority bits being present in the field after the CPU delivers those interrupts to the guest. This is fine since the field is cleared in vmx_inject_pir() when it checks for accumulated posted interrupts to inject into the host. There is a condition, however, when those stale bits present a problem: When a high-level interrupt is posted and delivered to the guest, followed by the guest exiting on HLT at a lowered PPR, without a subsequent VM entry. In that case, the processor will be in a lower priority but with stale high-prio bits left in the pending field. If two interrupts, one lower than the PPR, one higher than the PPR (but lower than the priority represented by the stale bit(s)), are sent to the vCPU, in that order, it will fail the notification check on the second interrupt and miss the necessary wake-up.

The solution is rather simple: When setting the pir_desc->pending field as part of interrupt posting, attempt to clear the priority bits if the pending bit is absent. A failure of the CAS for the updating to pir_desc->pending is still fine, as an atomic OR of the priority, plus unconditional wake-up notification remains an acceptable fallback.

Comments

Comment by Jira Bot
Created at 2018-05-29T19:15:44.343Z

illumos-joyent commit 21e2c666e809045ac5b02a65fa8b9c6708909804 (branch master, by Patrick Mooney)

OS-6930 bhyve PIR can still miss PIR wake-ups
Reviewed by: John Levon <john.levon@joyent.com>
Reviewed by: Bryan Cantrill <bryan@joyent.com>
Approved by: Robert Mustacchi <rm@joyent.com>