|Priority:||5 - Low|
|Created by:||Jerry Jelinek [X]|
|Reported by:||Jerry Jelinek [X]|
|Assigned to:||Hans Rosenfeld [X]|
Fixed: A fix for this issue is checked into the tree and tested.
(Resolution Date: 2018-08-17T15:15:37.350Z)
2018-08-30 Zolom Swamp (Release Date: 2018-08-30)
Currently when a
reboot is issued from within a bhyve VM we have to tear down the zone and reboot the entire zone. It would be better if we could re-use the
init process (
zhyve) we already have. Normally this could be done in the
restart_init function which can be called when the zone's
init exits, but for
zhyve we currently get an EFAULT out of
exec_init when it's called by
restart_init. This can only happen when we try the
copyout to construct the stack. If we can fix this we could re-use the existing process and hang on to the memory we've already allocated for the VM.
Marking as low priority since this is not a blocker for releasing bhyve.
I don't think we need to necessarily reuse the same process - reusing the zone would be sufficient. The goal is to avoid tearing down the guest memory so as to avoid the long teardown and setup time, as well as the risk that the memory isn't available due to someone else allocating memory.
It's perhaps worth pointing out that this means we wouldn't pick up any new zonecfg changes across this kind of reboot. Not sure if that'll be a problem?
I was thinking that we probably need two forms of reboot - hard and soft for lack of better terms. A hard reboot will do a full teardown and fresh setup of the zone. This can only be initiated from the global zone (or tooling that acts in the global zone). That's ok, because you have to be in the global zone to change the config too. Any reboot that is initiated in the guest is a soft reboot.
This implies that a complete implementation may also need to have new flags on
zoneadm reboot and
zoneadm shutdown -r to distinguish between hard and soft reboots.
I was thinking along similar lines, where a reboot initiated from within the VM kept the same process, but an external reboot behaved as we do now. I don't really see a need for additional
zoneadm reboot options at this time, since there is no infrastructure to leverage that.
I believe this was addressed in OS-6612?
No, OS-6612 just allows the zone to be rebooted from within the VM, but we currently have to exit the old bhyve process and start a new one from scratch.
Pardon me. We are leaving the memory resources allocated though, right?
No, we reboot the zone when init exits, so all of the bhyve resources are deallocated and then reallocated. Preserving all of the resources is the whole point of this ticket.
Preventing the EFAULT when re-exec'ing init was handled in OS-7089. The remaining changes to bhyve to allow VM reboots reusing the allocated resources are handled here.
Testing: I've been using this change for a while. I can reboot VMs with various guest OSs from within and the already allocated resources will be reused. There remains an issue (OS-7110) with the UEFI firmware dying early in reboot due to an unexpected exception, but for now this worked around by clearing the lower 1G of VM memory.
Additional testing: John asked for this to be tested in conjunction with GPGPU passthru. I didn't run the load tests, but the NVIDIA samples run fine before and after a in-VM reboot. It is no different than before.
illumos-joyent commit a72f92c76cef0e035c3ae9af38ea6789c0676527 (branch master, by Hans Rosenfeld)
OS-6620 bhyve reboot should reuse existing process
Reviewed by: Patrick Mooney <firstname.lastname@example.org>
Reviewed by: John Levon <email@example.com>
Reviewed by: Jerry Jelinek <firstname.lastname@example.org>
Approved by: Patrick Mooney <email@example.com>