Issue Type: | Improvement |
---|---|
Priority: | 4 - Normal |
Status: | Resolved |
Created at: | 2018-04-02T22:32:58.641Z |
Updated at: | 2018-04-05T23:54:36.462Z |
Created by: | Former user |
---|---|
Reported by: | Former user |
Assigned to: | Former user |
Fixed: A fix for this issue is checked into the tree and tested.
(Resolution Date: 2018-04-05T23:54:36.452Z)
2018-04-12 Promised Land (Release Date: 2018-04-12)
The FPU save/restore logic from upstream performs a swap of host/guest FPU state surrounding every call to VMRUN. This is presumably to ensure that such state is manipulated inside the critical_enter/critical_exit section, where it cannot be preempted. Since illumos has the convenience of savectx/restorectx handlers available, we could choose to be lazy about when we do these FPU state swaps.
I wrote up a dtrace script for quantifying the scope of this FPU-state-saving elision:
BEGIN { secs = 0; } vmm:vm_run:entry { self->vmx = 0; self->fpu = 0; } vmm:vmx_run:entry { self->vmx++; } vmm:restore_guest_fpustate:entry { self->fpu++; } vmm:vm_run:return { @ = quantize(self->vmx - self->fpu); @total = quantize(self->vmx - self->fpu); @summed = sum(self->vmx - self->fpu); } tick-1s { printa(@); trunc(@); secs++; } END { printf("TOTAL:\n"); printa(@total); printa("FPU ops saved: %@u\n", @summed); if (secs > 0) { normalize(@summed, secs); printa("(%@u/s)\n", @summed); } }
Running this with a varied workload, including CPU-bound and network-bound tasks, the results seem like they're promising:
8 2 :END TOTAL: value ------------- Distribution ------------- count -1 | 0 0 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 58000 1 | 15 2 | 9 4 | 11 8 | 19 16 | 26 32 | 30 64 | 19 128 | 19 256 | 123 512 | 66 1024 | 18 2048 | 14 4096 | 4 8192 | 5 16384 | 1 32768 | 0 FPU ops saved: 258410 8 2 :END (6984/s)
Further data capture on real-world workloads would be valuable.
illumos-joyent commit 6a175f35f25ea47a4b116ad2dd1a0600fdf5a2bc (branch master, by Patrick Mooney)
OS-6864 bhyve should guard against going off-cpu
OS-6865 bhyve could be lazy about FPU state
Reviewed by: Jerry Jelinek <jerry.jelinek@joyent.com>
Reviewed by: John Levon <john.levon@joyent.com>
Approved by: John Levon <john.levon@joyent.com>