Issue Type: | Improvement |
---|---|
Priority: | 4 - Normal |
Status: | Resolved |
Created at: | 2018-08-23T19:16:37.389Z |
Updated at: | 2019-03-15T15:37:07.003Z |
Created by: | Former user |
---|---|
Reported by: | Former user |
Assigned to: | Former user |
Fixed: A fix for this issue is checked into the tree and tested.
(Resolution Date: 2019-03-15T15:36:55.754Z)
2019-03-28 Operation Righteous Cowboy Lightning (Release Date: 2019-03-28)
When the bhyve porting effort was underway, all effort was focused on the Intel side, ignoring AMD support. With Epyc coming down the pipeline, it would be nice to finish the porting effort so that bhyve works there as well.
A community member on IRC (jubal) tried a test PI on their relatively old Opteron system (CPU part: OS2439YDS6DGN) and reported that they were able to boot a centos-7 VM without any issues.
The two goals of this test were to a) stress the EPYC hardware enough to create instability b) to get some baseline benchmark performance numbers.
The lab machine odyssey has 1.4TB disk, 256gigs of ram, and 32 cores (64 virtual cpus).
I created 6 VM's with varying disk, cpu, and ram that consumed 2/3 of total system memory, 56 of the 64 vcpus, and approximately half of the 1.4 TB of disk space.
I ran benchmarks on a single system at a time for optimal baselines as seen below from a 16cpu / 32gb ram system.
I ran a variety of load generation to stress the overall system and see if anything fell over. Nothing did.
If there was more time I would have liked to generate a more "organic" application based load. I'll work on tooling up automation to deploy something quickly, so that in the future we can make better tests in short time frames.
In terms of baselines the cpu performance in particular looks quite good.
Raw results:
#sysbench cpu
[root@befce8ba-ce72-4e2c-bee8-baf5b38130c2 ~]# rm -f /rent/result.txt && for i in {1..10}; do sysbench cpu --threads=8 run | grep "events per second"; cat result1.txt | awk '{ sum += $1 } END { if (NR > 0) print sum / NR }'
Result: 10380.8
#sysbench ram
rm -f /rent/result1.txt && for i in {1..10};do sysbench memory --threads=8 run | grep "MiB/sec" | awk '{print $4}' | sed -e 's/(//g' >> /rent/result1.txt; done;cat result1.txt | awk '{ sum += $1 } END { if (NR > 0) print sum / NR }'
Result: 1122.03
#fio disk
Jobname | Read BW | Read IOPS | Write BW | Write IOPS |
readblock128ksize5gfiooutput | 1629741 | 12732.35934 | 0 | 0 |
writeblock128ksize5gfiooutput | 0 | 0 | 1347784 | 10529.56298 |
rwblock128ksize5gfiooutput | 285407 | 2229.747387 | 285463 | 2230.182927 |
randreadblock8kfile1gfiooutput | 40496 | 5062.063106 | 0 | 0 |
randwriteblock8kfile1gfiooutput | 0 | 0 | 1006310 | 125788.8676 |
randrwblock8kfile1gfiooutput | 28815 | 3601.896514 | 28659 | 3582.492874 |
randreadblock64kfile1gfiooutput | 246202 | 3846.912421 | 0 | 0 |
randwriteblock64kfile1gfiooutput | 0 | 0 | 1086607 | 16978.23834 |
randrwblock64kfile1gfiooutput | 110949 | 1733.588273 | 111819 | 1747.185044 |
randreadblock128kfile1gfiooutput | 299935 | 2343.249428 | 0 | 0 |
randwriteblock128kfile1gfiooutput | 0 | 0 | 1203876 | 9405.281286 |
randrwblock128kfile1gfiooutput | 151756 | 1185.59719 | 155203 | 1212.529274 |
randreadblock256kfile1gfiooutput | 527187 | 2059.326295 | 0 | 0 |
randwriteblock256kfile1gfiooutput | 0 | 0 | 1502257 | 5868.194842 |
randrwblock256kfile1gfiooutput | 227555 | 888.888889 | 240349 | 938.866577 |
randreadblock8kfile5gfiooutput | 41407 | 5175.924244 | 0 | 0 |
randwriteblock8kfile5gfiooutput | 0 | 0 | 778452 | 97306.60728 |
randrwblock8kfile5gfiooutput | 28153 | 3519.142633 | 28119 | 3514.956692 |
randreadblock64kfile5gfiooutput | 229528 | 3586.375974 | 0 | 0 |
randwriteblock64kfile5gfiooutput | 0 | 0 | 1580132 | 24689.57203 |
randrwblock64kfile5gfiooutput | 107183 | 1674.738648 | 106916 | 1670.573342 |
randreadblock128kfile5gfiooutput | 295606 | 2309.427154 | 0 | 0 |
randwriteblock128kfile5gfiooutput | 0 | 0 | 984578 | 7692.018779 |
randrwblock128kfile5gfiooutput | 146336 | 1143.255918 | 146365 | 1143.479232 |
randreadblock256kfile5gfiooutput | 415738 | 1623.979066 | 0 | 0 |
randwriteblock256kfile5gfiooutput | 0 | 0 | 1145984 | 4476.502732 |
randrwblock256kfile5gfiooutput | 167820 | 655.547035 | 167231 | 653.246421 |
randrwratio80block8ksize5gfiooutput | 33761 | 4220.243417 | 8468 | 1058.574777 |
randrwratio80block64ksize5gfiooutput | 185075 | 2891.797634 | 46378 | 724.660074 |
randrwratio80block128ksize5gfiooutput | 236896 | 1850.753029 | 59948 | 468.35013 |
randrwratio80block256ksize5gfiooutput | 334709 | 1307.458387 | 84855 | 331.466069 |
With vmx_call_trap
rearranged to vmm_call_trap
, I wanted to make sure NMI handling was still ok, which it was:
fffffe007aeed440 fffffffff79488ff () fffffe007aeed470 unix:av_dispatch_nmivect+34 () fffffe007aeed480 unix:nmiint+152 () fffffe007aeed570 vmm:vmm_call_trap+3e () fffffe007aeed670 vmm:vmx_run+7d2 () fffffe007aeed760 vmm:vm_run+21f () fffffe007aeedc20 vmm:vmmdev_do_ioctl+775 () fffffe007aeedcc0 vmm:vmm_ioctl+12f () fffffe007aeedd00 genunix:cdev_ioctl+39 () fffffe007aeedd50 specfs:spec_ioctl+60 () fffffe007aeedde0 genunix:fop_ioctl+55 () fffffe007aeedf00 genunix:ioctl+9b () fffffe007aeedf10 unix:brand_sys_syscall+21a ()
Using the typical variety of guests (Linux, Windows, FreeBSD, illumos) on an Intel machine, operation seemed normal as expected. The PI was also run nested under KVM on a Threadripper machine where it seemed to function fine.
illumos-joyent commit 2453029c010976e95241a5f5244e86d44dc6194c (branch master, by Patrick Mooney)
OS-7170 bhyve should support AMD
Reviewed by: John Levon <john.levon@joyent.com>
Reviewed by: Hans Rosenfeld <hans.rosenfeld@joyent.com>
Approved by: Robert Mustacchi <rm@joyent.com>