OS-8143: Keep the shadow PCIR_COMMAND synced with the real one for pass through.

Details

Issue Type:Improvement
Priority:4 - Normal
Status:Resolved
Created at:2020-03-23T21:52:10.259Z
Updated at:2020-03-24T18:20:24.839Z

People

Created by:Michael Zeller
Reported by:Michael Zeller
Assigned to:Michael Zeller

Resolution

Fixed: A fix for this issue is checked into the tree and tested.
(Resolution Date: 2020-03-24T18:20:24.829Z)

Fix Versions

2020-03-26 Oh Henry (Release Date: 2020-03-26)

Related Links

Labels

bhyve

Description

This is a follow up to OS-8117. This pulls in an initially skipped userland diff for pci passthru.
The diff comes from FreeBSD commit 8e4b03df46ab (r348779).

Additionally we want to grab https://github.com/freebsd/freebsd/commit/bdad744823ebf970815d691c9ec71236f6f0d90b ahead of the next re-sync as the commit is directly related to r348779.

Comments

Comment by Michael Zeller
Created at 2020-03-24T18:09:52.049Z

To test this change I booted a debug platform with the patch on an intel NUC.

I setup loader.conf.local to look like:

[root@hollownest ~]# cat /opt/local/bin/mount-usb^C
[root@hollownest ~]# cat /usb
usbkey/   usbstick/
[root@hollownest ~]# cat /usbstick/boot/loader.conf.local
ppt_aliases_load=YES
ppt_aliases_type=file
ppt_aliases_name=/overlay/etc/ppt_aliases
ppt_aliases_flags="name=/etc/ppt_aliases"

ppt_matches_load=YES
ppt_matches_type=file
ppt_matches_name=/overlay/etc/ppt_matches
ppt_matches_flags="name=/etc/ppt_matches"

With each file looking like:

[root@hollownest ~]# cat /usbstick/overlay/etc/ppt_matches
pci8086,390a
[root@hollownest ~]# cat /usbstick/overlay/etc/ppt_aliases
ppt "/pci@0,0/pci8086,a118@1d/pci8086,390a@0"

This sets aside the nvme device in the NUC for use with a bhyve VM.

[root@hollownest ~]# pptadm list -a -o dev,vendor,device,path
DEV        VENDOR DEVICE PATH
/dev/ppt0  8086   f1a5   /pci@0,0/pci8086,a118@1d/pci8086,390a@0

Using the above device I updated the VM with the following json payload:

{
  "pci_devices": [
    {
      "path": "/devices/pci@0,0/pci8086,a118@1d/pci8086,390a@0",
      "pci_slot": "0:8:0"
    }
  ]
}

Inside the VM I provisioned an ext4 filesystem and wrote some data to a file.

[root@deepnest ~]# lsblk /dev/nvme1n1 -f
NAME FSTYPE FSVER LABEL UUID                                 FSAVAIL FSUSE% MOUNTPOINT
nvme1n1
│
└─nvme1n1p1
     ext4   1.0         280d8aaf-b635-4c38-8c4b-46d20edc8e7e  221.8G     0% /ppt
[root@deepnest ~]# echo hello > /ppt/testfile
[root@deepnest ~]# cat /ppt/testfile
hello

Additionally I had sjorge in the community test the patch on his hardware. He was able to passthrough the previously problematic NIC with multiple BARs to his Windows VM and everything worked as expected.


Comment by Jira Bot
Created at 2020-03-24T18:19:01.088Z

illumos-joyent commit 83d6c1c819dce16c1f12a8ae9f0d8319900f4695 (branch master, by Michael Zeller)

OS-8143 Keep the shadow PCIR_COMMAND synced with the real one for pass through.

Reviewed by: Hans Rosenfeld <hans.rosenfeld@joyent.com>
Reviewed by: Jason King <jason.king@joyent.com>
Reviewed by: Jorge Schrauwen <jorge@blackdot.be>
Approved by: Jason King <jason.king@joyent.com>