OS-7974: bhyve does not persist nic_driver

Details

Issue Type:Bug
Priority:4 - Normal
Status:Resolved
Created at:2019-08-29T21:03:14.467Z
Updated at:2019-09-06T20:17:45.422Z

People

Created by:Former user
Reported by:Former user
Assigned to:Former user

Resolution

Fixed: A fix for this issue is checked into the tree and tested.
(Resolution Date: 2019-09-06T20:17:45.409Z)

Labels

bhyve

Description

When a NIC is added to a machine, the model needs to be specified as vm.nics[*].model for each NIC being added or vm.nic_driver needs to be specified.

When AdminUI provisions a bhyve or kvm instance, the payload that it sends includes vm.nic_driverl, but does not specify vm.nics[*].model. That is:

{
    "uuid": "838a8d1f-0f21-4d9f-d89b-8128cec12d79",
    "image": {
      "v": 2,
      "uuid": "d6e56c0e-1e51-11e9-8ff1-2fecf38a5566",
      "owner": "9dce1460-0c4c-4417-ab8b-25ca478c5a78",
      "name": "centos-7",
      "version": "20190122",
      "state": "active",
      "disabled": false,
      "public": true,
      "published_at": "2019-01-22T14:27:25Z",
      "type": "zvol",
      "os": "linux",
      "files": [
        {
          "sha1": "eea7d4cf1c19d35464f6ed4ef2f8ccf232644cba",
          "size": 507103646,
          "compression": "gzip"
        }
      ],
      "description": "CentOS 7.5 64-bit image with just essential packages installed, built to run on KVM virtual machines. For kvm and bhyve.",
      "homepage": "https://docs.joyent.com/images/linux/centos",
      "requirements": {
        "networks": [
          {
            "name": "net0",
            "description": "public"
          }
        ],
        "ssh_key": true
      },
      "users": [
        {
          "name": "root"
        }
      ],
      "nic_driver": "virtio",
      "disk_driver": "virtio",
      "cpu_type": "host",
      "image_size": 10240,
      "tags": {
        "role": "os"
      }
    },
    "alias": "bhyve1",
    "billing_id": "16a8b78c-be87-4800-bd57-573f27f72988",
    "brand": "bhyve",
    "cpu_cap": 100,
    "cpu_shares": 256,
    "customer_metadata": {
      ...
    },
    "firewall_enabled": false,
    "internal_metadata": {
      "root_pw": "EksAcPet3Ol-"
    },
    "max_lwps": 1000,
    "max_physical_memory": 4096,
    "max_swap": 8192,
    "nics": [
      {
        "belongs_to_type": "zone",
        "belongs_to_uuid": "838a8d1f-0f21-4d9f-d89b-8128cec12d79",
        "mac": "90:b8:d0:a2:e8:1a",
        "owner_uuid": "b17e72f4-83b2-4e13-869d-e463ad3d706a",
        "primary": true,
        "state": "provisioning",
        "created_timestamp": "2019-08-29T20:41:44.114Z",
        "modified_timestamp": "2019-08-29T20:41:44.114Z",
        "ip": "165.225.170.118",
        "gateway": "165.225.168.1",
        "mtu": 1500,
        "netmask": "255.255.252.0",
        "nic_tag": "external",
        "resolvers": [
          "8.8.8.8",
          "8.8.4.4"
        ],
        "vlan_id": 3105,
        "routes": {},
        "network_uuid": "feb7b2c5-0063-42f0-a4e6-b812917397f7",
        "cn_uuid": "00000000-0000-0000-0000-0025909437d8"
      }
    ],
    "owner_uuid": "b17e72f4-83b2-4e13-869d-e463ad3d706a",
    "ram": 4096,
    "vcpus": 1,
    "zfs_io_priority": 20,
    "archive_on_delete": true,
    "resolvers": [
      "8.8.8.8",
      "8.8.4.4"
    ],
    "disks": [
      {
        "image_uuid": "d6e56c0e-1e51-11e9-8ff1-2fecf38a5566",
        "image_name": "centos-7",
        "image_size": 10240,
        "boot": true
      },
      {
        "size": 51200
      }
    ],
    "disk_driver": "virtio",
    "nic_driver": "virtio",
    "cpu_type": "host",
    "imgapiPeers": []
  }
}

There are no problems creating this instance. However, if all NICs are removed then one is added, we see that it fails with:

missing .model option for NIC: {"mac":"90:b8:d0:3f:39:50","primary":true,"mtu":1500,"nic_tag":"external","vlan_id":3105,"network_uuid":"feb7b2c5-0063-42f0-a4e6-b812917397f7","ips":["165.225.168.81/22"],"gateways":["165.225.168.1"]}

The AdminUI job included:

  "add_nics": [
    {
      "belongs_to_type": "zone",
      "belongs_to_uuid": "838a8d1f-0f21-4d9f-d89b-8128cec12d79",
      "mac": "90:b8:d0:3f:39:50",
      "owner_uuid": "b17e72f4-83b2-4e13-869d-e463ad3d706a",
      "primary": true,
      "state": "provisioning",
      "created_timestamp": "2019-08-29T20:43:00.423Z",
      "modified_timestamp": "2019-08-29T20:43:00.423Z",
      "ip": "165.225.168.81",
      "gateway": "165.225.168.1",
      "mtu": 1500,
      "netmask": "255.255.252.0",
      "nic_tag": "external",
      "resolvers": [
        "8.8.8.8",
        "8.8.4.4"
      ],
      "vlan_id": 3105,
      "routes": {},
      "network_uuid": "feb7b2c5-0063-42f0-a4e6-b812917397f7",
      "cn_uuid": "00000000-0000-0000-0000-0025909437d8"
    }
  ],

That's odd... vm.nic_driver was specified during creation so we shouldn't need to specify vm.nics[*].model. On the CN where this was provisioned, we find that it wasn't persisted.

$ vmadm get 838a8d1f-0f21-4d9f-d89b-8128cec12d79 | json -k | grep driver
  "disk_driver",

Comments

Comment by Former user
Created at 2019-08-29T21:06:29.662Z

The root cause of this is that in smartos-live/src/vm/node_modules/proptable.js:

1156     }, nic_driver: {
1157         payload: {
1158             allowed: {
1159                 'kvm': ['create', 'receive', 'update']
1160             },
1161             type: 'string'
1162         },
1163         zonexml: 'zone.attr.nic-driver'

Contrast that to disk_driver which is persisted:

 736     }, disk_driver: {
 737         payload: {
 738             allowed: {
 739                 'bhyve': ['create', 'receive', 'update'],
 740                 'kvm': ['create', 'receive', 'update']
 741             },
 742             type: 'string'
 743         },
 744         zonexml: 'zone.attr.disk-driver'

We need need to add the bhyve line to nic_driver.payload.allowed.


Comment by Former user
Created at 2019-09-06T07:47:44.291Z

Per the analysis of Mike, I added the missing bhyve line. Here is the diff:

diff --git a/src/vm/node_modules/proptable.js b/src/vm/node_modules/proptable.js
index d9b7fdec..9042627d 100644
--- a/src/vm/node_modules/proptable.js
+++ b/src/vm/node_modules/proptable.js
@@ -1156,6 +1156,7 @@ exports.properties = {
     }, nic_driver: {
         payload: {
             allowed: {
+                'bhyve': ['create', 'receive', 'update'],
                 'kvm': ['create', 'receive', 'update']
             },
             type: 'string'

Here is how I tested this change:

# vmadm get 2652235b-5abb-e4dc-d5dd-aa72c2706c20 | json nic_driver ## before changes
# vmadm get 1fcf77a6-d8f5-e4d9-aa05-b554519967a7 | json nic_driver ## after changes
virtio
#
# echo '{ "nic_driver" : "virtio" }' | vmadm update 2652235b-5abb-e4dc-d5dd-aa72c2706c20
Successfully updated VM 2652235b-5abb-e4dc-d5dd-aa72c2706c20
# vmadm get 2652235b-5abb-e4dc-d5dd-aa72c2706c20 | json nic_driver
virtio
#
# vmadm update 2652235b-5abb-e4dc-d5dd-aa72c2706c20 <<EOF
> {
> "add_nics": [
>     {
>       "belongs_to_type": "zone",
>       "belongs_to_uuid": "2652235b-5abb-e4dc-d5dd-aa72c2706c20",
>       "mac": "90:b8:d0:bd:cf:8f",
>       "owner_uuid": "49aad7bc-b8c7-4dc4-802f-c87f0479176f",
>       "primary": true,
>       "state": "provisioning",
>       "ip": "64.30.138.144",
>       "gateway": "64.30.138.129",
>       "mtu": 1500,
>       "netmask": "255.255.255.224",
>       "nic_tag": "external",
>       "resolvers": [
>         "8.8.8.8",
>         "4.2.2.2"
>       ],
>       "vlan_id": 1120,
>       "network_uuid": "5220e7d4-ddcc-451c-bc07-6a5fd0be6e37",
>       "cn_uuid": "3585428c-11d9-11e4-ad8f-002590e4f0ac"
>     }
>   ]
> }
> EOF
Successfully updated VM 2652235b-5abb-e4dc-d5dd-aa72c2706c20
#

Comment by Jira Bot
Created at 2019-09-06T20:13:05.826Z

smartos-live commit b2222a797cfd65ee44b224ea90f3317725396253 (branch master, by Mohamed Khalfella)

OS-7974 bhyve does not persist nic_driver
Reviewed by: Mike Gerdts <mike.gerdts@joyent.com>
Approved by: Mike Gerdts <mike.gerdts@joyent.com>