OS-7975: bhyve should require exactly one primary nic

Details

Issue Type:Bug
Priority:4 - Normal
Status:Resolved
Created at:2019-08-29T21:11:54.050Z
Updated at:2019-10-07T16:56:52.073Z

People

Created by:Former user
Reported by:Former user
Assigned to:Former user

Resolution

Fixed: A fix for this issue is checked into the tree and tested.
(Resolution Date: 2019-09-05T21:25:01.483Z)

Labels

bhyve

Description

If a bhyve instance has no NIC that is the primary NIC, no NIC is seen by the guest.

In usr/src/lib/brnad/bhyve/zone.c:

398                 / Allow at most one "primary" net /
399                 if (is_env_true("net", net, "primary")) {
400                         if (primary != NULL) {
401                                 (void) printf("Error: "
402                                     "multiple primary nets: %s %s\n",
403                                     primary, net);
404                                 return (-1);
405                         }
406                         primary = net;
407                         pcifn = 0;
408                 } else {
409                         pcifn = nextpcifn;
410                         nextpcifn++;
411                 }
111

All of the non-primary NICs are assigned to PCI function 1 - 7. If there is no primary NIC, no NIC gets assigned to function 0. In order for the guest OS to enumerate functions 1 - 7, function 0 must be present.

Comments

Comment by Former user
Created at 2019-09-03T07:50:18.326Z

I tried to keep the changes as small as possible. With this change, if there is no nic marked as primary, the last nic will be assigned function 0. This solution has one caveat, for instances with no primary nic, every time a nic is added to an instance, it will show up in function 0. In general, that should not be an issue for bhyve instances running production as of now. Since we are hitting this bug, we know that all production bhyve instances do have an interface tagged as primary.

Here is how I tested these changes:


Comment by Former user
Created at 2019-09-04T04:55:29.816Z

The code has been updated as suggested by Mike. Here is what has been tested:

{"log":"add_arg: argv[25]='6:1,virtio-net-viona,net0'\n","stream":"zoneadmd","time":"2019-09-04T04:37:00.302767000Z"}
{"log":"Error: no primary net has been specified\n","stream":"zoneadmd","time":"2019-09-04T04:37:00.302779000Z"}
{"log":"add_arg: argv[25]='6:1,virtio-net-viona,net0'\n","stream":"zoneadmd","time":"2019-09-04T04:41:33.771487000Z"}
{"log":"get_zcfg_var: '_ZONECFG_net_net1_primary=<null>'\n","stream":"zoneadmd","time":"2019-09-04T04:41:33.771499000Z"}
{"log":"add_arg: argv[26]='-s'\n","stream":"zoneadmd","time":"2019-09-04T04:41:33.771510000Z"}
{"log":"add_arg: argv[27]='6:2,virtio-net-viona,net1'\n","stream":"zoneadmd","time":"2019-09-04T04:41:33.771522000Z"}
{"log":"Error: no primary net has been specified\n","stream":"zoneadmd","time":"2019-09-04T04:41:33.771533000Z"}

Comment by Former user
Created at 2019-09-05T17:51:37.077Z

Updated smartos-live repositories to the latest gmake update

[root@build01 ~/smartos-live/output]# cat gitstatus.json
[
    {
        "repo": "smartos-live",
        "branch": "master",
        "commit_date": "1567118453",
        "rev": "7eb0eafe29887ff7ba5488f083304e8f6cb0382e",
        "url": "git://github.com/joyent/smartos-live"
    },
    {
        "repo": "illumos-joyent",
        "branch": "master",
        "commit_date": "1567693274",
        "rev": "65294ed2cdc03ab5daf93b7ad97f0683063b1e01",
        "url": "https://github.com/joyent/illumos-joyent.git"
    },
    {
        "repo": "illumos-extra",
        "branch": "master",
        "commit_date": "1567438874",
        "rev": "e2b3be80e2c9ecb3a9fa1a9448621b1190b21823",
        "url": "https://github.com/joyent/illumos-extra.git"
    },
    {
        "repo": "kvm",
        "branch": "master",
        "commit_date": "1567438869",
        "rev": "529dae2817a28d9f46759acceb6d9488b8b05d94",
        "url": "https://github.com/joyent/illumos-kvm.git"
    },
    {
        "repo": "kvm-cmd",
        "branch": "master",
        "commit_date": "1566942834",
        "rev": "41e7c3e192eb39bb5ced7667fdc12ad2b33e646d",
        "url": "https://github.com/joyent/illumos-kvm-cmd.git"
    },
    {
        "repo": "mdata-client",
        "branch": "master",
        "commit_date": "1482175115",
        "rev": "dbb3fdec26373f8e86c9ea0057ebba48581ff476",
        "url": "https://github.com/joyent/mdata-client.git"
    },
    {
        "repo": "ur-agent",
        "branch": "master",
        "commit_date": "1552929732",
        "rev": "476bac8f579eb60bf9c6ba6e0c27d6da1076a705",
        "url": "https://github.com/joyent/sdc-ur-agent.git"
    }
]
[root@build01 ~/smartos-live/output]# cd ../projects/illumos
[root@build01 ~/smartos-live/projects/illumos]# git show 65294ed2cdc03ab5daf93b7ad97f0683063b1e01
commit 65294ed2cdc03ab5daf93b7ad97f0683063b1e01 (HEAD -> master)
Author: Mohamed Khalfella <mohamed.khalfella@joyent.com>
Date:   Thu Sep 5 04:13:13 2019 +0000

    OS-7975 bhyve should require exactly one primary nic

diff --git a/usr/src/lib/brand/bhyve/zone/boot.c b/usr/src/lib/brand/bhyve/zone/boot.c
index 6e6f3dc640..51aab83daf 100644
--- a/usr/src/lib/brand/bhyve/zone/boot.c
+++ b/usr/src/lib/brand/bhyve/zone/boot.c
@@ -10,7 +10,7 @@
  */

 /*
- * Copyright (c) 2018, Joyent, Inc.
+ * Copyright (c) 2019, Joyent, Inc.
  */

 /*
@@ -382,7 +382,8 @@ add_nets(int *argc, char **argv)
        char slotconf[MAXNAMELEN];
        char *primary = NULL;

-       if ((nets = get_zcfg_var("net", "resources", NULL)) == NULL) {
+       if ((nets = get_zcfg_var("net", "resources", NULL)) == NULL ||
+           strcmp(nets, "") == 0) {
                return (0);
        }

@@ -423,6 +424,12 @@ add_nets(int *argc, char **argv)
                }
        }

+       /* Make sure there is a "primary" net */
+       if (primary == NULL) {
+               (void) printf("Error: no primary net has been specified\n");
+               return (-1);
+       }
+
        return (0);
 }

[root@build01 ~/smartos-live/projects/illumos]#

Tested:


Comment by Jira Bot
Created at 2019-09-05T21:14:10.960Z

illumos-joyent commit 1cc204b97b9317e681958da0e91abeb27bcc6f82 (branch master, by Mohamed Khalfella)

OS-7975 bhyve should require exactly one primary nic
Reviewed by: Mike Gerdts <mike.gerdts@joyent.com>
Approved by: Mike Gerdts <mike.gerdts@joyent.com>