OS-6836: bhyve Assertion failed: mr->name == memp->name, file mem.c, line 265


Issue Type:Bug
Priority:4 - Normal
Created at:2018-03-26T18:28:16.105Z
Updated at:2018-03-27T16:32:44.209Z


Created by:Former user
Reported by:Former user

Related Issues




sjorge had a bhyve instance (not in a zone) trip over an assertion every time he tries to boot openbsd.

> ::status
debugging core file of bhyve (64-bit) from carbon
initial argv: bhyve -m 1g -c 1 -l com1,stdio -P -H -s 1,lpc -s3,virtio-blk,/zones/images/inst
threading model: native threads
status: process terminated by SIGABRT (Abort), pid=15154 uid=0 code=-1
panic message: Assertion failed: mr->name == memp->name, file mem.c, line 265
> $C
fffffd7fedfce7d0 libc.so.1`_lwp_kill+0xa()
fffffd7fedfce800 libc.so.1`raise+0x20(6)
fffffd7fedfce850 libc.so.1`abort+0x98()
fffffd7fedfceaa0 0xfffffd7fef22109a()
fffffd7fedfceaf0 unregister_mem+0x6b(fffffd7fedfceb00)
fffffd7fedfcebc0 modify_bar_registration+0x178(677900, 1, 0)
fffffd7fedfcebe0 unregister_bar+0x17(677900, 1)
fffffd7fedfcec60 pci_emul_cmdsts_write+0x178(677900, 4, 10001c, 4)
fffffd7fedfcece0 pci_cfgrw+0x2f0(66a4c0, 0, 0, 0, 4, 0)
fffffd7fedfced80 pci_emul_cfgdata+0xe5(66a4c0, 0, 0, cfc, 4, fffffd7fedfcee3c)
fffffd7fedfcee90 emulate_inout+0x3be(66a4c0, 0, 4600c0, 0)
fffffd7fedfceee0 vmexit_inout+0x54(66a4c0, 4600c0, fffffd7fedfceefc)
fffffd7fedfcef60 vm_loop+0xcf(66a4c0, 0, fff0)
fffffd7fedfcefb0 fbsdrun_start_thread+0x4f(663b00)
fffffd7fedfcefe0 libc.so.1`_thrp_setup+0x8a(fffffd7fef114a40)
fffffd7fedfceff0 libc.so.1`_lwp_start()
> fffffd7fedfceb00::print struct mem_range
    name = 0x677913 "virtio-net-viona-pci-4"
    flags = 0
    handler = 0
    arg1 = 0
    arg2 = 0
    base = 0
    size = 0x2000
> fffffd7fffdffcb0::dump -l 0t164
                   \/ 1 2 3  4 5 6 7  8 9 a b  c d e f  v123456789abcdef
fffffd7fffdffcb0:  62687976 65002d6d 00316700 2d630031  bhyve.-m.1g.-c.1
fffffd7fffdffcc0:  002d6c00 636f6d31 2c737464 696f002d  .-l.com1,stdio.-
fffffd7fffdffcd0:  50002d48 002d7300 312c6c70 63002d73  P.-H.-s.1,lpc.-s
fffffd7fffdffce0:  332c7669 7274696f 2d626c6b 2c2f7a6f  3,virtio-blk,/zo
fffffd7fffdffcf0:  6e65732f 696d6167 65732f69 6e737461  nes/images/insta
fffffd7fffdffd00:  6c6c3632 2e667300 2d730034 2c766972  ll62.fs.-s.4,vir
fffffd7fffdffd10:  74696f2d 6e65742d 76696f6e 612c6f62  tio-net-viona,ob
fffffd7fffdffd20:  73643100 2d6c0062 6f6f7472 6f6d2c2f  sd1.-l.bootrom,/
fffffd7fffdffd30:  7573722f 73686172 652f6268 7976652f  usr/share/bhyve/
fffffd7fffdffd40:  75656669 2d63736d 2d726f6d 2e62696e  uefi-csm-rom.bin
fffffd7fffdffd50:  006f6273 64363200 4d414e50 4154483d  .obsd62.MANPATH=

From that, the command line is:

bhyve -m 1g -c 1 -l com1,stdio -P -H -s 1,lpc \
    -s3,virtio-blk,/zones/images/install62.fs -s 4,virtio-net-viona,obsd1 \
    -l bootrom,/usr/share/bhyve/uefi-csm-rom.bin obsd62


Comment by Former user
Created at 2018-03-26T18:34:16.663Z
$ thoth upload core.bhyve.15154 
thoth: using database at (configured from Manta)
thoth: creating 276baea08d4775f0c75cb5c3273e762a

Comment by Former user
Created at 2018-03-26T19:26:34.813Z

Reproduced easily with:

[root@buglets ~]# imgadm import 003f435e-3f5b-11e7-9078-9bf7d2ba26fb
Importing 003f435e-3f5b-11e7-9078-9bf7d2ba26fb (openbsd-6@20170523) from "https://images.joyent.com"
Gather image 003f435e-3f5b-11e7-9078-9bf7d2ba26fb ancestry
Must download and install 1 image (214.9 MiB)
Download 1 image      [=============================>] 100% 214.92MB   4.53MB/s    47s
Downloaded image 003f435e-3f5b-11e7-9078-9bf7d2ba26fb (214.9 MiB)
...-9078-9bf7d2ba26fb [=============================>] 100% 214.92MB  19.04MB/s    11s
Imported image 003f435e-3f5b-11e7-9078-9bf7d2ba26fb (openbsd-6@20170523)
[root@buglets ~]# zfs clone zones/003f435e-3f5b-11e7-9078-9bf7d2ba26fb@final zones/openbsd
[root@buglets ~]# dladm create-vnic -l e1000g0 obsd0
[root@buglets ~]# bhyve -m 1g -c 1 -l com1,stdio -P -H -s 1,lpc -s 3,virtio-blk,/dev/zvol/rdsk/zones/openbsd -s 4,virtio-net-viona,obsd0 -l bootrom,/usr/share/bhyve/uefi-csm-rom.bin obsd
booting hd0a:/bsd: 7057136+2225160+262496+0+675840 [72+718560+484603]=0xae6ea8
entry point at 0x1001000 [7205c766, 34000004, 24448b12, 81c0a304]
[ using 1203880 bytes of bsd ELF symbol table ]
Copyright (c) 1982, 1986, 1989, 1991, 1993
	The Regents of the University of California.  All rights reserved.
Copyright (c) 1995-2017 OpenBSD. All rights reserved.  https://www.OpenBSD.org

OpenBSD 6.1 (GENERIC.MP) #20: Sat Apr  1 13:45:56 MDT 2017
real mem = 1034493952 (986MB)
avail mem = 998522880 (952MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 3.0 @ 0x3fb6d000 (9 entries)
bios0: vendor BHYVE version "1.00" date 03/14/2014
bios0: bhyve BHYVE
acpi0 at bios0: rev 2
acpi0: sleep states S5
acpi0: wakeup devices
acpitimer0 at acpi0: 3579545 Hz, 32 bits
acpihpet0 at acpi0: 10000000 Hz
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz, 2904.09 MHz
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: TSC frequency 2904086610 Hz
cpu0: smt 0, core 0, package 0
mtrr: CPU supports MTRRs but not enabled by BIOS
cpu0: apic clock running at 134MHz
ioapic0 at mainbus0: apid 1 pa 0xfec00000, version 11, 24 pins
acpimcfg0 at acpi0 addr 0xe0000000, bus 0-255
acpiprt0 at acpi0: bus 0 (PC00)
"PNP0303" at acpi0 not configured
"PNP0F03" at acpi0 not configured
"PNP0501" at acpi0 not configured
"PNP0501" at acpi0 not configured
pvbus0 at mainbus0: bhyve
pci0 at mainbus0 bus 0
0:3:0: mem address conflict 0xc0000000/0x2000
0:4:0: mem address conflict 0xc0002000/0x2000
Assertion failed: mr->name == memp->name, file mem.c, line 265
                                                              Abort (core dumped)
[root@buglets ~]# 

Comment by Former user
Created at 2018-03-27T16:08:14.597Z

This seems not to be unique to SmartOS. The same symptoms (two mem address conflict errors followed by crash) are also reported on FreeBSD.

Comment by Former user
Created at 2018-03-27T16:32:44.209Z

Both @accountid:62561aa04f1d57006a24d403 and I found it odd that an assertion was being tripped at removal (unregister_mem()) rather than insertion (register_mem()). See OS-6841.