OS-7338: vmadm support for flexible disk space


Issue Type:Improvement
Priority:4 - Normal
Created at:2018-10-31T15:56:03.059Z
Updated at:2018-11-12T12:53:38.479Z


Created by:Former user
Reported by:Former user
Assigned to:Former user


Fixed: A fix for this issue is checked into the tree and tested.
(Resolution Date: 2018-11-07T18:02:06.893Z)

Fix Versions

2018-11-08 EGOT (Release Date: 2018-11-08)


As described in RFD 154, VM.flexible_disk_size should be (optionally) used to to calculate the zfs quota and reservation on bhyve zones' datasets.


The fix for TRITON-243 changed the base quota (VM.quota) on the zone's dataset from 10 GiB to 1 GiB. Further, it ensures that VM.quota is reserved for the zone's dataset and that it can write no more than VM.quota to that dataset. This is good because we remove one aspect of overprovisioning from the system. The downside is that zfs snapshot -r zones/$uuid@$snapname (see OS-7182) now returns ENOSPC because there is not enough unreserved space. With the constraint that we must not overprovision, that is the right thing to do.

RFD 154 is intended to address this very problem by allowing a flexible disk space instance to have space that is allocated to the instance that is not allocated to any disk.  Such space is available for snapshots, growth of existing disks, or creation of new disks.  A proof of concept implementation of flexible_disk_size has proven to be compatible with OS-7182 and TRITON-243.


The value of the zfs quota and reservation properties should be calculated as follows:

zfs.reservation = zfs.quota = VM.quota + VM.flexible_disk_size + DISK_RESV - DISK_SIZE

Where DISK_SIZE and DISK_RESV are defined as:

VariableValue is sum of values returned by
DISK_SIZEzfs list -Hpr -t volume -o volsize zones/$UUID
DISK_RESVzfs list -Hpr -t volume -o refreserv zones/$UUID


Comment by Former user
Created at 2018-11-07T17:19:56.847Z

I tested the fix by running the tests (including several new ones) in /usr/vm/test. That all looked good.  I also verified that by bumping the flexible_disk_size up that snapshots are possible via zfs snapshot and imgadm create.

[root@buglets ~]# zfs list -o name,quota,refquota,refreservation,written,available -r -t all zones/$uuid
NAME                                              QUOTA  REFQUOTA  REFRESERV  WRITTEN  AVAIL
zones/ac685e55-5b81-6f19-bb31-a6b87dae44a8        11.3G        1G         1G      75K  1024M
zones/ac685e55-5b81-6f19-bb31-a6b87dae44a8/disk0      -         -      10.3G     103M  10.2G
[root@buglets ~]# zfs snapshot -r zones/$uuid@1
cannot create snapshot 'zones/ac685e55-5b81-6f19-bb31-a6b87dae44a8@1': out of space
cannot create snapshot 'zones/ac685e55-5b81-6f19-bb31-a6b87dae44a8/disk0@1': out of space
no snapshots were created
[root@buglets ~]# vmadm update $uuid flexible_disk_size=$((15 * 1024))
Successfully updated VM ac685e55-5b81-6f19-bb31-a6b87dae44a8
[root@buglets ~]# zfs snapshot -r zones/$uuid@1
[root@buglets ~]# zfs destroy -r zones/$uuid@1
[root@buglets ~]# imgadm create -i $uuid name=centos6 version=test2
Inheriting from origin image 11891f44-a374-11e8-9c35-c7bfcc724881 (centos-6 20180819)
      "v": 2,
      "uuid": "63acd585-fa0a-4acb-a772-da67b2fdc1a5",
      "name": "centos6",
      "version": "test2",
      "type": "zvol",
      "os": "linux",
      "requirements": {
        "networks": [
            "name": "net0",
            "description": "public"
        "ssh_key": true,
        "brand": "bhyve"
      "users": [
          "name": "root"
      "nic_driver": "virtio",
      "disk_driver": "virtio",
      "cpu_type": "host",
      "image_size": 10240,
      "origin": "11891f44-a374-11e8-9c35-c7bfcc724881"
Snapshotting to "zones/ac685e55-5b81-6f19-bb31-a6b87dae44a8/disk0@final"
Sending image file to "centos6-test2.zvol"
Saving manifest to "centos6-test2.imgmanifest"
[root@buglets ~]#

Comment by Jira Bot
Created at 2018-11-07T18:01:52.859Z

smartos-live commit 6927a12fec8a30bb041d5c79906d884d282c0dfe (branch master, by Mike Gerdts)

OS-7338 vmadm support for flexible disk space
Reviewed by: Pedro Palazón Candel <pedro@joyent.com>
Reviewed by: Dylan Yep <dylan.yep@joyent.com>
Approved by: Dylan Yep <dylan.yep@joyent.com>