Issue Type: | Bug |
---|---|
Priority: | 4 - Normal |
Status: | Resolved |
Created at: | 2018-05-16T21:09:41.606Z |
Updated at: | 2018-10-24T18:44:32.917Z |
Created by: | Ryan Kitchen |
---|---|
Reported by: | Ryan Kitchen |
Assigned to: | Former user |
Fixed: A fix for this issue is checked into the tree and tested.
(Resolution Date: 2018-10-19T19:19:03.349Z)
2018-10-25 Dot Com (Release Date: 2018-10-25)
Enabling the cloud firewall does not appear to affect Bhyve instances. More details can be found here: https://jira.joyent.us/browse/QA-262
When the firewall is enabled an LX host blocks traffic as expected, and honors additonal rules.
When the firewall is enabled on bhyve the default rules do not take effect, and rules to block specific traffic also do not block it.
In order for bhyve to hook into cloud firewall we need to implement firewall hook support into viona; much like was done for KVM/VND. We can probably use vnd_hook()
as a reference implementation – though the details may differ. I'm not sure how long this will take but it's my opinion that the "smart checksum" and LSO work is higher priority as it should benefit both inter and intra-host network performance greatly, and the user can also use iptables inside the guest until we implement cloud firewall support.
I have a working solution, updated to the viona-ipf
branch on GitHub. The consensus is that it will be more beneficial to wait for OS-2340, OS-6778, OS-6794, and OS-6884 to integrate (all are currently under review) before submitting this code for review (a minor amount of tweaking will be required after that, but should be fairly minimal) because of the robustness improvements to the viona driver they include.
For testing (this was all in a SmartOS instance running under Vmware fusion), I created two bhyve instances running 56b9596c-a3bf-11e8-b5c9-0f79b1e762c5 (centos-7@20180819) whose network interfaces were created on an etherstub.
I started with a default set of rules:
[root@viona ~]# fwadm list UUID ENABLED RULE 2bad8e10-4be9-439c-ab53-639f687f3563 true FROM any TO all vms ALLOW icmp TYPE all 5cefcd83-f170-4799-8601-1c9c9c9e2d40 true FROM any TO all vms ALLOW tcp PORT 22 c22c3437-09ee-46bf-a989-6dfc0fb70c4b true FROM any TO all vms ALLOW icmp6 TYPE all
I then started pings from each VM to the other VM and verified that both pings were successful. I then disabled the ICMP rule via fwadm disable 2bad8e10-4be9-439c-ab53-639f687f3563
. I then verified that the pings stopped between the instances. I then re-enabled the rule, and saw that the pings then resumed.
I then tried to scp a 2.2Mb file (vim) from one instance to the other with 5cefcd83-f170-4799-8601-1c9c9c9e2d40
enabled, which was successful. I then disabled 5cefcd83-f170-4799-8601-1c9c9c9e2d40
and tried to scp again (which failed with a connection timeout after a while).
I then re-enabled the rule and tried it again (which succeeded).
We've also setup a test rig w/ two CentOS VMs (both were given a tag of 'test').
They were created with cloud firewall disabled.
Running iperf3:
[ ID] Interval Transfer Bandwidth Retr [ 4] 0.00-10.00 sec 2.52 GBytes 2.16 Gbits/sec 552 sender [ 4] 0.00-10.00 sec 2.51 GBytes 2.16 Gbits/sec receiver
Once cloud firewall was enabled, my existing ssh sessions to the two VMs died and attempting to run iperf3 failed (as expected). Then rules for SSH as well as iperf3 were added:
fwadm add -g -e 'FROM any TO all vms ALLOW tcp PORT 22' fwadm add -g -e 'FROM any to tag "test" ALLOW tcp PORT 5201' fwadm add -g -e 'FROM any to tag "test" ALLOW udp PORT 5201'
And ssh connections were again successful, and iperf3 was able to run (note this run was for longer -- 60s):
[ ID] Interval Transfer Bandwidth Retr [ 4] 0.00-60.00 sec 13.9 GBytes 1.99 Gbits/sec 5764 sender [ 4] 0.00-60.00 sec 13.9 GBytes 1.99 Gbits/sec receiver
Letting iperf3 run for a bit over two hours (restarting after every 10s run) w/ the above rules in place:
Average 1.79 Gbits/s Stddev=0.16 Time=8220s (137m)
illumos-joyent commit 504b3e4c2ecff2fc5ce62559b4ba11c33e33027c (branch master, by Jason King)
OS-6964 Cloud Firewall not affecting bhyve Instances
Reviewed by: Ryan Zezeski <rpz@joyent.com>
Reviewed by: Patrick Mooney <patrick.mooney@joyent.com>
Approved by: Patrick Mooney <patrick.mooney@joyent.com>