OS-8334: Bring OpenSSL 3 to the platform

Details

Issue Type:Improvement
Priority:4 - Normal
Status:Resolved
Created at:2021-11-10T19:20:18.526Z
Updated at:2021-12-15T16:18:21.401Z

People

Created by:Dan McDonald
Reported by:Dan McDonald

Resolution

Fixed: A fix for this issue is checked into the tree and tested.
(Resolution Date: 2021-12-15T16:18:21.395Z)

Fix Versions

2021-12-16 Hideo (Release Date: 2021-12-16)

Related Links

Description

This bug will track the importation of OpenSSL 3.0 into SmartOS.  Initial experiments suggest that it will NOT be able to support either platform-Node (currently v0.10) or KBMD.  Related issues will track those respectively.

Comments

Comment by Dan McDonald
Created at 2021-11-10T19:48:42.990Z

First off, the OpenSSL 3 for the platform must include OpenSSL 1.1.1 API support to allow illumos to build (confirmed by Andy Fiddaman).

The following illumos-extra deliverables have been modified or updated:


Comment by Dan McDonald
Created at 2021-11-10T19:54:19.079Z
Updated at 2021-11-11T15:27:10.753Z

Also, because of the increased size of OpenSSL 3's libraries, the ramdisk size will need a 25k bump as well.

This issue affects three repositories:


Comment by Dan McDonald
Created at 2021-11-12T21:35:54.786Z

Test request:  use platform curl:

smartos-build-2(~)[0]% curl --version
curl 7.51.0 (i386-pc-solaris2.11) libcurl/7.51.0 OpenSSL/3.0.0 zlib/1.2.3 libidn2/0.11
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp smb smbs smtp smtps telnet tftp 
Features: IDN IPv6 Largefile NTLM NTLM_WB SSL libz UnixSockets 
smartos-build-2(~)[0]% which curl
/bin/curl
smartos-build-2(~)[0]% curl https://kebe.com/~danmcd/index.html | tail -5
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  5038  100  5038    0     0  19096      0 --:--:-- --:--:-- --:--:-- 19083
(<a href="mailto:danmcd@kebe.com"><tt>danmcd@kebe.com</tt></a>)
<a href="https://www.illumos.org">
<img border="0" alt="Powered by illumos" title="Powered by illumos" src="https://kebe.com/illumos-logos/Illumos-web-32px.png">
</a>
</html>
smartos-build-2(~)[0]% which mget
/export/home/danmcd/node_modules/manta/bin//mget
smartos-build-2(~)[0]% mget /Joyent_Dev/public/SmartOS/smartos-latest.iso | digest -a md5
...martos-latest.iso [=======================>] 100% 553.96MB  11.40MB/s    48s
2f7ef41c6b4e95f360e785d8d155906c
smartos-build-2(~)[0]% curl https://us-east.manta.joyent.com/Joyent_Dev/public/SmartOS/smartos-latest.iso | digest -a md5
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  553M  100  553M    0     0  6539k      0  0:01:26  0:01:26 --:--:-- 6983k
2f7ef41c6b4e95f360e785d8d155906c
smartos-build-2(~)[0]% curl https://us-east.manta.joyent.com/Joyent_Dev/public/SmartOS/smartos-latest.iso | openssl md5
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  553M  100  553M    0     0  12.0M      0  0:00:45  0:00:45 --:--:-- 14.9M
MD5(stdin)= 2f7ef41c6b4e95f360e785d8d155906c
smartos-build-2(~)[0]% 

Comment by Dan McDonald
Created at 2021-11-12T21:36:54.990Z

Test request: ipmitool:

smartos-build-2(~)[0]% ipmitool -I lanplus -H curly-ipmi -U ADMIN -P XXXX sol activate
[SOL Session operational.  Use ~? for help][root@curly (kebecloud) ~]# 
[root@curly (kebecloud) ~]# 
[root@curly (kebecloud) ~]# ~. [terminated ipmitool]
                                                    smartos-build-2(~)[0]% 
smartos-build-2(~)[0]% which ipmitool
/usr/sbin/ipmitool
smartos-build-2(~)[0]% ipmitool -I lanplus -H curly-ipmi -U ADMIN -P XXXX power status
Chassis Power is on
smartos-build-2(~)[0]% 

Comment by Dan McDonald
Created at 2021-11-12T21:41:52.243Z

Test request: imgadm (NOTE: This is a compute node in Kebecloud, not standalone smartos, nor a non-global zone like the above tests)

[root@larry (kebecloud) ~]# imgadm list | grep minimal-64-lts
5417ab20-3156-11ea-8b19-2b66f5e7a439  minimal-64-lts                  19.4.0                            smartos  zone-dataset  2020-01-07
[root@larry (kebecloud) ~]# imgadm avail | grep minimal-64-lts
c2c31b00-1d60-11e9-9a77-ff9f06554b0f  minimal-64-lts                  18.4.0                                            smartos  zone-dataset  2019-01-21
5417ab20-3156-11ea-8b19-2b66f5e7a439  minimal-64-lts                  19.4.0                                            smartos  zone-dataset  2020-01-07
[root@larry (kebecloud) ~]# imgadm import c2c31b00-1d60-11e9-9a77-ff9f06554b0f
Importing c2c31b00-1d60-11e9-9a77-ff9f06554b0f (minimal-64-lts@18.4.0) from "http://imgapi.kebecloud.work.kebe.com"
Gather image c2c31b00-1d60-11e9-9a77-ff9f06554b0f ancestry
Must download and install 1 image (27.5 MiB)
Download 1 image                  [==============================================================>] 100%  27.53MB                  
Downloaded image c2c31b00-1d60-11e9-9a77-ff9f06554b0f (27.5 MiB)
...00-1d60-11e9-9a77-ff9f06554b0f [==============================================================>] 100%  27.53MB  10.11MB/s     2s
Imported image c2c31b00-1d60-11e9-9a77-ff9f06554b0f (minimal-64-lts@18.4.0)
[root@larry (kebecloud) ~]# imgadm list | grep minimal-64-lts
c2c31b00-1d60-11e9-9a77-ff9f06554b0f  minimal-64-lts                  18.4.0                            smartos  zone-dataset  2019-01-21
5417ab20-3156-11ea-8b19-2b66f5e7a439  minimal-64-lts                  19.4.0                            smartos  zone-dataset  2020-01-07
[root@larry (kebecloud) ~]# 

Comment by Dan McDonald
Created at 2021-11-17T15:06:13.221Z

All Kebecloud nodes have been using the OS-8334 branch from Monday Nov 15, and will until the 20211118 release branches.  Lots of compiles and other normal business on the CNs.


Comment by Dan McDonald
Created at 2021-12-06T22:18:14.810Z

Kebecloud nodes again are on the OS-8334 branch, and will be until these reviews are complete.


Comment by Dan McDonald
Created at 2021-12-06T22:29:50.342Z

KBMD shows few differences.  NOTE: smartos-bios is running the current SmartOS public release, and smartos-build is running the OS-8334 bits.

smartos-build-2(~/gcc10-with-gcc7)[1]% ssh root@smartos-bios dis /usr/lib/kbm/kbmd > /tmp/before
(root@smartos-bios) Password: 
(root@smartos-bios) Password: 
smartos-build-2(~/gcc10-with-gcc7)[0]% dis /usr/lib/kbm/kbmd > /tmp/after   
smartos-build-2(~/gcc10-with-gcc7)[0]% wc -l /tmp/{before,after}
  392502 /tmp/before
  392502 /tmp/after
  785004 total
smartos-build-2(~/gcc10-with-gcc7)[0]% diff /tmp/{before,after} | wc -l
    3504

A cursory inspection shows these diffs to be global-symbol offset renamings, not even code-offset ones, just symbol ones.


Comment by Brian Bennett
Created at 2021-12-07T00:24:24.289Z

imgadm on a compute node will be configured to use the local Triton imgapi, which does not use https. We need to check this on standalone SmartOS too.


Comment by Dan McDonald
Created at 2021-12-07T04:10:35.705Z
Updated at 2021-12-07T15:47:51.670Z

So I did:

imgadm avail |& tee /zones/root/{stock,openssl}

Depending on the PI (stock == 20211202, openssl == current OpenSSL 3 branches), on a standalone SmartOS VM.

[root@smartos-bios ~]# wc -l /zones/root/{stock,openssl3}
     576 /zones/root/stock
     576 /zones/root/openssl3
    1152 total
[root@smartos-bios ~]# diff /zones/root/{stock,openssl3}
574c574
< real        1.5
---
> real        0.8
576c576
< sys         0.1
---
> sys         0.0
[root@smartos-bios ~]# 

Identical across two PIs, and the OpenSSL 3 one might even be quicker.

NOTE:  This is a regression test of sorts, because imgadm is in Node, and Node is (along with kbmd) an OpenSSL 1.0.2 user still.


Comment by Jira Bot
Created at 2021-12-07T20:19:06.137Z

illumos-extra commit 3cdfa8ec06541fc7166f63a873c5749597a96d09 (branch master, by Dan McDonald)

OS-8334 Bring OpenSSL 3 to the platform

Reviewed by: Brian Bennett <brian.bennett@joyent.com>
Reviewed by: Mike Zeller <mike.zeller@joyent.com>
Approved by: Brian Bennett <brian.bennett@joyent.com>