OS-5192: need faster clock_gettime

Details

Issue Type:Improvement
Priority:4 - Normal
Status:Resolved
Created at:2016-02-29T18:37:39.000Z
Updated at:2016-05-17T16:23:11.000Z

People

Created by:Patrick Mooney [X]
Reported by:Patrick Mooney [X]
Assigned to:Patrick Mooney [X]

Resolution

Fixed: A fix for this issue is checked into the tree and tested.
(Resolution Date: 2016-05-17T16:23:11.000Z)

Fix Versions

2016-05-26 Pootie-Poot (Release Date: 2016-05-26)

Description

It seems that a number of micro-benchmarks abusing clock_gettime as part of metric collection. This often causes the results to speak more about the syscall expense of clock_gettime rather than the component being measured. In order for native SmartOS and LX to perform adequately, a VDSO or comm-page like mechanism is needed.

Comments

Comment by Patrick Mooney [X]
Created at 2016-04-06T17:03:23.000Z

With a rough draft of the functionality created, I ran some performance tests using this benchmark logic:

#include <stdio.h>
#include <unistd.h>
#include <malloc.h>
#include <sys/time.h>

#define TESTSAMPLES     100000
#define TESTTYPE        CLOCK_MONOTONIC

int main(int argc, char* argv[])
{
        int i;
        struct timespec *tp;

        tp = malloc(sizeof (struct timespec) * TESTSAMPLES);

        clock_gettime(TESTTYPE, &tp[0]);
        for (i = 0; i < TESTSAMPLES; i++) {
                clock_gettime(TESTTYPE, &tp[i]);
        }


        for (i = 1; i < TESTSAMPLES; i++) {
                time_t diff = tp[i].tv_nsec - tp[i-1].tv_nsec;
                if (tp[i].tv_sec > tp[i-1].tv_sec) {
                        diff += NANOSEC;
                }
                printf("%ld\n", diff);
        }
}

The test machine was a 12C 3ghz Xeon practically free of workload.

Original CLOCK_MONOTONIC 32-bit

           value  ------------- Distribution ------------- count
              64 |                                         0
             128 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@  97350
             256 |@                                        2437
             512 |                                         15
            1024 |                                         1
            2048 |                                         129
            4096 |                                         67
            8192 |                                         0

AVG: 251.86465864658646

Original CLOCK_MONOTONIC 64-bit

           value  ------------- Distribution ------------- count
              64 |                                         0
             128 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 99531
             256 |                                         72
             512 |                                         3
            1024 |                                         0
            2048 |                                         164
            4096 |                                         229
            8192 |                                         0

AVG: 227.6469864698647

Original CLOCK_REALTIME 32-bit

           value  ------------- Distribution ------------- count
              64 |                                         0
             128 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@  97350
             256 |@                                        2437
             512 |                                         15
            1024 |                                         1
            2048 |                                         129
            4096 |                                         67
            8192 |                                         0

AVG: 251.86465864658646

Original CLOCK_REALTIME 64-bit

           value  ------------- Distribution ------------- count
              64 |                                         0
             128 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@        83598
             256 |@@@@@@                                   15953
             512 |                                         42
            1024 |                                         12
            2048 |                                         306
            4096 |                                         87
            8192 |                                         0
           16384 |                                         1
           32768 |                                         0

AVG: 295.33428334283343

Comment by Patrick Mooney [X]
Created at 2016-04-06T19:55:54.000Z

Updated CLOCK_MONOTONIC 32-bit

           value  ------------- Distribution ------------- count
              32 |                                         0
              64 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 99965
             128 |                                         24
             256 |                                         10
             512 |                                         0

AVG: 79.00284002840029

Update CLOCK_MONOTONIC 64-bit

           value  ------------- Distribution ------------- count
              16 |                                         0
              32 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 99748
              64 |                                         235
             128 |                                         14
             256 |                                         2
             512 |                                         0

AVG: 56.34843348433484

Updated CLOCK_REALTIME 32-bit

           value  ------------- Distribution ------------- count
              32 |                                         0
              64 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 99977
             128 |                                         13
             256 |                                         2
             512 |                                         7
            1024 |                                         0

AVG: 70.37834378343783

Updated CLOCK_REALTIME 64-bit

           value  ------------- Distribution ------------- count
              16 |                                         0
              32 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 99960
              64 |                                         24
             128 |                                         6
             256 |                                         8
             512 |                                         1
            1024 |                                         0

AVG: 55.62782627826278

Comment by Patrick Mooney [X]
Created at 2016-04-06T20:16:21.000Z

A test of the Linux clock_gettime (granted, on different hardware) shows that there's still a need for improvement.

           value  ------------- Distribution ------------- count
               4 |                                         0
               8 |@@@@@@@@@@@@@@@@@@@@@@@@@                62312
              16 |@@@@@@@@@@@@@@@                          37644
              32 |                                         40
              64 |                                         0
             128 |                                         1
             256 |                                         0
             512 |                                         0
            1024 |                                         1
            2048 |                                         0
            4096 |                                         0
            8192 |                                         1
           16384 |                                         0

AVG: 15.074720747207472

Comment by Bot Bot [X]
Created at 2016-05-16T21:23:08.000Z

illumos-joyent commit d19cb13 (branch master, by Patrick Mooney)

OS-5192 need faster clock_gettime
Reviewed by: Jerry Jelinek <jerry.jelinek@joyent.com>
Reviewed by: Robert Mustacchi <rm@joyent.com>
Reviewed by: Joshua M. Clulow <jmc@joyent.com>
Reviewed by: Ryan Zezeski <ryan@zinascii.com>