ZAPI-671: Want early validation of special tags

Resolution

Fixed: A fix for this issue is checked into the tree and tested.
(Resolution Date: 2017-10-27T03:55:14.000Z)

Description

When setting tags that have special internal significance (eg for the new TCNS service), their values should be validated to ensure they conform to the structure required.

We also likely want to restrict the namespaces that are reserved for these special uses so that if you attempt to set a tag under one of them that is not recognised, your update request is rejected.

In particular, right now we want:
Also, for docker, all of these should be restricted namespaces (they are reserved docker labels, which are mapped to sdc tags by prefixing with docker:label:):
We don't currently support any validated tags under these docker namespaces, so any matching tags should return an error along the lines of "Special tag X not supported" instead of being set.

sdc-docker should also check for the triton.* special namespace in labels and either translate them directly to tags without the prefix, or else VMAPI should support aliasing triton.* tags as docker:label:triton.*.

This should be done in an extensible manner to make sure we can easily add new things later, and we should be careful to make certain we return useful errors that get propagated back to a CloudAPI user using the node-smartdc tools.

Comments

Comment by Bot Bot [X]
Created at 2015-10-14T11:45:01.000Z

sdc-vmapi commit a328bf7 (branch master, by Marsell Kukuljevic)

ZAPI-671 - add validation of tags for triton and docker.


Comment by Angela Fong
Created at 2015-10-15T00:16:02.000Z
Hi @marsell - I came across this case which is a legit docker label but failed the validation:

Creating mesos_zookeeper_1
(ValidationFailed) problem creating container: Invalid VM parameters: "tags" (Invalid) Special tag "docker:label:com.docker.compose.service" not supported

This should be allowed.

Comment by Marsell Kukuljevic
Created at 2015-10-15T00:58:33.000Z
That string matches one of the regexes listed in the above description. Fair enough, but if that's the case, can you elaborate on what should and should not be let through?

Comment by Angela Fong
Created at 2015-10-15T04:48:03.000Z
Updated at 2015-10-15T05:39:40.000Z
Sorry, now that I've re-read the ticket description, I can see that it's not clear about what should be prohibited for the docker reserved namespace validation. For the three docker namespaces, they should be prevented only when they are updated through the vmapi add/set/delete metadata (tags) operations for an existing container. If they are set as part of the docker run or docker create workflow (or simply from within a provisioning workflow), they are legitimate tags generated by sdc-docker, instead of something that are specified by the end user and can bypass the validation.

I don't know if there is a good way to identify the caller or the context. If it gets too tricky, we should discuss the logic further and comment out the docker namespace check for now because this currently prevents docker-compose from working on east-3b.

Comment by Bot Bot [X]
Created at 2015-10-15T12:00:33.000Z

sdc-vmapi commit 0933189 (branch master, by Marsell Kukuljevic)

ZAPI-671 - disable docker tag validation for now.


Comment by Marsell Kukuljevic
Created at 2015-10-15T12:16:18.000Z
After thinking about it a bit, I think I'm missing enough pieces of the puzzle that I'm disabling the docker tag validation for the time being.

Just to clarify, is this what should happen?


Comment by Angela Fong
Created at 2015-10-15T14:27:02.000Z
Yes to all three, plus one more:

Comment by Bot Bot [X]
Created at 2015-10-15T16:05:58.000Z

sdc-vmapi commit 8c94cc9 (branch release-20151015, by Marsell Kukuljevic)

ZAPI-671 - disable docker tag validation for now.


Comment by Bot Bot [X]
Created at 2015-10-21T02:28:43.000Z

sdc-vmapi commit 448d5dd (branch master, by Marsell Kukuljevic)

ZAPI-671 - allow the use of docker tags when provisioning, but otherwise
    disallow their modification or deletion.


Comment by Angela Fong
Created at 2015-10-26T19:21:56.000Z
Hi @marsell, this is looking good. One ask - can we externalize the TRITON_TAG_RE and DOCKER_TAG_RE as SAPI configurations so that we can add/modify them as needed without changing the code? Granted, the person who makes the change has to understand what he is doing. Let me know what you think. Thanks.

Comment by Marsell Kukuljevic
Created at 2015-10-26T21:20:16.000Z
Okay. Should vmapi provide the current regexes as defaults?

Comment by Angela Fong
Created at 2015-10-26T22:13:58.000Z
Yes, those regexes should be there by default.

Comment by Bot Bot [X]
Created at 2015-10-27T12:56:27.000Z

sdc-vmapi commit b08ac4a (branch master, by Marsell Kukuljevic)

ZAPI-671 - allow default regexes for validation of docker and triton
    tags to be changed through sapi.


Comment by Bot Bot [X]
Created at 2015-10-28T13:20:07.000Z

sdc-cloudapi commit 7adf320 (branch master, by Marsell Kukuljevic)

ZAPI-671 - add tests covering triton and docker tags.


Comment by Trent Mick
Created at 2015-10-28T18:12:23.000Z
@marsell Updates to https://github.com/joyent/sdc-vmapi/blob/master/docs/index.md#vmapi-configuration-file please, and a "SAPI Configuration" section would be nice (a la CNAPI: https://github.com/joyent/sdc-cnapi/blob/master/docs/index.md#sapi-configuration).

Comment by Bot Bot [X]
Created at 2015-10-29T12:41:51.000Z

sdc-vmapi commit 5347d67 (branch master, by Marsell Kukuljevic)

ZAPI-671 - add some documentation regarding config file entries and
    sapi metadata.