TRITON-2497 Static IP support for sdc-docker #158

Smithx10 · 2021-03-29T14:34:25Z

Addresses #157

bahamat

Need to fix make check, but otherwise this looks good.

teutat3s · 2021-09-01T11:55:50Z

make check is looking green here. If I understand the file https://github.com/joyent/sdc-docker/blob/master/tools/check-docs.sh correctly, it only checks in the docs/ directory, where no changes were made - or did you mean missing docs?

bahamat · 2021-09-01T16:13:37Z

@teutat3s It's a bit confusing, because the target is check::. The double colon means there are multiple targets with the same name and running make check executes all of them. The other checks are defined in deps/eng/tools/mk/Makefile.targ (which is a submodule).

Specifically, the following sub-checks fail:

check-jsl
check-jsstyle
check-copyright

teutat3s · 2021-09-01T16:36:59Z

@bahamat thanks for clarifying, I'll see what I can do to contribute and fix these failing sub-checks.

teutat3s · 2021-11-16T13:04:33Z

@Smithx10 I tried addressing the make check errors in Smithx10#1
Feedback welcome!

Static addresses: fix make check

teutat3s · 2021-11-19T12:31:09Z

@bahamat make check should be all green now.

EDIT: probably this is required before merging this?
TritonDataCenter/node-triton-tags#5

bahamat · 2021-11-19T16:35:10Z

Ok, this is looking good so far.

I'd like to see at least some integration tests created, and the test run to see the results.

The following tests should be created:

create instance with IP on owned network gets created properly
create instance with multiple IPs on different owned networks
create instance with multiple IPs on same owned network
create instance with IP on non-owned network gets rejected properly
create instance with IP on owned network and IP on non-owned network gets rejected properly

In order to do this, you'll need an image built that includes TritonDataCenter/node-triton-tags#5. The easiest way to do that is to change package.json in this commit to point triton-tags at https://github.com/Smithx10/node-triton-tags#e4c543ffad2e36e180e7446b5527223cb413c39f and our Jenkins will build an image for you and put it in the experimental channel.

teutat3s · 2021-12-14T18:41:09Z

@bahamat should the image already be built or does it take a bit?

[root@headnode (dc-1) ~]# sdcadm avail -C experimental docker
SERVICE  IMAGE                                 VERSION
docker   5f7af963-c87c-432a-b04f-be2784298487  docker@PR-158-20211119T162303Z-gf1a5a92

bahamat · 2021-12-14T18:58:25Z

@teutat3s I think automatic builds only happen when they come from joyent owned repos (otherwise it could be dangerous). I've kicked off a build so you should see it in a few minutes.

teutat3s · 2021-12-15T15:30:43Z

Something seems to be broken in that image:

EDIT: Maybe a sync with joyent/node-triton-tags's master branch is necessary?

Click to expand

[ Dec 15 15:17:35 Enabled. ]
[ Dec 15 15:17:35 Rereading configuration. ]
[ Dec 15 15:17:41 Executing start method ("/opt/smartdc/docker/smf/method/docker start"). ]
+ . /lib/svc/share/smf_include.sh
++ SMF_EXIT_OK=0
++ SMF_EXIT_NODAEMON=94
++ SMF_EXIT_ERR_FATAL=95
++ SMF_EXIT_ERR_CONFIG=96
++ SMF_EXIT_MON_DEGRADE=97
++ SMF_EXIT_MON_OFFLINE=98
++ SMF_EXIT_ERR_NOSMF=99
++ SMF_EXIT_ERR_PERM=100
+ PATH=/usr/sbin:/usr/bin
+ export PATH
+ case "$1" in
+ exit 0
+ /usr/bin/ctrun -l child -o noorphan /opt/smartdc/docker/build/node/bin/node --abort_on_uncaught_exception /opt/smartdc/docker/lib/docker.js
[ Dec 15 15:17:41 Method "start" exited with status 0. ]
[2021-12-15T15:17:43.234Z]  INFO: docker/57712 on 88e4c263-658f-4b02-a698-d5637a256f81: Loading config from "/opt/smartdc/docker/etc/config.json"
Uncaught Error: Cannot find module './cmon-groups-tag'

FROM
Function.Module._load (module.js:478:5)
Module.require (module.js:504:17)
require (internal/module.js:20:19)
Object.<anonymous> (/opt/smartdc/docker/node_modules/triton-tags/lib/index.js:35:21)
Module._compile (module.js:577:32)
Object.Module._extensions..js (module.js:586:34)
Module.load (module.js:494:32)
tryModuleLoad (module.js:453:12)
Function.Module._load (module.js:445:3)
Module.require (module.js:504:17)
require (internal/module.js:20:19)
Object.<anonymous> (/opt/smartdc/docker/lib/backends/sdc/utils.js:16:19)
Module._compile (module.js:577:32)
Object.Module._extensions..js (module.js:586:34)
Module.load (module.js:494:32)
tryModuleLoad (module.js:453:12)
Function.Module._load (module.js:445:3)
Module.require (module.js:504:17)
require (internal/module.js:20:19)
Object.<anonymous> (/opt/smartdc/docker/lib/backends/sdc/images.js:57:13)
Module._compile (module.js:577:32)
Object.Module._extensions..js (module.js:586:34)
Module.load (module.js:494:32)
tryModuleLoad (module.js:453:12)
Function.Module._load (module.js:445:3)
Module.require (module.js:504:17)
require (internal/module.js:20:19)
Object.<anonymous> (/opt/smartdc/docker/lib/backends/sdc/build.js:28:14)
Module._compile (module.js:577:32)
Object.Module._extensions..js (module.js:586:34)
Module.load (module.js:494:32)
tryModuleLoad (module.js:453:12)
Function.Module._load (module.js:445:3)
Module.require (module.js:504:17)
require (internal/module.js:20:19)
Object.<anonymous> (/opt/smartdc/docker/lib/backends/sdc/index.js:17:13)
Module._compile (module.js:577:32)
Object.Module._extensions..js (module.js:586:34)
Module.load (module.js:494:32)
tryModuleLoad (module.js:453:12)
Function.Module._load (module.js:445:3)
Module.require (module.js:504:17)
require (internal/module.js:20:19)
new App (/opt/smartdc/docker/lib/docker.js:75:19)
main (/opt/smartdc/docker/lib/docker.js:649:15)
Object.<anonymous> (/opt/smartdc/docker/lib/docker.js:653:1)
Module._compile (module.js:577:32)
Object.Module._extensions..js (module.js:586:10)
Module.load (module.js:494:32)
tryModuleLoad (module.js:453:12)
Function.Module._load (module.js:445:3)
Module.runMain (module.js:611:10)
run (bootstrap_node.js:394:7)
startup (bootstrap_node.js:160:9)
bootstrap_node.js:507:3
[ Dec 15 15:17:44 Stopping because all processes in service exited. ]
[ Dec 15 15:17:44 Executing stop method (:kill). ]

teutat3s · 2021-12-15T15:53:15Z

@bahamat would you be so kind and trigger another build? Or do you have an idea what else could have gone wrong with that image?

bahamat · 2021-12-15T18:01:22Z

Build started. Should be ready in about 5 minutes from now.

teutat3s · 2021-12-16T08:52:32Z

Same result. Could there be a step missing in the build process? It seems the .pegjs files are not rendered?

[root@88e4c263-658f-4b02-a698-d5637a256f81 (dc-1:docker0) ~]# ls /opt/smartdc/docker/node_modules/triton-tags/lib
cmon-groups-tag.pegjs  cns-svc-tag.pegjs  index.js

EDIT:
It seems those files are only rendered when cutting a new release, is there a workaround to get those into the experimental image?
https://github.com/joyent/node-triton-tags/blob/6d78099450aae993bededf03bf71f085ac700d55/Makefile#L33-L41

danmcd · 2025-08-11T19:44:57Z

Bug now filed: https://smartos.org/bugview/TRITON-2497

danmcd

I see that #157 is the "issue" that is now TRITON-2497. I know you've had this in production for $YEARS, but I'll wanna make sure we get enough current-people to review this, AND to make sure it's documented properly as well.

lib/backends/sdc/networks.js

danmcd · 2025-08-11T19:49:38Z

package.json

    "tape": "^4.4.0",
    "trace-event": "1.2.0",
-    "triton-tags": "1.4.0",
+    "triton-tags": "git+https://github.com/Smithx10/node-triton-tags#ed3272116c846473c3fceee4c630dde1f40589f5",


I relabeled TritonDataCenter/node-triton-tags#5 to also be part of TRITON-2497

Once v1.4.2 of node-triton-tags lands you can fix this.

ALSO, like with node-triton-tags, change the "Author" to be "Triton Data Center (tritondatacenter.com)" (or "Edgecast Cloud (edgecast.io)").

danmcd · 2025-08-11T19:56:09Z

Also, there's an upstream merge problem here that needs fixing, probably package.json.

Smithx10 · 2025-08-12T04:46:21Z

lib/backends/sdc/networks.js

-                    payload.networks = [ {uuid: network.uuid, primary: true} ];
+                    payload.networks =
+                        [ {ipv4_uuid: network.uuid, primary: true} ];
+                    if (container.NetworkingConfig.EndpointsConfig[networkMode]


I think it's possible for us to land here without a NetworkingConfig. I think we may need to check for null's all the way down. Not sure how this is typically handled in node.

if (container.NetworkingConfig != null && container.NetworkingConfig.EndpointsConfig != null && container.NetworkingConfig.EndpointsConfig[networkMode] != undefined) {

package.json

teutat3s · 2025-09-04T14:31:08Z

If you would like to reproduce the copy-right check, the command is:

deps/eng/tools/check-copyright -q

danmcd · 2025-09-04T15:57:23Z

The copyright check is failing... We probably want to update the deps/eng submodule to the latest commit and need to add one line with Copyright 2025 Edgecast Cloud LLC. before yours. Pending confirmation by Dan.

Absolutely, commit TritonDataCenter/eng@abf5ffd

Have you worked with submodules before @Smithx10 ? I find it a PITA and it took be A While (TM) to get it so I ask only out of my own horrid experiences.

Smithx10 · 2025-09-04T16:17:52Z

The copyright check is failing... We probably want to update the deps/eng submodule to the latest commit and need to add one line with Copyright 2025 Edgecast Cloud LLC. before yours. Pending confirmation by Dan.

Absolutely, commit TritonDataCenter/eng@abf5ffd

Have you worked with submodules before @Smithx10 ? I find it a PITA and it took be A While (TM) to get it so I ask only out of my own horrid experiences.

I usually just vendor deps, but not really... I went into the submodule and git checkout'd the latest commit. This could be completely wrong. Let me know if that looks right or what to do instead.

I'll need to squash / force push before merging so no biggie.

teutat3s · 2025-09-04T16:35:53Z

test/integration/helpers.js

                'Name': '',
                'MaximumRetryCount': 0
-            }
+            },


Check wants a copyright update here, too:

deps/eng/tools/check-copyright -q error: test/integration/helpers.js: copyright year not updated to 2025: ' * Copyright 2022 MNX Cloud, Inc.'

teutat3s · 2025-09-04T16:37:44Z

The eng bump looks good to me, thanks. We'll squash and merge to integrate these changes, a force push should not be required.

danmcd

Yeah, @teutat3s is correct: Fix the one file, and all of us should be able to use its output for final testing. Will approve on successful build, will squash-and-merge upon @teutat3s and/or you reporting test success with Jenkins-built artifacts.

Smithx10 · 2025-09-04T20:06:14Z

The copyright check is failing... We probably want to update the deps/eng submodule to the latest commit and need to add one line with Copyright 2025 Edgecast Cloud LLC. before yours. Pending confirmation by Dan.

Did we still want the Edgecast copyright added?

danmcd · 2025-09-04T20:08:06Z

The copyright check is failing... We probably want to update the deps/eng submodule to the latest commit and need to add one line with Copyright 2025 Edgecast Cloud LLC. before yours. Pending confirmation by Dan.

danmcd

WEIRD it needs the comma?!?

Anyway, re-upping, and hopefully this build I just kicked works.

danmcd · 2025-09-04T20:32:35Z

WEIRD it needs the comma?!?

Anyway, re-upping, and hopefully this build I just kicked works.

Same error:

16:28:38  error: test/integration/helpers.js: copyright year not updated to 2025: ' * Copyright 2022 MNX Cloud, Inc.'

Lemme look again.

danmcd · 2025-09-04T20:44:32Z

Lemme look again.

Okay, it's a kind of bug in eng's check-copyright. Would've been a similar problem in the old days if there was an old Joyent followed by a new non-MNX; except now it's old MNX and new non-Edgecast.

Your other copyright changes aren't triggering because there's no MNX AND no Edgecast copyright. I'm attaching a commit you should add.
0001-Workaround-bug-in-eng-s-check-copyright.patch

danmcd

Kicked off build again! :)

danmcd · 2025-09-04T21:02:33Z

Kicked off build again! :)

AND IT WORKED!

Here are the build artifacts in Manta. Please re-test or test-anew with them?

kebe(~)[0]% mls -lt /Joyent_Dev/public/builds/docker/PR-158-20250904T205202Z-g794be7e/docker
-rwxr-xr-x 1 Joyent_Dev            33 Sep 04 16:56 latest-build-stamp
-rwxr-xr-x 1 Joyent_Dev        264490 Sep 04 16:56 npm-ls.json
-rwxr-xr-x 1 Joyent_Dev           120 Sep 04 16:56 build-environment
-rwxr-xr-x 1 Joyent_Dev      26847762 Sep 04 16:56 docker-pkg-PR-158-20250904T205202Z-g794be7e.tar.gz
-rwxr-xr-x 1 Joyent_Dev      80741629 Sep 04 16:56 docker-zfs-PR-158-20250904T205202Z-g794be7e.zfs.gz
-rwxr-xr-x 1 Joyent_Dev           633 Sep 04 16:56 docker-zfs-PR-158-20250904T205202Z-g794be7e.imgmanifest
-rwxr-xr-x 1 Joyent_Dev         15257 Sep 04 16:56 docker-zfs-PR-158-20250904T205202Z-g794be7e.pkgaudit
kebe(~)[0]%

Smithx10 · 2025-09-05T12:08:52Z

I think also we will need to bump the version, right?

Smithx10 · 2025-09-05T12:20:28Z

In order for folks to use this, sdc-vmapi will need node-triton-tags 1.5.0
TritonDataCenter/sdc-vmapi#97

danmcd · 2025-09-05T14:41:04Z

I think also we will need to bump the version, right?

Probably to 0.8.0 from 0.7.11. A cursory search of the TritonDataCenter repositories suggest nobody cites this package version anywhere else, so I hope this means knock-on effects are minimal.

Needs package.json bump, and discussion about other affected components.

danmcd

Thank you.

teutat3s · 2025-09-15T11:48:38Z

package.json

  "name": "sdc-docker",
-  "version": "0.7.11",
+  "version": "0.8.0",
  "author": "MNX Cloud (mnx.io)",


Minor nit: we usually update the author line to:

"author": "Edgecast Cloud (edgecast.io)",

teutat3s · 2025-09-15T11:49:35Z

Nice progress here and in sdc-vmapi, I'll start testing this today and report back.

teutat3s · 2025-09-17T13:27:40Z

Testing results for

fabrics enabled
vmapi updated to master-20250910T155332Z-g54d7b54
docker updated to PR-158-20250915T114917Z-gb9bd128
provisioning on headnode enabled

Trying to provision a docker container with a single external network in this environment fails with:

docker run --label triton.network.public=external --label triton.network.public_ipv4=172.25.2.17 --network external --name test-TRITON-2497 --publish 80:80 alpine:latest
docker: Error response from daemon: (Validation) both networks are of uuid: b400e665-1bb4-4859-983d-bae1c9e56926, networks must be distinct

The new code currently seems to assume that --network would be used only for a private fabric network.

An additional note is that currently both triton.network.public and triton.network.public_ipv4 labels need to be provided, which would be fine by me as long as we add a note clarifying that to docs/api.

teutat3s · 2025-09-19T14:27:19Z

The above was reproduced in Triton with fabrics disabled:

Trying to provision a docker container with a single external network in this environment fails with:

kebe-sub(~)[125]% docker --tls run --label triton.network.public=external --label triton.network.public_ipv4=172.24.4.199 --network external --name test-TRITON-2497 --publish 80:80 alpine:latest
docker: Error response from daemon: (Validation) both networks are of uuid: 54416e88-328e-4b41-a026-48a7d4b78827, networks must be distinct (b91a69ad-e3fb-4aa0-a456-8bf3a68fc1fb).
See 'docker run --help'.

P.S. for testing a docker endpoint with a self signed certificate, this works:

eval "$(triton env)"
unset DOCKER_TLS_VERIFY
docker --tls info

or

unset DOCKER_TLS_VERIFY
export DOCKER_TLS=1
docker info

Smithx10 · 2025-10-07T16:52:23Z

From what I understand currently is we'd like the behavior to be:

Providing the same UUID for triton.network.public and EndPointConfig is illegal. We don't support 2 interfaces on the same network.

Usage of "triton.network.public" requires a container with a published port.

"The new code currently seems to assume that --network would be used only for a private fabric network":

To have a single External Network you'd run:
docker run --network $NonFabricNetwork --ip $address

for example:

## Public Network
❯ itt network get sdcdockertest_apicreate_external_alice1
{
    "id": "8637495a-4bfc-4629-80f5-4222dff37c23",
    "name": "sdcdockertest_apicreate_external_alice1",
    "public": true,
    "suffixes": [
        "svc.ab459a6d-3a63-47c9-908c-a9c074e727ea.coal.cns.joyent.us",
        "inst.ab459a6d-3a63-47c9-908c-a9c074e727ea.coal.cns.joyent.us"
    ]
}

## Run Docker Container on Network
docker run --network sdcdockertest_apicreate_external_alice1 --ip 10.0.31.5 --name mytest nginx:latest

## Show Docker Container with the assigned address.
❯ itt inst get mytest | grep primaryIp
    "primaryIp": "10.0.31.5",

This means the "public" labels are used to only define the "public" network.

Did you try with --ip?

@teutat3s when I tested prior running --network and --label triton.network.public of the same UUID we had 2 interfaces on the same network.

I believe @danmcd didn't think the behavior should happen if we can avoid it.

Bruce Smith added 2 commits March 29, 2021 10:31

support static ip

0dd6ee3

IPAM Config only exists when IP is given

353e197

bahamat suggested changes May 21, 2021

View reviewed changes

teutat3s added 2 commits November 16, 2021 13:03

Fix make check

73bed19

Merge branch 'master' into static_addresses

703c276

Merge pull request #1 from greenbaum/static_addresses

f1a5a92

Static addresses: fix make check

Use changes from TritonDataCenter/node-triton-tags#5

5a0a6eb

teutat3s force-pushed the static_addresses branch from a04acd3 to 5a0a6eb Compare December 15, 2021 15:52

danmcd changed the title ~~support static ip~~ TRITON-2497 Static IP support for sdc-docker Aug 11, 2025

danmcd reviewed Aug 11, 2025

View reviewed changes

Smithx10 commented Aug 12, 2025

View reviewed changes

Smithx10 and others added 3 commits August 12, 2025 19:29

handle copy and possible null

f1ee7ad

Merge branch 'master' into static_addresses

1447c07

bump tag version

063037b

danmcd reviewed Aug 20, 2025

View reviewed changes

package.json Outdated Show resolved Hide resolved

bump tag version

0840a0f

bump eng and fix typo

e06d615

Smithx10 dismissed danmcd’s stale review via e06d615 September 4, 2025 16:11

teutat3s reviewed Sep 4, 2025

View reviewed changes

danmcd reviewed Sep 4, 2025

View reviewed changes

json trailing ,

85561bf

fix copyright

f6a571b

danmcd previously approved these changes Sep 4, 2025

View reviewed changes

eng copyright workaround

794be7e

Smithx10 dismissed danmcd’s stale review via 794be7e September 4, 2025 20:49

danmcd previously approved these changes Sep 4, 2025

View reviewed changes

bump version

b9bd128

danmcd approved these changes Sep 5, 2025

View reviewed changes

teutat3s reviewed Sep 15, 2025

View reviewed changes

TRITON-2497 Static IP support for sdc-docker #158

Are you sure you want to change the base?

TRITON-2497 Static IP support for sdc-docker #158

Uh oh!

Conversation

Smithx10 commented Mar 29, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bahamat left a comment

Choose a reason for hiding this comment

Uh oh!

teutat3s commented Sep 1, 2021

Uh oh!

bahamat commented Sep 1, 2021

Uh oh!

teutat3s commented Sep 1, 2021

Uh oh!

teutat3s commented Nov 16, 2021

Uh oh!

teutat3s commented Nov 19, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bahamat commented Nov 19, 2021

Uh oh!

teutat3s commented Dec 14, 2021

Uh oh!

bahamat commented Dec 14, 2021

Uh oh!

teutat3s commented Dec 15, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

teutat3s commented Dec 15, 2021

Uh oh!

bahamat commented Dec 15, 2021

Uh oh!

teutat3s commented Dec 16, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

danmcd commented Aug 11, 2025

Uh oh!

danmcd left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

danmcd Aug 11, 2025

Choose a reason for hiding this comment

Uh oh!

danmcd Aug 14, 2025

Choose a reason for hiding this comment

Uh oh!

danmcd Aug 14, 2025

Choose a reason for hiding this comment

Uh oh!

danmcd commented Aug 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Smithx10 Aug 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

teutat3s commented Sep 4, 2025

Uh oh!

danmcd commented Sep 4, 2025

Uh oh!

Smithx10 commented Sep 4, 2025

Uh oh!

teutat3s Sep 4, 2025

Choose a reason for hiding this comment

Uh oh!

teutat3s commented Sep 4, 2025

Uh oh!

danmcd left a comment

Choose a reason for hiding this comment

Uh oh!

Smithx10 commented Sep 4, 2025

Uh oh!

danmcd commented Sep 4, 2025

Uh oh!

danmcd left a comment

Smithx10 commented Mar 29, 2021 •

edited

Loading

teutat3s commented Nov 19, 2021 •

edited

Loading

teutat3s commented Dec 15, 2021 •

edited

Loading

teutat3s commented Dec 16, 2021 •

edited

Loading

danmcd commented Aug 11, 2025 •

edited

Loading

Smithx10 Aug 12, 2025 •

edited

Loading

danmcd commented Sep 4, 2025 •

edited

Loading

teutat3s commented Sep 19, 2025 •

edited

Loading