Add network validation script executed in the sagemaker_ui_post_startup script #713
+376
−0
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…up script **Description** This change introduces the network validation script which tests if certain AWS services are reachable by making read only API calls with a set timeout. If the call exceeds the timeout, the script infers that it was caused by a bad network setup such as not having access to the internet/ VPC endpoint to make the call. API calls that resolve (succeed or fail) within the timeout are inferred as having the proper network setup. AWS services for Compute Connections and Git are checked in this script. More specifically, the script lists the datazone connections to see which services need to be checked. The unreachable services are aggregated and are displayed by writing to the post-startup-status.json, which displays the error notification in the IDE. **Testing** Tested in a SMUS portal containing internet, no internet, and no internet with VPC Endpoints to Datazone and s3.
reganbaum
reviewed
Jun 21, 2025
| # Initialize SERVICE_COMMANDS with always-needed STS and S3 checks | ||
| declare -A SERVICE_COMMANDS=( | ||
| ["STS"]="aws sts get-caller-identity" | ||
| ["S3"]="aws s3api list-buckets --max-items 1" |
There was a problem hiding this comment.
Can we use list-objects at the project S3 path instead? This will result in a 4xx error as the project role doesn't have permissions to list buckets by default. While I understand the goal here is to check for what services are reachable, it would be better to invoke an API we expect to succeed so logs aren't polluted and 4xx metrics aren't impacted.
| if [[ "$type" == "SPARK" ]]; then | ||
| # If sparkGlueProperties present, add Glue check | ||
| if echo "$item" | jq -e '.props.sparkGlueProperties' > /dev/null; then | ||
| SERVICE_COMMANDS["Glue"]="aws glue get-crawlers --max-items 1" |
There was a problem hiding this comment.
Same comment here, can we use glue get-catalogs or get-databases instead?
| # Check for emr-serverless in sparkEmrProperties.computeArn for EMR Serverless check | ||
| emr_arn=$(echo "$item" | jq -r '.props.sparkEmrProperties.computeArn // empty') | ||
| if [[ "$emr_arn" == *"emr-serverless"* ]]; then | ||
| SERVICE_COMMANDS["EMR Serverless"]="aws emr-serverless list-applications --max-results 1" |
There was a problem hiding this comment.
Same thing here. Can we use get-application or another emr-s API that the project role does have permission to call?
|
|
||
| # Optionally add CodeConnections if S3 storage flag is true (Git storage) | ||
| if [[ "$is_s3_storage" == "1" ]]; then | ||
| SERVICE_COMMANDS["CodeConnections"]="aws codeconnections list-hosts --max-results 1" |
There was a problem hiding this comment.
I haven't seen the managed policy updates for S3 storage; will those include this API?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This change introduces the network validation script which tests if certain AWS services are reachable by making read only API calls with a set timeout. If the call exceeds the timeout, the script infers that it was caused by a bad network setup such as not having access to the internet/ VPC endpoint to make the call. API calls that resolve (succeed or fail) within the timeout are inferred as having the proper network setup.
AWS services for Compute Connections and Git are checked in this script. More specifically, the script lists the datazone connections to see which services need to be checked.
The unreachable services are aggregated and are displayed by writing to the post-startup-status.json, which displays the error notification in the IDE.
Testing
Tested in a SMUS portal containing internet, no internet, and no internet with VPC Endpoints to Datazone and s3.
Description
[Provide a brief description of the changes]
Type of Change
Release Information
Does this change need to be included in patch version releases? By default, any pull requests will only be added to the next SMD image minor version release once they are merged in template folder. Only critical bug fix or security update should be applied to new patch versions of existed image minor versions.
If yes, please explain why:
[Explain the criticality of this change and why it should be included in patch releases]
How Has This Been Tested?
Tested in a SMUS portal containing internet, no internet, and no internet with VPC Endpoints to Datazone and s3.
Checklist:
Test Screenshots (if applicable):
Related Issues
[Link any related issues here]
Additional Notes
[Any additional information that might be helpful for reviewers]