Sandcastle build config updates #12904

jjspace · 2025-09-19T19:41:31Z

Description

This PR contains updates to the build process for Sandcastle. These were largely inspired/required by the ability to build sandcastle for the ion-sdk repo/library.

New buildStatic and createSandcastleConfig allow building sandcastle from outside the package directory. This should reduce the pseudo-dependency that the sandcastle package has on knowing it's nested in the CesiumJS repo.
- This function should also now make it much more clear which files/resources are "external" by just setting all the necessary paths to access them wherever it's hosted.
- I still need to actually relocate the configs into the larger CesiumJS build processes but I wanted to confirm the approach first. I think we'll be able to remove all the vite configs except the dev one after this.
The engine and widgets packages are now bundled into a single file to make it easier to load them in the browser.
- Using this and an importmap we're also now using the version of cesium that imports/re-exports from engine/widgets to avoid duplicate cesium versions in the browser
- This also enables easier importing of the ion-sdk packages when we need to
- The types import was also made dynamic based on build-time paths. This can probably still be improved in the future but works and I've tested it with the ion-sdk
Development sandcastles will no longer be included in the production build. This was a small config change while I was already in there

Issue number and link

Part of #12894

Testing plan

This was mostly an internal, build system, related change so there's not much to test in the app.
Make sure npm run build-sandcastle works from the project root
Make sure things are built to the correct, expected locations

Author checklist

I have submitted a Contributor License Agreement
I have added my name to CONTRIBUTORS.md
I have updated CHANGES.md with a short summary of my change
I have added or updated unit tests to ensure consistent code coverage
I have updated the inline documentation, and included code examples where relevant
I have performed a self-review of my code

github-actions · 2025-09-19T19:41:51Z

Thank you for the pull request, @jjspace!

✅ We can confirm we have a CLA on file for you.

ggetz · 2025-09-22T22:28:18Z

Thanks for the refactoring @jjspace!

I'm not sure if this is covered by the TODO item in your PR description, but the deployed app is not loading:

In broad strokes, the approach you describe makes sense to me. I'm taking a closer look at the code itself now.

ggetz

I think this is definitely a step in the right direction. I'm glad to see static files being handled separately from the vite build where possible.

I have a few clarifying questions, and I think we should be able to simplify the process a bit more. Let me know what you think.

ggetz · 2025-09-22T22:35:35Z

packages/sandcastle/scripts/typescriptCompile.js

+ * @param {string} configPath Absolute path to the config file to build
+ * @returns {number} exit code from the TS build process
+ */
+export default function typescriptCompile(configPath) {


I'm a bit confused why this script is needed. Can't we run tsc directly?

We could run it as a child process but that felt more messy and potentially unreliable when called from outside the package. I had a version that did that based off this answer. This way felt like the more reliable/stable method to actually use the package's dependencies.

I'm a bit concerned simply because it's so atypical and therefore poses more of a maintenance burden. The tsc CLI is documented, but there doesn't seem to be much for the Node API. Plus it's either an unstable API or the docs haven't been updated to reflect its stability.

I don't mind wrapping a helper function for typescript in it's own script. But if we do so:

I'd lean towards running a child processes.

Can we move the script to the top-level cesium package? Nothing needs to be sandcastle specific, right?

Switching to the tsc cli is a bit tricky too because we need to figure out where the binary actually is in the node_modules folder. I think I figured out an ok way to do this and just updated with a child process.

Can we move the script to the top-level cesium package? Nothing needs to be sandcastle specific, right?

The function itself is not necessarily Sandcastle specific but using it is. It's required to be called before the vite build kicks off so it's nicer in the Sandcastle package itself. If we moved it up then that'd be another dependency sandcastle has on cesium.

ggetz · 2025-09-22T22:38:21Z

packages/sandcastle/vite.config.js

 // https://vite.dev/config/
-const baseConfig: UserConfig = {
+/** @type {UserConfig} */
+const baseConfig = {


Why now JS instead of TS?

To enable easy imports outside of the sandcastle package without requiring it be compiled with typescript first.
/scripts/build.js -> /packages/sandcastle/buildStatic.js -> /packages/sandcastle/vite.config.js

ggetz · 2025-09-22T23:12:52Z

packages/sandcastle/scripts/buildStatic.js

+
+  config.define = {
+    ...config.define,
+    __VITE_TYPE_IMPORT_PATHS__: JSON.stringify(typePaths),


It seems like we'er bending over backwards to get the type definition locations. I'm thinking this would be a lot easier if we build out the definitions relative to the bundled files—Then, we wouldn't need to configure the path to Source/Cesium.d.ts and instead could get the path at runtime based on the resolved module URLs from the import maps:

const cesiumModulePath = import.meta.resolve("cesium"); const tsdPath = new URL("./Cesium.d.ts", cesiumModulePath).href;

I like the idea of this change but it would involve some larger changes with the rest of our build systems. Happy to take a look at it at some point but I think I'd prefer to leave this as is for this current PR. Maybe it's rather verbose in the config but at least it should be clear what it's doing.

ggetz · 2025-09-22T23:15:11Z

packages/sandcastle/src/SandcastleEditor.tsx

+        transformTypes(typesContent) {
+          // TODO: this feels a little messy and still very targeted at our own modules, is there a way to improve?
+          // I was experimenting with setting the transform from Vite but that doesn't work with functions
+
+          // Monaco expects the import statements to be inside the module so
+          // move the module declaration to the top of the "file"
+          if (typesContent.trim().startsWith("import")) {
+            const declareModuleLine = typesContent.match(
+              /declare module "([\w@\-\/]+)" {/gm,
+            )?.[0];
+            if (declareModuleLine) {
+              return `${declareModuleLine}
+              ${typesContent.replace(declareModuleLine, "")}`;
+            }
+          }
+          return typesContent;


I'm not clear on all the things that are happening here and in transformTypes in general, but is it possible to build all this into the tsd files directly?

is it possible to build all this into the tsd files directly?

I was going to say "no" because I assumed this was just the result of the tsd-jsdoc output. But apparently this specific part was our own modifications after the jsdoc stuff was generated.

I was also worried about modifying the types files we generate and export with the library but long story short I think this change is fine and things seem to still "just work" as they currently do. I've pushed a new commit adjusting the build and removing most of these transforms

Short story long:

Our types files for the packages themselves were roughly generated in the format

import { ... } from '@cesium/engine'; declare module '@cesium/widgets' { // ... }

When this file was added to the monaco extraLibs to enable the intellisense it wasn't making the connection to the @cesium/engine module. So viewer.cesiumWidget was not a known type. Switching the order to have the import statements inside the module declaration file made it work for monaco; that's what most of these transforms were in the code.

Reading into Typescript Modules a bit and there are multiple ways to declare modules that all behave a little different. Normally (I believe) an entire file is treated as a single module itself. When using the declare module syntax this becomes an Ambient Module. However if there are any import or export statements also at the "top level" of the file then the declare module syntax becomes a module augmentation even though the syntax is identical. 🤦‍♀️

In other words, our current types exported from the packages were module augmentations (though technically only the widgets and ion-sdk packages, engine had no imports and was only ambient). My latest change to move the imports inside the declare makes both packages ambient.
In some quick testing this doesn't seem to make any difference to the TS server when importing widgets in a new npm project in VSCode. I'm definitely not an expert on this part of TS though so I could be wrong.

Also worth reading a bit into the "correct" structures for declaration files for libraries and this page on the actual module .d.ts file

ggetz · 2025-09-22T23:16:33Z

packages/sandcastle/vite-plugins.js

+ * @param {string[]} [filenames]
+ * @returns {PluginOption}
+ */
+export const insertImportMap = (


Huh, is there no standard plugin for this? If not, are we perhaps missing something that serves a similar function in vite?

There seems to be a couple plugins that do this but none are popular or seem to be actively maintained. This code is loosely based on this one but mostly just using the Plugin API. It's a small requirement and could be made to fit our use cases so I felt it was better to just fully "own" the code and implement the plugin ourselves.

At first brush, that all sounds fine. The potential gotcha I'm worried about is that

but none are popular or seem to be actively maintained

is because we're missing a simpler or more integrated way of getting to the same result. Nothing is top of mind, but I may investigate further.

My ask is just that we've done the due diligence on alternative approaches.

My ask is just that we've done the due diligence on alternative approaches.

AFAIK I have. This seems like the cleanest and easiest solution for what we need. If we didn't want to request external files then the import map wouldn't be needed and everything could be bundled by vite. I assume that's why this is not a more "common" process.

ggetz · 2025-09-22T23:20:21Z

packages/sandcastle/vite.config.dev.ts

+  // IN DEV THIS DOES NOT COPY FILES it simply sets up in-memory server routes
+  // to the correct locations on disk in the parent directories
+  // This config should not be used for the actual build, only development of Sandcastle itself


I think my question is more along the lines of: If this is only being used for the development server, why are we using the viteStaticCopy plugin at all? Is there a more standard way to configure these paths? Should they be aliased instead?

If this is only being used for the development server, why are we using the viteStaticCopy plugin at all?

Largely because we already have it as a dependency. It didn't feel necessary to add another plugin or system when this one works well for our usecase. It's also how we tell people to set up a vite application in our own example repo

Is there a more standard way to configure these paths? Should they be aliased instead?

Not that I've been able to find. This is once again a slightly "special" case for us because we want to statically host files on the dev server that are not in the public directory. Most vite articles say to "just add your files to public and it'll work" but that kinda defeats our purpose of using the local files directly. The alternative seems to be to set up a custom plugin with middleware/routing on the serve part of vite to "host" the expected files at the expected paths. We could implement this ourselves but the vite static copy plugin already does all this heavy lifting for us. server.proxy does not work, it proxies to a different hostname which is not what we want. resolve.alias does not work either, it seems to only care about the built files which these are not.

ggetz · 2025-09-22T23:26:16Z

packages/sandcastle/README.md

 - `npm run build`: alias for `npm run build-app`
 - `npm run build-app`: build to static files in `/Apps/Sandcastle2` for hosting/access from the root cesium dev server
- `npm run build-ci`: build to static files in `/Apps/Sandcastle2` and configure paths as needed for CI deployment
+  - If the env variable `BASE_URL` is set it will be prefixed on all required paths in the application. This is useful for building to a "nested" url like we do in CI


This is the app's base public path, correct? I'm not sure that's entirely clear in the wording, especially since CESIUM_BASE_URL is in play.

jjspace · 2025-09-23T15:22:04Z

I'm not sure if this is covered by the TODO item in your PR description, but the deployed app is not loading:

Mismatch in my local CI build test not including the trailing / in the path that the CI job does. Fixed now!
I'm looking through your other comments now.

javagl · 2025-09-23T19:50:33Z

Also see #12911

Use proper URLs for import statements

javagl · 2025-09-25T13:20:44Z

I just reopened #12910 : There is another import at

cesium/packages/sandcastle/scripts/buildGallery.js

Line 306 in 20f3edc

const config = await import(configPath);

that will have to become
const config = await import(pathToFileURL(configPath).href);

(I could create a PR, but... let's not over-formalize the process here...)

ggetz

@jjspace I took another pass, this time a bit closer. Let me know the status of this PR, as I think you mentioned there are still updates incoming?

packages/sandcastle/scripts/buildStatic.js

ggetz · 2025-09-29T17:51:34Z

packages/sandcastle/standalone.html

Why does bucket.html live in templates, but not standalone.html?

Largely historical reasons. I believe there were some issues with routes before. Possibly resolved now but I would like to leave it alone in this pr as it's working as is.

packages/sandcastle/vite.config.js

ggetz · 2025-09-29T18:06:45Z

packages/sandcastle/vite.config.js

+function getCesiumVersion() {
+  const data = readFileSync(join(__dirname, "../../package.json"), "utf-8");
+  const { version } = JSON.parse(data);
+  return version;
+}


Should the value of Cesium version be passed as an option rather than autodetected here? getCesiumVersion is now defined in multiple places, despite serving the same purpose.

Another option would be to set a env variable with the value of cesium version. We already do this for building the docs.

I'm leaning towards this route as I think we could streamline several areas of our build processes and CI workflows if we use an environment variable instead. We could extract and set the env var as part of the postinstall step to ensure it's only run once and before all of the other build steps.

Should the value of Cesium version be passed as an option rather than autodetected here?

Yes, I thought the same thing, already updated.

Another option would be to set a env variable with the value of cesium version

I think my new approach/opinion is to leave any env variable reliance out of the sandcastle scripts. This could be used in the main build/gulpfile scripts but I think that can be separate from these changes if we want to go that route

ggetz · 2025-09-29T18:23:06Z

packages/sandcastle/scripts/buildStatic.js

+  config.define = {
+    ...config.define,
+    __VITE_TYPE_IMPORT_PATHS__: JSON.stringify(typePaths),
+    __COMMIT_SHA__: JSON.stringify(commitSha ?? undefined),


Is there a reason not to use process.env.GITHUB_SHA directly?

We may not always want to use it. For example in Prod we don't want to show it but it's currently set in the github workflow. Doing it this way makes it customizable and explicit from wherever we call this function. Using the env variable directly creates more, almost, "side effect" logic that people have to be aware of

ggetz · 2025-09-29T18:35:46Z

packages/sandcastle/scripts/buildStatic.js

+
+  const copyPlugin = viteStaticCopy({
+    targets: [
+      { src: "templates/Sandcastle.(d.ts|js)", dest: "templates" },


Can the TS build output to the public directory instead to save a step?

ggetz · 2025-09-29T18:42:21Z

packages/sandcastle/vite-plugins.js

+export const cesiumPathReplace = (cesiumBaseUrl) => {
+  return {
+    name: "custom-cesium-path-plugin",
+    config(config) {
+      config.define = {
+        ...config.define,
+        __CESIUM_BASE_URL__: JSON.stringify(cesiumBaseUrl),
+      };
+    },
+    transformIndexHtml(html) {
+      return html.replaceAll("__CESIUM_BASE_URL__", `${cesiumBaseUrl}`);
+    },
+  };
+};


What all do we need __CESIUM_BASE_URL__ after using import maps? Consider also that we can dynamically get the paths relative to the Cesium ESM import at runtime based on the import map.

We need the cesium base url for all of the assets and files that CesiumJS loads. This is setting up the app the same way we recommend/require everyone else to do. I think we should leave it like this

ggetz · 2025-09-29T18:48:02Z

packages/sandcastle/scripts/buildStatic.js

+ */
+
+/**
+ * @typedef {Object<string, ImportObject>} ImportList


Suggested change

* @typedef {Object<string, ImportObject>} ImportList

* @typedef {Record<string, ImportObject>} ImportList

This is JS doc not TS, Record doesn't exist in JS. Intellisense would probably pick it up fine in most environments and I'm already stretching the rule a little with the @import lines but it was intentional to use Object not Record here.

ggetz · 2025-09-29T19:00:35Z

packages/sandcastle/vite.config.prod.ts

+    { src: join(__dirname, `${cesiumSource}/ThirdParty`), dest: cesiumBaseUrl },
+    { src: join(__dirname, `${cesiumSource}/Workers`), dest: cesiumBaseUrl },
+    { src: join(__dirname, `${cesiumSource}/Assets`), dest: cesiumBaseUrl },
+    { src: join(__dirname, `${cesiumSource}/Widgets`), dest: cesiumBaseUrl },
+    { src: join(__dirname, `${cesiumSource}/*.(js|cjs)`), dest: cesiumBaseUrl },


Should we copy the Cesium build artifacts to S3 as part of the GitHub Actions workflow? That would be more consistent with our other deployments.

It might be possible to do that but then it would leave the built files not working by themselves. This way it creates a single directory that can be copied to S3, just like the other build processes do. Plus this way we can test it locally by doing PROD=true npm run build-sandcastle and then hosting those static files with something like npx http-server

(FYI This file and configuration has been removed in favor of incorporating it into the gulpfile like our other build steps)

jjspace · 2025-09-29T20:50:38Z

Let me know the status of this PR, as I think you mentioned there are still updates incoming?

@ggetz yes, sorry for the delay it took a bit longer getting things together than I thought it would. I've just pushed an update to extract the build configs out to the top level and eliminate all of the "pseudo-dependency" on being in the cesium repo. This includes now exporting the buildStatic and buildGallery functions for use in the cesium build scripts/gulpfile.
I'm looking through your new comments now, I think some might already be addressed by this change

jjspace · 2025-09-30T16:25:16Z

I need to finalize how this works in the release zip file. Currently it can't "install" the sandcastle package because it's not included. We need to build it before the zip and/or sort out which files to include.

jjspace added 5 commits September 19, 2025 11:29

central sandcastle build function, bundle esm packages

f490f63

remove duplicated configs

814cb73

dont include dev sandcastles in prod

f7c4d62

add commit sha to build for CI

aabcb3d

build TS for sandcastle

f3f4075

jjspace requested a review from ggetz September 19, 2025 19:41

Merge remote-tracking branch 'origin/main' into sandcastle-build-updates

4de8de1

ggetz assigned jjspace Sep 22, 2025

ggetz requested changes Sep 23, 2025

View reviewed changes

fix ci build double //

dcc4ec1

jjspace and others added 2 commits September 23, 2025 14:55

move import inside module declaration in package types

97f606d

Use proper URLs for import statements

4aab565

javagl mentioned this pull request Sep 23, 2025

Use proper URLs for import statements #12911

Merged

jjspace added 2 commits September 23, 2025 16:16

make sure imports are strings

2fcefac

Merge pull request #12911 from CesiumGS/sandcastle-build-updates-windows

20f3edc

Use proper URLs for import statements

jjspace mentioned this pull request Sep 26, 2025

Follow up for import urls #12930

Merged

6 tasks

jjspace added 2 commits September 26, 2025 12:52

Merge remote-tracking branch 'origin/main' into sandcastle-build-updates

d350020

clarify build option

63c8ccb

ggetz requested changes Sep 29, 2025

View reviewed changes

further isolate sandcastle build process

1bd41e9

jjspace added 2 commits September 29, 2025 17:23

switch from ts api to tsc cli

d83fb76

remove extra logs

26a011d

switch deployed sandcastle directory

0f7a9c8

jjspace mentioned this pull request Sep 30, 2025

Sandcastle full release #12937

Merged

6 tasks

Merge remote-tracking branch 'origin/main' into sandcastle-build-updates

c708661

	* @typedef {Object<string, ImportObject>} ImportList
	* @typedef {Record<string, ImportObject>} ImportList

Sandcastle build config updates #12904

Are you sure you want to change the base?

Sandcastle build config updates #12904

Uh oh!

Conversation

jjspace commented Sep 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Issue number and link

Testing plan

Author checklist

Uh oh!

github-actions bot commented Sep 19, 2025

Uh oh!

ggetz commented Sep 22, 2025

Uh oh!

ggetz left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jjspace commented Sep 23, 2025

Uh oh!

javagl commented Sep 23, 2025

Uh oh!

javagl commented Sep 25, 2025

Uh oh!

ggetz left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

jjspace commented Sep 19, 2025 •

edited

Loading

jjspace Sep 29, 2025 •

edited

Loading