Skip to content

Commit 43a5df7

Browse files
sebstoclaude
sebsto
and
claude
authored
[plugin] Add swift-static-sdk cross-compile option (no container) (#689)
Fixes #332. Adds `--cross-compile swift-static-sdk` to `lambda-build`. It compiles a statically linked, musl-based binary directly on the host using the Static Linux SDK, with no Docker or `container` involved. The resulting `bootstrap` runs as-is on the Lambda `provided.al2023` runtime. ## What it does - New `StaticLinuxSDKBuildBackend` that shells out to `swift build --swift-sdk <triple>`, slotting into the existing `BuildBackend` abstraction. No changes to the archive or deploy paths. - `--architecture` selects the target: `arm64` maps to `aarch64-swift-linux-musl`, `x64` to `x86_64-swift-linux-musl`. Because the SDK genuinely cross-compiles, either architecture can be built from either host. - The SDK must be installed beforehand. The plugin cannot download it (the sandbox limits network to Docker), so the backend detects a missing SDK and fails with install guidance instead of hanging or erroring cryptically. - Detection uses `swift build --swift-sdk <triple> --show-bin-path`, which resolves the SDK exactly as the real build does. It does not parse `swift sdk list`, whose output is SDK bundle ids that never contain the target triple. - The nested build runs in a dedicated `--scratch-path` under `.build`. The command plugin holds the workspace lock on the default `.build` for its whole run, so a nested build against the same directory would deadlock waiting for that lock. A separate scratch path avoids it. - The plugin rejects flags that only make sense for a container build when this method is selected (`--base-docker-image`, `--swift-version`, `--disable-docker-image-update`, `--base-oci-image`, and `--archive-format oci`), rather than silently ignoring them. ## A note on binary size Building the same trivial Lambda both ways (release, stripped), the static-linux-sdk binary is about 16 MB and the container (glibc) binary is about 19 MB. The two are closer than you might expect, and the reason is worth recording: both statically link the Swift standard library via `--static-swift-stdlib`, so their `.text` sections are within a few KB of each other (around 11 MB each). The C library is the small part. musl links statically and stays compact; glibc is not statically linked at all in the container build, it stays dynamic. So the size floor for both is the Swift runtime, not libc, and static musl comes out slightly smaller rather than larger. If we want to reduce size further, the lever is the Swift stdlib footprint, not the choice of libc. ## Testing - Unit tests for backend selection, the musl triple mapping, and config parsing. - Verified end to end against the 6.4 dev snapshot: builds a statically linked aarch64 ELF, packages it into a zip containing `bootstrap`, and writes a build manifest recording `arm64`. - Verified the incompatible-flag errors fire, and that the plain and `--archive-format zip` paths still build with no false positives. Semver: minor. --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 831fa0e commit 43a5df7

9 files changed

Lines changed: 350 additions & 56 deletions

File tree

‎Plugins/AWSLambdaBuilder/Plugin.swift‎

Lines changed: 38 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -44,12 +44,45 @@ struct AWSLambdaBuilder: CommandPlugin {
4444
)
4545
}
4646

47-
// Resolve the container CLI that matches the requested cross-compilation method. The plugin
48-
// sandbox can only run tools it resolves up front, so we must pick the right binary here:
47+
// Resolve the tool that matches the requested cross-compilation method. The plugin sandbox
48+
// can only run tools it resolves up front, so we must pick the right binary here:
49+
// `swift` for `--cross-compile swift-static-sdk` (no container runtime needed),
4950
// `container` for `--cross-compile container`, `docker` otherwise.
5051
let crossCompileMethod = crossCompileArgument.first?.lowercased()
51-
let containerCLIToolName = crossCompileMethod == "container" ? "container" : "docker"
52-
let containerToolPath = try context.tool(named: containerCLIToolName).url
52+
let crossCompileToolName: String
53+
switch crossCompileMethod {
54+
case "swift-static-sdk":
55+
crossCompileToolName = "swift"
56+
// The Static Linux SDK builds without a container, so the docker/container-specific
57+
// options do not apply. Reject them here rather than let them be silently ignored.
58+
// These flags are forwarded verbatim to the helper, so inspect the raw arguments.
59+
let incompatibleWithStaticSDK = [
60+
"--base-docker-image",
61+
"--swift-version",
62+
"--disable-docker-image-update",
63+
"--base-oci-image",
64+
]
65+
for flag in incompatibleWithStaticSDK where arguments.contains(flag) {
66+
throw BuilderErrors.invalidArgument(
67+
"'\(flag)' cannot be used with '--cross-compile swift-static-sdk'; it targets a "
68+
+ "container-based build. Remove it, or choose '--cross-compile docker' or 'container'."
69+
)
70+
}
71+
// The OCI image build requires a container CLI, so it is incompatible too. Match the
72+
// value that follows --archive-format rather than a bare "oci" token anywhere.
73+
if let formatIndex = arguments.firstIndex(of: "--archive-format"),
74+
arguments.indices.contains(formatIndex + 1),
75+
arguments[formatIndex + 1].lowercased() == "oci"
76+
{
77+
throw BuilderErrors.invalidArgument(
78+
"'--archive-format oci' cannot be used with '--cross-compile swift-static-sdk'; "
79+
+ "building an OCI image requires a container CLI. Use '--cross-compile docker' or 'container'."
80+
)
81+
}
82+
case "container": crossCompileToolName = "container"
83+
default: crossCompileToolName = "docker"
84+
}
85+
let crossCompileToolPath = try context.tool(named: crossCompileToolName).url
5386
let zipToolPath = try context.tool(named: "zip").url
5487

5588
// Resolve the output directory. The default lives under the plugin's work directory, whose
@@ -90,7 +123,7 @@ struct AWSLambdaBuilder: CommandPlugin {
90123
"--package-display-name", context.package.displayName,
91124
"--package-directory", context.package.directoryURL.path(),
92125
"--configuration", configurationArgument.first ?? "release",
93-
"--cross-compile-tool-path", containerToolPath.path,
126+
"--cross-compile-tool-path", crossCompileToolPath.path,
94127
"--zip-tool-path", zipToolPath.path,
95128
]
96129
// Re-inject the cross-compilation method (normalised to --cross-compile) so the helper can

‎Sources/AWSLambdaPluginHelper/lambda-build/BuildArchitecture.swift‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,15 @@ enum BuildArchitecture: String, Codable, CustomStringConvertible {
3232
#endif
3333
}
3434

35+
/// The Static Linux SDK (musl) target triple for this architecture, passed to
36+
/// `swift build --swift-sdk`.
37+
var muslTriple: String {
38+
switch self {
39+
case .x64: return "x86_64-swift-linux-musl"
40+
case .arm64: return "aarch64-swift-linux-musl"
41+
}
42+
}
43+
3544
/// Parses the `--architecture` value, defaulting to the host architecture when omitted.
3645
static func parse(_ value: String?) throws -> Self {
3746
guard let value else {

‎Sources/AWSLambdaPluginHelper/lambda-build/BuildBackends/StaticLinuxSDKBuildBackend.swift‎

Lines changed: 126 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,126 @@
1+
//===----------------------------------------------------------------------===//
2+
//
3+
// This source file is part of the SwiftAWSLambdaRuntime open source project
4+
//
5+
// Copyright SwiftAWSLambdaRuntime project authors
6+
// Copyright (c) Amazon.com, Inc. or its affiliates.
7+
// Licensed under Apache License v2.0
8+
//
9+
// See LICENSE.txt for license information
10+
// See CONTRIBUTORS.txt for the list of SwiftAWSLambdaRuntime project authors
11+
//
12+
// SPDX-License-Identifier: Apache-2.0
13+
//
14+
//===----------------------------------------------------------------------===//
15+
16+
#if canImport(FoundationEssentials)
17+
import FoundationEssentials
18+
#else
19+
import Foundation
20+
#endif
21+
22+
/// Cross-compiles products with the Static Linux SDK (musl), producing a statically-linked binary
23+
/// that runs on Amazon Linux without a container runtime.
24+
///
25+
/// Unlike ``ContainerBuildBackend`` this shells out to `swift` directly on the host — no docker or
26+
/// Apple `container` involved. The target architecture is selected with `--swift-sdk <musl-triple>`,
27+
/// so this backend genuinely cross-compiles (e.g. building an arm64 binary on an x64 host and vice
28+
/// versa), independent of the host architecture.
29+
///
30+
/// The SDK must be installed beforehand (`swift sdk install …`). The plugin's network sandbox
31+
/// (`.docker` scope) forbids downloading it at build time, so this backend detects a missing SDK
32+
/// and fails with install guidance rather than attempting to fetch it.
33+
@available(LambdaSwift 2.0, *)
34+
struct StaticLinuxSDKBuildBackend: BuildBackend {
35+
/// The target CPU architecture, mapped to a musl target triple via ``BuildArchitecture/muslTriple``.
36+
let architecture: BuildArchitecture
37+
38+
/// Path to the `swift` executable resolved by the plugin (the toolchain location differs across
39+
/// hosts, so we never hardcode `/usr/bin/swift` here).
40+
let swiftToolPath: URL
41+
42+
let name = "swift-static-sdk"
43+
44+
func build(
45+
packageIdentity: String,
46+
packageDirectory: URL,
47+
products: [String],
48+
buildConfiguration: BuildConfiguration,
49+
noStrip: Bool,
50+
verboseLogging: Bool
51+
) throws -> [String: URL] {
52+
53+
// verify the swift binary exists at the resolved path
54+
guard FileManager.default.fileExists(atPath: self.swiftToolPath.path()) else {
55+
throw BuilderErrors.swiftToolNotFound(self.swiftToolPath.path())
56+
}
57+
58+
let triple = self.architecture.muslTriple
59+
60+
// Build into a dedicated scratch path, NOT the package's default `.build`. This plugin runs
61+
// as a SwiftPM command plugin, which holds the workspace lock on `.build` for its whole
62+
// duration; a nested `swift build` targeting the same `.build` would block forever waiting
63+
// for that lock. A separate scratch path sidesteps the deadlock (the container backend does
64+
// not hit this because its build runs inside the container, not against the host `.build`).
65+
let scratchPath = packageDirectory.appending(path: ".build").appending(path: "lambda-static-sdk")
66+
67+
// Resolve the build output path with the same `--swift-sdk` selector the build uses. This
68+
// doubles as the SDK preflight: SwiftPM resolves the SDK exactly as a real build would, so
69+
// if no SDK targets the triple this fails, and we surface actionable install guidance. We
70+
// cannot download the SDK ourselves (the plugin sandbox limits network to Docker).
71+
//
72+
// `swift sdk list` is deliberately NOT used here: it prints SDK bundle identifiers (e.g.
73+
// `swift-…_static-linux-0.1.0`), which do not contain the target triple, so matching on the
74+
// triple gives false negatives even when the SDK is installed.
75+
let binPath: String
76+
do {
77+
binPath = try Utils.execute(
78+
executable: self.swiftToolPath,
79+
arguments: [
80+
"build", "-c", buildConfiguration.rawValue,
81+
"--swift-sdk", triple,
82+
"--scratch-path", scratchPath.path(),
83+
"--show-bin-path",
84+
],
85+
customWorkingDirectory: packageDirectory,
86+
logLevel: verboseLogging ? .debug : .silent
87+
).trimmingCharacters(in: .whitespacesAndNewlines)
88+
} catch {
89+
throw BuilderErrors.staticSDKNotInstalled(triple)
90+
}
91+
let buildOutputPath = URL(fileURLWithPath: binPath)
92+
93+
print("-------------------------------------------------------------------------")
94+
print("building \"\(packageIdentity)\" with the Static Linux SDK (\(triple))")
95+
print("-------------------------------------------------------------------------")
96+
97+
var builtProducts = [String: URL]()
98+
for product in products {
99+
print("building \"\(product)\"")
100+
var buildArguments = [
101+
"build", "-c", buildConfiguration.rawValue,
102+
"--product", product,
103+
"--swift-sdk", triple,
104+
"--scratch-path", scratchPath.path(),
105+
"--static-swift-stdlib",
106+
]
107+
if !noStrip {
108+
buildArguments += ["-Xlinker", "-s"]
109+
}
110+
try Utils.execute(
111+
executable: self.swiftToolPath,
112+
arguments: buildArguments,
113+
customWorkingDirectory: packageDirectory,
114+
logLevel: verboseLogging ? .debug : .output
115+
)
116+
117+
let productPath = buildOutputPath.appending(path: product)
118+
guard FileManager.default.fileExists(atPath: productPath.path()) else {
119+
print("expected '\(product)' binary at \"\(productPath.path())\"")
120+
throw BuilderErrors.productExecutableNotFound(product)
121+
}
122+
builtProducts[product] = productPath
123+
}
124+
return builtProducts
125+
}
126+
}

‎Sources/AWSLambdaPluginHelper/lambda-build/Builder.swift‎

Lines changed: 40 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -35,9 +35,13 @@ struct Builder {
3535
}
3636

3737
// Select the build backend: build natively when already on an Amazon Linux host,
38-
// otherwise cross-compile using the backend chosen by --cross-compile.
38+
// otherwise cross-compile using the backend chosen by --cross-compile. An explicit
39+
// --cross-compile swift-static-sdk always cross-compiles (it can target either
40+
// architecture), so it takes precedence over the native path even on Amazon Linux.
3941
let backend: any BuildBackend
40-
if self.isAmazonLinux(.al2) || self.isAmazonLinux(.al2023) {
42+
if configuration.crossCompileMethod == .swiftStaticSdk {
43+
backend = try configuration.makeCrossCompileBackend()
44+
} else if self.isAmazonLinux(.al2) || self.isAmazonLinux(.al2023) {
4145
// A native build compiles for the host architecture only; it cannot target another one.
4246
// Recording a mismatched architecture in the manifest would recreate the very bug the
4347
// --architecture flag exists to prevent, so reject an explicit cross-architecture request.
@@ -152,7 +156,10 @@ struct Builder {
152156
--cross-compile <method> The cross-compilation method to use.
153157
Values: docker, container, swift-static-sdk, custom-sdk
154158
(default is docker)
155-
Note: swift-static-sdk and custom-sdk are not yet supported.
159+
swift-static-sdk requires a pre-installed Static Linux
160+
SDK (musl); it needs no docker/container. Install it with
161+
'swift sdk install <url>'.
162+
Note: custom-sdk is not yet supported.
156163
--archive-format <format> The packaging format for the build artifact.
157164
Values: zip, oci
158165
(default is zip)
@@ -329,15 +336,26 @@ struct BuilderConfiguration: CustomStringConvertible {
329336
/// everything a backend needs (the resolved tool path, base image, and image-update
330337
/// preference), so the factory lives here rather than on ``CrossCompileMethod``.
331338
func makeCrossCompileBackend() throws -> any BuildBackend {
332-
let cli = try self.makeContainerCLI()
333-
return ContainerBuildBackend(
334-
cli: cli,
335-
toolPath: self.crossCompileToolPath,
336-
baseImage: self.baseDockerImage,
337-
disableImageUpdate: self.disableDockerImageUpdate,
338-
architecture: self.architecture,
339-
method: self.crossCompileMethod
340-
)
339+
switch self.crossCompileMethod {
340+
case .docker, .container:
341+
return ContainerBuildBackend(
342+
cli: try self.makeContainerCLI(),
343+
toolPath: self.crossCompileToolPath,
344+
baseImage: self.baseDockerImage,
345+
disableImageUpdate: self.disableDockerImageUpdate,
346+
architecture: self.architecture,
347+
method: self.crossCompileMethod
348+
)
349+
case .swiftStaticSdk:
350+
// The Static Linux SDK build shells out to `swift`, not a container CLI. The plugin
351+
// resolves the swift toolchain and forwards its path via --cross-compile-tool-path.
352+
return StaticLinuxSDKBuildBackend(
353+
architecture: self.architecture,
354+
swiftToolPath: self.crossCompileToolPath
355+
)
356+
case .customSdk:
357+
throw BuilderErrors.unsupportedCrossCompileMethod(self.crossCompileMethod)
358+
}
341359
}
342360

343361
/// Resolves the ``ContainerCLI`` argument flavor for the configured cross-compile method.
@@ -402,6 +420,8 @@ enum BuilderErrors: Error, CustomStringConvertible {
402420
case unsupportedCrossCompileMethod(CrossCompileMethod)
403421
case unsupportedArchiveFormat(ArchiveFormat)
404422
case containerCLINotFound(CrossCompileMethod)
423+
case swiftToolNotFound(String)
424+
case staticSDKNotInstalled(String)
405425
case failedWritingDockerfile
406426
case failedParsingDockerOutput(String)
407427
case processFailed([String], Int32)
@@ -439,6 +459,14 @@ enum BuilderErrors: Error, CustomStringConvertible {
439459
+ "For information on how to install and use Swift cross-compilation SDKs, visit: "
440460
+ "https://www.swift.org/documentation/articles/static-linux-getting-started.html"
441461
}
462+
case .swiftToolNotFound(let path):
463+
return "The 'swift' executable was not found at the expected path '\(path)'."
464+
case .staticSDKNotInstalled(let triple):
465+
return
466+
"No Static Linux SDK targeting '\(triple)' is installed. "
467+
+ "Install it with 'swift sdk install <url>' and try again. "
468+
+ "For information on how to install and use Swift cross-compilation SDKs, visit: "
469+
+ "https://www.swift.org/documentation/articles/static-linux-getting-started.html"
442470
case .failedWritingDockerfile:
443471
return "failed writing dockerfile"
444472
case .failedParsingDockerOutput(let output):

‎Sources/AWSLambdaPluginHelper/lambda-build/CrossCompileMethod.swift‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -34,8 +34,8 @@ enum CrossCompileMethod: String, CustomStringConvertible {
3434

3535
var isSupported: Bool {
3636
switch self {
37-
case .docker, .container: return true
38-
case .swiftStaticSdk, .customSdk: return false
37+
case .docker, .container, .swiftStaticSdk: return true
38+
case .customSdk: return false
3939
}
4040
}
4141

0 commit comments

Comments
 (0)