Skip to content

Creates a smaller build of TypeScript which just has the language services #33584

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Sep 30, 2019
3 changes: 2 additions & 1 deletion .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ scripts/buildProtocol.js
scripts/ior.js
scripts/authors.js
scripts/configurePrerelease.js
scripts/configureTSCBuild.js
scripts/open-user-pr.js
scripts/open-cherry-pick-pr.js
scripts/processDiagnosticMessages.d.ts
Expand All @@ -45,4 +46,4 @@ TEST-results.xml
package-lock.json
tests
.vscode
.git
.git
4 changes: 3 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,8 @@ scripts/buildProtocol.js
scripts/ior.js
scripts/authors.js
scripts/configurePrerelease.js
scripts/configureLanguageServiceBuild.js
scripts/createLanguageServiceBuild.js
scripts/open-user-pr.js
scripts/open-cherry-pick-pr.js
scripts/processDiagnosticMessages.d.ts
Expand Down Expand Up @@ -92,4 +94,4 @@ tests/cases/user/webpack/webpack
tests/cases/user/puppeteer/puppeteer
tests/cases/user/axios-src/axios-src
tests/cases/user/prettier/prettier
.eslintcache
.eslintcache
6 changes: 5 additions & 1 deletion .npmignore
Original file line number Diff line number Diff line change
Expand Up @@ -31,4 +31,8 @@ yarn.lock
CONTRIBUTING.md
TEST-results.xml
.dockerignore
Dockerfile
Dockerfile
.DS_Store
.eslintrc.json
.yarnrc
tmp
4 changes: 4 additions & 0 deletions Gulpfile.js
Original file line number Diff line number Diff line change
Expand Up @@ -590,6 +590,10 @@ const configureExperimental = () => exec(process.execPath, ["scripts/configurePr
task("configure-experimental", series(buildScripts, configureExperimental));
task("configure-experimental").description = "Runs scripts/configurePrerelease.ts to prepare a build for experimental publishing";

const createLanguageServicesBuild = () => exec(process.execPath, ["scripts/createLanguageServicesBuild.js"]);
task("create-language-services-build", series(buildScripts, createLanguageServicesBuild));
task("create-language-services-build").description = "Runs scripts/createLanguageServicesBuild.ts to prepare a build which only has the require('typescript') JS.";

const publishNightly = () => exec("npm", ["publish", "--tag", "next"]);
task("publish-nightly", series(task("clean"), task("LKG"), task("clean"), task("runtests-parallel"), publishNightly));
task("publish-nightly").description = "Runs `npm publish --tag next` to create a new nightly build on npm";
Expand Down
87 changes: 87 additions & 0 deletions scripts/configureLanguageServiceBuild.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
/// <reference types="node"/>
import { normalize, dirname, join } from "path";
import { readFileSync, writeFileSync, unlinkSync, existsSync } from "fs";
import assert = require("assert");
import { execSync } from "child_process";
const args = process.argv.slice(2);

/**
* A minimal description for a parsed package.json object.
*/
interface PackageJson {
name: string;
bin: {};
main: string;
scripts: {
prepare: string
postpublish: string
}
}

function main(): void {
if (args.length < 1) {
console.log("Usage:");
console.log("\tnode configureTSCBuild.js <package.json location>");
return;
}

// Acquire the version from the package.json file and modify it appropriately.
const packageJsonFilePath = normalize(args[0]);
const packageJsonValue: PackageJson = JSON.parse(readFileSync(packageJsonFilePath).toString());

// Remove the bin section from the current package
delete packageJsonValue.bin;
// We won't be running eslint which would run before publishing
delete packageJsonValue.scripts.prepare;
// No infinite loops
delete packageJsonValue.scripts.postpublish;

// Set the new name
packageJsonValue.name = "@typescript/language-services";

writeFileSync(packageJsonFilePath, JSON.stringify(packageJsonValue, /*replacer:*/ undefined, /*space:*/ 4));

// Remove the files which aren't use when just using the API
const toRemove = [
// JS Files
"tsserver.js",
"tsserverlibrary.js",
"typescriptServices.js",
"typingsInstaller.js",
"tsc.js",
// DTS files
"typescriptServices.d.ts",
"tsserverlibrary.d.ts"
];

// Get a link to the main dependency JS file
const lib = join(dirname(packageJsonFilePath), packageJsonValue.main);
const libPath = dirname(lib);

// Remove the sibling JS large files referenced above
toRemove.forEach(file => {
const path = join(libPath, file);
if (existsSync(path)) unlinkSync(path);
});

// Remove VS-specific localization keys
execSync("rm -rf loc", { cwd: dirname(packageJsonFilePath) });

// Remove runnable file reference
execSync("rm -rf bin", { cwd: dirname(packageJsonFilePath) });

///////////////////////////////////

// This section verifies that the build of TypeScript compiles and emits

const ts = require(lib);
const source = "let x: string = 'string'";

const results = ts.transpileModule(source, {
compilerOptions: { module: ts.ModuleKind.CommonJS }
});

assert(results.outputText.trim() === "var x = 'string';", `Running typescript with ${packageJsonValue.name} did not return the expected results, got: ${results.outputText}`);
}

main();
46 changes: 46 additions & 0 deletions scripts/createLanguageServiceBuild.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
/// <reference types="node"/>
import { join } from "path";
import { readFileSync, unlinkSync } from "fs";
import { tmpdir } from "os";
import { execSync, ExecSyncOptions } from "child_process";
import chalk from "chalk";

interface PackageJson {
name: string;
version: string
}

const exec = (cmd: string, opts?: ExecSyncOptions) => {
console.log(chalk.gray(`> ${cmd} ${opts ? JSON.stringify(opts) : ""}`));
execSync(cmd, opts);
};

const step = (msg: string) => {
console.log("\n\n" + chalk.bold("- ") + msg);
};

function main(): void {
console.log(chalk.bold("## Creating the language services build of TypeScript"));
process.stdout.write(chalk.grey("> node /scripts/createLanguageServiceBuild.ts"));

// Create a tarball of the current version
step("Packing the current TypeScript via npm.");
exec("npm pack");

const packageJsonValue: PackageJson = JSON.parse(readFileSync("package.json", "utf8"));
const tarballFileName = `${packageJsonValue.name}-${packageJsonValue.version}.tgz`;

const unzipDir = tmpdir();
step(`Extracting the built version into a temporary folder. ${unzipDir}/package`);
exec(`tar -xvzf ${tarballFileName} -C ${unzipDir}`);
unlinkSync(tarballFileName);

step(`Updating the build metadata`);
const packagePath = join(unzipDir, "package");
exec(`node scripts/configureLanguageServiceBuild.js ${join(packagePath, "package.json")}`);

step(`Deploying the language service`);
exec("npm publish --access public", { cwd: packagePath });
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have any other scripts that auto-publish? I don’t know what our publish process is like, but I assume it runs in CI. I would lean toward adding automation there rather than scripted into postpublish, as principle of least surprise.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IOW, the code seems fine (but I’ve never looked at our packing process before either, so I’m not the best one to review), but I’m skeptical of the trigger

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope, it doesn't happen on CI (I am happy to move it to a new step in there) and/or look at moving to deploy on CI ( e.g microsoft/types-publisher#659 ) and all we do is ship a tag to indicate we want a release

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

@weswigham weswigham Sep 27, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do! It definitely happens on CI! Anyone caught not publishing on CI will be [redacted]!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Specifically, we publish a prepared tarball directly, using npm publish $(System.ArtifactsDirectory)/typescript.tgz --tag $(PUBLISH_TAG)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So a postpublish script is actually wholly nonfunctional, since we don't even publish in a clone of our repo 😄

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hah! OK cool, perfect!

I'll that and add the extra step on CI 👯‍♂

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool - updated the deployment docs, and removed the auto-publish. Will give this a few hours for last feedback and merge today. Then once that's in I'll add the step to the pipeline.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This script is not executable in the release pipeline, as far as I'm aware. Releases are not executed within the context of the repo. (There's good reason for this, too, it means at no point does potentially malicious code get a chance to execute in a context where we have our publishing key set!) The code in the repo does not exist in the release pipeline. To publish a second package, you would need to publish a second pipeline artifact with the secondary tarball (which is just done with a few commands in the build pipeline, but I imagine you'll need to add a call to the configure script in addition to replicating those), and publish in a similar way as we do the first (with the npm publish release pipeline task).

}

main();