Typings directory when in-lining project references

[lavcraft]

Troubleshooting

My reproducible example - https://github.com/lavcraft/rollup-typescript-project-references-example-problems

You need

1. clone
2. npm run ci

Follow to next instructions

1. Does tsc have the same output? If so, please explain why this is incorrect behavior

Run npm run -w @project/sdk build - tsc variant works fine and generate d.ts files into correct directories

1. Does your Rollup plugin order match this plugin's compatibility? If not, please elaborate

I guess order is correct:

        nodeResolve(),
        typescript2()

1. Can you create a minimal example that reproduces this behavior? Preferably, use this environment for your reproduction

My example - https://github.com/lavcraft/rollup-typescript-project-references-example-problems
Follow to README or read next lines:

Two commands for feel difference between tsc and rollup in my case:

TSC baseline

1. npm run -w @project/sdk build - tsc variant
2. And see d.ts files in dirs

tree packages/sdk/build
packages/sdk/build
├── actions
│   ├── create.d.ts
│   ├── create.d.ts.map
│   └── create.js
├── models
│   ├── user.d.ts
│   ├── user.d.ts.map
│   └── user.js
└── tsconfig.tsbuildinfo

All d.ts files existed in right location (except d.ts from project from project reference, but it is ok for tsc i guess)

Rollup variant

1. rm -rf packages/sdk/build && npm run -w @project/sdk rollup
2. And see d.ts files in dirs

tree packages/sdk/build
packages/sdk/build
├── actions
│   ├── create.js
│   └── create.js.map
├── models
│   ├── user.js
│   └── user.js.map
├── sdk      <------------------------ THIS DIR SUSPICIOUS
│   └── src
│       ├── actions
│       │   ├── create.d.ts
│       │   └── create.d.ts.map
│       └── models
│           ├── user.d.ts
│           └── user.d.ts.map
└── shared <------------------------ THIS DIR SUSPICIOUS
    └── src
        └── test
            ├── util.d.ts
            └── util.d.ts.map

Created to excess directories with typings - sdk and shared

Please help me to explain right behaviour or workaround. Because my next step is apply dts plugin for obtain valid d.ts files without internal paths. But some difficulties happened when all this d.ts located in strange directories :)

Environment

Versions

  System:
    OS: macOS 13.3.1
    CPU: (8) arm64 Apple M1 Pro
    Memory: 52.64 MB / 32.00 GB
    Shell: 5.9 - /bin/zsh
  Binaries:
    Node: 18.16.0 - ~/.nvm/versions/node/v18.16.0/bin/node
    npm: 9.5.1 - ~/.nvm/versions/node/v18.16.0/bin/npm
    pnpm: 8.6.1 - /opt/homebrew/bin/pnpm
  npmPackages:
    typescript: ^5.1.5 => 5.1.5

rollup.config.js

:

import commonjs from '@rollup/plugin-commonjs';
import {nodeResolve} from '@rollup/plugin-node-resolve';
import typescript2 from 'rollup-plugin-typescript2';
import fs from 'fs';
import path from 'path';
import * as url from 'url';

const __dirname = url.fileURLToPath(new URL('.', import.meta.url));
/**
 * @type {import('rollup').RollupOptions}
 */
const config = {
    input: {
        ...files('actions'),
        ...files('models'),
    },
    output: {
        sourcemap: true,
        dir: 'build',
        preserveModules: false,
        format: 'esm'
    },
    plugins: [
        nodeResolve(),
        typescript2({
            // build: true,
            // useTsconfigDeclarationDir: true,
            // declarationDir: './types',
        }),
        // commonjs(),
    ],
    external: [
        'undici',
        'querystring',

        'mobx',
        'mobx-react-lite',
        'mobx-utils',
        'react',
    ],
    watch: {
        chokidar: true,
        exclude: ['build/**', '**/*.d.ts', '**/*.js']
    }
};

function files(dir = '.', exclude = ['__tests__']) {
    return fs
        .readdirSync(path.resolve(__dirname, `src/${dir}`))
        .reduce((prev, current) => {
            if (exclude.includes(current)) return prev;
            const targetName = current.replace('.ts', '');
            prev[`${dir}/${targetName}`] = `./src/${dir}/${current}`;
            return prev;
        }, {});
}
export default [config];

[agilgur5]

I'll need to look into this more as it's been a while since I've seen this kind of issue, but really quick off the top of my head (and on my phone): setting rootDir: "src/" in tsconfig was often the solution with directory placement.
If the rootDir is up a directory, that would make sense why src is in the typings. In this case, it seems like it might be defaulting to the root of the monorepo?

Workspaces (I don't think I've used NPM's variant of them before) and project references could complicate things a bit though.

I'll come back with more details and previous issue references. EDIT: See #275 (comment)

EDIT2: nvm, the rootDir is correct, see below comment

[agilgur5]

Oh there are indeed some complications in your case. I see you have configured TS paths, and that it goes up two directories. That would indeed set your rootDir to be packages.

Unfortunately, aliases are not truly supported by TS. #201 has pretty significant details and several workarounds to support aliases. IIRC, there are some related issues on monorepos specifically that I might be able to pull up as well (#237 is one)

Using one of those workarounds can probably fix your issue. Also not sure if @shared is meant to be compiled in or meant as a separate external package reference when published.

[lavcraft]

Thanks for your comment @agilgur5

According comment about rootDir - try it and what will happen when i set rootDir (previously deleted because found some recommendation in issues)

rootDir Problem — different behaviour with tsc and rollup

Try to use

1. npm run -w @project/sdk build

2. npm run -w @project/sdk rollup

In rollup variant number 2 we will see next error:

[!] (plugin rpt2) RollupError: src/actions/create.ts:1:20 - error TS6059: File '<...>/rollup-ws-problem/packages/shared/src/test/util.ts' is not under 'rootDir' '<...>/rollup-ws-problem/packages/sdk/src'. 'rootDir' is expected to contain all source files.

But in tsc variant number 1 - it works fine

I can avoid this mistake when delete rootDir

[lavcraft]

Thanks for this workarounds:
#237
#201

But in my opinion my example highlight another kind of problem. I try to process resulting declarations, but it's hard, because located in strange directories (also located strange dir when i set declarationDir, but also copied into declaration dir with <workspace_dir>/src prefix)

I guess it's not reliable behaviour (i may be wrong). I am trying to find a consistent solution to avoid problems in the future.

In my example with rollup, shared workspace sources are inlined into the resulting sdk perfectly. But types have some problems, because previously when i have used @rollup/plugin-typescript, I process resulting d.ts files via dts plugin with respectExternal prop and inline all my shared d.ts into sdk project and replace paths in declarations. But with typescript2 this workaround doesn't work

[lavcraft]

@agilgur5 could you help me with one more advice for avoiding excess .d.ts files generation in <workspace_dir_name/src/**/*.d.ts dirs? Workarounds from previous message related to other issues or not working in this case i guess

[agilgur5]

In rollup variant number 2 we will see next error:

[!] (plugin rpt2) RollupError: src/actions/create.ts:1:20 - error TS6059: File '<...>/rollup-ws-problem/packages/shared/src/test/util.ts' is not under 'rootDir' '<...>/rollup-ws-problem/packages/sdk/src'. 'rootDir' is expected to contain all source files.

Sorry, I probably should have clarified in my second comment that you could ignore my first comment after I saw you actually are trying to import from a sibling directory. I've hid both of our comments on that to cause less confusion. Will also hide this comment.

Will respond in a separate comment to the rest.

[agilgur5]

goal? in-lining?

In my example with rollup, shared workspace sources are inlined into the resulting sdk perfectly.
inline all my shared d.ts

Ah ok, so you are trying to in-line the JS without in-lining the types? Or in-line both?
In-lining both is what rpt2 is currently doing in your repro example.

In your repro example, you might notice that npm run -w @project/sdk build, the tsc variant, just treats @shared as an external:

// create.js
import { test } from "@shared/test/util";
export function create(user) {
    console.log(`user = ${user}`);
    test(() => { });
}

tsc will not in-line @shared and does not produce declarations for @shared, as that's a separate project.

With Rollup, you can mark @shared as an external to not in-line it.
Unfortunately, however, rpt2 is still producing declarations for @shared. I was able to get it to not produce declarations for @shared when I added it to rpt2's exclude, however it still outputs sdk's declarations in build/sdk/src/.

root cause analysis

I spent a good amount of time trying to debug this and unfortunately hit a bit of a wall 😕

This seems to be because rpt2's internal TS LanguageService is still interpreting your rootDir as packages instead of sdk/src.
I was able to confirm that by setting rootDir: 'src/' and then tracing the error with verbosity: 3. rpt2 is getting an error during semantic diagnostics on create.ts, which it leaves entirely up to the TS LanguageService.

So for some reason the LanguageService is interpreting your project reference differently from tsc. Unfortunately, the LanguageService API is barely documented, so folks writing TS compiler integrations (like rpt2) have to effectively "reverse engineer" some of the behavior. This seems to be one of those instances where we have to do that, as it's unclear why it's acting differently right now when given the same tsconfig.
I dug into the TS Compiler source code a bit but this area of it is really dense with variables referencing variables referencing variables in a very long and deep chain.

As a note, it had this behavior even when I removed the aliases and just manually referenced the other project, so this does not seem to be due to integration with TS paths.

EDIT: It actually was due to the paths aliases 😕 . See below comment.

temporary conclusion

Given the significant analysis / reverse-engineering and debugging this requires, I don't think this will be solved soon (unless someone happens to make a really good guess). As such, I would recommend using a workaround for now

[agilgur5]

OH. I got it to work.

As a note, it had this behavior even when I removed the aliases and just manually referenced the other project, so this does not seem to be due to integration with TS paths.

It seems like it may indeed be paths related. Specifically, I changed your paths to use the output directory, shared/build/*, instead of shared/src/*:

     "paths": {
       "@shared/*": [
-         "../../shared/src/*"
+         "../../shared/build/*"
       ]
     }

and then it worked with no other changes needed 😮
it output .js and .d.ts to build/actions and build/models with no extra build/sdk/src directory for the declarations.

adding /@shared/ to external will get rid of the Rollup warning as well.

EDIT: sent a PR with fixes: lavcraft/rollup-typescript-project-references-example-problems#1