Wire things up in a flake

[blaggacao]

I've read the TL;DR.

But I'm looking at a wall to climb, now.

As others have pointed out interop is cheap with: https://github.com/edolstra/flake-compat

[blaggacao]

... wall is in the context of a service provider to divnix/std#165

[blaggacao]

This comment looks like particularly relevant to your flake TL; DR; → NixOS/flake-compat#39 (comment)

[olebedev]

Hi @blaggacao,

This comment looks like particularly relevant to your flake TL; DR; → NixOS/flake-compat#39 (comment)

From my understanding, this comment is about to don't copying the whole repo into the Nix store but this is not about what I have been talking in this comment. I was more talking about the flakes implementation and how they are coupled with git tool and rely on it to infer a diff and what files have been affected by a change set.

[blaggacao]

In this case, I don't think I grok the issue, yet.

But it also seems like the undesired behaviour is entirely attached to the use of code paths involved in the new flakes commands.

So a forward-compatible flake layout, per se, wouldn't be affected, at all.

There might be actually a couple of additional requirements to weave js2nix into an integration framework, such as not claiming hard coded contracts (on files such as packages.nix and it's output schema). But maybe that's not even actually the case but just implied (in the eye of the reader) by the Readme's posture. I haven't dug deep enough, yet.

[typetetris]

Maybe there are some misunderstandings.

I read "Wire things up in a flake" as "Please provide a flake.nix/flake.lock for js2nix".

Together with flake-compat this shouldn't hurt monorepo users
too much, as they can continue to ignore flakes and only have the slight discomfort of having one copy
of the js2nix source code in the store. No "flake code paths", which are not already taken, should be
taken in nix/nix-daemon in this case, I think.

The results produced by js2nix should be easily usable in a flake, if the code generation is used to avoid
import from derivation.

Am I missing something?

[olebedev]

Oh, I haven't read this in that way. I am fully on board with having flake.{nix,lock}files along the way! More than happy to review PRs if someone interested in contributing into it. Otherwise, I will try to allocate some time for that, but note, for the @Canva use case it's rather theoretical, ephemeral benefit since we don't use flakes. That is, I most likely I will do it in the blind, without relying on a particular use case, so contribution of those who use flakes in practice is very much appreciated :)

[typetetris]

With flake.{nix,lock} and flake-compat, flake.nix would be the source of truth.

So maybe that is not a great idea, if you don't want to look at some flake documentation and doing the occasional nix --extra-experimental-features "flake nix-command" flake update command.

Having separate flake.nix and default.nix/shell.nix (no flake-compat here) would maybe diverge to fast to not be frustrating for everyone.

[olebedev]

The only one aspect of flake.{nix,lock} + flake-compat usage I haven't thought about before is the ability to define the dependencies at the call site. The idea of flakes is that you describe dependencies of your Nix package (flake) in the repository so you don't have to deal with its dependencies at the call site, but if you want, it's easy to do with flakes as well.

This is great, however, IIUC flake-compat in default.nix file removes capability of passing dependencies with callPackage function call and will anyways read the flake.lock file to determine the project dependency closure regardless of what you passing in callPackage function call, no?

For example, at the call-site, I would like to have the run time dependency of Node.js to be the 12 version:

with import <nixpkgs> { };

let
  js2nix = callPackage (builtins.fetchGit {
    url = "ssh://git@github.com/canva-public/js2nix.git";
    ref = "main";
  }) {
    nodejs = nodejs_12-x;
  };
  nodeModules = js2nix.makeNodeModules ./package.json {
    tree = js2nix.load ./yarn.lock { };
  };
in nodeModules

But, if we use flake-compat in the default.nix, it won't be possible to override, right?

I that is correct understanding of how the flake-compat work, I would suggest to have the default.nix file as an entry point for both the project flake and for the non-flake consumers.

Meaning, the default.nix file returns a function that expects project dependencies passed thought, like any other package in the nixpkgs repository, and in the flake.nix we do callPackage of the default.nix with the dependencies declared in flake.{nix,lock} but not the other way around; i.e. when we use flake-compat in the default.nix.

This will resolve the issue with dependencies injection for non-flake consumers along with make the project a first-class flake for those who use flakes at call sites. And the most importantly we remove the divergence you have been talking above.

@typetetris, @blaggacao, what do you think?

[blaggacao]

{inputs, system}:

let
  inherit (inputs) nixpkgs js2nix;
  inherit (nixpkgs.legacyPackages.${system}) callPackage;
  js2nix' = callPackage js2nix {
    nodejs = nodejs_12-x;
  };
  nodeModules = js2nix'.makeNodeModules ./package.json {
    tree = js2nix'.load ./yarn.lock { };
  };
in nodeModules

Should * work with a top __functor outputs that makes a js2nix flake a callPackage-able package function.

* At least the nixpkgs library version of functionArgs (the implementation detail behind callPackage) is aware of functors.

Iirc, I've made a patch to flake-compat a while back to not remove functors from the output.

[olebedev]

It removes the __functor property here https://github.com/edolstra/flake-compat/blob/master/default.nix#L193.

I am wondering, would it work if we just do pkgs.callPackage ./default.nix from flake.nix? This should remove the divergence problem and make flake and non-flake folks happy, no?

[blaggacao]

Oh, I did remove the functor, then. Must have been for another reason in the Nix CLI jungle. 😄

That approach would also work and in most cases people can operate with .overrideAttrs if they need to amend any (package function) inputs.