Let CoursierModules provide a self-information about depenencies

[lefou]

Many modules have and use dependencies. But it's rather hard to gather complete dependency information about all modules in a build. It may be rather simple to get dependencies from classic ScalaModule, as we know they have the ivyDeps target and friends. But many modules have hidden dependencies. E.g. Many worker and tool provider modules depend on additional libraries which are more or less hidden. Often, those modules have some version target, e.g. scalaVersion or proguardVersion, which will internally be used to resolve and fetch additional dependencies. In most cases, CoursierModule.resolveDeps is used for this, hence most modules already inherit from this trait.

On the other side, not only interested users or security auditors are interested in the big picture of used dependencies. There are many tools out there, that can provide useful benefits from analyzing the dependencies. Those tools can help find potential updates, security issues, licensing issues, classpath incompatibilities and many more useful stuff. Unfortunately, each tools integration need to re-invent ways to gather those dependency information. This leads to incomplete or incorrectly gathered information but also to wasted work and time.

Wouldn't it be nice to have an additional target, which can be used to just get all dependency information?

I think we agree, that the answer is "yes". It could be as simple as follows (although the choosen names are just picked to illustrate the idea, and I encourage you to provide suggestions)

def dependencyInformation: T[DepInfos]

In the following I hope we can discuss a good structure to provide this information. Here are some information we might want to include:

• compile, runtime, compile-only dependencies of Java, Scala and all derived modules like Scala.js, Kotlin, ...
• tools dependencies like Scala/Zinc compiler dependencies, but also everything we load in isolated classloaders like Protoc, Proguard, Twirl, Jacoco, Aspectj, Kotlin, and many more dependencies
• dependencies of the build tools itself, e.g. Mill plugins

Maybe, we can try to adapt or re-use the structure of already existing tools. E.g. the GitHub Dependency Graph (https://docs.github.com/en/rest/dependency-graph) comes to mind.

What do you think?

[lefou]

I just want to mention that we already support this functionality for the classic Java/Scala dependencies with the ivyDepsTree target, which can render a dependency tree based on functionality from coursier. But this information is limited to what ivyDeps and friends support and is currently just some rendered output.

[lefou]

As a start, I have some simple map-like structure in mind, which maps various kinds of dependencies to dependencies trees. But I have no idea, what the best way to classify various worker is. I think we may want to keep an open structure, but on the other side, some well defined names would make post-processing such trees much easier for tools.

Map(,
  "compile" -> // transitively resolved ivyDeps tree
  "runtime" -> // transitively resolvd runIvyDeps tree
  "worker" -> // don't know whats the best name for worker
  "tools" ->  // or tools
  "a-contrete-tool-name" -> // or should be list each tool separately?

[DamianReeves]

I just want to point out that in addition to the dependency information there is sometimes a need to grab metadata from the dependencies, such as:

• Licenses used
• Repository
• Authors
• Etc available from the POM, ivy files