feat(forge): [ALPHA] add soldeer as an optional package manager.

[mario-eth]

Motivation

This pull request is related to issue #5966, which essentially outlines the reasons I believe a dedicated dependency manager for Solidity is necessary.

To summarize, relying on git submodules is not a reliable solution. As a security researcher, I've observed numerous projects defaulting to using npmjs to avoid dealing with git submodules. Considering npmjs was designed for the JavaScript ecosystem, why not create a package manager specifically tailored for Solidity?

Solution

My proposal involves developing an open-source Rust-based dependency manager that functions similarly to crates.io or npmjs for Solidity. I'm introducing Soldeer, a straightforward package manager with four key features:

• install: This function allows the installation of dependencies from https://soldeer.xyz/, or from a custom link provided by the user. The dependency must be pre-zipped. For instance, you could zip your dependency, host it on a CDN, and share the link with your team.
• update: This function is designed to update the existing dependencies specified in the project.
• login: Enables users to log into the repository, allowing them to push new dependencies.
• push: Allows users to push a directory to their account on https://soldeer.xyz/ as a new project version.

Integration

Soldeer is built as a library that can be also used standalone.
For foundry, i've just integrated the library and based on the commands, i redirect the logic to the library. The integration is not that big.
For reviewing probably one will need to check the open-source github repository to see how it works.
https://github.com/mario-eth/soldeer

Future Plans:

• Integrate Solidity-based tools such as Slither and Mythril runners to identify and report errors before pushing changes.
• Implement audit report features for each version directly on the platform, allowing users to easily verify whether a specific version has been audited.
• Collecting feedback and listening to the community.

ALPHA STAGE

The current implementation of Soldeer is intended to serve as an alternative to git submodules, rather than a replacement, until it matures sufficiently to become the default package manager.

[mario-eth]

I've updated the pull request with version 0.2.6 of soldeer.
This version does the following things:

1. Previously i was using sdependencies tag to store the dependencies in the foundry.toml but after a discussion with the foundry team, they suggested that it should be renamed to dependencies.
2. The dependency config in foundry.toml has been changed to
   [DEP-NAME]~[DEP-VERSION] = {version = "", url=""} respecting Cargo.toml.
   I had to include the version in the dependency key because one project might want to use 1 dependency with multiple versions and Cargo.toml does not support that, the only way to do it is to have different keys pointing to different versions.

[mario-eth]

Updated to version 0.2.7.
Now the dependency name is the key in the config file, previously was [DEP-NAME]~[DEP-VERSION]. This was needed to respect the cargo way.
If the need for multiple versions for the same dependency, an issue has been opened for that mario-eth/soldeer#34

[onbjerg]

if possible, can we also add a simple integration test? see e.g. https://github.com/foundry-rs/foundry/tree/master/crates/forge/tests/it

[mario-eth]

if possible, can we also add a simple integration test? see e.g. https://github.com/foundry-rs/foundry/tree/master/crates/forge/tests/it

I have some issues on how to approach this, most of the integrations are related with cheatcodes as i can see.
Do you want me to do a full integration test? like running the soldeer command for install for example, then check if it was installed correctly and so on?

[onbjerg]

@mario-eth Sorry about that I meant to link https://github.com/foundry-rs/foundry/tree/master/crates/forge/tests/cli which has some CLI tests

[mario-eth]

if possible, can we also add a simple integration test? see e.g. https://github.com/foundry-rs/foundry/tree/master/crates/forge/tests/it

Added cli tests here

[onbjerg]

Some comment on the dependency format

I also think we ideally ditch the "v" prefix so we can support semver operators later, see e.g. https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#specifying-dependencies-from-cratesio

[mario-eth]

Some comment on the dependency format

I also think we ideally ditch the "v" prefix so we can support semver operators later, see e.g. https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#specifying-dependencies-from-cratesio

Removed the v as well.

[onbjerg]

i think this is a great first step, lot of things we can add on top and expand upon. keeping it simple for now is a good idea imo, and as long as we agree on how dependencies should be specified long term, the internals do not matter as much, as the experience will be stable for the user

ptal @mattsse

[mattsse]

sorry for the delay, I'm okay with proceeding here.
only have a few nits

[mattsse]

I want this to be a new type that wraps the betreemap

[mario-eth]

Updated, does that work? or maybe i misunderstood?

[mattsse]

last pedantic nit

[mattsse]

let's try this!

[dalechyn]

@mario-eth, before this is merged, wanted to ask on the concept of nested dependencies.

I'm using soldeer to install @uniswap-v2-periphery-1.1.0-beta.0, yet it's not usable since it has @uniswap/lib dependency which is not being resolved automatically.

Moreover, the imports from such and similar contracts often use other "package" names, so that if you even install one, you have to remap them manually.

Is that something that's being tackled or planned to do after the merge?

[mario-eth]

@mario-eth, before this is merged, wanted to ask on the concept of nested dependencies.

I'm using soldeer to install @uniswap-v2-periphery-1.1.0-beta.0, yet it's not usable since it has @uniswap/lib dependency which is not being resolved automatically.

Moreover, the imports from such and similar contracts often use other "package" names, so that if you even install one, you have to remap them manually.

Is that something that's being tackled or planned to do after the merge?

Hey,
Unfortunately soldeer does not track nested dependencies for now, this is just the first step to making an actual package manager.

The reason why it does not track it's because, in your example, what version of @uniswap/lib-dependency does the uniswap-v2-periphery-1.1.0-beta.0 uses? We can't assume it's the last one.

While this is the first step, we must start define how we want these dependencies to be tracked, a step towards that i tried to do in soldeer, in the remappings, i tried to map the package+version as a path that it will be hardcoded in the solidity files so that instead of using
import '@openzeppelin-contracts/token/ERC20.sol'
we should do
import '@openzeppelin-contracts-5.0.2/token/ERC20.sol'
In this way we can infer the dependency version.

[RensR]

Any timeline when this would be released? Really exited to not use submodules

[mario-eth]

Bumped the Soldeer version to v0.2.15.

I had to solve two bugs:

• One was regarding replacing versions in the config files. I was using append instead of overwrite, which would have broken the config if the new version string length was smaller than the old one.
• When pushing, zipping new files would have created an empty directory due to the zip dependency bump.

[mattsse]

okay last rebase then I'm sending it