CONTRIBUTING.md

Contributing to the Cardano Ledger

Before You Contribute

cardano-ledger is a core Cardano component with downstream impact across the whole ecosystem. To protect the core team's review bandwidth and ensure all changes align with the project's direction, we do not accept unsolicited contributions.

Comment on the issue you'd like to work on and wait for a maintainer to explicitly assign it to you. Pull requests for work that was not approved in advance will be closed without review.

The reason for this process is that reviewing code in such a security-critical project is time-expensive, and we need to make sure both your time and the reviewers' time are well spent.

Usage of AI tools for adding contributions to the repository is reserved for maintainers and core contributors only. Any pull request that was generated by AI or with use of AI will be immediately rejected.

For security-related issues please consult the disclosure policy for security vulnerabilities.

Roles and responsibilities

The @cardano-ledger group is responsible for helping with reviewing and merging pull requests, adjudicating technical (or other) disputes, releasing the ledger packages on CHaP.

@hamishmack can help with issues regarding this repository's continuous integration and nix infrastructure.

For security related issues please consult the security file in the Cardano engineering handbook.

Development

We use trunk based developement. Normal development will branch off of master and be merged back to master.

Recommended git configuration

Once you cloned the repository, it is recommended to run the following from the repository's root:

git config blame.ignoreRevsFile .git-blame-ignore-revs

This way git blame will ignore the commits specified in the .git-blame-ignore-revs file. This can come in handy if you want to exclude large commits with only formatting changes. You can ignore the above however, if you tend to look at git blame through GitHub. In that case, you don't have to do anything, as GitHub will pick up .git-blame-ignore-revs automatically and ignore the specified commits.

If you want to add further revisions to the ignore-revs file, just prepend the full commit hash that you want git blame to ignore and add the commit's title and date as a comment for clarity.

Releasing and versioning

See documentation on the adopted release and versioning processes in ledger.

Releasing the ledger packages to CHaP

Ledger packages are released to CHaP.

Also see the CHaP README for instructions.

Building

See the Readme for instructions on building.

GHC 9.x transition

We are transitioning to use GHC 9.x rather than GHC 8.10. We need to retain 8.10 compatibility until we are sure that the Cardano node can switch over to 9.8 without any problems. At that point we can drop it.

The main nix develop shell will now give you a GHC 9.8 compiler, but you can get a GHC 8.10 shell by calling

nix develop .#ghc8107

(this pattern can also be used to test any supported GHC version)

Updating dependencies

Our Haskell packages come from two package repositories:

• Hackage
• CHaP (which is essentially another Hackage)

The "index state" of each repository is pinned to a particular time in cabal.project. This tells Cabal to treat the repository "as if" it was the specified time, ensuring reproducibility. If you want to use a package version from repository X which was added after the pinned index state time, you need to bump the index state for X. This is not a big deal, since all it does is change what packages cabal considers to be available when doing solving, but it will change what package versions cabal picks for the plan, and so will likely result in significant recompilation, and potentially some breakage. That typically just means that we need to fix the breakage (and add a lower-bound on the problematic package), or add an upper-bound on the problematic package.

Note that cabal itself keeps track of what index states it knows about, so when you bump the pinned index state you may need call cabal update in order for cabal to be happy.

The Nix code which builds our packages also cares about the index state. This is represented by inputs managed by nix flake: You can update these by running:

• nix flake update haskellNix/hackage for Hackage
• nix flake update CHaP for CHaP (Cardano Haskell Packages)

or this, if you are on an older nix flake:

• nix flake lock --update-input haskellNix/hackage for Hackage
• nix flake lock --update-input CHaP for CHaP (Cardano Haskell Packages)

If you fail to do this you may get an error like this from Nix:

error: Unknown index-state 2021-08-08T00:00:00Z, the latest index-state I know about is 2021-08-06T00:00:00Z. You may need to update to a newer hackage.nix.

Use of source-repository-packages

We can use Cabal's source-repository-package mechanism to pull in un-released package versions. However, we must avoid this for dependencies of packages that are subject to release process. In particular, we should not release our packages to CHaP while we depend on a source-repository-package.

If we are stuck in a situation where we need a long-running fork of a package, we should release it to CHaP instead (see the CHaP README for more).

If you do add a source-repository-package, you need to provide a --sha256 comment in cabal.project so that Nix knows the hash of the content.

Warnings

While building most compilation warnings will be turned into an error due to -Werror flag. However during development it might be a bit inconvenient thus can be disabled on per project basis:

cabal configure <package-name> --ghc-options="-Wwarn"
cabal build <package-name>

Publishing specifications

PDF specs are stored as attachments to github releases We can create a release that builds and attaches the latest specs, by triggering the push-docs github action. This github action can be triggered by pushing a tag of the pattern: cardano-ledger-spec-YYYY-MM-DD, for example: cardano-ledger-spec-2023-01-17

For example, if we decide it's time to publish new versions of docs, we can do the following to publish the PDFs under release cardano-ledger-spec-2023-03-21:

git tag cardano-ledger-spec-2023-03-21
git push origin cardano-ledger-spec-2023-03-21

This will create a new release that will be available as latest. Make sure that the YYYY-MM-DD part in the tag name is alphabetically greater than the rest, otherwise the release won't be tagged as latest. Using the current date should ensure that this is the case.

CDDL files

For each era, the serialization protocol is defined by a corresponding cddl file, located at: eras/<era>/impl/cddl/data/<era>.cddl.

These files are generated using cuddle, based on the Haskell definitions in: eras/<era>/impl/testlib/Test/Cardano/Ledger/<era>/CDDL.hs.

To modify the cddl files for a given era:

1. Edit the CDDL.hs file for the desired era to reflect your changes
2. Regenerate the cddl files by running ./scripts/gen-cddl.sh

Testing the Haskell programs

The tests can be run with cabal. For example the Shelley tests can be run with:

cabal test cardano-ledger-shelley-test

It can be helpful to use the --test-show-details=streaming option for seeing the output of the tests while they run:

cabal test cardano-ledger-shelley-test --test-show-details=streaming

Running Specific Tests

The test suites use Tasty, which allows for running specific tests. This is done by passing the -p flag to the test program, followed by an awk pattern. You can alternatively use the TASTY_PATTERN environment variable with a pattern. For example, the Shelley golden tests can be run with:

cabal test cardano-ledger-shelley-test --test-options="-p golden"

or

TASTY_PATTERN=golden cabal test cardano-ledger-shelley-test

Tasty allows for more complex patterns. For instance, to run only the Byron update mechanism tests for the ledger that classify traces, we can pass the -p $1 ~ /Ledger/ && $2 ~ /Update/ && $3 ~ /classified/ option. Here each $i refers to a level in the tests names hierarchy. Passing -l to tasty will list the available test names.

When testing using cabal, pay special attention to escaping the right symbols, e.g.:

cabal test byron-spec-ledger:test:byron-spec-ledger-test --test-options "-p \"\$1 ~ /Ledger/ && \$2 ~ /Update/ && \$3 ~ /classified/\""

Replaying QuickCheck Failures

When a QuickCheck test fails, the seed which produced the failure is reported. The failure can be replayed with:

cabal test cardano-ledger-shelley-test --test-options "--quickcheck-replay=42"

(where 42 is an example seed).

Test Scenarios

Most of the test suites are grouped into test scenarios. For example, the Shelley test suite contains ContinuousIntegration, Development, Nightly, and Fast, which can be run with the --scenario flag. For example:

cabal test cardano-ledger-shelley-test --test-options --scenario=Fast

Default and Nightly builds

Most test suites have two different sets of tests: default and "nightly" (which take longer to run). The latter are being run when the environment variable NIGHTLY is set:

NIGHTLY=true cabal test cardano-ledger-shelley-test

Conformance Testing Against the Agda Formal Ledger Specification

The Haskell package cardano-ledger-conformance contains the glue code that enables running tests against the Haskell code extracted from the Agda specification (specified as an SRP in the top-level cabal.project of cardano-ledger).

• To run the complete test suite, execute:

  nix develop
  cabal update
  cabal test cardano-ledger-conformance

At the moment, the conformance part of some of the tests is disabled. That is, the test will run but it will not be compared against the specification.

Conformance-disabled tests are specified in the Haskell source code (of cardano-ledger) using the combinator disableInConformanceIt (instead of it). The output of disabled tests contains the string "[disabled in conformance]" when run.

Below are instructions for some common use cases and workflow examples.

How To:

• Run a specific test:

  1. Enter the nix develop shell.
  2. Execute

     cabal test cardano-ledger-conformance --test-options='--match "PATTERN"'

     where PATTERN is the name of the test (or some part of it).

• Re-enable and run a disabled test:

  1. Replace disableInConformance by fspecify in the test.
  2. Run

     nix develop
     cabal test cardano-ledger-conformance

  Note that fspecify re-enables conformance and makes the test "focused". When there are focused tests, cabal test will only run those.

• Change the source of the Agda-extracted Haskell package used in conformance to a local folder:

  1. Enter the nix develop shell
  2. Comment out the source-repository-package section in cabal.project that points to formal-ledger-specifications
  3. Add the path to the Agda-extracted Haskell package cardano-ledger-executable-spec to the packages section in cabal.project
  4. Execute the tests.

• Change the source of the Agda-extracted Haskell package used in conformance to a remote repo: See To update the referenced Agda ledger spec. Note that these instructions are valid for any *-artifacts branch.

Worked Out Examples:

• (Preferred Workflow) Extracting the Haskell code locally (with Agda caching) and adding it as an absolute path:

  In a local copy of the formal-ledger-specifications repo (FLS_REPO):

  • Extract the Haskell package using:

    rm -rf dist/hs _build; nix develop --command fls-shake hs

    This generates the cardano-ledger-executable-spec Haskell package in REPO/dist/hs Removing the dist/hs and _build folders ensures that no obsolete modules are left together with the extracted code.

  In a local copy of cardano-ledger (with the cabal.project file unmodified):

  1. Enter the nix develop shell
  2. Modify the file cabal.project by:
     • Commenting out the source-repository-package that points to formal-ledger-specifications.
     • Adding the path FLS_REPO/dist/hs to the packages section.

• Extracting the Haskell code locally (without Agda caching) and adding it as an absolute path: Same steps as the bullet point above except:

  • Instead of nix develop --command fls-shake hs run nix-build -A hs-src, which returns a path to the nix store, e.g. /nix/store/9pv3x44dfnwrz0jjrh9mlxa9y143i987-hs-src-0.1.
  • In (2) replace FLS_REPO/dist/hs by /nix/store/9pv3x44dfnwrz0jjrh9mlxa9y143i987-hs-src-0.1/hs (Note the added suffix hs).

nix build Infrastructure

The artifacts in this repository can be built and tested using nix. This is additionally used by the Hydra CI to test building, including cross-compilation for other systems.

To add a new Haskell package

To add a new Haskell package, you should do the following:

1. Create the project in the usual way. It should have an appropriate .cabal file.
2. Test that you can build your new project by running the following: nix build .#<project_name>:lib:<lib_name>. If you have executables, then you may also try building these using the .#<project_name>:exe:<exe_name> attribute path. A good way to see what's available is to execute :lf . in nix repl. This will allow you to explore the potential attribute names by using tab completion on "packages.<your_system>".

To add a new LaTeX specification

To add a new LaTeX specification, the easiest way is to copy from one of the existing specifications. You will want the Makefile and default.nix (say from the Shelley ledger spec).

1. Copy these files into the root of your new LaTeX specification.
2. Modify the DOCNAME in the Makefile.
3. Update default.nix to:
   1. Update the buildInputs to add in any LaTeX packages you need in your document, and remove any unneeded ones.
   2. Alter the meta description field to reflect the nature of this document.
4. Add a link to the package near the bottom of flake.nix, following the existing examples.

To build the Haskell code from the Agda ledger spec

See Contributing in the formal-ledger-specifications repo.

To update the referenced Agda ledger spec

To update the version of the Agda spec that the conformance tests are using:

1. Locate the master-artifacts branch in the formal-ledger-specifications repo
2. Identify the SHA of the commit that you need, belonging to that branch
3. In the cardano-ledger repository:
   • Update the cabal.project file by replacing the tag field in the source-repository-package stanza with SHA.
   • Update the flake.lock:

     nix flake update formal-ledger-specifications --override-input formal-ledger-specifications github:IntersectMBO/formal-ledger-specifications/SHA

If the commit you need in formal-ledger-specifications is not on master, open a PR for your branch in the formal-ledger-specifications repository. This will create a branch with the updated generated code, which you can then use as described above. You will not be able to merge in cardano-leder master a reference to a commit not yet merged in formal-ledger-specifications.

Additional documentation

You can find additional documentation on the nix infrastructure used in this repo in the following places:

• The haskell.nix user guide
• The nix-tools repository
• The iohk-nix repository

Note that the user guide linked above is incomplete and does not correctly refer to projects built using iohk-nix, as this one is. A certain amount of trial and error may be required to make substantive changes!

Working Conventions

Code formatting

We use fourmolu for formatting. You can either use it via a script or use pre-commit. pre-commit is provided via a separate devShell which uses the default shell as a base and adds pre-commit on top. You can use it by calling nix develop .#pre-commit. If you use direnv in some form, you can make this your default shell by adding a flake parameter to use flake in your .envrc. I.e: use flake .#pre-commit.

When running fourmolu manually via the fourmolize.sh script, you can instruct the script to run it only on changed files (compared to origin/master) by providing the --changes flag. If you omit it, then fourmolu will format everything.

Compiler warnings

The CI builds Haskell code with -Werror, so will fail if there are any compiler warnings.

If the warnings are stupid, we can turn them off, e.g. sometimes it makes sense to add -Wno-orphans.

Commit messages

Summarize changes in around 50 characters or less.

Provide more detailed explanatory text, if necessary. Wrap it to about 72 characters or so. In some contexts, the first line is treated as the subject of the commit and the rest of the text as the body. The blank line separating the summary from the body is critical (unless you omit the body entirely); various tools like log, shortlog and rebase can get confused if you run the two together.

Explain the problem that this commit is solving, and use one commit per conceptual change. Focus on why you are making this change as opposed to how (the code explains that). Are there side effects or other unintuitive consequences of this change? Here's the place to explain them.

Further paragraphs come after blank lines.

• Bullet points are okay, too

• Typically a hyphen or asterisk is used for the bullet, preceded by a single space, with blank lines in between, but conventions vary here

If you use an issue tracker, put references to them at the bottom, like this:

Resolves: #123 See also: #456, #789

Commit signing

Commits are required to be signed.

Pull Requests

Keep commits to a single logical change where possible. The reviewer will be happier, and you’ll be happier if you ever have to revert it. If you can’t do this (say because you have a huge mess), best to just have one commit with everything in it.

Keep your PRs to a single topic. Including unrelated changes makes things harder for your reviewers, slowing them down, and makes it harder to integrate new changes.

If you’re working on something that’s likely to conflict with someone else, talk to them. It’s not a race.

Performance

Memory

The ledger-state tool is helpful for obverserving the memory overhead of the ledger state.

Profiling

A good way to profile the ledger code is to use the db-analyser to replay block validation from mainnet.

First, inside the ouroboros repository base directory, open a nix shell with profiling enabled:

~/ouroboros-network$ nix-shell --arg config "{ haskellNix.profiling = true; }"

Configure cabal to build everything with profiling enabled:

cabal configure --enable-profiling --profiling-detail=all-functions

Now we need to run a node to build up the dataabase. This can be done in the cardano-node repository by running:

nix run .#mainnet/node

This will take a very long time. You can stop the node once it is past any slots that you care about.

Change directories back to the ouroboros-network repository. Download the mainnet config files.

Create a snapshot at the slot that you wish the profiling to start. We use 45288084 in this example:

cabal run db-analyser -- --db ~/io/cardano-node/state-node-mainnet/db-mainnet/ --minimum-block-validation cardano --configByron mainnet-byron-genesis.json --configShelley mainnet-shelley-genesis.json --nonce 1a3be38bcbb7911969283716ad7aa550250226b76a61fc51cc9a9a35d9276d81 --configAlonzo mainnet-alonzo-genesis.json --only-immutable-db --store-ledger 45288084

The value of the nonce used above can be discovered in the config.

Finally,

Run the block validation, say for 1000 slots, with:

cabal run db-analyser -- --db <PATH_TO_NODE>/cardano-node/state-node-mainnet/db-mainnet/ --minimum-block-validation cardano --configByron mainnet-byron-genesis.json --configShelley mainnet-shelley-genesis.json --configAlonzo mainnet-alonzo-genesis.json --only-immutable-db --analyse-from 45288084 --num-blocks-to-process 1000 --trace-ledger +RTS -pj -l-agu -RTS

This produces the profiling file db-analyser.prof.

Architectural Decision Records

See ADR-1.

Odds and Ends

See the wiki for some other odds and ends.