Add a Local CID Database for IPFS

[wswsmao]

Hi,

In relation to IPFS, as described in the documentation here, when users upload an image using the ctr-remote i ipfs-push command, they receive a CID (Content Identifier). Currently, users are required to manually record this CID for future image pulls.

I was wondering if we can add a local database for IPFS that stores the Image Reference as the key and the corresponding CID as the value whenever an ipfs-push operation is performed. Subsequent IPFS operations would then prioritize looking up this database, simplifying the user experience.

I would be happy to implement this feature.

Best regards,
abush wang

[ktock]

How does this work in multi-node environment like Kubernetes?

[wswsmao]

How does this work in multi-node environment like Kubernetes?

The local database solution does have some limitations. How about using IPFS's MFS (Mutable File System)?

We can organize the image files in a MFS directory structured by the image reference as the path. This way, we can query the image content using this path.

I conducted a simple experiment by modifying the push functionality under the ipfs/ directory. After pushing the content using the aforementioned method, I obtained the following results:

$ ipfs files ls /containerd/docker.io/abushwang/ocs9:org/
blobs
root

$ ipfs files read /containerd/docker.io/abushwang/ocs9:org/root | jq
{
  "mediaType": "application/vnd.oci.image.manifest.v1+json",
  "digest": "sha256:5e143112093612584c032f830a6e797ed83ade577c80c34ff68b30ff0eefbfaa",
  "size": 563,
  "urls": [
    "ipfs://bafkreic6cqyrecjwcjmeyazpqmfg46l63a5n4v34qdbu75ulgd7q5357vi"
  ]
}

$ ipfs files ls /containerd/docker.io/abushwang/ocs9:org/blobs
sha256:1bc36c278520b1a817d5a60016ed8b365d75b3769fbaefe0b2f1c357fd68df0f
sha256:5e143112093612584c032f830a6e797ed83ade577c80c34ff68b30ff0eefbfaa
sha256:ba7df7c2d4c09ddea132cac1bcc9d77c461db75fb49d681837695bbec4df6142

With this setup, we can use the relevant IPFS get APIs to download the image just like we would with a conventional image registry. @ktock

[wswsmao]

How does this work in multi-node environment like Kubernetes?

The local database solution does have some limitations. How about using IPFS's MFS (Mutable File System)?

We can organize the image files in a MFS directory structured by the image reference as the path. This way, we can query the image content using this path.

I conducted a simple experiment by modifying the push functionality under the ipfs/ directory. After pushing the content using the aforementioned method, I obtained the following results:

$ ipfs files ls /containerd/docker.io/abushwang/ocs9:org/
blobs
root

$ ipfs files read /containerd/docker.io/abushwang/ocs9:org/root | jq
{
  "mediaType": "application/vnd.oci.image.manifest.v1+json",
  "digest": "sha256:5e143112093612584c032f830a6e797ed83ade577c80c34ff68b30ff0eefbfaa",
  "size": 563,
  "urls": [
    "ipfs://bafkreic6cqyrecjwcjmeyazpqmfg46l63a5n4v34qdbu75ulgd7q5357vi"
  ]
}

$ ipfs files ls /containerd/docker.io/abushwang/ocs9:org/blobs
sha256:1bc36c278520b1a817d5a60016ed8b365d75b3769fbaefe0b2f1c357fd68df0f
sha256:5e143112093612584c032f830a6e797ed83ade577c80c34ff68b30ff0eefbfaa
sha256:ba7df7c2d4c09ddea132cac1bcc9d77c461db75fb49d681837695bbec4df6142

With this setup, we can use the relevant IPFS get APIs to download the image just like we would with a conventional image registry. @ktock

This is the draft version:
49798ee

[wswsmao]

How does this work in multi-node environment like Kubernetes?

The local database solution does have some limitations. How about using IPFS's MFS (Mutable File System)?
We can organize the image files in a MFS directory structured by the image reference as the path. This way, we can query the image content using this path.
I conducted a simple experiment by modifying the push functionality under the ipfs/ directory. After pushing the content using the aforementioned method, I obtained the following results:

$ ipfs files ls /containerd/docker.io/abushwang/ocs9:org/
blobs
root

$ ipfs files read /containerd/docker.io/abushwang/ocs9:org/root | jq
{
  "mediaType": "application/vnd.oci.image.manifest.v1+json",
  "digest": "sha256:5e143112093612584c032f830a6e797ed83ade577c80c34ff68b30ff0eefbfaa",
  "size": 563,
  "urls": [
    "ipfs://bafkreic6cqyrecjwcjmeyazpqmfg46l63a5n4v34qdbu75ulgd7q5357vi"
  ]
}

$ ipfs files ls /containerd/docker.io/abushwang/ocs9:org/blobs
sha256:1bc36c278520b1a817d5a60016ed8b365d75b3769fbaefe0b2f1c357fd68df0f
sha256:5e143112093612584c032f830a6e797ed83ade577c80c34ff68b30ff0eefbfaa
sha256:ba7df7c2d4c09ddea132cac1bcc9d77c461db75fb49d681837695bbec4df6142

With this setup, we can use the relevant IPFS get APIs to download the image just like we would with a conventional image registry. @ktock

This is the draft version: 49798ee

When the local MFS is updated, the updated information can be synchronized to other nodes using the ipfs name publish command. This allows those nodes to locate the images uploaded to the MFS node via a unified path.