Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add storacha action provider #402

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions typescript/agentkit/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -43,9 +43,11 @@
"@alloralabs/allora-sdk": "^0.1.0",
"@coinbase/coinbase-sdk": "^0.20.0",
"@jup-ag/api": "^6.0.39",
"@ipld/dag-ucan": "^3.4.5",
"@privy-io/server-auth": "^1.18.4",
"@solana/spl-token": "^0.4.12",
"@solana/web3.js": "^1.98.0",
"@storacha/client": "^1.1.5",
"md5": "^2.3.0",
"opensea-js": "^7.1.18",
"reflect-metadata": "^0.2.2",
Expand Down
8 changes: 8 additions & 0 deletions typescript/agentkit/src/action-providers/storacha/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# Storacha Action Provider

This action provider provides actions for uploading/retriving files to/from [Storacha](https://storacha.network/)

## Use Cases
- Agent backup data to IPFS
- Agent upload data with content-addressing hash (CID) for verifiability
- Agents Swarm use Storacha as hot storage and inform each other on reasoning, chain of thought and output.
161 changes: 161 additions & 0 deletions typescript/agentkit/src/action-providers/storacha/client.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,161 @@
import {

Check failure on line 1 in typescript/agentkit/src/action-providers/storacha/client.ts

View workflow job for this annotation

GitHub Actions / lint-typescript

Replace `⏎····Signer,⏎····type·DID·as·W3DID,⏎` with `·Signer,·type·DID·as·W3DID·`
Signer,
type DID as W3DID,
} from "@storacha/client/principal/ed25519";
import * as Proof from "@storacha/client/proof";
import type {

Check failure on line 6 in typescript/agentkit/src/action-providers/storacha/client.ts

View workflow job for this annotation

GitHub Actions / lint-typescript

Replace `⏎····Client,⏎····EmailAddress,⏎····FileLike,⏎····ProgressStatus,⏎` with `·Client,·EmailAddress,·FileLike,·ProgressStatus·`
Client,
EmailAddress,
FileLike,
ProgressStatus,
} from "@storacha/client/types";

export type DownloadProgress = {
percent: number;

Check failure on line 14 in typescript/agentkit/src/action-providers/storacha/client.ts

View workflow job for this annotation

GitHub Actions / lint-typescript

Delete `··`
transferredBytes: number;

Check failure on line 15 in typescript/agentkit/src/action-providers/storacha/client.ts

View workflow job for this annotation

GitHub Actions / lint-typescript

Replace `····` with `··`

/**

Check failure on line 17 in typescript/agentkit/src/action-providers/storacha/client.ts

View workflow job for this annotation

GitHub Actions / lint-typescript

Delete `··`
Note: If it's not possible to retrieve the body size, it will be `0`.

Check failure on line 18 in typescript/agentkit/src/action-providers/storacha/client.ts

View workflow job for this annotation

GitHub Actions / lint-typescript

Expected JSDoc block to be aligned
*/
totalBytes: number;

Check failure on line 20 in typescript/agentkit/src/action-providers/storacha/client.ts

View workflow job for this annotation

GitHub Actions / lint-typescript

Replace `····` with `··`
};

import * as DID from "@ipld/dag-ucan/did";

Check failure on line 23 in typescript/agentkit/src/action-providers/storacha/client.ts

View workflow job for this annotation

GitHub Actions / lint-typescript

Import in body of module; reorder to top
import type { ServiceAbility } from "@storacha/client/types";

Check failure on line 24 in typescript/agentkit/src/action-providers/storacha/client.ts

View workflow job for this annotation

GitHub Actions / lint-typescript

Import in body of module; reorder to top

import { create } from "@storacha/client";
import { StoreMemory } from "@storacha/client/stores/memory";

// enable sync methods
import * as ed from "@noble/ed25519";
import { sha512 } from "@noble/hashes/sha512";

export type StorachaInitParams = {
keyString: string;
proofString: string;
store?: StoreMemory;
};

export type FileParams<T> = {
files: T[];
uploadProgressCallback?: (data: DownloadProgress) => void;
};

//@ts-ignore
ed.etc.sha512Sync = (...m) => sha512(ed.etc.concatBytes(...m));

export interface StorachaConfig {
client: Client;
spaceDid: W3DID;
}

// @w3ui use indexed DB with unextractable `CryptoKey`s.
// https://github.com/storacha/w3ui/blob/main/packages/core/src/index.ts#L69

export const createClient = async (options: any): Promise<Client> => {
const store = new StoreMemory();

const client = await create({
...options,
store,
});
return client;
};

export const authWithEmail = async (client: Client, email: EmailAddress) => {
const account = await client.login(email);

return account;
};

export const listFiles = async ({ client, spaceDid }: StorachaConfig) => {
await client.setCurrentSpace(spaceDid);
return await client.capability.upload.list({ cursor: "", size: 25 });
};

export const uploadFiles = async (
config: StorachaConfig,
{ files, uploadProgressCallback }: FileParams<FileLike>,
) => {
const { client } = config;
let link;
const onUploadProgress = (progress: ProgressStatus) => {
uploadProgressCallback?.({
transferredBytes: progress.loaded,
totalBytes: progress.total,
percent: progress.loaded / progress.total,
});
};
if (files.length == 1) {
const [file] = files;
link = await client.uploadFile(file, {
onUploadProgress,
});
} else {
// seems wont return actual progress
link = await client.uploadDirectory(files, {
onUploadProgress,
});
}

if (uploadProgressCallback) {
uploadProgressCallback({
transferredBytes: link.byteLength,
totalBytes: link.byteLength,
percent: 1,
});
}
return link;
};

export const createDelegation = async (
config: StorachaConfig,
{
userDid,
}: {
userDid: string;
},
) => {
const { client } = config;

const audience = DID.parse(userDid);

const abilities = [
"space/blob/add",
"space/index/add",
"filecoin/offer",
"upload/add",
] as ServiceAbility[];
const expiration = Math.floor(Date.now() / 1000) + 60 * 60 * 24; // 24 hours from now
const delegation = await client.createDelegation(audience, abilities, {
expiration,
});

const archive = await delegation.archive();
return archive.ok;
};

export const initStorachaClient = async ({
keyString,
proofString,
store = new StoreMemory(),
}: StorachaInitParams) => {
const principal = Signer.parse(keyString);
const client = await createClient({
principal,
});

const proof = await Proof.parse(proofString);
const space = await client.addSpace(proof);

await client.setCurrentSpace(space.did());

console.log(
`storcha init: principal ${principal.did()} space ${space.did()}`,
);

return {
client,
space,
};
};
2 changes: 2 additions & 0 deletions typescript/agentkit/src/action-providers/storacha/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
export * from "./schemas";
export * from "./storachaActionProvider";
17 changes: 17 additions & 0 deletions typescript/agentkit/src/action-providers/storacha/schemas.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
import { z } from "zod";

/**
* Input schema for retrieving file stored on storacha.
*/
export const StorachaRetrieveFileSchema = z
.object({})
.strip()
.describe("Input schema for retrieving file on storacha");

/**
* Input schema for uploading file to storacha.
*/
export const StorachaUploadFilesSchema = z
.object({})
.strip()
.describe("Input schema for uploading file on storacha");
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
import { StorachaActionProvider } from "./storachaActionProvider";

import type {
Client,
} from "@storacha/client";

// TODO base64
const MOCK_CONFIG = {
key: "test-key",
proof: "test-proof",
};


describe("StorachaActionProvider", () => {
let mockClient: jest.Mocked<Client>;
let provider: StorachaActionProvider;

beforeEach(() => {
mockClient = {
uploadFile: jest.fn(),
uploadDirectory: jest.fn(),
} as unknown as jest.Mocked<Client>;

provider = new StorachaActionProvider(MOCK_CONFIG);
});

describe("Constructor", () => {
it("should initialize with config values", () => {
expect(() => new StorachaActionProvider(MOCK_CONFIG)).not.toThrow();
});

it("should initialize with environment variables", () => {
process.env.STORACHA_KEY = MOCK_CONFIG.key;
process.env.STORACHA_PROOF = MOCK_CONFIG.proof;

expect(() => new StorachaActionProvider()).not.toThrow();
});

it("should throw error if no config or env vars", () => {
delete process.env.STORACHA_KEY;
delete process.env.STORACHA_PROOF;

expect(() => new StorachaActionProvider()).toThrow("STORACHA_KEY is not configured.");
});
});


});
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
import { ActionProvider } from "../actionProvider";

import { z } from "zod";
import { CreateAction } from "../actionDecorator";
import { Network } from "../../network";
import {
StorachaRetrieveFileSchema,
StorachaUploadFilesSchema,
} from "./schemas";
import { initStorachaClient } from "./client";
import type { Client } from "@storacha/client";

/**
* Configuration options for the StorachaActionProvider.
*/
export interface StorachaActionProviderConfig {
/**
* Storacha Private Key
*/
key?: string;

/**
* Storacha Space Proof
*/
proof?: string;


}


/**
* StorachaActionProvider is an action provider for interacting with Storacha
*
* @augments ActionProvider
*/
export class StorachaActionProvider extends ActionProvider {

private config: StorachaActionProviderConfig;
private client?: Client;

constructor(config: StorachaActionProviderConfig = {}) {
super("storacha", []);

config.key ||= process.env.STORACHA_PRIVATE_KEY;
config.proof ||= process.env.STORACHA_PROOF;

if (!config.key) {
throw new Error("STORACHA_PRIVATE_KEY is not configured.");
}

if (!config.proof) {
throw new Error("STORACHA_PROOF is not configured.");
}
this.config = config;


}



private createGatewayUrl() {
// TODO
}

private getClient = async () => {

if (!this.client) {
const { client } = await initStorachaClient({
keyString: this.config.key!,
proofString: this.config.proof!,
});

this.client = client;
}

return this.client;
}


/**
* Upload Files to Storacha
*
* @param args - The arguments containing file path
* @returns The root CID of the uploaded files
*/
@CreateAction({
name: "upload",
description: `
This tool will upload files to Storacha

A successful response will return a message with root data CID for in the JSON payload:
[{"cid":"bafybeib"}]
`,
schema: StorachaUploadFilesSchema,
})
async uploadFiles(args: z.infer<typeof StorachaUploadFilesSchema>): Promise<string> {

// TODO

return '';
}


/**
* Checks if the Storacha action provider supports the given network.
* Storacha actions don't depend on blockchain networks, so always return true.
*
* @param _ - The network to check (not used)
* @returns Always returns true as Storacha actions are network-independent
*/
supportsNetwork(_: Network): boolean {
return true;
}

}
Loading
Loading