feat(storage): add dataSetId to piece confirmation callbacks

[virajbhartiya]

• Add onPiecesConfirmed(dataSetId, pieces) callback exposing dataset ID when pieces are confirmed
• Add onPiecesAdded(transaction, pieces) callback with piece CIDs
• Deprecate onPieceAdded and onPieceConfirmed
• Update docs and examples to demonstrate new callbacks

Closes #409

[virajbhartiya]

@rvagg @SgtPooki can you please review

[SgtPooki]

@virajbhartiya this needs a rebase

[vercel]

@virajbhartiya is attempting to deploy a commit to the FilOz Team on Vercel.

A member of the Team first needs to authorize it.

[Copilot]

Pull Request Overview

This PR adds enhanced callback APIs for storage uploads that expose dataset IDs and piece information when pieces are confirmed on-chain.

• Added PieceIdentifiers interface combining pieceId and pieceCid
• Introduced onPiecesAdded and onPiecesConfirmed callbacks with richer data
• Deprecated onPieceAdded and onPieceConfirmed callbacks

Reviewed Changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
packages/synapse-sdk/src/types.ts	Defines new PieceIdentifiers interface and adds onPiecesAdded/onPiecesConfirmed callbacks while deprecating old ones
packages/synapse-sdk/src/storage/context.ts	Implements new callbacks in upload flow, passing dataset ID and piece arrays to callbacks
packages/synapse-sdk/src/test/storage-upload.test.ts	Adds test coverage for new callback signatures and validates they receive correct arguments
utils/example-storage-e2e.js	Updates example to demonstrate new callbacks with piece arrays and dataset IDs
examples/cli/src/commands/upload.ts	Migrates CLI upload command to use new callback APIs
docs/src/content/docs/guides/storage.mdx	Updates storage guide documentation with new callback examples
docs/src/content/docs/developer-guides/storage/storage-context.mdx	Updates developer guide with comprehensive examples of new callback usage and type definitions

Comments suppressed due to low confidence (2)

packages/synapse-sdk/src/storage/context.ts:1083

• Same issue as lines 1029-1032: the mapping of confirmedPieceIds to pieceCids by index could be incorrect if the arrays have different lengths. Validation should occur before firing the callbacks to prevent passing incorrect data to onPiecesConfirmed. Consider adding:

if (confirmedPieceIds.length !== pieceCids.length) {
  throw createError('StorageContext', 'addPieces', `Expected ${pieceCids.length} confirmed piece IDs but got ${confirmedPieceIds.length}`)
}

        }
        item.resolve(pieceId)
      })
    } catch (error) {

packages/synapse-sdk/src/storage/context.ts:1032

• The mapping of confirmedPieceIds to pieceCids by index could be incorrect if the arrays have different lengths. This validation happens later (line 1092-1095) but after callbacks are fired, potentially passing incorrect data to onPiecesConfirmed. Consider validating array lengths match before creating piecesForConfirmed, or use the same validation pattern here as used later:

if (confirmedPieceIds.length !== pieceCids.length) {
  throw createError('StorageContext', 'addPieces', `Expected ${pieceCids.length} confirmed piece IDs but got ${confirmedPieceIds.length}`)
}

            : baseMetadataObj

        // Convert to MetadataEntry[] for PDP operations (requires ordered array)
        const finalMetadata = objectToEntries(metadataObj)

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[virajbhartiya]

@SgtPooki rebased

[SgtPooki]

I think the comments from Hugo need looked at.. if we just use this.dataSetId in the checks where it exists we should be able to remove the null check

[SgtPooki]

nit: I'm not sure how large we expect the batches to grow, or how sensitive we need to be to performance, but we're looping over each batch three times here and may want to look (separate from this PR) into a reduce or for...of loop

[rvagg]

Looking pretty good @virajbhartiya, some suggestions in there that it would be good for you to update. There are some disagreements we're having that it's not up to you to resolve, although you could use PieceRecord to make that disagreement disappear. Just do your best to resolve comments where there's no disagreement and we'll go from there.

[rvagg]

Looks good! Just that one doc fix and I think this is OK to go.