Skip to content

Commit ad7423c

Browse files
stevensJourneystevensJourney
committed
Merge branch 'attachmentshelper' of github.com:powersync-ja/powersync-swift into attachmentshelper
2 parents 330d4bf + f9a3d10 commit ad7423c

File tree

1 file changed

+117
-109
lines changed

1 file changed

+117
-109
lines changed

‎Sources/PowerSync/attachments/README.md

Lines changed: 117 additions & 109 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
# PowerSync Attachment Helpers
22

3-
A [PowerSync](https://powersync.com) library to manage attachments in Swift apps.
3+
A [PowerSync](https://powersync.com) library to manage attachments (such as images or files) in Swift apps.
44

5-
## Alpha Release
5+
### Alpha Release
66

77
Attachment helpers are currently in an alpha state, intended strictly for testing. Expect breaking changes and instability as development continues.
88

@@ -14,18 +14,18 @@ An `AttachmentQueue` is used to manage and sync attachments in your app. The att
1414

1515
### Key Assumptions
1616

17-
- Each attachment should be identifiable by a unique ID.
18-
- Attachments are immutable.
19-
- Relational data should contain a foreign key column that references the attachment ID.
20-
- Relational data should reflect the holistic state of attachments at any given time. An existing local attachment will be deleted locally if no relational data references it.
17+
- Each attachment is identified by a unique ID
18+
- Attachments are immutable once created
19+
- Relational data should reference attachments using a foreign key column
20+
- Relational data should reflect the holistic state of attachments at any given time. An existing local attachment will deleted locally if no relational data references it.
2121

22-
### Example
22+
### Example Implementation
2323

2424
See the [PowerSync Example Demo](../../../Demo/PowerSyncExample) for a basic example of attachment syncing.
2525

26-
In this example below, the user captures photos when checklist items are completed as part of an inspection workflow.
26+
In the example below, the user captures photos when checklist items are completed as part of an inspection workflow.
2727

28-
The schema for the `checklist` table:
28+
1. First, define your schema including the `checklist` table and the local-only attachments table:
2929

3030
```swift
3131
let checklists = Table(
@@ -40,34 +40,14 @@ let checklists = Table(
4040
let schema = Schema(
4141
tables: [
4242
checklists,
43-
createAttachmentTable(name: "attachments") // Includes the table which stores attachment states
43+
// Add the local-only table which stores attachment states
44+
// Learn more about this function below
45+
createAttachmentTable(name: "attachments")
4446
]
4547
)
4648
```
4749

48-
The `createAttachmentTable` function defines a `local-only` attachment state storage table. See the [Implementation Details](#implementation-details) section for more details.
49-
50-
#### Steps to Implement
51-
52-
1. Implement a `RemoteStorageAdapter` which interfaces with a remote storage provider. This will be used for downloading, uploading, and deleting attachments.
53-
54-
```swift
55-
class RemoteStorage: RemoteStorageAdapter {
56-
func uploadFile(data: Data, attachment: Attachment) async throws {
57-
// TODO: Make a request to the backend
58-
}
59-
60-
func downloadFile(attachment: Attachment) async throws -> Data {
61-
// TODO: Make a request to the backend
62-
}
63-
64-
func deleteFile(attachment: Attachment) async throws {
65-
// TODO: Make a request to the backend
66-
}
67-
}
68-
```
69-
70-
2. Create an instance of `AttachmentQueue`. This class provides default syncing utilities and implements a default sync strategy. It can be subclassed for custom functionality.
50+
2. Create an `AttachmentQueue` instance. This class provides default syncing utilities and implements a default sync strategy. It can be subclassed for custom functionality:
7151

7252
```swift
7353
func getAttachmentsDirectoryPath() throws -> String {
@@ -98,18 +78,37 @@ let queue = AttachmentQueue(
9878
) }
9979
)
10080
```
101-
10281
- The `attachmentsDirectory` specifies where local attachment files should be stored. `FileManager.default.urls(for: .documentDirectory, in: .userDomainMask).first?.appendingPathComponent("attachments")` is a good choice.
10382
- The `remoteStorage` is responsible for connecting to the attachments backend. See the `RemoteStorageAdapter` protocol definition.
10483
- `watchAttachments` is closure which generates a publisher of `WatchedAttachmentItem`. These items represent the attachments that should be present in the application.
10584

106-
3. Call `startSync()` to start syncing attachments.
85+
3. Implement a `RemoteStorageAdapter` which interfaces with a remote storage provider. This will be used for downloading, uploading, and deleting attachments.
86+
87+
```swift
88+
class RemoteStorage: RemoteStorageAdapter {
89+
func uploadFile(data: Data, attachment: Attachment) async throws {
90+
// TODO: Make a request to the backend
91+
}
92+
93+
func downloadFile(attachment: Attachment) async throws -> Data {
94+
// TODO: Make a request to the backend
95+
}
96+
97+
func deleteFile(attachment: Attachment) async throws {
98+
// TODO: Make a request to the backend
99+
}
100+
}
101+
```
102+
103+
4. Start the sync process:
107104

108105
```swift
109106
queue.startSync()
110107
```
111108

112-
4. To create an attachment and add it to the queue, call `saveFile()`. This method saves the file to local storage, creates an attachment record, queues the file for upload, and allows assigning the newly created attachment ID to a checklist item.
109+
5. Create and save attachments using `saveFile()`. This method will
110+
save the file to the local storage, create an attachment record which queues the file for upload
111+
to the remote storage and allows assigning the newly created attachment ID to a checklist item:
113112

114113
```swift
115114
try await queue.saveFile(
@@ -132,39 +131,19 @@ try await queue.saveFile(
132131
}
133132
```
134133

135-
#### (Optional) Handling Errors
136-
137-
The attachment queue automatically retries failed sync operations. Retries continue indefinitely until success. A `SyncErrorHandler` can be provided to the `AttachmentQueue` constructor. This handler provides methods invoked on a remote sync exception. The handler can return a Boolean indicating if the attachment sync should be retried or archived.
138-
139-
```swift
140-
class ErrorHandler: SyncErrorHandler {
141-
func onDownloadError(attachment: Attachment, error: Error) async -> Bool {
142-
// TODO: Return if the attachment sync should be retried
143-
}
134+
## Implementation Details
144135

145-
func onUploadError(attachment: Attachment, error: Error) async -> Bool {
146-
// TODO: Return if the attachment sync should be retried
147-
}
136+
### Attachment Table Structure
148137

149-
func onDeleteError(attachment: Attachment, error: Error) async -> Bool {
150-
// TODO: Return if the attachment sync should be retried
151-
}
152-
}
153-
154-
// Pass the handler to the queue constructor
155-
let queue = AttachmentQueue(
156-
db: db,
157-
attachmentsDirectory: attachmentsDirectory,
158-
remoteStorage: remoteStorage,
159-
errorHandler: ErrorHandler()
160-
)
161-
```
138+
The `createAttachmentsTable` function creates a local-only table for tracking attachment states.
162139

163-
## Implementation Details
140+
An attachments table definition can be created with the following options:
164141

165-
### Attachment Table
142+
| Option | Description | Default |
143+
|--------|-----------------------|---------------|
144+
| `name` | The name of the table | `attachments` |
166145

167-
The default columns in `AttachmentTable`:
146+
The default columns are:
168147

169148
| Column Name | Type | Description |
170149
| ------------ | --------- | ------------------------------------------------------------------------------------------------------------------ |
@@ -177,11 +156,9 @@ The default columns in `AttachmentTable`:
177156
| `has_synced` | `INTEGER` | Internal tracker which tracks if the attachment has ever been synced. This is used for caching/archiving purposes. |
178157
| `meta_data` | `TEXT` | Any extra meta data for the attachment. JSON is usually a good choice. |
179158

180-
### Attachment State
181-
182-
The `AttachmentQueue` class manages attachments in your app by tracking their state.
159+
### Attachment States
183160

184-
The state of an attachment can be one of the following:
161+
Attachments are managed through the following states:
185162

186163
| State | Description |
187164
| ----------------- | ------------------------------------------------------------------------------ |
@@ -191,64 +168,95 @@ The state of an attachment can be one of the following:
191168
| `SYNCED` | The attachment has been synced |
192169
| `ARCHIVED` | The attachment has been orphaned, i.e., the associated record has been deleted |
193170

194-
### Syncing Attachments
171+
### Sync Process
172+
195173

196-
The `AttachmentQueue` sets a watched query on the `attachments` table for records in the `QUEUED_UPLOAD`, `QUEUED_DELETE`, and `QUEUED_DOWNLOAD` states. An event loop triggers calls to the remote storage for these operations.
174+
The `AttachmentQueue` implements a sync process with these components:
197175

198-
In addition to watching for changes, the `AttachmentQueue` also triggers a sync periodically. This will retry any failed uploads/downloads, particularly after the app was offline. By default, this is every 30 seconds but can be configured by setting `syncInterval` in the `AttachmentQueue` constructor options or disabled by setting the interval to `0`.
176+
1. **State Monitoring**: The queue watches the attachments table for records in `QUEUED_UPLOAD`, `QUEUED_DELETE`, and `QUEUED_DOWNLOAD` states. An event loop triggers calls to the remote storage for these operations.
199177

200-
#### Watching State
178+
2. **Periodic Sync**: By default, the queue triggers a sync every 30 seconds to retry failed uploads/downloads, in particular after the app was offline. This interval can be configured by setting `syncInterval` in the `AttachmentQueue` constructor options, or disabled by setting the interval to `0`.
201179

202-
The `watchedAttachments` publisher provided to the `AttachmentQueue` constructor is used to reconcile the local attachment state. Each emission of the publisher should represent the current attachment state. The updated state is constantly compared to the current queue state. Items are queued based on the difference.
180+
3. **Watching State**: The `watchedAttachments` flow in the `AttachmentQueue` constructor is used to maintain consistency between local and remote states:
181+
- New items trigger downloads - see the Download Process below.
182+
- Missing items trigger archiving - see Cache Management below.
203183

204-
- A new watched item not present in the current queue is treated as an upstream attachment creation that needs to be downloaded.
205-
- An attachment record is created using the provided watched item. The filename will be inferred using a default filename resolver if it has not been provided in the watched item.
206-
- The syncing service will attempt to download the attachment from the remote storage.
207-
- The attachment will be saved to the local filesystem. The `localURI` on the attachment record will be updated.
208-
- The attachment state will be updated to `SYNCED`.
209-
- Local attachments are archived if the watched state no longer includes the item. Archived items are cached and can be restored if the watched state includes them in the future. The number of cached items is defined by the `archivedCacheLimit` parameter in the `AttachmentQueue` constructor. Items are deleted once the cache limit is reached.
184+
#### Upload Process
210185

211-
#### Uploading
186+
The `saveFile` method handles attachment creation and upload:
212187

213-
The `saveFile` method provides a simple method for creating attachments that should be uploaded to the backend. This method accepts the raw file content and metadata. This function:
188+
1. The attachment is saved to local storage
189+
2. An `AttachmentRecord` is created with `QUEUED_UPLOAD` state, linked to the local file using `localURI`
190+
3. The attachment must be assigned to relational data in the same transaction, since this data is constantly watched and should always represent the attachment queue state
191+
4. The `RemoteStorage` `uploadFile` function is called
192+
5. On successful upload, the state changes to `SYNCED`
193+
6. If upload fails, the record stays in `QUEUED_UPLOAD` state for retry
214194

215-
- Persists the attachment to the local filesystem.
216-
- Creates an attachment record linked to the local attachment file.
217-
- Queues the attachment for upload.
218-
- Allows assigning the attachment to relational data.
195+
#### Download Process
219196

220-
The sync process after calling `saveFile` is:
197+
Attachments are scheduled for download when the `watchedAttachments` flow emits a new item that is not present locally:
221198

222-
- An `AttachmentRecord` is created or updated with a state of `QUEUED_UPLOAD`.
223-
- The `RemoteStorageAdapter` `uploadFile` function is called with the `Attachment` record.
224-
- The `AttachmentQueue` picks this up and, upon successful upload to the remote storage, sets the state to `SYNCED`.
225-
- If the upload is not successful, the record remains in the `QUEUED_UPLOAD` state, and uploading will be retried when syncing triggers again. Retries can be stopped by providing an `errorHandler`.
199+
1. An `AttachmentRecord` is created with `QUEUED_DOWNLOAD` state
200+
2. The `RemoteStorage` `downloadFile` function is called
201+
3. The received data is saved to local storage
202+
4. On successful download, the state changes to `SYNCED`
203+
5. If download fails, the operation is retried in the next sync cycle
226204

227-
#### Downloading
205+
### Delete Process
228206

229-
Attachments are scheduled for download when the `watchedAttachments` publisher emits a `WatchedAttachmentItem` not present in the queue.
207+
The `deleteFile` method deletes attachments from both local and remote storage:
230208

231-
- An `AttachmentRecord` is created or updated with the `QUEUED_DOWNLOAD` state.
232-
- The `RemoteStorageAdapter` `downloadFile` function is called with the attachment record.
233-
- The received data is persisted to the local filesystem.
234-
- If this is successful, update the `AttachmentRecord` state to `SYNCED`.
235-
- If any of these fail, the download is retried in the next sync trigger.
209+
1. The attachment record moves to `QUEUED_DELETE` state
210+
2. The attachment must be unassigned from relational data in the same transaction, since this data is constantly watched and should always represent the attachment queue state
211+
3. On successful deletion, the record is removed
212+
4. If deletion fails, the operation is retried in the next sync cycle
236213

237-
#### Deleting Attachments
214+
### Cache Management
238215

239-
Local attachments are archived and deleted (locally) if the `watchedAttachments` publisher no longer references them. Archived attachments are deleted locally after cache invalidation.
216+
The `AttachmentQueue` implements a caching system for archived attachments:
240217

241-
In some cases, users might want to explicitly delete an attachment in the backend. The `deleteFile` function provides a mechanism for this. This function:
218+
1. Local attachments are marked as `ARCHIVED` if the `watchedAttachments` flow no longer references them
219+
2. Archived attachments are kept in the cache for potential future restoration
220+
3. The cache size is controlled by the `archivedCacheLimit` parameter in the `AttachmentQueue` constructor
221+
4. By default, the queue keeps the last 100 archived attachment records
222+
5. When the cache limit is reached, the oldest archived attachments are permanently deleted
223+
6. If an archived attachment is referenced again while still in the cache, it can be restored
224+
7. The cache limit can be configured in the `AttachmentQueue` constructor
242225

243-
- Deletes the attachment on the local filesystem.
244-
- Updates the record to the `QUEUED_DELETE` state.
245-
- Allows removing assignments to relational data.
226+
### Error Handling
246227

247-
#### Expire Cache
228+
1. **Automatic Retries**:
229+
- Failed uploads/downloads/deletes are automatically retried
230+
- The sync interval (default 30 seconds) ensures periodic retry attempts
231+
- Retries continue indefinitely until successful
248232

249-
When PowerSync removes a record, as a result of coming back online or conflict resolution, for instance:
233+
2. **Custom Error Handling**:
234+
- A `SyncErrorHandler` can be implemented to customize retry behavior (see example below)
235+
- The handler can decide whether to retry or archive failed operations
236+
- Different handlers can be provided for upload, download, and delete operations
250237

251-
- Any associated `AttachmentRecord` is orphaned.
252-
- On the next sync trigger, the `AttachmentQueue` sets all orphaned records to the `ARCHIVED` state.
253-
- By default, the `AttachmentQueue` only keeps the last `100` attachment records and then expires the rest.
254-
- In some cases, these records (attachment IDs) might be restored. An archived attachment will be restored if it is still in the cache. This can be configured by setting `cacheLimit` in the `AttachmentQueue` constructor options.
238+
Example of a custom `SyncErrorHandler`:
239+
240+
```swift
241+
class ErrorHandler: SyncErrorHandler {
242+
func onDownloadError(attachment: Attachment, error: Error) async -> Bool {
243+
// TODO: Return if the attachment sync should be retried
244+
}
245+
246+
func onUploadError(attachment: Attachment, error: Error) async -> Bool {
247+
// TODO: Return if the attachment sync should be retried
248+
}
249+
250+
func onDeleteError(attachment: Attachment, error: Error) async -> Bool {
251+
// TODO: Return if the attachment sync should be retried
252+
}
253+
}
254+
255+
// Pass the handler to the queue constructor
256+
let queue = AttachmentQueue(
257+
db: db,
258+
attachmentsDirectory: attachmentsDirectory,
259+
remoteStorage: remoteStorage,
260+
errorHandler: ErrorHandler()
261+
)
262+
```

0 commit comments

Comments
 (0)