Commit a96d971
committed
[IMP] fs_attachment_s3_migration: checksum-based deduplication
Before this change, each attachment was processed individually causing
duplicate S3 uploads for files with identical content. The migration
domain was incomplete (missing type, checksum, db_datas filters), and
writing to the datas field risked data loss if source files were
unreadable.
After this change, attachments are grouped by checksum within each
batch - uploading each unique file once and updating all related
attachment records to point to the same S3 path. The domain now
properly excludes URL attachments, DB-stored files, and already
migrated files. Record updates use _force_write_store_fname instead
of datas writes, eliminating data loss risk. Added configurable
force-DB rules respect and updated error logging.
Task: 50481 parent ec02e7a commit a96d971
File tree
6 files changed
+438
-158
lines changed- fs_attachment_s3_migration
- data
- models
- tests
- views
- wizard
6 files changed
+438
-158
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
9 | 9 | | |
10 | 10 | | |
11 | 11 | | |
12 | | - | |
| 12 | + | |
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
19 | 19 | | |
20 | 20 | | |
21 | 21 | | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
22 | 29 | | |
23 | 30 | | |
24 | 31 | | |
| |||
0 commit comments