Job History and container logs indicate backup failure but looks like success in archive

[ryang3d]

I have been unable to get a (200-300GB) backup to complete successfully, at least according to the Job History tab and container logs. But when I look at the output from the backup job, it looks like it actually backed everything up.

Aside from the screenshot below and docker container logs (pasted below), I can't quite figure out why it is reporting the job as failed (or if there was an actual failure to be concerned about).

Also, maybe a separate bug, but when I set BORGITORY_DEBUG=true, Borgitory refuses to load, so I can't really use that for debugging....

Job History Screenshot:

Docker Container Logs:

2025-10-26 18:00:04,266 - borgitory.services.jobs.job_manager - INFO - Saving task backup to database - Status: failed, Return Code: 1, Output Lines: 1929416

2025-10-26 18:00:06,595 - borgitory.services.jobs.job_manager - ERROR - Critical task backup failed, stopping job

2025-10-26 18:00:06,595 - borgitory.services.jobs.job_manager - INFO - Saving all tasks to database after critical failure - Job: 8fbf1002-b00b-49bb-a6f8-18c97d3df6fe

2025-10-26 18:00:12,107 - borgitory.services.jobs.job_manager - INFO - Successfully saved all tasks to database after critical failure

2025-10-26 18:00:12,123 - borgitory.services.jobs.job_database_manager - INFO - Updated database job 8fbf1002-b00b-49bb-a6f8-18c97d3df6fe status to failed

2025-10-26 18:01:28,172 - borgitory.services.jobs.job_render_service - INFO - Using database data for completed/failed job 8fbf1002-b00b-49bb-a6f8-18c97d3df6fe

2025-10-26 18:01:32,039 - borgitory.services.jobs.job_render_service - INFO - Using database data for completed/failed job 8fbf1002-b00b-49bb-a6f8-18c97d3df6fe

2025-10-26 18:57:16,676 - borgitory.services.jobs.job_render_service - INFO - Using database data for completed/failed job 8fbf1002-b00b-49bb-a6f8-18c97d3df6fe

2025-10-26 20:26:14,715 - borgitory.services.jobs.job_render_service - INFO - Using database data for completed/failed job 8fbf1002-b00b-49bb-a6f8-18c97d3df6fe

End of Backup Job output:

------------------------------------------------------------------------------
Repository: /mnt/repos/Backups2
Archive name: backup-20251026-175152
Archive fingerprint: d28cab5d4075bed42dee995cf1349cb720c2c5e3c144dce3977cb21b168ce326
Time (start): Sun, 2025-10-26 17:51:52
Time (end):   Mon, 2025-10-27 00:59:09
Duration: 7 hours 7 minutes 17.18 seconds
Number of files: 964568
Utilization of max. archive size: 0%
------------------------------------------------------------------------------
                       Original size      Compressed size    Deduplicated size
This archive:              214.90 GB            102.88 GB             92.43 GB
All archives:              214.90 GB            102.88 GB             92.43 GB
                       Unique chunks         Total chunks
Chunk index:                  619576              1015199
------------------------------------------------------------------------------

[mlapaglia]

hmm, borgitory uses the return code from the process, -1 in this case, to detect success/failure. i wonder why it returned a failure response.. can you get more of the logs, a few lines before what you have would show the failure reason.

[ryang3d]

hmm, borgitory uses the return code from the process, -1 in this case, to detect success/failure. i wonder why it returned a failure response.. can you get more of the logs, a few lines before what you have would show the failure reason.

The reason I only included those logs are because there were no signs of failure above that. It is literally just a list of files that match an exclude pattern that I'm using:

Added pattern: --pattern=-*/.cache*
followed by countless lines that look something like /mnt/backup/root/home/user/.cache/pip/http/0/f/9/5/2/blahblahblah

I don't think I included this previously, but probably not that useful - right below the Output section under Job History (in red text):
Error: Backup failed with return code 1: All archives: 221.27 GB 105.72 GB 95.01 GB Unique chunks Total chunks Chunk index: 621434 1017158 --------------------------------

It does seem like the Job History output is being truncated though. Would it be possible to access untruncated versions to look deeper for possible errors/failure points?

[ryang3d]

Turns out I may have been trying to back up a bunch of stuff that really shouldn't have been (including potentially corrupted files and system files). After sorting through things, I've managed to clean my backups in Borgitory up and finally got them all to complete successfully. Let's close this, as I'm finally ready to admit user error =)

For a little added info: After sorting through the logs a bit more, I did start to see some errors (permissions, no such file, etc) which helped determine what else I could filter out of my backups; I also realized that I should have been using "!" instead of "-" for some of my patterns, which really seemed to help the most.

[ryang3d]

Re-opening this; After cleaning up my backups, and getting 2 out of 3 of my schedules to Succeed ("Completed"), I am still seeing "Failed" reports when there was a single file that changed during the otherwise successful backup (as far as I can tell), with something like "/mnt/backup/root/var/lib/program/program.db: file changed while we backed it up" visible in the Job History Output.