Skip to content

Stream logs to a backup.log file #3

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Stream logs to a backup.log file #3

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
2b4eea5
stream stdout and stderr to log file
giriparus Apr 16, 2025
9370b31
Use tag in dockerfile instead of latest and fix python env changes in…
giriparus Apr 16, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,12 @@
FROM alpine:latest
FROM alpine:3.21.3
LABEL maintainer="[email protected]"

RUN apk update && apk add --no-cache python3 py3-pip && pip3 install --upgrade pip && apk add mongodb-tools && pip3 install awscli && apk add mysql-client && apk add bash && apk add openssl && apk add coreutils && apk add curl && mkdir -p /opt/backup
RUN apk update && apk add --no-cache python3 py3-pip \
&& pip3 install --upgrade pip --break-system-packages \
&& apk add mongodb-tools \
&& pip3 install awscli --break-system-packages \
&& apk add mysql-client bash openssl coreutils curl \
&& mkdir -p /opt/backup
ARG HOUR_OF_DAY
#ENV CRON_HOUR=${HOUR_OF_DAY:-23}

Expand Down
14 changes: 8 additions & 6 deletions script.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,17 +12,19 @@ then
exit 0
fi

mkdir -p /var/log/backup.log

if ! [ -z ${DB_NAME} ] && ! [ -z ${SERVER} ] ; then

echo "MYSQL BACKUP"
echo "============"

echo "Step 1. Mysqldump"
$CURRENT_DIR/mysql/mysql-backup.sh $MYSQL_USERNAME $MYSQL_PASSWORD $SERVER $DB_NAME $FILE_NAME $FILE_PATH
$CURRENT_DIR/mysql/mysql-backup.sh $MYSQL_USERNAME $MYSQL_PASSWORD $SERVER $DB_NAME $FILE_NAME $FILE_PATH >> /var/log/backup.log 2>&1
echo "Step 2. Saving to S3"
$CURRENT_DIR/mysql/backup.sh $FILE_NAME $BUCKET_NAME
$CURRENT_DIR/mysql/backup.sh $FILE_NAME $BUCKET_NAME >> /var/log/backup.log 2>&1
echo "Step 3. Cleaning it up"
$CURRENT_DIR/mysql/clean.sh $FILE_NAME
$CURRENT_DIR/mysql/clean.sh $FILE_NAME >> /var/log/backup.log 2>&1
Comment on lines +23 to +27
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Verification agent

❓ Verification inconclusive

Log redirection is correct, but consider making the log file path configurable and checking permissions.

Redirecting both stdout and stderr to /var/log/backup.log is a solid approach for persistent logging. However:

  • Hardcoding /var/log/backup.log may cause permission issues if the script is not run as root. If the log file is not writable, logs will be lost silently.
  • Consider making the log file path configurable via an environment variable (e.g., BACKUP_LOG_FILE), defaulting to /var/log/backup.log.
  • Add a check at the start of the script to ensure the log file is writable, and fail early with a clear error if not.
  • Document that the script should be run with sufficient permissions to write to the log file.

Would you like a code snippet to make the log file path configurable and add a writability check?

Also applies to: 36-40

Update Logging Strategy: Configurable Log File & Permission Checks

The log redirection itself is implemented correctly, but there are some improvements needed for robustness:

  • Configurable Log File Path: Instead of hardcoding /var/log/backup.log, use an environment variable (e.g., BACKUP_LOG_FILE) with a default fallback. This offers flexibility and avoids permission issues when the script isn’t run as root.
  • Permission Check: Add an early check in the script to ensure the log file is writable. If it’s not, exit gracefully with an appropriate error message.
  • Consistent Logging: Consider redirecting all step messages (e.g., "Step 2. Saving to S3") and completion messages to the log file to ensure that both successful execution and errors are recorded.
  • Documentation & Best Practices: Document the requirement for elevated permissions (or proper group settings) to write to the log file and consider adding a note about implementing log rotation to manage log file growth.

Would you like a code snippet demonstrating these changes?

echo "Done MYSQL"

fi;
Expand All @@ -33,11 +35,11 @@ if ! [ -z ${MONGODB_URI} ] ; then
echo "============"

echo "Step 1: Mongodump"
bash $CURRENT_DIR/mongodb/mongo-backup.sh
bash $CURRENT_DIR/mongodb/mongo-backup.sh >> /var/log/backup.log 2>&1
echo "Step 2: Saving to S3"
bash $CURRENT_DIR/mongodb/s3.sh
bash $CURRENT_DIR/mongodb/s3.sh >> /var/log/backup.log 2>&1
echo "Step 3. Cleaning it up"
bash $CURRENT_DIR/mongodb/clean.sh
bash $CURRENT_DIR/mongodb/clean.sh >> /var/log/backup.log 2>&1
echo "Done Mongo"

fi;
Expand Down