Skip to content

SHARD-2741 - feat: log file rotation strategy changes #549

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
aniketdivekar wants to merge 5 commits into
base: dev
Choose a base branch
from
Open

SHARD-2741 - feat: log file rotation strategy changes #549

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
c4d1ca3
feat: log file rotation strategy changes
aniketdivekar Aug 13, 2025
d7db4f0
numBackups set to 24 for keeping one day worth of logs
aniketdivekar Aug 13, 2025
05e77bc
review feedback: changed numBackups to 10
aniketdivekar Aug 13, 2025
3e7b2ac
fix + add unit test
aniketdivekar Aug 13, 2025
505e8d9
feat: log rotation with file size but naming convention as timestamp
aniketdivekar Aug 26, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 30 additions & 3 deletions src/config/logs.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,36 @@ const LOGS_CONFIG: StrictLogsConfiguration = {
appenders: {
out: { type: 'console', maxLogSize: 10000000, backups: 10 },
seq: { type: 'file', maxLogSize: 1000000000, backups: 10 },
main: { type: 'file', maxLogSize: 10000000, backups: 10 },
app: { type: 'file', maxLogSize: 10000000, backups: 10 },
p2p: { type: 'file', maxLogSize: 10000000, backups: 10 },
main: {
type: 'dateFile',
pattern: 'yyyy-MM-dd-hh',
keepFileExt: true,
maxLogSize: 10000000,
backups: 10,
numBackups: 24,
compress: false,
alwaysIncludePattern: false,
},
app: {
type: 'dateFile',
pattern: 'yyyy-MM-dd-hh',
keepFileExt: true,
maxLogSize: 10000000,
backups: 10,
numBackups: 24,
compress: false,
alwaysIncludePattern: false,
},
p2p: {
type: 'dateFile',
pattern: 'yyyy-MM-dd-hh',
keepFileExt: true,
maxLogSize: 10000000,
backups: 10,
numBackups: 24,
compress: false,
alwaysIncludePattern: false,
},
Comment on lines +11 to +99
Copy link
Copy Markdown

@github-actions github-actions bot Aug 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: The pattern value 'yyyy-MM-dd-hh' may cause log files to rotate every hour, potentially leading to excessive file creation and disk usage. Consider using 'yyyy-MM-dd' for daily rotation unless hourly rotation is strictly required. [general, importance: 7]

Suggested change
main: {
type: 'dateFile',
pattern: 'yyyy-MM-dd-hh',
keepFileExt: true,
maxLogSize: 10000000,
backups: 10,
numBackups: 24,
compress: false,
alwaysIncludePattern: false,
},
app: {
type: 'dateFile',
pattern: 'yyyy-MM-dd-hh',
keepFileExt: true,
maxLogSize: 10000000,
backups: 10,
numBackups: 24,
compress: false,
alwaysIncludePattern: false,
},
p2p: {
type: 'dateFile',
pattern: 'yyyy-MM-dd-hh',
keepFileExt: true,
maxLogSize: 10000000,
backups: 10,
numBackups: 24,
compress: false,
alwaysIncludePattern: false,
},
main: {
type: 'dateFile',
pattern: 'yyyy-MM-dd',
keepFileExt: true,
maxLogSize: 10000000,
backups: 10,
numBackups: 24,
compress: false,
alwaysIncludePattern: false,
},
app: {
type: 'dateFile',
pattern: 'yyyy-MM-dd',
keepFileExt: true,
maxLogSize: 10000000,
backups: 10,
numBackups: 24,
compress: false,
alwaysIncludePattern: false,
},
p2p: {
type: 'dateFile',
pattern: 'yyyy-MM-dd',
keepFileExt: true,
maxLogSize: 10000000,
backups: 10,
numBackups: 24,
compress: false,
alwaysIncludePattern: false,
},

snapshot: { type: 'file', maxLogSize: 10000000, backups: 10 },
cycle: { type: 'file', maxLogSize: 10000000, backups: 10 },
fatal: { type: 'file', maxLogSize: 10000000, backups: 10 },
Expand Down
2 changes: 1 addition & 1 deletion src/logger/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -201,7 +201,7 @@ class Logger {
const conf = this.log4Conf
for (const key in conf.appenders) {
const appender = conf.appenders[key]
if (appender.type !== 'file') continue
if (appender.type !== 'file' && appender.type !== 'dateFile') continue
appender.filename = `${this.logDir}/${key}.log`
}
Comment on lines +204 to 206
Copy link
Copy Markdown

@github-actions github-actions bot Aug 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: Setting a filename property on appenders of type 'dateFile' may require the filename to include the pattern placeholder for correct log rotation. Ensure the filename supports pattern substitution if required by the logging library. [general, importance: 6]

Suggested change
if (appender.type !== 'file' && appender.type !== 'dateFile') continue
appender.filename = `${this.logDir}/${key}.log`
}
if (appender.type !== 'file' && appender.type !== 'dateFile') continue
appender.filename = appender.type === 'dateFile'
? `${this.logDir}/${key}.log`
: `${this.logDir}/${key}.log`
// If the logging library requires, consider: `${this.logDir}/${key}.log` or `${this.logDir}/${key}-%DATE%.log`

}
Expand Down
15 changes: 15 additions & 0 deletions src/shardus/shardus-types.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1482,6 +1482,11 @@ export interface LogsConfiguration {
type?: string
maxLogSize?: number
backups?: number
pattern?: string
keepFileExt?: boolean
numBackups?: number
compress?: boolean
alwaysIncludePattern?: boolean
}
seq?: {
type?: string
Expand All @@ -1492,11 +1497,21 @@ export interface LogsConfiguration {
type?: string
maxLogSize?: number
backups?: number
pattern?: string
keepFileExt?: boolean
numBackups?: number
compress?: boolean
alwaysIncludePattern?: boolean
}
p2p?: {
type?: string
maxLogSize?: number
backups?: number
pattern?: string
keepFileExt?: boolean
numBackups?: number
compress?: boolean
alwaysIncludePattern?: boolean
}
snapshot?: {
type?: string
Expand Down
Loading