Enable JSON Formatting for Stdout/Stderr Logs with Customizable Schema

[andreufontb]

Description

Summary

In Kubernetes environments, it’s a common practice to use a centralized log collector to gather logs from the stdout and stderr streams of all pods. These logs are typically parsed and stored in a database (e.g., Elasticsearch) for further analysis.

To streamline this process, having all logs in JSON format would offer significant advantages:

1. Simplified Parsing: JSON logs eliminate the need for pod-specific parsers, reducing complexity and configuration overhead.
2. Standardized Logging: A unified format ensures consistency across all pods, making it easier to analyze and correlate logs.
3. Customizable Fields: The ability to define custom JSON schemas would enable users to tailor log formats to their specific needs, ensuring all logs have consistent and meaningful fields.

Proposal

• Provide an option to enable JSON formatting for stdout and stderr logs.
• Allow customization of the JSON schema (e.g., define specific fields like timestamp, log_level, message, etc.).

Benefits

• Improved Observability: Consistent, JSON-formatted logs facilitate easier integration with log collectors and analysis tools.
• Operational Efficiency: Reduces the time and effort required to configure and maintain log parsing pipelines.
• Flexibility: A customizable schema enables organizations to standardize their logging practices according to their requirements. like using ECS schema.

This feature would greatly enhance the logging capabilities in Kubernetes and improve interoperability with existing logging ecosystems.