Skip to content

Commit 5926203

Browse files
authored
chore(fluentd): add changelog (#720)
Signed-off-by: Steve Hipwell <steve.hipwell@gmail.com>
1 parent 4097435 commit 5926203

11 files changed

Lines changed: 417 additions & 74 deletions

File tree

‎.helmdocsignore‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +0,0 @@
1-
fluentd

‎charts/fluentd/CHANGELOG.md‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
# Fluentd Helm Chart Changelog
2+
3+
> [!NOTE]
4+
> All notable changes to this project will be documented in this file; the format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
5+
6+
<!--
7+
### Added - For new features.
8+
### Changed - For changes in existing functionality.
9+
### Deprecated - For soon-to-be removed features.
10+
### Removed - For now removed features.
11+
### Fixed - For any bug fixes.
12+
### Security - In case of vulnerabilities.
13+
-->
14+
15+
## [UNRELEASED]
16+
17+
### Added
18+
19+
- Add `variantVersion` value to specify the version of the variant to use. ([#720](https://github.com/fluent/helm-charts/pull/720)) @stevehipwell
20+
21+
### Changed
22+
23+
- Use SemVer for chart versioning (remove `v` prefix from `appVersion`). ([#720](https://github.com/fluent/helm-charts/pull/720)) @stevehipwell
24+
- Update _Fluentd_ app version to [`1.19.2`](https://github.com/fluent/fluentd/releases/tag/v1.19.2). ([#720](https://github.com/fluent/helm-charts/pull/720)) @stevehipwell
25+
- Update _elasticsearch7_ variant version to `1.6`. ([#720](https://github.com/fluent/helm-charts/pull/720)) @stevehipwell
26+
27+
### Fixed
28+
29+
- Fix test connection pod to actually run so CI can pass. ([#720](https://github.com/fluent/helm-charts/pull/720)) @stevehipwell
30+
31+
## [v0.5.3] - 2025-05-05
32+
33+
### Changed
34+
35+
- Upgrade the FluentD app version to 1.17.1. ([#570](https://github.com/fluent/helm-charts/pull/570)) @idogada-akamai
36+
- Since variant version 1.0 of elasticsearch7 for 1.17.1 doesn't exist in the Docker registry, I also changed its hard coded value to 1.2. ([#570](https://github.com/fluent/helm-charts/pull/570)) @idogada-akamai
37+
38+
<!--
39+
RELEASE LINKS
40+
-->
41+
[UNRELEASED]: https://github.com/fluent/helm-charts/tree/main/charts/fluentd
42+
[v0.5.3]: https://github.com/fluent/helm-charts/releases/tag/fluentd-0.5.3

‎charts/fluentd/Chart.yaml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,9 @@
11
apiVersion: v2
22
name: fluentd
3-
description: A Helm chart for Kubernetes
3+
description: Helm chart for deploying Fluentd on Kubernetes.
44
# type: application
55
version: 0.5.3
6-
appVersion: v1.17.1
6+
appVersion: 1.19.2
77
icon: https://www.fluentd.org/images/miscellany/fluentd-logo_2x.png
88
home: https://www.fluentd.org/
99
sources:

‎charts/fluentd/README.md‎

Lines changed: 125 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,131 @@
1-
# Fluentd Helm Chart
1+
# fluentd
22

3-
[Fluentd](https://www.fluentd.org/) is an open source data collector for unified logging layer. Fluentd allows you to unify data collection and consumption for a better use and understanding of data.
3+
![Version: 0.5.3](https://img.shields.io/badge/Version-0.5.3-informational?style=flat-square) ![AppVersion: 1.19.2](https://img.shields.io/badge/AppVersion-1.19.2-informational?style=flat-square)
44

5-
## Installation
5+
Helm chart for deploying Fluentd on Kubernetes. [Fluentd](https://www.fluentd.org/) is an open source data collector for unified logging layer. Fluentd allows you to unify data collection and consumption for a better use and understanding of data.
66

7-
To add the `fluent` helm repo, run:
7+
**Homepage:** <https://www.fluentd.org/>
88

9-
```sh
10-
helm repo add fluent https://fluent.github.io/helm-charts
11-
helm repo update
9+
## Maintainers
10+
11+
| Name | Email | Url |
12+
| ---- | ------ | --- |
13+
| edsiper | <eduardo@treasure-data.com> | |
14+
| dioguerra | <diogo.filipe.tomas.guerra@cern.ch> | |
15+
16+
## Source Code
17+
18+
* <https://github.com/fluent/fluentd/>
19+
* <https://github.com/fluent/fluentd-kubernetes-daemonset>
20+
21+
## Installing the Chart
22+
23+
### OCI Repository
24+
25+
To install the chart using the recommended OCI method you can use the following command.
26+
27+
```shell
28+
helm upgrade --install fluentd oci://ghcr.io/fluent/helm-charts/fluentd --version 0.5.3
1229
```
1330

14-
To install a release named `fluentd`, run:
31+
#### Verification
1532

16-
```sh
17-
helm install fluentd fluent/fluentd
33+
As the OCI chart release is signed by [Cosign](https://github.com/sigstore/cosign) you can verify the chart before installing it by running the following command.
34+
35+
```shell
36+
cosign verify --certificate-oidc-issuer https://token.actions.githubusercontent.com --certificate-identity-regexp 'https://github\.com/action-stars/helm-workflows/\.github/workflows/release\.yaml@.+' --certificate-github-workflow-repository fluent/helm-charts --certificate-github-workflow-name Release ghcr.io/fluent/helm-charts/fluentd:0.5.3
37+
```
38+
39+
### Non-OCI Repository
40+
41+
Alternatively you can use the legacy non-OCI method via the following commands.
42+
43+
```shell
44+
helm repo add fluent https://fluent.github.io/helm-charts/
45+
helm upgrade --install fluentd fluent/fluentd --version 0.5.3
1846
```
47+
48+
## Values
49+
50+
| Key | Type | Default | Description |
51+
|-----|------|---------|-------------|
52+
| affinity | object | `{}` | |
53+
| annotations | object | `{}` | |
54+
| autoscaling.customRules | list | `[]` | |
55+
| autoscaling.enabled | bool | `false` | |
56+
| autoscaling.maxReplicas | int | `100` | |
57+
| autoscaling.minReplicas | int | `1` | |
58+
| autoscaling.targetCPUUtilizationPercentage | int | `80` | |
59+
| configMapConfigs | list | `[]` | |
60+
| dashboards.enabled | string | `"true"` | |
61+
| dashboards.labels.grafana_dashboard | string | `"\"1\""` | |
62+
| dashboards.namespace | string | `""` | |
63+
| env | list | `[]` | |
64+
| envFrom | list | `[]` | |
65+
| fileConfigs."01_sources.conf" | string | `"## logs from podman\n<source>\n @type tail\n @id in_tail_container_logs\n @label @KUBERNETES\n path /var/log/containers/*.log\n pos_file /var/log/fluentd-containers.log.pos\n tag kubernetes.*\n read_from_head true\n <parse>\n @type multi_format\n <pattern>\n format json\n time_key time\n time_type string\n time_format \"%Y-%m-%dT%H:%M:%S.%NZ\"\n keep_time_key false\n </pattern>\n <pattern>\n format regexp\n expression /^(?<time>.+) (?<stream>stdout|stderr)( (.))? (?<log>.*)$/\n time_format '%Y-%m-%dT%H:%M:%S.%NZ'\n keep_time_key false\n </pattern>\n </parse>\n emit_unmatched_lines true\n</source>\n\n# expose metrics in prometheus format\n<source>\n @type prometheus\n bind 0.0.0.0\n port 24231\n metrics_path /metrics\n</source>"` | |
66+
| fileConfigs."02_filters.conf" | string | `"<label @KUBERNETES>\n <match kubernetes.var.log.containers.fluentd**>\n @type relabel\n @label @FLUENT_LOG\n </match>\n\n # <match kubernetes.var.log.containers.**_kube-system_**>\n # @type null\n # @id ignore_kube_system_logs\n # </match>\n\n <filter kubernetes.**>\n @type kubernetes_metadata\n @id filter_kube_metadata\n skip_labels false\n skip_container_metadata false\n skip_namespace_metadata true\n skip_master_url true\n </filter>\n\n <match **>\n @type relabel\n @label @DISPATCH\n </match>\n</label>"` | |
67+
| fileConfigs."03_dispatch.conf" | string | `"<label @DISPATCH>\n <filter **>\n @type prometheus\n <metric>\n name fluentd_input_status_num_records_total\n type counter\n desc The total number of incoming records\n <labels>\n tag ${tag}\n hostname ${hostname}\n </labels>\n </metric>\n </filter>\n\n <match **>\n @type relabel\n @label @OUTPUT\n </match>\n</label>"` | |
68+
| fileConfigs."04_outputs.conf" | string | `"<label @OUTPUT>\n <match **>\n @type elasticsearch\n host \"elasticsearch-master\"\n port 9200\n path \"\"\n user elastic\n password changeme\n # Don't wait for elastic to start up.\n verify_es_version_at_startup false\n </match>\n</label>"` | |
69+
| fullnameOverride | string | `""` | |
70+
| image.pullPolicy | string | `"IfNotPresent"` | |
71+
| image.repository | string | `"fluent/fluentd-kubernetes-daemonset"` | |
72+
| image.tag | string | `""` | |
73+
| imagePullSecrets | list | `[]` | |
74+
| ingress.annotations | object | `{}` | |
75+
| ingress.enabled | bool | `false` | |
76+
| ingress.hosts[0].port | int | `9880` | |
77+
| ingress.tls | list | `[]` | |
78+
| initContainers | list | `[]` | |
79+
| kind | string | `"DaemonSet"` | |
80+
| labels | object | `{}` | |
81+
| lifecycle | object | `{}` | |
82+
| livenessProbe.httpGet.path | string | `"/metrics"` | |
83+
| livenessProbe.httpGet.port | string | `"metrics"` | |
84+
| metrics.prometheusRule.additionalLabels | object | `{}` | |
85+
| metrics.prometheusRule.enabled | bool | `false` | |
86+
| metrics.prometheusRule.namespace | string | `""` | |
87+
| metrics.prometheusRule.rules | list | `[]` | |
88+
| metrics.serviceMonitor.additionalLabels.release | string | `"prometheus-operator"` | |
89+
| metrics.serviceMonitor.enabled | bool | `false` | |
90+
| metrics.serviceMonitor.metricRelabelings | list | `[]` | |
91+
| metrics.serviceMonitor.namespace | string | `""` | |
92+
| metrics.serviceMonitor.namespaceSelector | object | `{}` | |
93+
| metrics.serviceMonitor.relabelings | list | `[]` | |
94+
| minReadySeconds | string | `nil` | |
95+
| mountDockerContainersDirectory | bool | `true` | |
96+
| mountVarLogDirectory | bool | `true` | |
97+
| nameOverride | string | `""` | |
98+
| nodeSelector | object | `{}` | |
99+
| persistence.accessMode | string | `"ReadWriteOnce"` | |
100+
| persistence.enabled | bool | `false` | |
101+
| persistence.size | string | `"10Gi"` | |
102+
| persistence.storageClass | string | `""` | |
103+
| plugins | list | `[]` | |
104+
| podAnnotations | object | `{}` | |
105+
| podLabels | object | `{}` | |
106+
| podSecurityContext | object | `{}` | |
107+
| podSecurityPolicy.annotations | object | `{}` | |
108+
| podSecurityPolicy.enabled | bool | `true` | |
109+
| rbac.create | bool | `true` | |
110+
| readinessProbe.httpGet.path | string | `"/metrics"` | |
111+
| readinessProbe.httpGet.port | string | `"metrics"` | |
112+
| resources | object | `{}` | |
113+
| securityContext | object | `{}` | |
114+
| service.annotations | object | `{}` | |
115+
| service.enabled | bool | `true` | |
116+
| service.ports | list | `[]` | |
117+
| service.type | string | `"ClusterIP"` | |
118+
| serviceAccount.annotations | object | `{}` | |
119+
| serviceAccount.create | bool | `true` | |
120+
| serviceAccount.name | string | `nil` | |
121+
| terminationGracePeriodSeconds | string | `nil` | |
122+
| tolerations | list | `[]` | |
123+
| updateStrategy | object | `{}` | |
124+
| variant | string | `"elasticsearch7"` | |
125+
| variantVersion | string | `"1.6"` | |
126+
| volumeMounts | list | `[]` | |
127+
| volumes | list | `[]` | |
128+
19129
## Upgrading
20130

21131
### To 0.4.0
@@ -38,7 +148,7 @@ helm show values fluent/fluentd
38148

39149
### default-volumes
40150

41-
The default configurations bellow are required for the fluentd pod to be able to read the hosts container logs. The second section is responsible for allowing the user to load the "extra" configMaps either defined by the `fileConfigs` contained objects or, in addition, loaded externally and indicated by `configMapConfigs`.
151+
The default configurations bellow are required for the fluentd pod to be able to read the hosts container logs. The second section is responsible for allowing the user to load the "extra" configMaps either defined by the `fileConfigs` contained objects or, in addition, loaded externally and indicated by `configMapConfigs`.
42152

43153
```yaml
44154
- name: varlog
@@ -185,3 +295,7 @@ env:
185295
- name: FLUENT_ELASTICSEARCH_PORT
186296
value: "9200"
187297
```
298+
299+
----------------------------------------------
300+
301+
Autogenerated from chart metadata using [helm-docs](https://github.com/norwoodj/helm-docs/).

‎charts/fluentd/_description.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Helm chart for deploying Fluentd on Kubernetes. [Fluentd](https://www.fluentd.org/) is an open source data collector for unified logging layer. Fluentd allows you to unify data collection and consumption for a better use and understanding of data.

‎charts/fluentd/_docs.md‎

Lines changed: 169 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,169 @@
1+
## Upgrading
2+
3+
### To 0.4.0
4+
5+
Although the services will deploy and generally work, version 0.4.0 introduces some changes that are considered _breaking changes_. To upgrade, you should do the following to avoid any potential conflicts or problems:
6+
7+
- Add the `mountVarLogDirectory` and `mountDockerContainersDirectory` values and set them to the values you need; to follow the previous setup where these were mounted by default, set the values to `true`, e.g. `mountVarLogDirectory: true`
8+
- If you have the `varlog` mount point defined and enabled under both `volumes` and `volumeMounts`, set `mountVarLogDirectory` to true
9+
- If you have the `varlibdockercontainers` mount point defined and enabled under both `volumes` and `volumeMounts`, set `mountDockerContainersDirectory` to true
10+
- Remove the previous default volume and volume mount definitions - `etcfluentd-main`, `etcfluentd-config`, `varlog`, and `varlibdockercontainers`
11+
- Remove the `FLUENTD_CONF` entry from the `env:` list
12+
13+
## Chart Values
14+
15+
```sh
16+
helm show values fluent/fluentd
17+
```
18+
19+
## Value Details
20+
21+
### default-volumes
22+
23+
The default configurations bellow are required for the fluentd pod to be able to read the hosts container logs. The second section is responsible for allowing the user to load the "extra" configMaps either defined by the `fileConfigs` contained objects or, in addition, loaded externally and indicated by `configMapConfigs`.
24+
25+
```yaml
26+
- name: varlog
27+
hostPath:
28+
path: /var/log
29+
- name: varlibdockercontainers
30+
hostPath:
31+
path: /var/lib/docker/containers
32+
---
33+
- name: etcfluentd-main
34+
configMap:
35+
name: fluentd-main
36+
defaultMode: 0777
37+
- name: etcfluentd-config
38+
configMap:
39+
name: fluentd-config
40+
defaultMode: 0777
41+
```
42+
43+
### default-volumeMounts
44+
45+
The default configurations bellow are required for the fluentd pod to be able to read the hosts container logs. They should not be removed unless for some reason your container logs are accessible through a different path
46+
47+
```yaml
48+
- name: varlog
49+
mountPath: /var/log
50+
- name: varlibdockercontainers
51+
mountPath: /var/lib/docker/containers
52+
readOnly: true
53+
```
54+
55+
The section bellow is responsible for allowing the user to load the "extra" configMaps either defined by the `fileConfigs` contained objects or otherwise load externally and indicated by `configMapConfigs`.
56+
57+
```yaml
58+
- name: etcfluentd-main
59+
mountPath: /etc/fluent
60+
- name: etcfluentd-config
61+
mountPath: /etc/fluent/config.d/
62+
```
63+
64+
### default-fluentdConfig
65+
66+
The `fileConfigs` section is organized by sources -> filters -> destinations. Flow control must be configured using fluentd routing with tags or labels to guarantee that the configurations are executed as intended. Alternatively you can use numeration on your files to control the configurations loading order.
67+
68+
```yaml
69+
01_sources.conf: |-
70+
<source>
71+
@type tail
72+
@id in_tail_container_logs
73+
@label @KUBERNETES
74+
path /var/log/containers/*.log
75+
pos_file /var/log/fluentd-containers.log.pos
76+
tag kubernetes.*
77+
read_from_head true
78+
<parse>
79+
@type multi_format
80+
<pattern>
81+
format json
82+
time_key time
83+
time_type string
84+
time_format "%Y-%m-%dT%H:%M:%S.%NZ"
85+
keep_time_key false
86+
</pattern>
87+
<pattern>
88+
format regexp
89+
expression /^(?<time>.+) (?<stream>stdout|stderr)( (.))? (?<log>.*)$/
90+
time_format '%Y-%m-%dT%H:%M:%S.%NZ'
91+
keep_time_key false
92+
</pattern>
93+
</parse>
94+
emit_unmatched_lines true
95+
</source>
96+
97+
02_filters.conf: |-
98+
<label @KUBERNETES>
99+
<match kubernetes.var.log.containers.fluentd**>
100+
@type relabel
101+
@label @FLUENT_LOG
102+
</match>
103+
104+
# <match kubernetes.var.log.containers.**_kube-system_**>
105+
# @type null
106+
# @id ignore_kube_system_logs
107+
# </match>
108+
109+
<filter kubernetes.**>
110+
@type record_transformer
111+
enable_ruby
112+
<record>
113+
hostname ${record["kubernetes"]["host"]}
114+
raw ${record["log"]}
115+
</record>
116+
remove_keys $.kubernetes.host,log
117+
</filter>
118+
119+
<match **>
120+
@type relabel
121+
@label @DISPATCH
122+
</match>
123+
</label>
124+
125+
03_dispatch.conf: |-
126+
<label @DISPATCH>
127+
<filter **>
128+
@type prometheus
129+
<metric>
130+
name fluentd_input_status_num_records_total
131+
type counter
132+
desc The total number of incoming records
133+
<labels>
134+
tag ${tag}
135+
hostname ${hostname}
136+
</labels>
137+
</metric>
138+
</filter>
139+
140+
<match **>
141+
@type relabel
142+
@label @OUTPUT
143+
</match>
144+
</label>
145+
146+
04_outputs.conf: |-
147+
<label @OUTPUT>
148+
<match **>
149+
@type elasticsearch
150+
host "elasticsearch-master"
151+
port 9200
152+
path ""
153+
user elastic
154+
password changeme
155+
</match>
156+
</label>
157+
```
158+
159+
## Backwards Compatibility - v0.1.x
160+
161+
The old fluentd chart used the ENV variables and the default fluentd container definitions to set-up automatically many aspects of fluentd. It is still possible to trigger this behaviour by removing this charts current `.Values.env` configuration and replace by:
162+
163+
```yaml
164+
env:
165+
- name: FLUENT_ELASTICSEARCH_HOST
166+
value: "elasticsearch-master"
167+
- name: FLUENT_ELASTICSEARCH_PORT
168+
value: "9200"
169+
```

0 commit comments

Comments
 (0)