diff --git a/.hugo_build.lock b/.hugo_build.lock deleted file mode 100644 index e69de29..0000000 diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..261eeb9 --- /dev/null +++ b/LICENSE @@ -0,0 +1,201 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/assets/scss/style.scss b/assets/scss/style.scss index 3390372..175e033 100644 --- a/assets/scss/style.scss +++ b/assets/scss/style.scss @@ -319,6 +319,10 @@ header{ color: $white; mix-blend-mode: normal; } + p{ + font-style:normal; + color:$white; + } } .brand-list{ @@ -1745,6 +1749,10 @@ header{ } +#hs-form-iframe-0{background: none!important;} +.label { color:white !important } +.hbspt-form iframe.hs-form-iframe {background: none!important;} + .highlight>div{ padding: 10px } diff --git a/config.toml b/config.toml index 7c9d9b9..e29b440 100644 --- a/config.toml +++ b/config.toml @@ -1,24 +1,25 @@ baseURL = "/" languageCode = "en-us" title = "Fluentbit, High Performance" - +googleAnalytics = "G-93T2VC0VBC" [params] email = "" logo = "/images/logo.svg" logo2 = "/images/logo-white.svg" footer_logo = "/images/footer-logo.svg" -copyright = "© 2022 Fluent Bit. All rights reserved" -lastVersion = "v1.8.12" -releasedOn = "Jan 24, 2022" - +copyright = "© 2015-2024 The Fluent Bit Authors. Fluent Bit is a CNCF sub-project under the umbrella of Fluentd" +lastVersion = "v3.2.3" +releasedOn = "Dec 19, 2024" +noteUrl = "announcements/v3.2.3" + [menu] [[menu.main]] identifier = "documentation" url = "https://docs.fluentbit.io/manual" name = "DOCUMENTATION" weight = 1 - tab = "extranal" + tab = "extranal" [[menu.main]] identifier = "how it works" url = "/how-it-works/" @@ -28,22 +29,28 @@ releasedOn = "Jan 24, 2022" identifier = "release notes" url = "/announcements/" name = "RELEASE NOTES" - weight = 3 + weight = 3 [[menu.main]] identifier = "Enterprise" url = "/enterprise/" name = "Enterprise" - weight = 4 + weight = 4 [[menu.main]] identifier = "Community" url = "/community/" name = "Community" - weight = 5 + weight = 5 [[menu.main]] identifier = "blog" url = "/blog/" name = "Blog" - weight = 7 + weight = 7 + + [[menu.main]] + identifier = "opentelemetry" + url = "/opentelemetry/" + name = "OpenTelemetry" + weight = 7 [[menu.main]] identifier = "slack" @@ -53,11 +60,9 @@ releasedOn = "Jan 24, 2022" pre = "/images/slack.svg" slack = "/images/slack-w.svg" - + [params.latestVer] light = "true" - heading = "Fluent Bit v1.8.12 is out!" - subHeading = "New release on Jan 24, 2022," text = "Check out the Release Notes, read the Updated Documentation or jump directly to the Downloads Section." smallText = "We are part of a wide community, no vendor lock-in." btnText = "Download Now" @@ -65,7 +70,7 @@ releasedOn = "Jan 24, 2022" bottomText = ":: Read Documentation" bottomUrl = "https://docs.fluentbit.io/manual/" - + [[params.footer.bottom]] url = "#" name = "Terms & Conditions" @@ -85,22 +90,21 @@ releasedOn = "Jan 24, 2022" [[params.social]] url = "https://launchpass.com/fluent-all" - icon = "/images/slack-color.svg" + icon = "/images/slack-color.svg" name = "slack" [[params.social]] url = "https://twitter.com/fluentbit" - icon = "/images/twitter-color.svg" + icon = "/images/twitter-color.svg" name = "twitter" # [[params.social]] # url = "https://www.youtube.com/results?search_query=fluentbit" -# icon = "fa fa-youtube-play" +# icon = "fa fa-youtube-play" # name = "youtube" [permalinks] - # posts = "blog/:year/:month/:title/" - posts = "blog/:title/" + posts = "blog/:year/:month/:day/:title/" pages = ":slug/" @@ -150,7 +154,7 @@ releasedOn = "Jan 24, 2022" lineNos = true lineNumbersInTable = true noClasses = true - style = 'api' + style = 'api' tabWidth = 20 # style= 'monika api fruity monokai native paraiso-dark rrt solarized-dark solarized-dark256 swapoff vim' @@ -158,7 +162,7 @@ releasedOn = "Jan 24, 2022" endLevel = 3 ordered = true startLevel = 2 - + [markup.goldmark.renderer] unsafe= true diff --git a/content/_index.md b/content/_index.md index 472866a..8f51ef7 100644 --- a/content/_index.md +++ b/content/_index.md @@ -1,41 +1,40 @@ --- title: 'fluentbit' description: '' -hero: +hero: enabled: true position: 1 heading: | An End to End Observability Pipeline - tag: "Deployed Over One Billion Times" + tag: "Deployed Over Ten Billion Times" subHeading: | - Fluent Bit is a super fast, lightweight, and highly scalable logging and metrics processor and forwarder. It is the preferred choice for cloud and containerized environments. + Fluent Bit is a super fast, lightweight, and highly scalable logging, metrics, and traces processor and forwarder. It is the preferred choice for cloud and containerized environments. + btnText: TRY NOW + btnUrl: "https://play.instruqt.com/Fluent/tracks/fluent-bit-getting-started-101" btnText: DOWNLOAD NOW btnUrl: "https://docs.fluentbit.io/manual/installation/getting-started-with-fluent-bit" - version: v1.8.12 - releasedDate: Jan 24, 2022 noteText: Read the notes » - noteUrl: "/announcements/" image: "/images/hero.svg" mobileImage: "/images/hero-m.svg" -howitworks: +howitworks: enabled: true position: 5 - image: /images/how-back.svg + image: /images/how-back.svg btnText: Read how it works btnUrl: "/how-it-works/" info: Fluent Bit enables you to collect logs and metrics from multiple sources, enrich them with filters, and distribute them to any defined destination. - lists: + lists: - Optimized data parsing and routing - Prometheus and OpenTelemetry compatible - Stream processing functionality - Built in buffering and error-handling capabilities -brand: - enabled: true +brand: + enabled: true position: 2 heading: Trusted by some of the world’s largest and most complex organizations - subHeading: -performance: + subHeading: +performance: enabled: true position: 3 sectionId: "performance" @@ -59,7 +58,7 @@ performance: icon: /images/extensible.svg description: "Integration with all your technology - cloud native services, containers, streaming processors, and data backends." heightlightText: "80+ Plugins for inputs, filters, analytics tools and outputs." -ctaBoxed: +ctaBoxed: enabled: true position: 4 image: /images/cloun-native.svg @@ -88,7 +87,7 @@ highPerformance: description: Filter out data from
expensive backends image: /images/reduce-cost.svg imagePosition: - - heading: Tested at 100K /s
at 125 MB/s. + - heading: Tested at 100K /s
at 125 MB/s. align: center description: Check out our benchmarks" image: /images/tested.svg @@ -107,7 +106,7 @@ testimonials: enabled: false position: 5 heading: Our Reviews - subHeading: + subHeading: features: enabled: true position: 5 @@ -121,4 +120,4 @@ cta: image: "/images/cta.png" heading: Stay Updated subHeading: Leave your email and get connected with our lastest news, relases and more. ---- \ No newline at end of file +--- diff --git a/content/announcements/_index.md b/content/announcements/_index.md index b2be05b..8cc9a12 100644 --- a/content/announcements/_index.md +++ b/content/announcements/_index.md @@ -1,111 +1,12 @@ --- title: 'A better knowledge base' -description: 'We provides the means for the collection, organization and computerized retrieval of knowledge' +description: 'We provide the means for the collection, organization and computerized retrieval of knowledge' headerTheme: light url: 'announcements' heroBg: "/images/hero.jpg" latestVer: true --- - -###### KNOWLEDGE BASE - -### Release Notes v1.8.12 - -[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD and OSX. We are proud to announce the availability of **Fluent Bit v1.8.12**. - -For people upgrading from previous versions you **must read** the Upgrading Notes section of our documentation: - -[https://docs.fluentbit.io/manual/installation/upgrade_notes](https://docs.fluentbit.io/manual/installation/upgrade_notes) - -#### News - -[Fluent Bit](https://fluentbit.io) v1.8.12 is the stable release!, new changes on this version: - -**Core** - -* tls: openssl: fix error handling for OpenSSL apis (#4584) -* input chunk: correctly calculate the difference when chunks grow (#4318) (#4461) -* parser: if multiline parser is set, get the reference -* network: properly react to being awoken due to a timeout -* network: preserve addrlinfo list release address -* network: argument type error correction -* network: added `net.dns.prefer_ipv4` result priorization option -* build: add a sample config file for Windows -* upstream: moved event injection and implemented timeout wakeup mechanism -* upstream: add new option `net.connect_timeout_log_error` (#4473) -* input_chunk: warn the user about tag truncation due to limits (#4467) -* dockerfiles: backport systemd lib fixes from #3177 to 1.8 (#4567) - -**Tests:** - -* tests: internal: timeout: added coroutine hang test case (#4605) -* tests: runtime: add tests for multiline filter -* tests: runtime: in_tail: new multiline + json + regex test - -**Libs:** - -* lib: mbedtls: upgrade from v2.27 -> v2.28 -* lib: monkey: sync inject enhancement and list fixes (#4623) -* lib: monkey: sync fix for config reader - -**Plugins** - -* [Forward (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/forward/) - * Clear out_tag before using it (#4670) - * Add new option `tag_prefix` (#4455) (#4466) -* [Tail (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/tail/) - * Enhance handling of high number of static files (#3947) - * Fix multiline api buffer ingestion (#4554) -* [Exec (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/exec/) - * Don’t eat last character of plugin output (#4655) -* [Rewrite_Tag (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/rewrite_tag/) - * Set ‘keep’ before emitting the record (#4518) -* [Multiline (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/multiline/) - * Implement buffered mode using in_emitter -* [S3 (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/s3/) - * Add support for external_id -* [Kinesis_Firehose (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/kinesis_firehose/) - * Add support for external_id -* [Cloudwatch_Logs (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/cloudwatch_logs/) - * Add support for external_id - * Fix invalid memory access bug #4425 -* [Splunk (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/splunk/) - * New `http_debug_bad_request` option - * Auto-set http response buffer size if `http_buffer_size` is not set or zero -* [Forward (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/forward/) - * Add missing ra check (#4511) -* [Stackdriver (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/stackdriver/) - * Protect against undefined metadata (#4664) -* [Kinesis_Streams (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/kinesis_streams/) - * Add support for external_id - -{{< contributor-list >}} - -#### Contributors - -On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! - -* [Ramya Krishnamoorthy](https://github.com/krispraws) -* [Leonardo Alminana](https://github.com/leonardo-albertovich) -* [Takahiro Yamashita](https://github.com/nokute78) -* [Pat](https://github.com/patrick-stephens) -* [Fujimoto Seiji](https://github.com/fujimotos) -* [Wesley Pettit](https://github.com/PettitWesley) -* [Eduardo Silva](https://github.com/edsiper) -* [Jorik Jonker](https://github.com/jonkerj) - -{{< /contributor-list >}} - -#### Join us - -We want to hear about you, our community is growing and you can be part of it!, you can contact us at: - -* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) -* Mailing list: [https://groups.google.com/forum/#!forum/fluent-bit](https://groups.google.com/forum/#!forum/fluent-bit) -* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) -* IRC: irc.freenode.net #fluentbit -* Twitter: [@fluentbit](https://twitter.com/fluentbit) - +{{% content "announcements/v3.2/v3.2.3.md" %}} {{% button href="/announcements/older-versions/" position="center" text-transform="" class="btn-gradient" margin="0px 0px 30px" %}}See Older Versions{{% /button %}} diff --git a/content/announcements/older-versions.md b/content/announcements/older-versions.md index 5cfea39..3e314e2 100644 --- a/content/announcements/older-versions.md +++ b/content/announcements/older-versions.md @@ -2,6 +2,9 @@ title: 'Older Versions' description: | All versions are neatly organized to make it easy to find what you are looking for. + + Security Policy: support for current series ends three months after a new series is released. + headerTheme: light permalink: '/announcements/older-versions' heroBg: "/images/hero.jpg" diff --git a/content/announcements/v1.8/_index.md b/content/announcements/v1.8/_index.md index d209277..15795b6 100644 --- a/content/announcements/v1.8/_index.md +++ b/content/announcements/v1.8/_index.md @@ -1,5 +1,5 @@ --- -title: 'Fluent Bit v1.8 Series' +title: 'Fluent Bit v1.8 Series (EOL Dec 5, 2022)' description: "[Fluent Bit v1.8](https://github.com/fluent/fluent-bit/tree/1.8) is the new **stable branch** for production usage. Based on bug reports or specific minor feature requests, we do quick releases upon demand. Below is a list of the notes for each version." url: "/announcements/v1.8/" herobg: "/images/hero@2x.jpg" diff --git a/content/announcements/v1.8/v1.8.13.md b/content/announcements/v1.8/v1.8.13.md new file mode 100644 index 0000000..5b05330 --- /dev/null +++ b/content/announcements/v1.8/v1.8.13.md @@ -0,0 +1,125 @@ +--- +title: 'v1.8.13' +description: 'We provides the means for the collection, organization and computerized retrieval of knowledgeand Lightweight Data Forwarder for Linux, BSD, macOS and Windows.' +url: "/announcements/v1.8.13/" +release_date: 2022-03-01 +publishdate: 2022-03-01 +ver: v1.8.13 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +### Release Notes v1.8.13 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD and OSX. We are proud to announce the availability of **Fluent Bit v1.8.13**. + +For people upgrading from previous versions you **must read** the Upgrading Notes section of our documentation: + +[https://docs.fluentbit.io/manual/installation/upgrade_notes](https://docs.fluentbit.io/manual/installation/upgrade_notes) + +#### News + +[Fluent Bit](https://fluentbit.io) v1.8.13 is the stable release!, new changes on this version: + +**Core** + + - tls: add support to load system certificates on Windows + - metrics_exporter: use flb_hs_push_health_metrics when health check is registered + - parser: initialize multiline parser_ctx to NULL + - hash: add new API to check if hash exists + - input: initialize cmetrics to zero + - ra_key: reverse order of map lookup when extracting values + - output_thread: fix size of message `0xdeadbeef` to stop the thread + - http_server: health: modify qid of metrics + - http_server: add function to ingest health metrics + - filter: initialize cmetrics to zero + - base64: fork mbedtls 2.25.0 base64 utility to avoid 2.26.0+ performance hit + - plugins: in_http: memory leak correction + - output: allow to define a number of workers + - output: initialize cmetrics to zero + - http_client: fix wrong host header when using proxy (#4816) + - pack: add java_sql_timestamp format (#4822) + - pack: ensure minimum allocation is 256 bytes (unit test) + - pack: enhance memory allocation strategy for large JSON payloads + +**Libs** + +- lib: chunkio: upgrade to v1.1.6 +- lib: monkey: sync capacity improvement and channel release fix + +**Plugins** + + - [Storage_Backlog (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/storage_backlog/) + - Do not abort if chunk cannot be processed + - [Tail (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/tail/) + - Do not use st_dev on windows + - Performance improvement on management of file list + - [Systemd (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/systemd/) + - Skip broken entry + - [Rewrite_Tag (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/rewrite_tag/) + - Fix match aborting issue (#4276) + - [Expect (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/expect/) + - Exit with 255 on failure (#4846) + - [HTTP (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/http/) + - Set 2 workers by default + - [Prometheus_Remote_Write (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/prometheus_remote_write/) + - Set 2 workers by default + - [Stdout (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/stdout/) + - Set 1 worker by default + - [S3 (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/s3/) + - Add default 1 workers + - Migrate to performant base64 implementation + - [Kinesis_Firehose (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/kinesis_firehose/) + - Add default 1 worker + - Migrate to performant base64 implementation + - [Tcp (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/tcp/) + - Set 2 workers by default + - [Prometheus_Exporter (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/prometheus_exporter/) + - Add new option 'add_timestamp', disabled by default + - [File (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/file/) + - Set 1 worker by default + - [Splunk (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/splunk/) + - Set 2 workers by default + - [Forward (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/forward/) + - Set 2 workers by default + - [Stackdriver (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/stackdriver/) + - Set 2 workers by default + - Check for proper http request key + - Add new metric 'fluentbit_stackdriver_requests_total' (#2698) + - Correct env variable for sas (#4763) + - [Null (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/null/) + - Set 1 worker by default + - [Elasticsearch (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/es/) + - Set 2 workers by default + - [Kinesis_Streams (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/kinesis_streams/) + - Add default 1 worker + - Migrate to performant base64 implementation + +{{< contributor-list >}} + +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +* [Ramya Krishnamoorthy](https://github.com/krispraws) +* [Leonardo Alminana](https://github.com/leonardo-albertovich) +* [Takahiro Yamashita](https://github.com/nokute78) +* [Pat](https://github.com/patrick-stephens) +* [Fujimoto Seiji](https://github.com/fujimotos) +* [Wesley Pettit](https://github.com/PettitWesley) +* [Eduardo Silva](https://github.com/edsiper) +* [Jorik Jonker](https://github.com/jonkerj) + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Mailing list: [https://groups.google.com/forum/#!forum/fluent-bit](https://groups.google.com/forum/#!forum/fluent-bit) +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* IRC: irc.freenode.net #fluentbit +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v1.8/v1.8.14.md b/content/announcements/v1.8/v1.8.14.md new file mode 100644 index 0000000..2ac7f5a --- /dev/null +++ b/content/announcements/v1.8/v1.8.14.md @@ -0,0 +1,57 @@ +--- +title: 'v1.8.14' +description: 'We provides the means for the collection, organization and computerized retrieval of knowledgeand Lightweight Data Forwarder for Linux, BSD, macOS and Windows.' +url: "/announcements/v1.8.14/" +release_date: 2022-03-18 +publishdate: 2022-03-18 +ver: v1.8.14 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +### Release Notes v1.8.14 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD and OSX. We are proud to announce the availability of **Fluent Bit v1.8.14**. + +The __v1.8__ series is on maintenance mode only, where it receives critical fixes for users that cannot upgrade to the latest __v1.9__ series. + + +#### Fixes + + - Plugins + - [Systemd (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/systemd/) + - Backport new `lowercase` option (#4992) + - [Tail (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/tail/) + - On file rotation, remove entry from hash tables + - [Cloudwatch_Logs (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/cloudwatch_logs/) + - Fix build issue from #4826 + - Only create log group if it does not already exist to prevent throttling + - [Loki (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/loki/) + - Reduce accessor noise + + +{{< contributor-list >}} + +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +- [Eduardo Silva](https://github.com/edsiper) +- [zhenyami](https://github.com/zhenyami) +- [chitoku-k](https://github.com/chitoku-k) +- [Pat](https://github.com/patrick-stephens) +- [Wesley Pettit](https://github.com/PettitWesley) + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Mailing list: [https://groups.google.com/forum/#!forum/fluent-bit](https://groups.google.com/forum/#!forum/fluent-bit) +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* IRC: irc.freenode.net #fluentbit +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v1.8/v1.8.15.md b/content/announcements/v1.8/v1.8.15.md new file mode 100644 index 0000000..b488e2d --- /dev/null +++ b/content/announcements/v1.8/v1.8.15.md @@ -0,0 +1,46 @@ +--- +title: 'v1.8.15' +description: 'We provides the means for the collection, organization and computerized retrieval of knowledgeand Lightweight Data Forwarder for Linux, BSD, macOS and Windows.' +url: "/announcements/v1.8.15/" +release_date: 2022-03-23 +publishdate: 2022-03-23 +ver: v1.8.15 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +### Release Notes v1.8.15 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD and OSX. We are proud to announce the availability of **Fluent Bit v1.8.15**. + +The __v1.8__ series is on maintenance mode only, where it receives critical fixes for users that cannot upgrade to the latest __v1.9__ series. + + +#### Fixes + + - Core + - task: fixed wrong assumed type for data in flb_task_retry_count + - io: fixed write event monitoring for recycled keep_alive connections + - upstream: replaced non thread safe release call in flb_upstream_conn_release + +{{< contributor-list >}} + +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +- [Leonardo Alminana](https://github.com/leonardo-albertovich) + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Mailing list: [https://groups.google.com/forum/#!forum/fluent-bit](https://groups.google.com/forum/#!forum/fluent-bit) +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* IRC: irc.freenode.net #fluentbit +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v1.9/_index.md b/content/announcements/v1.9/_index.md new file mode 100644 index 0000000..92370b3 --- /dev/null +++ b/content/announcements/v1.9/_index.md @@ -0,0 +1,13 @@ +--- +title: 'Fluent Bit v1.9 Series (EOL Dec 5, 2022)' +description: "[Fluent Bit v1.9](https://github.com/fluent/fluent-bit/tree/1.9) is the new **stable branch** for production usage. Based on bug reports or specific minor feature requests, we do quick releases upon demand. Below is a list of the notes for each version." +url: "/announcements/v1.9/" +herobg: "/images/hero@2x.jpg" +latestVer: true +releaseNotes: + heading: "Release Notes v1.9.9" + version: "v1.9.9" + text: "Fluent Bit is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD and OSX. We are proud to announce the availability of Fluent Bit v1.9.9.
+ For people upgrading from previous versions you must read the Upgrading Notes section of our documentation: + https://docs.fluentbit.io/manual/installation/upgrade_notes" +--- \ No newline at end of file diff --git a/content/announcements/v1.9/v1.9.0.md b/content/announcements/v1.9/v1.9.0.md new file mode 100644 index 0000000..753890a --- /dev/null +++ b/content/announcements/v1.9/v1.9.0.md @@ -0,0 +1,339 @@ +--- +title: 'v1.9.0' +description: 'We provides the means for the collection, organization and computerized retrieval of knowledgeand Lightweight Data Forwarder for Linux, BSD, macOS and Windows.' +url: "/announcements/v1.9.0/" +release_date: 2022-03-11 +publishdate: 2022-03-11 +ver: v1.9.0 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +### Release Notes v1.9.0 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD and OSX. We are proud to announce the availability of **Fluent Bit v1.9.0**. + +For people upgrading from previous versions you **must read** the Upgrading Notes section of our documentation: + +[https://docs.fluentbit.io/manual/installation/upgrade_notes](https://docs.fluentbit.io/manual/installation/upgrade_notes) + +#### Introduction + +Fluent Bit v1.9 is the start of the new stable series of the project. During the last months our primary focus has been around scalability and performance, metrics and overall usability. + +In the performance area, we have created a new __priority queue__ mechanism to handle the processing order of internal events in the events loop, this lead to scale event more in high +demand scenarios, this is an internal change that will benefit majority of users, no user config changes are needed. + +We have been hard working on extending __metrics__ support in Fluent Bit, meaning the input and output metrics plugins, where now is possible to perform end-to-end metrics collection +and delivery. We fully support __Prometheus & OpenMetrics__ and we are also shipping experimental __OpenTelemetry__ metrics support (spoiler: traces will come shortly!). A new Nginx Metrics scraper plugin +is part of this release that supports Nginx OSS and Nginx+ enterprise edition. + +In the Loggin side, we have a new experimental __Kafka__ input plugin, enhanced performance for __Tail__, a new Windows __winevtlog__ that supports non-classic channels, new __OpenSearch__ output plugin +and a hot new __Apache Arrow__ encoding support for our __S3__ output. + +Below you will find a more complete summary of the changes on this release. + +> \> note that not all commits or specific changes are listed, for full details please refer to the GIT history + +#### News + +[Fluent Bit](https://fluentbit.io) v1.9.0 is the stable release!, new changes on this version: + +- __Core__ + - output_thread: fix shutdown message byte size for 32 bits + - output_thread: fixed multiple initialization of local_thread_instance in emulated TLS + - base64: fork mbedtls 2.25.0 base64 utility to avoid 2.26.0+ performance hit + - signv4: fix NULL dereference + - input: re-architect coroutines messaging + - input: initialize cmetrics to zero + - bucket_queue: implement priority bucket queue for priority event loop + - metrics_exporter: use flb_hs_push_health_metrics when health check is registered + - metrics_exporter: new cmetrics context exporter + - random: use getentropy if available. + - network : enable transport layer protocol selection for dns + - input chunk: fix linkage error when storage backlog is disabled + - input chunk: correctly calculate the difference when chunks grow (#4318) + - plugin_proxy: use new flb_event_chunk structure + - winevtlog: add new `winevtlog` input plugin (#4179) + - output: allow to define a number of workers + - output: initialize cmetrics to zero + - output: initialize network defaults for output instances (#4050) (#4088) + - output: added multithreaded worker support for proxy plugins + - output: don't return uninitialized variable + - hash: add new API to check if hash exists + - event: expose new FLB_EVENT_TYPE_ macro + - event: new 'flb_event_' interface + - network: properly react to being awoken due to a timeout + - network: preserve addrlinfo list release address + - network: added dns result priorization option + - network: fixed async usage when not in a coroutine error + - network: added dns context to hold active and drop lists + - network: added async DNS UDP timeout management + - network: refactored the async DNS client + - pack_gelf: make time precision explicit(#3727) + - pack_gelf: ensure proper cleanup in case of failure. + - lua: Add test module for lua util functions + - health: use input coroutine flag to correctly use async network I/O + - ra: allow slash character (#4370) + - ra: allow dash character as an IDENTIFIER(#4236) + - core: integrate priority event loop + - ml_parser_cri: add missing arg skip_empty + - aws_util: support auto_retry_requests for failed connections + - aws_util: added index recognition for flb_get_s3_key + - tls: improve TLS handshake error message (#4561) + - tls: Load system certificates on Windows + - tls: openssl: fix error handling for OpenSSL API #4098 (#4100) + - installation: add script to install in one liner (#4740) + - plugin: add missing char cast + - plugin: add support for config format + - config_map: support deprecated property + - filter: initialize cmetrics to zero + - filter: allow case-insensitive filter name + - codeowners: update to include packaging directory + - ci: update the yaml configuration file for the smoke test. + - regex: log allocation failures in Onigmo + - git: add clangd files to .gitignore (#4299) + - custom: add new 'custom' plugin type support + - upstream: avoid dns timeout null event injection (#4956) + - upstream: replaced timeout wakeup mechanism + - upstream: add new option 'net.connect_timeout_log_error' (#4473) + - upstream: added dns result priorization option + - upstream: added dns resolver configuration switch + - upstream: detect shutdown and reduce log noise + - sha512: use openssl implementation if available (#4303) + - in stdin: Initialize memory to 0 + - engine: unify flush protocol for thread and engine + - engine: add flb_mp_count_remaining to flb_mp + - engine: fix linkage error when storage backlog is disabled + - engine: do not retry if the engine is shutting down + - engine: enforce and reduce shutdown time + - engine: dispatch: update task by using new API + - ra_key: reverse order of map lookup when extracting values + - ra_key: check subkeys length (#3896) + - event_loop: implement priority event loop + - sds: fix missing length set when using flb_sds_snprintf() + - sds: new flb_sds_trim() function + - oauth2: release upstream connection on exit (#4095) + - ml: support FLB_ML_TYPE_MAP to parse (#4034) + - mp: add new function to validate metric chunks + - str: use flb_strndup to implement flb_strdup (#4302) + - error: register custom invalid error + - input_chunk: warn the user about tag truncation due to limits (#4467) + - input_chunk: don't pause ingestion if filesystem storage is enabled (#4221) (#4411) + - mem: fix alloc_size attribute and remove extra ifs (#4305) + - pack: added java_sql_timestamp format (#4811) + - pack: ensure minimum allocation is 256 bytes (unit test) + - pack: enhance memory allocation strategy for large JSON payloads + - pack: only count records when packing plain JSON + - file: add new utility module + - utils: expose missing flb_utils_get_os_name() + - utils: lowercase text errors + - utils: detect and replace ill-formed utf-8 bytes (#4346) + - utils: fix return value handling of uuid generator + - utils: new machine-id reader and OS detector + - parser: support `skip_empty_values` (#1486) If enabled, the parser ignores empty value of the record. + - parsers: add rabbitmq neo4j and dns custom parsers (#4324) + - conf: set default flush intervals to '1' + - cmake: add support for jemalloc page size config (#5020) + - cmake: Enable CMAKE_EXPORT_COMPILE_COMMANDS + - storage: don't ingest metrics when storage.metrics is off + - storage: added extra argument to flb_sched_timer_cb_create call + - config: set default flush interval to '1' + - config: support lowercase proxy environment variables + - config: added dns result priorization option + - config: increase coro stack size if it is less than pagesize (#3716) (#4434) + - config: rename coro stack size define + - config: added dns resolver configuration switch + - config: add new 'is_shutting_down' field + - config: add new 'grace_count' structure field + - io: check connection error only on net_io_read + - io: fixed a bug in net_io_write_async + - io: keep u_conn->coro as NULL unless awaiting event + - aws: create shared compression utility restructure aws cmake to make maintaining nested directories easier + - aws: Add imds tests and mock flb_aws_client via flb_aws_client_mock struct + - aws: Include IMDSv2 support for EC2 instances + - aws: util: set max response buffer size for the http client (#4047) + - aws: Rename struct process stdin,stdout,stderr to fix build + - http_client: fix wrong host header when using proxy (#4810) + - http_client: remove ability to see HTTP request for non-dev env + - task: process direct routes if they are set + - task: use new flb_event_chunk structure + - windows: build legacy td-agent-bit package (#4862) + - windows: cmake: Add version information metadata with version.rc (#4531) + + - __Binary (fluent-bit)__ + + - bin: on '-J', print all JSON config schema + - bin: add support for 'custom' plugin help + - bin: report nightly build info if found + - bin: register config format env variables + - bin: do not overwrite context on static config + - bin: add support for static config + - bin: release config file optarg and always stop the context + - bin: use config_format to compose pipeline from the command line + - bin: read configuration by using new config_format API + - bin: exit with status code + - bin: load 'custom' plugins with '-C' + - bin: add SIGFPE handler + +- __Libraries__ + - lib: cmetrics: upgrade to v0.3.0 + - lib: libbacktrace: upgrade to 2446c66 + - lib: monkey: fix priority initialization for kqueue backend + - lib: xxhash: upgrade to v0.8.1 + - lib: chunkio: upgrade to v1.1.6 + - lib: mbedtls: upgrade from v2.26 to v2.27 + + - __Plugins__ + - [Winlog (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/winlog/) + - Support use_ansi config property (#4129) + - [Kafka (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/kafka/) + - Initial implementation (experimental) + - [Nginx_Exporter_Metrics (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/nginx_exporter_metrics/) + - Validate allocation and set proper null byte + - New nginx metrics exporter (#3800) + - [Systemd (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/systemd/) + - Add `lowercase` option (#4908) + - Update to use config_map. + - Skip broken entry + - [Prometheus_Scrape (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/prometheus_scrape/) + - New prometheus scrape input plugin + - [Storage_Backlog (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/storage_backlog/) + - Do not abort if chunk cannot be processed + - Lookup chunk event type and use it when mapping the chunk + - Properly handle when storage is not set + - [Tail (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/tail/) + - On file rotation, remove entry from hash tables + - Performance improvement on management of file list + - Enhance handling of high number of static files (#3947) + - [Health (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/health/) + - Add support for configmap. (#4907) + - [Forward (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/forward/) + - Increase the buffer size to buffer_max_size + - Fix handling of tcp connections on sigterm (#2610) + - [Proc (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/proc/) + - Fix possible free before allocation. (#5063) + - Add support for configmap (#4902) + - [Nginx_Metrics (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/nginx_metrics/) + - Add nginx plus metrics support (#4379) + - [Http (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/http/) + - Memory leak correction (#4890) + - Add lack of initialization and error handling (#4130) + - [Exec (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/exec/) + - Don't eat last character of plugin output (#4496) + - [Node_Exporter_Metrics (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/node_exporter_metrics/) + - Ensure zero with epsilon on double type factor + - [Windows_Exporter_Metrics (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/windows_exporter_metrics/) + - New windows exporter metrics plugin + - [Opentelemetry (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/opentelemetry/) + - Add experimental support for OpenTelemetry metrics (#5034) + - [Aws (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/aws/) + - Expose metadata into main context env + - Retrieve metadata on initialization + - [Lua (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/lua/) + - Reimplement using mpack (#4524) + - Extract code into flb_lua module + - Calculate table size using table.maxn (#3433) + - [Multiline (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/multiline/) + - Flush before return and new option 'debug_flush' + - [Checklist (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/checklist/) + - New filter to flag records based on matching strings (#3869) + - [Nightfall (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/nightfall/) + - Add `nightfall` filter (#4791) + - [Opensearch (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/opensearch/) + - Initial implementation of `opensearch` output plugin + - [Stdout (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/stdout/) + - Set 1 worker by default + - [TCP (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/tcp/) + - Set 2 workers by default + - [Kafka (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/kafka/) + - Fix broken configuration (#4996) + - [Loki (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/loki/) + - Reduce accessor noise + - Improve performance by removing record counter + - Add comment about label key removal (#4539) + - [File (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/file/) + - Set 1 worker by default + - Add option create output directory if does not exist (#4335) + - Add explicit default format case (#4152) + - [Stackdriver (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/stackdriver/) + - Set 2 workers by default + - Add new metric `fluentbit_stackdriver_requests_total` (#2698) + - [Null (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/null/) + - Set 1 worker by default + - Add `format` option, it supports `json_date_format` and `json_date_key` options. This option has + been added for development and performance test purposes. + - [Elasticsearch (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/es/) + - Set 2 workers by default + - [Calyptia (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/calyptia/) + - Process and append aws metadata + - [S3 (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/s3/) + - Migrate to shared compression lib + - Add default 1 workers + - Migrate to performant base64 implementation + - [Kinesis_Firehose (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/kinesis_firehose/) + - Integrate with shared compression lib + - Add default 1 worker + - [Prometheus_Remote_Write (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/prometheus_remote_write/) + - Add custom headers support + - [Cloudwatch (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/cloudwatch/) + - Fix integer overflow on 32 bit systems when converting tv_sec to millis (#3640) + - [Forward (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/forward/) + - Set 2 workers by default + - [HTTP (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/http/) + - Set 2 workers by default + - [Bigquery (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/bigquery/) + - Support aws auth via gcp workload identity federation (#4282) + - Add `skipinvalidrows` and `ignoreunknownvalues` params (#4158) + - [Influxdb (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/influxdb/) + - Fix retry handling when http call fails + - Enable http debug callbacks + - Rename option `header` as `http_header` + - Migrate to configmap and append new options + - Add support for native metrics + - [Splunk (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/splunk/) + - Set 2 workers by default + - New `http_debug_bad_request` option and auto buffer size + - Auto-set http response buffer size + - [Prometheus_Remote_Write (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/prometheus_remote_write/) + - Reduce log noise for happy path (#4974) + - [Skywalking (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/skywalking/) + - Introduce apache skywalking output plugin (#3592) + - [Prometheus_Remote_Write (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/prometheus_remote_write/) + - Set 2 workers by default + - [Opentelemetry Metrics (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/opentelemetry/) + - Set default metrics uri to `/v1/metrics` + - [Prometheus_Exporter (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/prometheus_exporter/) + - Add new option `add_timestamp`, disabled by default + - [Kinesis_Streams (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/kinesis_streams/) + - Add default 1 worker + - Migrate to performant base64 implementation + +{{< contributor-list >}} + +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +* [Ramya Krishnamoorthy](https://github.com/krispraws) +* [Leonardo Alminana](https://github.com/leonardo-albertovich) +* [Takahiro Yamashita](https://github.com/nokute78) +* [Pat](https://github.com/patrick-stephens) +* [Fujimoto Seiji](https://github.com/fujimotos) +* [Wesley Pettit](https://github.com/PettitWesley) +* [Eduardo Silva](https://github.com/edsiper) +* [Jorik Jonker](https://github.com/jonkerj) + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Mailing list: [https://groups.google.com/forum/#!forum/fluent-bit](https://groups.google.com/forum/#!forum/fluent-bit) +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* IRC: irc.freenode.net #fluentbit +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v1.9/v1.9.1.md b/content/announcements/v1.9/v1.9.1.md new file mode 100644 index 0000000..c125bc8 --- /dev/null +++ b/content/announcements/v1.9/v1.9.1.md @@ -0,0 +1,68 @@ +--- +title: 'v1.9.1' +description: 'We provides the means for the collection, organization and computerized retrieval of knowledgeand Lightweight Data Forwarder for Linux, BSD, macOS and Windows.' +url: "/announcements/v1.9.1/" +release_date: 2022-03-25 +publishdate: 2022-03-25 +ver: v1.9.1 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +### Release Notes v1.9.1 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD and OSX. We are proud to announce the availability of **Fluent Bit v1.9.1**. + +#### Changes + + - Core + - engine: print to log Fluent Bit version and Git commit + - bin: do not print hash on start + - task: fixed wrong assumed type for data in flb_task_retry_count + - parser: always destroy or link config format context + - storage: enhance version and init message + - config_map: add check if flb_env_var_translate failed (#5124) + - io: fixed write event monitoring for recycled keep_alive connections + - config_format: on exception return NULL + - config_format: add new optional head for linked list + - upstream: replaced non thread safe release call in flb_upstream_conn_release + - config: add new config format list + + - Plugins + - [Nightfall (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/nightfall/) + - Fix for loop variable syntax (#5119) + - Capitalize flag in cmakelists (#5107) + - [Kubernetes (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/kubernetes/) + - Fix leak on journal mode when excluding records + - [Opensearch (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/opensearch/) + - Fix double free on index header (#5132) + - [Kafka (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/kafka/) + - Fix broken config map (#5097) + +{{< contributor-list >}} + +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +- [Patrick Stephens](https://github.com/patrick-stephens) +- [Eduardo Silva](https://github.com/edsiper) +- [Takahiro Yamashita](https://github.com/nokute78) +- [Leonardo Alminana](https://github.com/leonardo-albertovich) +- [Victor Chen](https://github.com/victor88121) +- [Hiroshi Hatake](https://github.com/cosmo0920) +- [DavidKorczynski](https://github.com/DavidKorczynski) + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Mailing list: [https://groups.google.com/forum/#!forum/fluent-bit](https://groups.google.com/forum/#!forum/fluent-bit) +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* IRC: irc.freenode.net #fluentbit +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v1.9/v1.9.2.md b/content/announcements/v1.9/v1.9.2.md new file mode 100644 index 0000000..9b89f1c --- /dev/null +++ b/content/announcements/v1.9/v1.9.2.md @@ -0,0 +1,76 @@ +--- +title: 'v1.9.2' +description: 'We provides the means for the collection, organization and computerized retrieval of knowledgeand Lightweight Data Forwarder for Linux, BSD, macOS and Windows.' +url: "/announcements/v1.9.2/" +release_date: 2022-04-08 +publishdate: 2022-04-08 +ver: v1.9.2 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +### Release Notes v1.9.2 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD and OSX. We are proud to announce the availability of **Fluent Bit v1.9.2**. + +#### Changes + + - __Core__ + - time: set ext type 0 as event time(#5215) + - tls: mbedtls: implement proper retry for read and write operations + - lib: mbedtls: Upgrade mbedtls version to v2.28.0 + - aws: extract initialization of aws_provider commonly used + - aws: check AWS EC2 IMDSv2 client error + - multiline: cri: Use non-greedy parsing for parsing time + - build: macos: create certfiticates from keychain on configure + + - __Plugins__ + - [Parser (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/parser/) + - Mark `unescape_key` as a deprecated property + - [Grep (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/grep/) + - Support config map(#5209) + - [Websocket (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/websocket/) + - Add missing json_date_ properties to config_map + - [Lib (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/lib/) + - Change windows-setup.cmake to use flb_out_lib (#5011) + - [S3 (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/s3/) + - Support content_type, storage_class when multipart upload enabled + - Add storage_class support + - [HTTP (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/http/) + - Add sigv4 authentication options + - [Cloudwatch_Logs (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/cloudwatch_logs/) + - Handle dataalreadyacceptedexception response + - [Stackdriver (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/stackdriver/) + - Implement 502 status error for network exceptions + - Implement new metric fluentbit_stackdriver_proc_records_total + +{{< contributor-list >}} + +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +- [Matthew Fala](https://github.com/matthewfala) +- [Eduardo Silva](https://github.com/edsiper) +- [Patrick Stephens](https://github.com/patrick-stephens) +- [SteRoh](https://github.com/SteRoh) +- [Hideo Matsumoto](https://github.com/pekeq) +- [Takahiro Yamashita](https://github.com/nokute78) +- [Hiroshi Hatake](https://github.com/cosmo0920) +- [Wesley Pettit](https://github.com/PettitWesley) +- [Daniel Lenar](https://github.com/danlenar) +- [zhenyami](https://github.com/zhenyami) + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Mailing list: [https://groups.google.com/forum/#!forum/fluent-bit](https://groups.google.com/forum/#!forum/fluent-bit) +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* IRC: irc.freenode.net #fluentbit +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v1.9/v1.9.3.md b/content/announcements/v1.9/v1.9.3.md new file mode 100644 index 0000000..beb66d4 --- /dev/null +++ b/content/announcements/v1.9/v1.9.3.md @@ -0,0 +1,102 @@ +--- +title: 'v1.9.3' +description: 'We provides the means for the collection, organization and computerized retrieval of knowledgeand Lightweight Data Forwarder for Linux, BSD, macOS and Windows.' +url: "/announcements/v1.9.3/" +release_date: 2022-04-08 +publishdate: 2022-04-08 +ver: v1.9.3 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +### Release Notes v1.9.3 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD and OSX. We are proud to announce the availability of **Fluent Bit v1.9.3**. + +#### Changes + + - __Core__ + - filter: add input instance to filter callback + - luajit: new api flb_luajit_load_buffer() + - lib: librdkafka: upgrade from v1.7.0 to v1.8.2 + - lib: chunkio: upgrade to v1.2.0 + - lib: cmetrics: upgrade to v0.3.1 + - fstore: use new chunkio api + - storage: use new chunkip api + - parser: fix null dereference + - strptime: typecast to avoid signed overflow + - ra: support flb_ra_update_kv_pair (#3835) + - input: close event channel on destroy (#5351) + - output: fix potential NULL deref + - csv: implement CSV support interface (#5269) + - pack: use build time parameters for msgpack to json buffer + + + + - __Plugins__ + - [Prometheus_Scrape (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/prometheus_scrape/) + - Fix invalid type reference in log call format string (#5322) + - [Exec (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/exec/) + - Fix using interval from config map (#5227) + - [Storage_Backlog (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/storage_backlog/) + - Skip corrupted chunks + - [Tcp (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/tcp/) + - Close server_fd on destroy + - [Docker (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/docker/) + - Fix error handling if some resources not found (#5189) + - Add missing include/exclude property + - [Nest (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/nest/) + - Fix wildcard config to allow multiple entries (#5264) + - Check if flb_strndup returns null (#5315) + - [Record_Modifier (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/record_modifier/) + - Add error check for flb_strndup(#5103) (#5122) + - [Lua (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/lua/) + - Add return for discarding all records (#5251) (#5271) + - Add new option `code` to load script from a string + - [Multiline (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/multiline-stacktrace/) + - Add fluentd log driver partial message support (#5285) + - Implement buffered mode using in_emitter + - [Grep (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/grep/) + - Check if flb_strndup returns null (#5316) + - [Stackdriver (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/stackdriver/) + - Ensure url encoding for the oauth2 request, and send to the more modern endpoint. (#3198) + - Add static labels defined in configuration (#5176) + - [Bigquery (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/bigquery/) + - Url-encode the oauth request body. + - Switch to a more modern oauth2 token url. + - [Cloudwatch_Logs (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/cloudwatch/) + - Only create log group if it does not already exist to prevent throttling + +{{< contributor-list >}} + +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +- [Takahiro Yamashita](https://github.com/nokute78) +- [David Korczynski](https://github.com/DavidKorczynski) +- [Eduardo Silva](https://github.com/edsiper) +- [Patrick Stephens](https://github.com/patrick-stephens) +- [Wesley Pettit](https://github.com/PettitWesley) +- [Francisco Valente](https://github.com/franciscovalentecastro) +- [Igor Peshansky](https://github.com/igorpeshansky) +- [Laurent Rouquette](https://github.com/lrouquette) +- [Thiago Padilha](https://github.com/tarruda) +- [Leonardo Alminana](https://github.com/leonardo-albertovich) +- [Hiroshi Hatake](https://github.com/cosmo0920) +- [hsmatulisgoogle](https://github.com/hsmatulisgoogle) +- [Ben Buchwald](https://github.com/sparkyb) + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Mailing list: [https://groups.google.com/forum/#!forum/fluent-bit](https://groups.google.com/forum/#!forum/fluent-bit) +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* IRC: irc.freenode.net #fluentbit +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v1.9/v1.9.4.md b/content/announcements/v1.9/v1.9.4.md new file mode 100644 index 0000000..603cb3e --- /dev/null +++ b/content/announcements/v1.9/v1.9.4.md @@ -0,0 +1,94 @@ +--- +title: 'v1.9.4' +description: 'We provides the means for the collection, organization and computerized retrieval of knowledgeand Lightweight Data Forwarder for Linux, BSD, macOS and Windows.' +url: "/announcements/v1.9.4/" +release_date: 2022-06-06 +publishdate: 2022-06-06 +ver: v1.9.4 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +### Release Notes v1.9.4 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD and OSX. We are proud to announce the availability of **Fluent Bit v1.9.4**. + +#### Changes + + - __Core__ + - sp: remove unnecessary cmake requirement + - lib: monkey: sync changes for headers path + - fstore: ensure the chunk is up while modifying it + - utils: seek @ from tail (#5530) + - cpack: build: msi: Support MSI installer with CPack WiX generator (#5367) + - bitbake: fix file name + - build: export monkey headers path + - ra: add append API, flb_ra_append_kv_pair (#5390) + - time: add strict event time type checking + - http_client: add flb_http_get_header + - pack: modify to determine how many JSON messages are OK(#5336) + + - __Plugins__ + - [Forward (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/forward/) + - Support unix_perm (#5511) + - [Opentelemetry (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/opentelemetry/) + - Fixed the event type so OpenTelemetry metrics are properly routed + - [Systemd (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/systemd/) + - Don't insert rows if flb_sqldb_query fails + - Fix 'db.sync' property (#5426) + - [Syslog (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/syslog/) + - Don't release unix_path on exit + - [Lua (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/lua/) + - Fix record split in mpack implementation + - [Kubernetes (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/kubernetes/) + - Add new option `kube_token_ttl` (#4352) (#4487) + - [HTTP (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/http/) + - Implement "body_key" and "headers_key" options (#5439) + - [Kinesis_Firehose (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/kinesis_firehose/) + - Add debug message to check batching + - [Syslog (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/syslog/) + - Add formatter callback + - [Stackdriver (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/stackdriver/) + - Do not allocate metadata_server by config map + - Free metadata_server when 'metadata_server' is not set + - Initialize the metadata_server value to null + - [Azure (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/azure/) + - Fix typo which generates segv (#5379) + - [Cloudwatch_Logs (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/cloudwatch_logs/) + - Add debug message to check payload size + - Set 1 worker as default + - [Kinesis_Streams (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/kinesis_streams/) + - Add debug message to check batching + +{{< contributor-list >}} + +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + + +- [Takahiro Yamashita](https://github.com/nokute78) +- [Jorge Niedbalski](https://github.com/niedbalski) +- [Eduardo Silva](https://github.com/edsiper) +- [Thiago Padilha](https://github.com/tarruda) +- [Patrick Stephens](https://github.com/patrick-stephens) +- [Michael Voelker](https://github.com/novegit) +- [Leonardo Alminana](https://github.com/leonardo-albertovich) +- [Deepak Raja](https://github.com/m-deepakraja) +- [Daniel Lenar](https://github.com/danlenar) +- [Wesley Pettit](https://github.com/PettitWesley) +- [Hiroshi Hatake](https://github.com/cosmo0920) +- [strima](https://github.com/strima) + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Mailing list: [https://groups.google.com/forum/#!forum/fluent-bit](https://groups.google.com/forum/#!forum/fluent-bit) +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v1.9/v1.9.5.md b/content/announcements/v1.9/v1.9.5.md new file mode 100644 index 0000000..6c5ad7b --- /dev/null +++ b/content/announcements/v1.9/v1.9.5.md @@ -0,0 +1,62 @@ +--- +title: 'v1.9.5' +description: 'We provides the means for the collection, organization and computerized retrieval of knowledgeand Lightweight Data Forwarder for Linux, BSD, macOS and Windows.' +url: "/announcements/v1.9.5/" +release_date: 2022-06-22 +publishdate: 2022-06-22 +ver: v1.9.5 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +### Release Notes v1.9.5 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD and OSX. We are proud to announce the availability of **Fluent Bit v1.9.5**. + +#### Changes + + - __Core__ + - core: add compile flag to disable control chars (#5583) + - config: parsers: adding parser for Confluent schema registry + - dockerfiles: add FLUENT_BIT_VERSION environment variable to containers (#5585) + - config: enable YAML support by default (#5360) + + - __Libs__ + - cmetrics: upgrade to v0.3.4 + + - __Plugins__ + - [Opentelemetry (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/opentelemetry/) + - Add support for raw Traces + - [HTTP (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/http/) + - Add formatter function for testing + - [Tcp (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/tcp/) + - Add formatter function for testing + +{{< contributor-list >}} + +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + + +- [Patrick Stephens](https://github.com/patrick-stephens) +- [Sascha Grunert](https://github.com/saschagrunert) +- [Eduardo Silva](https://github.com/edsiper) +- John Preston +- [Jorge Niedbalski](https://github.com/niedbalski) +- [Aditya Prajapati](https://github.com/Syn3rman) +- [Takahiro Yamashita](https://github.com/nokute78) +- sho shimohata + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Mailing list: [https://groups.google.com/forum/#!forum/fluent-bit](https://groups.google.com/forum/#!forum/fluent-bit) +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v1.9/v1.9.6.md b/content/announcements/v1.9/v1.9.6.md new file mode 100644 index 0000000..41c53a5 --- /dev/null +++ b/content/announcements/v1.9/v1.9.6.md @@ -0,0 +1,101 @@ +--- +title: 'v1.9.6' +description: 'We provides the means for the collection, organization and computerized retrieval of knowledgeand Lightweight Data Forwarder for Linux, BSD, macOS and Windows.' +url: "/announcements/v1.9.6/" +release_date: 2022-07-07 +publishdate: 2022-07-07 +ver: v1.9.6 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +### Release Notes v1.9.6 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD and OSX. We are proud to announce the availability of **Fluent Bit v1.9.6**. + +#### Changes + + - __Core__ + - parser: declare loop index + - parser: free allocated types + - in_exec: fix name of oneshot parameter + - fstore: fix args to suppress warning + - multiline: clear last recently used parser to match next parser (#5524) + - storage: fix args to suppress warning + - build: Fix building error for c-ares library on macOS + - build: libyaml: Search libyaml with pkg-config and handle homebrew-ed libyaml (#5641) + - record_accessor: add flb_ra_translate_check that explicitly errors when keys are not found + - record_accessor: fix possible null-deref (#5101) + - input: return error code from cb_collect (#5605) + + - __Libs__ + - cmetrics: upgrade to v0.3.5 + - c-ares: upgrade to v1.18.1 + + - __Plugins__ + - [Opentelemetry (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/opentelemetry/) + - Change endpoints for metrics and traces and fix trace serialization + - On closing connection, release channel + - Set default OTLP/http port to `4318` + - [Netif (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/netif/) + - Add missing return type + - [Multiline (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/multiline/) + - Use flb_time api to get timestamp in millisecond + - [Throttle (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/throttle/) + - Add mutex to prevent race issue + - Fixed use after free (#4792) + - [Modify (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/modify/) + - Add error check for flb_strndup(#5103) + - [Expect (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/expect/) + - Support 'result_key' property + - [Kubernetes (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/kubernetes/) + - Add ability to change `kubelet_host` + - [Datadog (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/datadog/) + - Switch noisy info level to debug level (#5596) + - [Stdout (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/stdout/) + - Handle error case (#5603) + - [S3 (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/s3/) + - Update s3 putobjectsize to 1gb (#5688) + - [Syslog (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/syslog/) + - Fix TLS handling in config and code logic (fix #5643) (#5646) + - [Forward (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/forward/) + - Support `unix_path` + - [Cloudwatch_Logs (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/cloudwatch_logs/) + - Support for record_accessor for templating stream and group names + - [Datadog (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/datadog/)) + - do not append api-key to uri + - support v2 endpoint + +{{< contributor-list >}} + +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +- [Eduardo Silva](https://github.com/edsiper) +- weijian zhang +- [Wesley Pettit](https://github.com/PettitWesley) +- [Patrick Stephens](https://github.com/patrick-stephens) +- [Thomas Danielsson](https://github.com/tumd) +- [Takahiro Yamashita](https://github.com/nokute78) +- [DavidKorczynski](https://github.com/DavidKorczynski) +- [Hiroshi Hatake](https://github.com/cosmo0920) +- [Chris Burr](https://github.com/chrisburr) +- [Jaime Fullaondo](https://github.com/truthbk) +- [Lecaros](https://github.com/lecaros) +- [Furkan Türkal](https://github.com/Dentrax) +- [Sarad Mohanan](https://github.com/sar009) +- [Paulo Neves](https://github.com/ptsneves/) + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Mailing list: [https://groups.google.com/forum/#!forum/fluent-bit](https://groups.google.com/forum/#!forum/fluent-bit) +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v1.9/v1.9.7.md b/content/announcements/v1.9/v1.9.7.md new file mode 100644 index 0000000..bc896ff --- /dev/null +++ b/content/announcements/v1.9/v1.9.7.md @@ -0,0 +1,58 @@ +--- +title: 'v1.9.7' +description: 'We provide the means for the collection, organization and computerized retrieval of knowledge and Lightweight Data Forwarder for Linux, BSD, macOS and Windows.' +url: "/announcements/v1.9.7/" +release_date: 2022-07-07 +publishdate: 2022-07-07 +ver: v1.9.7 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +### Release Notes v1.9.7 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD and OSX. We are proud to announce the availability of **Fluent Bit v1.9.7**. + +#### Changes + + - __Core__ + - event loop: core event kqueue: don't use NOTE_SECONDS on macOS + - aws_util: use concurrent safe gmtime_r + - aws_util: use conccurent safe strtok_r + - aws_util: fix leak in user agent code + - aws_util: set AWS Fluent Bit user agent based on environment + + - __Plugins__ + - [Tail (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/tail/) + - Added event based file ingestion batch limit + - [S3 (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/s3/) + - Add gzip compression support for multipart uploads + - [Cloudwatch_Logs (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/cloudwatch_logs/) + - Skip counting empty events + +{{< contributor-list >}} + +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +- [Leonardo Alminana](https://github.com/leonardo-albertovich) +- [Hiroshi Hatake](https://github.com/cosmo0920) +- [Wesley Pettit](https://github.com/PettitWesley) +- [Zhonghui Hu](https://github.com/zhonghui12) +- Weijian Zhang +- Andy Wang +- [Eduardo Silva](https://github.com/edsiper) + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Mailing list: [https://groups.google.com/forum/#!forum/fluent-bit](https://groups.google.com/forum/#!forum/fluent-bit) +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v1.9/v1.9.8.md b/content/announcements/v1.9/v1.9.8.md new file mode 100644 index 0000000..04386c5 --- /dev/null +++ b/content/announcements/v1.9/v1.9.8.md @@ -0,0 +1,68 @@ +--- +title: 'v1.9.8' +description: 'We provide the means for the collection, organization and computerized retrieval of knowledge and Lightweight Data Forwarder for Linux, BSD, macOS and Windows.' +url: "/announcements/v1.9.8/" +release_date: 2022-09-06 +publishdate: 2022-09-06 +ver: v1.9.8 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +### Release Notes v1.9.8 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD and OSX. We are proud to announce the availability of **Fluent Bit v1.9.8**. + +#### Changes + + - __Core__ + - ra_key: save value when the data type is array (#5936) + - windows: enabled firehose, kinesis streams, and aws filter for Windows + - multiline: fix last_flush data type + - multiline: fix flush timeout logic (fixes #5504) + - pack: fix oss-fuzz crashon tokens state (#5975) + - pack: fix heap-double-free in flb_pack_state_reset (#5883) + + - __Libs__ + - cmetrics: upgrade from v0.3.5 to v0.3.6 + + - __Plugins__ + - [Tail (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/tail/) + - Add missed argument processed_bytes + - Fix path_key when the file is rotated (#5829) + - [Stackdriver (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/stackdriver/) + - Add name label to the metrics emitted + - [Kinesis_Firehose (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/kinesis_firehose/) + - Refactoring to fix windows build + - [Kafka (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/kafka/) + - Add support to force flush when shutdown. (fluent#5593) + +{{< contributor-list >}} + +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +- [Alexs L](https://github.com/sashashura) +- [Eduardo Silva](https://github.com/edsiper) +- [François-Régis Deboffles](https://github.com/frdeboffles) +- [Takahiro Yamashita](https://github.com/nokute78) +- jinyong.choi +- Alexander Shadchin +- [Ridwan Sharif](https://github.com/ridwanmsharif) +- [Hiroshi Hatake](https://github.com/cosmo0920) +- [Thiago Padilha](https://github.com/tarruda) +- [Harsh Rawat](https://github.com/harsh-rawat) + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Mailing list: [https://groups.google.com/forum/#!forum/fluent-bit](https://groups.google.com/forum/#!forum/fluent-bit) +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v1.9/v1.9.9.md b/content/announcements/v1.9/v1.9.9.md new file mode 100644 index 0000000..46d74bb --- /dev/null +++ b/content/announcements/v1.9/v1.9.9.md @@ -0,0 +1,64 @@ +--- +title: 'v1.9.9' +description: 'We provide the means for the collection, organization and computerized retrieval of knowledge and Lightweight Data Forwarder for Linux, BSD, macOS and Windows.' +url: "/announcements/v1.9.9/" +release_date: 2022-09-29 +publishdate: 2022-09-29 +ver: v1.9.9 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +### Release Notes v1.9.9 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD and OSX. We are proud to announce the availability of **Fluent Bit v1.9.9**. + +#### Changes + + - __Core__ + - ra_key: handle error case (#5997) + - cmake: adjust headers for recent changes in Chunk I/O + - input: accept new FLB_STORAGE_MEMRB storage type with metrics + - storage: wrap storage buffer mechanism names + - input_chunk: fix function name to retrieve timestamp + - input_chunk: add logic for new memory-ring-buffer (memrb) + - record_accessor: add NULL check (#5846) + - config_format: fluentbit: report undefined value error(#5880) (#5881) + - config_format: fluentbit: return NULL when some error occurs + +- __Libs__ + - onigmo: bundle new patched [fluent/onigmo](https://github.com/fluent/onigmo) version + - cmetrics: upgrade to v0.3.7 + - chunkio: upgrade to v1.3.0 + + - __Plugins__ + - [Opensearch (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/opensearch/) + - Fix trace_error truncating the response + - [Elasticsearch (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/es/) + - Fix trace_error truncating the response + +{{< contributor-list >}} + +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + + +- [Takahiro Yamashita](https://github.com/nokute78) +- [Eduardo Silva](https://github.com/edsiper) +- [DavidKorczynski](https://github.com/DavidKorczynski) +- [Hiroshi Hatake](https://github.com/cosmo0920) +- [Wesley Pettit](https://github.com/PettitWesley) + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Mailing list: [https://groups.google.com/forum/#!forum/fluent-bit](https://groups.google.com/forum/#!forum/fluent-bit) +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v2.0/_index.md b/content/announcements/v2.0/_index.md new file mode 100644 index 0000000..27f489b --- /dev/null +++ b/content/announcements/v2.0/_index.md @@ -0,0 +1,14 @@ +--- +title: 'Fluent Bit v2.0 Series (EOL Jul 19, 2023)' +description: "[Fluent Bit v1.9](https://github.com/fluent/fluent-bit/tree/2.0) is the new **stable branch** for production usage. Based on bug reports or +specific minor feature requests, we do quick releases upon demand. Below is a list of the notes for each version." +url: "/announcements/v2.0/" +herobg: "/images/hero@2x.jpg" +latestVer: true +releaseNotes: + heading: "Release Notes v2.0.1" + version: "v2.0.1" + text: "Fluent Bit is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD and OSX. We are proud to announce the availability of Fluent Bit v1.9.9.
+ For people upgrading from previous versions you must read the Upgrading Notes section of our documentation: + https://docs.fluentbit.io/manual/installation/upgrade_notes" +--- diff --git a/content/announcements/v2.0/v2.0.0.md b/content/announcements/v2.0/v2.0.0.md new file mode 100644 index 0000000..6ded66a --- /dev/null +++ b/content/announcements/v2.0/v2.0.0.md @@ -0,0 +1,296 @@ +--- +title: 'v2.0.0' +description: 'We provides the means for the collection, organization and computerized retrieval of knowledgeand Lightweight Data Forwarder for Linux, BSD, macOS and Windows.' +url: "/announcements/v2.0.0/" +release_date: 2022-10-25 +publishdate: 2022-10-25 +ver: v2.0.0 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +### Release Notes v2.0.0 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD and OSX. We are proud to announce the availability of **Fluent Bit v2.0.0**. + +For people upgrading from previous versions you **must read** the Upgrading Notes section of our documentation: + +[https://docs.fluentbit.io/manual/installation/upgrade_notes](https://docs.fluentbit.io/manual/installation/upgrade_notes) + +#### Introduction + +Fluent Bit v2.0 is the start of the new stable series of the project. During the last months our primary focus has been around extending support for Metrics, Traces and improving performance, among many others. + +#### News + +[Fluent Bit](https://fluentbit.io) v2.0.0 is the stable release!, new changes on this version: + +
+ +- Logs, Metrics, and Traces + - Metrics + - OpenTelemetry +- Performance +- Full OpenSSL support / deprecate mbedTLS +- TLS for input plugins +- Plugin Changes + - Loki + - Splunk + Metrics +- Developers Experience +- TAP +- Internal Metrics +- New Storage Metrics +- Memory Ring Buffer + - Metrics exposed +- Deprecating TD Agent Bit (packages) + +
+ +#### Logs, Metrics, and Traces + +Fluent Bit has always been agnostic of the data that its processes and moves around; one of the major use cases has been around Log Processing. Recently we introduced functionality to integrate Metrics and now with Fluent Bit v2, we are formally announcing the support for Traces. + +As a vendor-agnostic solution, Fluent Bit natively integrates with protocols and other ecosystems, so having Metrics and Traces support means that we fully integrate with systems like Prometheus and OpenTelemetry. + +
+ +#### Metrics + +As part of the metrics collectors (inputs) and outputs supported for Metrics, allow us to collect and deliver metrics in a smooth way, a common example of these components are: + +| Input | Description | +| -------------------------- | ------------------------------------------------------------ | +| node\_exporter\_metrics | A plugin that implements a subset of host metrics collection like the known external tool Prometheus Node Exporter. | +| nginx\_metrics | Scrape Nginx metrics endpoints. It supports the OSS and Nginx+ enterprise editions. | +| windows\_exporter\_metrics | Collect system metrics from a Windows system, for it support CPU collection. | +| fluentbit\_metrics | Collect internal Fluent Bit metrics and ingest them into the pipeline. | + +
+ +| Output | Description | +| ------------------------- | ------------------------------------------------------------ | +| prometheus\_exporter | Expose metrics in an HTTP endpoint in Prometheus text format. This mechanism is commonly used when you want to scrape metrics available by Fluent Bit by other a third part solution like Prometheus Server. | +| prometheus\_remote\_write | Deliver metrics to a remote endpoint by using the Prometheus Remote Write protocol. | +| splunk | New support for Splunk Metrics (HEC) | +| Influxdb | Send metrics to InfluxDB database | + +One of the biggest addition on this are is the support for OpenTelemetry metrics, in the input and output side, more details below. + +### OpenTelemetry + +OpenTelemetry is a framework, spec and protocol to unify the aspects of Telemetry collection and delivery. One of the major use cases in production is for Traces; Metrics and Logs support has been recently added. + +In Fluent Bit, we are announcing full support for OpenTelemetry, where now we can receive and send telemetry data by using OpenTelemetry protocol: + +**OpenTelemetry input plugin** + +This [new plugin](https://docs.fluentbit.io/manual/pipeline/inputs/opentelemetry), can receive OpenTelemetry Logs, Metrics and Traces, all of them are supported through this new implementation that allow easily integration with applications instrumented with OpenTelemetry SDKs. + +**OpenTelemetry output plugin** + +This [new plugin](https://docs.fluentbit.io/manual/pipeline/outputs/opentelemetry) allows to deliver any Log, Metric or Trace collected to a remote endpoint that supports OpenTelemetry, it could be any vendor platform or the OpenTelemetry collector. + +#### Performance + +New threaded mechanism allows input plugins to run in a separate thread which helps to desaturate the main pipeline and move certain loads to a different CPU core. This feature must be enabled manually in the configuration of each input plugin by adding the `threaded` property, e.g: + +``` +[INPUT] + name tail + path /var/log/containers/*.log + tag kube.* + threaded on +``` + +
+ +#### Full OpenSSL support / deprecate mbedTLS + +With this new version, all the network transport layer that needs security enabled will use OpenSSL instead of mbedTLS: networking & crypto functions. From now on, mbedTLS is officially deprecated. + +All interfaces and plugins that were using mbedTLS has been ported to use the new crypto and networking layer we have implemented. + +
+ +#### TLS for input plugins + +Using Fluent Bit as an aggregator is a common use case, but one component was missing: TLS support for ingestion (Transport Layer Security), which for many users it was a big blocker; the same problem has been reported by many Fluentd users looking forward to migrating to Fluent Bit. + +In this new version, we are announcing full-native TLS capabilities for input plugins. + +
+ +#### Plugins Changes + +##### Loki (output) + +In previous versions, the Loki project did not support out-of-order records, for performance reasons, Fluent Bit delivers messages in parallel which could cause exceptions on the transmission since some messages might not arrive in the order expected by Loki; to solve this problem, we used to force our Loki connector to just send one Chunk at a time, so the order was preserved. + +Since newer versions of Loki no longer have this restriction, we are removing the restriction too from our side and allowing concurrency again, which is a boost in performance. No changes are needed. + +**Note**: this improvement might be a breaking change if you are using an older version of Loki. If you are using **Loki \>= 2.4 ** you are good to go. + +To learn more about this Loki restriction and the enhancements, you can refer to the following blog post from Grafana team: + +[https://grafana.com/blog/2021/12/03/new-feature-in-loki-2.4-no-more-ordering-constraint/](https://grafana.com/blog/2021/12/03/new-feature-in-loki-2.4-no-more-ordering-constraint/) + +##### Splunk + Metrics (output) + +The current Splunk output connector now supports the delivery of Metrics. No changes in your configuration are needed, if you send metrics records to a Splunk output plugin, the data will be converted and delivered as expected by the Splunk HEC endpoint. + +
+ +#### Developers Experience + +There are many cases where a user would like to **extend** the functionality that Fluent Bit provides, usually for specific business reasons. + +For customizations, we currently support plugins that can be written in C, filters in Lua, or output plugins in Golang. But understanding our user’s needs and the restrictions of this limited functionality, we are going beyond that. + +**Fluent Bit v2** comes with support to integrate more plugin types with Golang and WebAssembly. The following table describes the supported languages by plugin type. + +| Language | Input / Source | Filter | Output / Destination | +| ----------- | -------------- | ------ | -------------------- | +| C | Yes | Yes | Yes | +| Golang | Yes | —- | Yes | +| WebAssembly | Yes | Yes | —- | + + +
+ +#### TAP + +One of the common questions from our users has been _“how can I see the data flowing through a pipeline?”_, usually, this comes up when there is a need to perform troubleshooting without stopping the agent. + +TAP is a very advanced functionality that allows inspecting the data that flows through a pipeline, it can be enabled on runtime by just using a simple HTTP call. + +More details about this functionality can be seen here in the official documentation: + +- [https://docs.fluentbit.io/manual/administration/troubleshooting#tap-functionality](https://docs.fluentbit.io/manual/administration/troubleshooting#tap-functionality) + +#### Internal Metrics + +Historically, a common way to extract internal metrics from Fluent Bit has been by enabling the built-in [HTTP service](https://docs.fluentbit.io/manual/administration/monitoring#http-server "Fluent Bit Monitoring - HTTP Server") which exposes the following endpoints in the [REST API](https://docs.fluentbit.io/manual/administration/monitoring#rest-api-interface "Fluent Bit Monitoring REST API"): + +| Endpoint | Description | +| -------------------------- | ------------------------------------------------------------ | +| /api/v1/metrics | JSON metrics for records and byte sizes associated. | +| /api/v1/metrics/prometheus | Same metrics as `/api/v1/metrics` but formatted in Prometheus Text format. | +| /api/v1/storage | JSON metrics for the storage component which expose metrics for Chunks. | + +One of the restrictions of this interface, is that those metrics are not part of the pipeline, which means, you can only access them remotely. But what about if you want to _send_ your metrics to a destination like Prometheus, InfluxDB or other ?. + +During the release cycle of the previous series of Fluent Bit, we introduced a new mechanism to collect and process internal metrics through a new input plugin plugin called `fluentbit_metrics`. This new mechanism allows to expose or send the metrics to a destination in many ways like: + +
+ +- Prometheus Exporter +- Prometheus Remote Write +- InfluxDB +- OpenTelemetry +- Standard output interface (stdout) + +
+ +#### New Storage Metrics + +As mentioned, the Storage metrics originally were exposed in a different endpoint which forced the user to scrape another endpoint and also faced the restriction that _storage_ metrics were only exposed in JSON. + +Starting from Fluent Bit v2, `fluentbit_metrics` exposes seven new metrics for the Storage layer: + +| Metrics name | Description | +| ------------------------------------------- | ------------------------------------------------------------ | +| `fluentbit_input_storage_overlimit` | It takes a value of 1 if the input plugin instance is over the limit imposed by a _mem\_buf\_limit_ configuration. Otherwise is just set to 0. | +| `fluentbit_input_storage_memory_bytes` | Ttotal number of bytes used by the chunks which are up in memory. | +| `fluentbit_input_storage_chunks` | Total number of chunks currently associated to the input plugin instance. This includes chunks in an 'up' and 'down' state. | +| `fluentbit_input_storage_chunks_up` | Total number of chunks that are 'up' in memory generated by the input plugin instance. | +| `fluentbit_input_storage_chunks_down` | Total number of chunks that are in a 'down' state. When the input plugin has enabled the filesystem storage type, not all chunks are loaded 'up' in memory, some of them will be on disk only and not loaded in memory until a later time. This metric represents the number of chunks on that state. | +| `fluentbit_input_storage_chunks_busy` | Total number of chunks that are in a _busy_ state. If a chunk is being used by an output plugin, likely under a flush operation, the chunk is marked as _busy_. | +| `fluentbit_input_storage_chunks_busy_bytes` | Total number of bytes being used by chunks in a _busy_ state. | + +
+ + +#### Memory Ring buffer + +The memory-ring-buffer mechanism is an optional buffering strategy for input plugins that relies only in memory. When this is enabled, the chunks management behaves as a ring buffer of a fixed size were setting up mem_buf_limit is mandatory to specify that limit. + +When the chunks are enqueued, and mem_buf_limit is reached, instead of pausing the input plugin as normally would happen with the default memory strategy, this new mechanism removes the oldest Chunks from the queue until to make sure there is enough space for the new incoming data. + +The configuration of this new strategy is as follows: + +``` + [INPUT] + name tail + path /var/log/* + # new configuration options for memory ring buffer + storage.type memrb + mem_buf_limit 20M +``` + +NOTE: by default Fluent Bit tries to enqueue Chunks of size around 2M, this is an internal size that cannot be changed, but there are scenarios where the Chunk size can be greater than that. So mem_buf_limit should be no less than 20M. + +__Metrics exposed__ + +In addition to the ring-buffer functionality, this PR implements an extension of metrics to count the number of dropped bytes and dropped chunks, e.g: + +``` + # HELP fluentbit_input_memrb_dropped_chunks Number of memrb dropped chunks. + # TYPE fluentbit_input_memrb_dropped_chunks counter + fluentbit_input_memrb_dropped_chunks{name="dummy.0"} 22 + # HELP fluentbit_input_memrb_dropped_bytes Number of memrb dropped bytes. + # TYPE fluentbit_input_memrb_dropped_bytes counter + fluentbit_input_memrb_dropped_bytes{name="dummy.0"} 572 +``` + +The metrics above are an example of how they are retrieved when exposed through the Prometheus Exporter output plugin. The following configuration was used: + +``` +[INPUT] + name dummy + # new configuration options for memory ring buffer + storage.type memrb + mem_buf_limit 20M + +[INPUT] + name fluentbit_metrics + tag metrics + +[OUTPUT] + name prometheus_exporter + match metrics + +[OUTPUT] + name http + match dummy* + host 127.0.0.1 + port 9999 +``` + +
+ +#### Deprecating td-agent-bit + +In the early days, Fluent Bit packages were called `td-agent-bit`, since it was the distribution of Fluent Bit by Treasure Data, the original sponsor of this project, but that’s sponsorship is no longer active. + +TD Agent Bit was widely adopted by the community and during the v1.9 release cycle, we informed the community that this package name will be deprecated, now with v2, we are finalizing that transition. + +From now on, all Fluent Bit packages and container images follow the project name `fluent-bit` as it should be. + +We encourage users to move to Fluent Bit v2 and adopt the new package name. Note that repositories keep being the same. + +{{< contributor-list >}} +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v2.0/v2.0.1.md b/content/announcements/v2.0/v2.0.1.md new file mode 100644 index 0000000..5572b15 --- /dev/null +++ b/content/announcements/v2.0/v2.0.1.md @@ -0,0 +1,299 @@ +--- +title: 'v2.0.1' +description: 'We provides the means for the collection, organization and computerized retrieval of knowledgeand Lightweight Data Forwarder for Linux, BSD, macOS and Windows.' +url: "/announcements/v2.0.1/" +release_date: 2022-10-25 +publishdate: 2022-10-25 +ver: v2.0.1 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +> NOTE: This is a minor hotfix release that covers for corner cases for metrics handling. We keep same +release notes as part of v2 + +### Release Notes v2.0.1 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD and OSX. We are proud to announce the availability of **Fluent Bit v2.0.0**. + +For people upgrading from previous versions you **must read** the Upgrading Notes section of our documentation: + +[https://docs.fluentbit.io/manual/installation/upgrade_notes](https://docs.fluentbit.io/manual/installation/upgrade_notes) + +#### Introduction + +Fluent Bit v2.0 is the start of the new stable series of the project. During the last months our primary focus has been around extending support for Metrics, Traces and improving performance, among many others. + +#### News + +[Fluent Bit](https://fluentbit.io) v2.0.0 is the stable release!, new changes on this version: + +
+ +- Logs, Metrics, and Traces + - Metrics + - OpenTelemetry +- Performance +- Full OpenSSL support / deprecate mbedTLS +- TLS for input plugins +- Plugin Changes + - Loki + - Splunk + Metrics +- Developers Experience +- TAP +- Internal Metrics +- New Storage Metrics +- Memory Ring Buffer + - Metrics exposed +- Deprecating TD Agent Bit (packages) + +
+ +#### Logs, Metrics, and Traces + +Fluent Bit has always been agnostic of the data that its processes and moves around; one of the major use cases has been around Log Processing. Recently we introduced functionality to integrate Metrics and now with Fluent Bit v2, we are formally announcing the support for Traces. + +As a vendor-agnostic solution, Fluent Bit natively integrates with protocols and other ecosystems, so having Metrics and Traces support means that we fully integrate with systems like Prometheus and OpenTelemetry. + +
+ +#### Metrics + +As part of the metrics collectors (inputs) and outputs supported for Metrics, allow us to collect and deliver metrics in a smooth way, a common example of these components are: + +| Input | Description | +| -------------------------- | ------------------------------------------------------------ | +| node\_exporter\_metrics | A plugin that implements a subset of host metrics collection like the known external tool Prometheus Node Exporter. | +| nginx\_metrics | Scrape Nginx metrics endpoints. It supports the OSS and Nginx+ enterprise editions. | +| windows\_exporter\_metrics | Collect system metrics from a Windows system, for it support CPU collection. | +| fluentbit\_metrics | Collect internal Fluent Bit metrics and ingest them into the pipeline. | + +
+ +| Output | Description | +| ------------------------- | ------------------------------------------------------------ | +| prometheus\_exporter | Expose metrics in an HTTP endpoint in Prometheus text format. This mechanism is commonly used when you want to scrape metrics available by Fluent Bit by other a third part solution like Prometheus Server. | +| prometheus\_remote\_write | Deliver metrics to a remote endpoint by using the Prometheus Remote Write protocol. | +| splunk | New support for Splunk Metrics (HEC) | +| Influxdb | Send metrics to InfluxDB database | + +One of the biggest addition on this are is the support for OpenTelemetry metrics, in the input and output side, more details below. + +### OpenTelemetry + +OpenTelemetry is a framework, spec and protocol to unify the aspects of Telemetry collection and delivery. One of the major use cases in production is for Traces; Metrics and Logs support has been recently added. + +In Fluent Bit, we are announcing full support for OpenTelemetry, where now we can receive and send telemetry data by using OpenTelemetry protocol: + +**OpenTelemetry input plugin** + +This [new plugin](https://docs.fluentbit.io/manual/pipeline/inputs/opentelemetry), can receive OpenTelemetry Logs, Metrics and Traces, all of them are supported through this new implementation that allow easily integration with applications instrumented with OpenTelemetry SDKs. + +**OpenTelemetry output plugin** + +This [new plugin](https://docs.fluentbit.io/manual/pipeline/outputs/opentelemetry) allows to deliver any Log, Metric or Trace collected to a remote endpoint that supports OpenTelemetry, it could be any vendor platform or the OpenTelemetry collector. + +#### Performance + +New threaded mechanism allows input plugins to run in a separate thread which helps to desaturate the main pipeline and move certain loads to a different CPU core. This feature must be enabled manually in the configuration of each input plugin by adding the `threaded` property, e.g: + +``` +[INPUT] + name tail + path /var/log/containers/*.log + tag kube.* + threaded on +``` + +
+ +#### Full OpenSSL support / deprecate mbedTLS + +With this new version, all the network transport layer that needs security enabled will use OpenSSL instead of mbedTLS: networking & crypto functions. From now on, mbedTLS is officially deprecated. + +All interfaces and plugins that were using mbedTLS has been ported to use the new crypto and networking layer we have implemented. + +
+ +#### TLS for input plugins + +Using Fluent Bit as an aggregator is a common use case, but one component was missing: TLS support for ingestion (Transport Layer Security), which for many users it was a big blocker; the same problem has been reported by many Fluentd users looking forward to migrating to Fluent Bit. + +In this new version, we are announcing full-native TLS capabilities for input plugins. + +
+ +#### Plugins Changes + +##### Loki (output) + +In previous versions, the Loki project did not support out-of-order records, for performance reasons, Fluent Bit delivers messages in parallel which could cause exceptions on the transmission since some messages might not arrive in the order expected by Loki; to solve this problem, we used to force our Loki connector to just send one Chunk at a time, so the order was preserved. + +Since newer versions of Loki no longer have this restriction, we are removing the restriction too from our side and allowing concurrency again, which is a boost in performance. No changes are needed. + +**Note**: this improvement might be a breaking change if you are using an older version of Loki. If you are using **Loki \>= 2.4 ** you are good to go. + +To learn more about this Loki restriction and the enhancements, you can refer to the following blog post from Grafana team: + +[https://grafana.com/blog/2021/12/03/new-feature-in-loki-2.4-no-more-ordering-constraint/](https://grafana.com/blog/2021/12/03/new-feature-in-loki-2.4-no-more-ordering-constraint/) + +##### Splunk + Metrics (output) + +The current Splunk output connector now supports the delivery of Metrics. No changes in your configuration are needed, if you send metrics records to a Splunk output plugin, the data will be converted and delivered as expected by the Splunk HEC endpoint. + +
+ +#### Developers Experience + +There are many cases where a user would like to **extend** the functionality that Fluent Bit provides, usually for specific business reasons. + +For customizations, we currently support plugins that can be written in C, filters in Lua, or output plugins in Golang. But understanding our user’s needs and the restrictions of this limited functionality, we are going beyond that. + +**Fluent Bit v2** comes with support to integrate more plugin types with Golang and WebAssembly. The following table describes the supported languages by plugin type. + +| Language | Input / Source | Filter | Output / Destination | +| ----------- | -------------- | ------ | -------------------- | +| C | Yes | Yes | Yes | +| Golang | Yes | —- | Yes | +| WebAssembly | Yes | Yes | —- | + + +
+ +#### TAP + +One of the common questions from our users has been _“how can I see the data flowing through a pipeline?”_, usually, this comes up when there is a need to perform troubleshooting without stopping the agent. + +TAP is a very advanced functionality that allows inspecting the data that flows through a pipeline, it can be enabled on runtime by just using a simple HTTP call. + +More details about this functionality can be seen here in the official documentation: + +- [https://docs.fluentbit.io/manual/administration/troubleshooting#tap-functionality](https://docs.fluentbit.io/manual/administration/troubleshooting#tap-functionality) + +#### Internal Metrics + +Historically, a common way to extract internal metrics from Fluent Bit has been by enabling the built-in [HTTP service](https://docs.fluentbit.io/manual/administration/monitoring#http-server "Fluent Bit Monitoring - HTTP Server") which exposes the following endpoints in the [REST API](https://docs.fluentbit.io/manual/administration/monitoring#rest-api-interface "Fluent Bit Monitoring REST API"): + +| Endpoint | Description | +| -------------------------- | ------------------------------------------------------------ | +| /api/v1/metrics | JSON metrics for records and byte sizes associated. | +| /api/v1/metrics/prometheus | Same metrics as `/api/v1/metrics` but formatted in Prometheus Text format. | +| /api/v1/storage | JSON metrics for the storage component which expose metrics for Chunks. | + +One of the restrictions of this interface, is that those metrics are not part of the pipeline, which means, you can only access them remotely. But what about if you want to _send_ your metrics to a destination like Prometheus, InfluxDB or other ?. + +During the release cycle of the previous series of Fluent Bit, we introduced a new mechanism to collect and process internal metrics through a new input plugin plugin called `fluentbit_metrics`. This new mechanism allows to expose or send the metrics to a destination in many ways like: + +
+ +- Prometheus Exporter +- Prometheus Remote Write +- InfluxDB +- OpenTelemetry +- Standard output interface (stdout) + +
+ +#### New Storage Metrics + +As mentioned, the Storage metrics originally were exposed in a different endpoint which forced the user to scrape another endpoint and also faced the restriction that _storage_ metrics were only exposed in JSON. + +Starting from Fluent Bit v2, `fluentbit_metrics` exposes seven new metrics for the Storage layer: + +| Metrics name | Description | +| ------------------------------------------- | ------------------------------------------------------------ | +| `fluentbit_input_storage_overlimit` | It takes a value of 1 if the input plugin instance is over the limit imposed by a _mem\_buf\_limit_ configuration. Otherwise is just set to 0. | +| `fluentbit_input_storage_memory_bytes` | Ttotal number of bytes used by the chunks which are up in memory. | +| `fluentbit_input_storage_chunks` | Total number of chunks currently associated to the input plugin instance. This includes chunks in an 'up' and 'down' state. | +| `fluentbit_input_storage_chunks_up` | Total number of chunks that are 'up' in memory generated by the input plugin instance. | +| `fluentbit_input_storage_chunks_down` | Total number of chunks that are in a 'down' state. When the input plugin has enabled the filesystem storage type, not all chunks are loaded 'up' in memory, some of them will be on disk only and not loaded in memory until a later time. This metric represents the number of chunks on that state. | +| `fluentbit_input_storage_chunks_busy` | Total number of chunks that are in a _busy_ state. If a chunk is being used by an output plugin, likely under a flush operation, the chunk is marked as _busy_. | +| `fluentbit_input_storage_chunks_busy_bytes` | Total number of bytes being used by chunks in a _busy_ state. | + +
+ + +#### Memory Ring buffer + +The memory-ring-buffer mechanism is an optional buffering strategy for input plugins that relies only in memory. When this is enabled, the chunks management behaves as a ring buffer of a fixed size were setting up mem_buf_limit is mandatory to specify that limit. + +When the chunks are enqueued, and mem_buf_limit is reached, instead of pausing the input plugin as normally would happen with the default memory strategy, this new mechanism removes the oldest Chunks from the queue until to make sure there is enough space for the new incoming data. + +The configuration of this new strategy is as follows: + +``` + [INPUT] + name tail + path /var/log/* + # new configuration options for memory ring buffer + storage.type memrb + mem_buf_limit 20M +``` + +NOTE: by default Fluent Bit tries to enqueue Chunks of size around 2M, this is an internal size that cannot be changed, but there are scenarios where the Chunk size can be greater than that. So mem_buf_limit should be no less than 20M. + +__Metrics exposed__ + +In addition to the ring-buffer functionality, this PR implements an extension of metrics to count the number of dropped bytes and dropped chunks, e.g: + +``` + # HELP fluentbit_input_memrb_dropped_chunks Number of memrb dropped chunks. + # TYPE fluentbit_input_memrb_dropped_chunks counter + fluentbit_input_memrb_dropped_chunks{name="dummy.0"} 22 + # HELP fluentbit_input_memrb_dropped_bytes Number of memrb dropped bytes. + # TYPE fluentbit_input_memrb_dropped_bytes counter + fluentbit_input_memrb_dropped_bytes{name="dummy.0"} 572 +``` + +The metrics above are an example of how they are retrieved when exposed through the Prometheus Exporter output plugin. The following configuration was used: + +``` +[INPUT] + name dummy + # new configuration options for memory ring buffer + storage.type memrb + mem_buf_limit 20M + +[INPUT] + name fluentbit_metrics + tag metrics + +[OUTPUT] + name prometheus_exporter + match metrics + +[OUTPUT] + name http + match dummy* + host 127.0.0.1 + port 9999 +``` + +
+ +#### Deprecating td-agent-bit + +In the early days, Fluent Bit packages were called `td-agent-bit`, since it was the distribution of Fluent Bit by Treasure Data, the original sponsor of this project, but that’s sponsorship is no longer active. + +TD Agent Bit was widely adopted by the community and during the v1.9 release cycle, we informed the community that this package name will be deprecated, now with v2, we are finalizing that transition. + +From now on, all Fluent Bit packages and container images follow the project name `fluent-bit` as it should be. + +We encourage users to move to Fluent Bit v2 and adopt the new package name. Note that repositories keep being the same. + +{{< contributor-list >}} +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v2.0/v2.0.10.md b/content/announcements/v2.0/v2.0.10.md new file mode 100644 index 0000000..0db3908 --- /dev/null +++ b/content/announcements/v2.0/v2.0.10.md @@ -0,0 +1,94 @@ +--- +title: 'v2.0.10' +description: 'We provide the means for the collection, organization and computerized retrieval of knowledgeand Lightweight Data Forwarder for Linux, BSD, macOS and Windows.' +url: "/announcements/v2.0.10/" +release_date: 2023-02-06 +publishdate: 2023-02-06 +ver: v2.0.10 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +> NOTE: This is a minor hotfix release that covers for corner cases for metrics handling. We keep same +release notes as part of v2 + +### Release Notes v2.0.10 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, +BSD and OSX. We are proud to announce the availability of **Fluent Bit v2.0.10**. + +#### Introduction + +Fluent Bit v2.0 is the start of the new stable series of the project. During the last months our primary focus has been around extending support for Metrics, Traces and improving performance, among many others. + +#### News + +[Fluent Bit](https://fluentbit.io) v2.0 is the stable release!, new changes on this version: + + - Core + - engine: added retry cleanup when a task cannot be retried + - output: backport of PR 6992 + - config_map: fix possible NULL-deref + - bitbake: fix missing version + - input: removed unnecessary code from flb_input_downstream_set + + - Plugins + - [UDP (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/udp/) + - Fixed event loop bugs in threaded mode + - [HTTP (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/http/) + - Fixed event loop bugs in threaded mode + - [Syslog (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/syslog/) + - Fixed event loop bugs in threaded mode + - [Winevtlog (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/winevtlog/) + - add missing break statement + - [MQTT (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/mqtt/) + - Fixed event loop bugs in threaded mode + - [Tcp (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/tcp/) + - Fixed event loop bugs in threaded mode + - [Opentelemetry (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/opentelemetry/) + - Uninitialized variable usage fix + - Fixed event loop bugs in threaded mode + - [Unix_Socket (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/unix_socket/) + - Fixed event loop bugs in threaded mode + - [Tail (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/tail/) + - Fixed argument order that caused a segfault in issue 6797 + - [Elasticsearch (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/elasticsearch/) + - Fixed event loop bugs in threaded mode + - Implement a plugin for elasticsearch bulk api (for 2.0) (#6685) + - [Forward (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/forward/) + - Fixed event loop bugs in threaded mode + - [Pgsql (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/pgsql/) + - Add new `connection_options` property + - [S3 (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/s3/) + - Backport of pr 6988 + - [Loki (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/loki/) + - Pr 6931 backport + - Add missing flb_sds_cat for float value + - Use flb_sds_pointer to propagate updated pointer. + - [Opentelemetry (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/opentelemetry/) + - Fixed bugs that caused issue 6899 + - Fixed memory corruptions and double frees + +{{< contributor-list >}} +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +- [Jonathan Gonzalez V](https://github.com/sxd) +- [Jeff Erbrecht](https://github.com/jefferbrecht) +- [Leonardo Alminana](https://github.com/leonardo-albertovich) +- [Takahiro Yamashita](https://github.com/nokute78) +- [Hiroshi Hatake](https://github.com/cosmo0920) +- [Eduardo Silva](https://github.com/edsiper) + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v2.0/v2.0.11.md b/content/announcements/v2.0/v2.0.11.md new file mode 100644 index 0000000..598fd9d --- /dev/null +++ b/content/announcements/v2.0/v2.0.11.md @@ -0,0 +1,55 @@ +--- +title: 'v2.0.11' +description: 'We provide the means for the collection, organization and computerized retrieval of knowledgeand Lightweight Data Forwarder for Linux, BSD, macOS and Windows.' +url: "/announcements/v2.0.11/" +release_date: 2023-02-06 +publishdate: 2023-02-06 +ver: v2.0.11 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +> NOTE: This is a minor hotfix release that covers for corner cases for metrics handling. We keep same +release notes as part of v2 + +### Release Notes v2.0.11 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, +BSD and OSX. We are proud to announce the availability of **Fluent Bit v2.0.11**. + +#### Introduction + +Fluent Bit v2.0 is the start of the new stable series of the project. During the last months our primary focus has been around extending support for Metrics, Traces and improving performance, among many others. + +#### News + +[Fluent Bit](https://fluentbit.io) v2.0 is the stable release!, new changes on this version: + + - Libs + - CTraces: upgrade to v0.3.0 + - CMetrics: upgrade to v0.6.0 + + - Packaging + - AL2023 backport support (#7110) + + +{{< contributor-list >}} +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + + +- [Pat](https://github.com/patrick-stephens) +- [Eduardo Silva](https://github.com/edsiper) + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v2.0/v2.0.2.md b/content/announcements/v2.0/v2.0.2.md new file mode 100644 index 0000000..9ee5b3b --- /dev/null +++ b/content/announcements/v2.0/v2.0.2.md @@ -0,0 +1,300 @@ +--- +title: 'v2.0.2' +description: 'We provides the means for the collection, organization and computerized retrieval of knowledgeand Lightweight Data Forwarder for Linux, BSD, macOS and Windows.' +url: "/announcements/v2.0.2/" +release_date: 2022-10-27 +publishdate: 2022-10-27 +ver: v2.0.2 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +> NOTE: This is a minor hotfix release that covers for corner cases for metrics handling. We keep same +release notes as part of v2 + +### Release Notes v2.0.2 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, +BSD and OSX. We are proud to announce the availability of **Fluent Bit v2.0.2**. + +For people upgrading from previous versions you **must read** the Upgrading Notes section of our documentation: + +[https://docs.fluentbit.io/manual/installation/upgrade_notes](https://docs.fluentbit.io/manual/installation/upgrade_notes) + +#### Introduction + +Fluent Bit v2.0 is the start of the new stable series of the project. During the last months our primary focus has been around extending support for Metrics, Traces and improving performance, among many others. + +#### News + +[Fluent Bit](https://fluentbit.io) v2.0 is the stable release!, new changes on this version: + +
+ +- Logs, Metrics, and Traces + - Metrics + - OpenTelemetry +- Performance +- Full OpenSSL support / deprecate mbedTLS +- TLS for input plugins +- Plugin Changes + - Loki + - Splunk + Metrics +- Developers Experience +- TAP +- Internal Metrics +- New Storage Metrics +- Memory Ring Buffer + - Metrics exposed +- Deprecating TD Agent Bit (packages) + +
+ +#### Logs, Metrics, and Traces + +Fluent Bit has always been agnostic of the data that its processes and moves around; one of the major use cases has been around Log Processing. Recently we introduced functionality to integrate Metrics and now with Fluent Bit v2, we are formally announcing the support for Traces. + +As a vendor-agnostic solution, Fluent Bit natively integrates with protocols and other ecosystems, so having Metrics and Traces support means that we fully integrate with systems like Prometheus and OpenTelemetry. + +
+ +#### Metrics + +As part of the metrics collectors (inputs) and outputs supported for Metrics, allow us to collect and deliver metrics in a smooth way, a common example of these components are: + +| Input | Description | +| -------------------------- | ------------------------------------------------------------ | +| node\_exporter\_metrics | A plugin that implements a subset of host metrics collection like the known external tool Prometheus Node Exporter. | +| nginx\_metrics | Scrape Nginx metrics endpoints. It supports the OSS and Nginx+ enterprise editions. | +| windows\_exporter\_metrics | Collect system metrics from a Windows system, for it support CPU collection. | +| fluentbit\_metrics | Collect internal Fluent Bit metrics and ingest them into the pipeline. | + +
+ +| Output | Description | +| ------------------------- | ------------------------------------------------------------ | +| prometheus\_exporter | Expose metrics in an HTTP endpoint in Prometheus text format. This mechanism is commonly used when you want to scrape metrics available by Fluent Bit by other a third part solution like Prometheus Server. | +| prometheus\_remote\_write | Deliver metrics to a remote endpoint by using the Prometheus Remote Write protocol. | +| splunk | New support for Splunk Metrics (HEC) | +| Influxdb | Send metrics to InfluxDB database | + +One of the biggest addition on this are is the support for OpenTelemetry metrics, in the input and output side, more details below. + +### OpenTelemetry + +OpenTelemetry is a framework, spec and protocol to unify the aspects of Telemetry collection and delivery. One of the major use cases in production is for Traces; Metrics and Logs support has been recently added. + +In Fluent Bit, we are announcing full support for OpenTelemetry, where now we can receive and send telemetry data by using OpenTelemetry protocol: + +**OpenTelemetry input plugin** + +This [new plugin](https://docs.fluentbit.io/manual/pipeline/inputs/opentelemetry), can receive OpenTelemetry Logs, Metrics and Traces, all of them are supported through this new implementation that allow easily integration with applications instrumented with OpenTelemetry SDKs. + +**OpenTelemetry output plugin** + +This [new plugin](https://docs.fluentbit.io/manual/pipeline/outputs/opentelemetry) allows to deliver any Log, Metric or Trace collected to a remote endpoint that supports OpenTelemetry, it could be any vendor platform or the OpenTelemetry collector. + +#### Performance + +New threaded mechanism allows input plugins to run in a separate thread which helps to desaturate the main pipeline and move certain loads to a different CPU core. This feature must be enabled manually in the configuration of each input plugin by adding the `threaded` property, e.g: + +``` +[INPUT] + name tail + path /var/log/containers/*.log + tag kube.* + threaded on +``` + +
+ +#### Full OpenSSL support / deprecate mbedTLS + +With this new version, all the network transport layer that needs security enabled will use OpenSSL instead of mbedTLS: networking & crypto functions. From now on, mbedTLS is officially deprecated. + +All interfaces and plugins that were using mbedTLS has been ported to use the new crypto and networking layer we have implemented. + +
+ +#### TLS for input plugins + +Using Fluent Bit as an aggregator is a common use case, but one component was missing: TLS support for ingestion (Transport Layer Security), which for many users it was a big blocker; the same problem has been reported by many Fluentd users looking forward to migrating to Fluent Bit. + +In this new version, we are announcing full-native TLS capabilities for input plugins. + +
+ +#### Plugins Changes + +##### Loki (output) + +In previous versions, the Loki project did not support out-of-order records, for performance reasons, Fluent Bit delivers messages in parallel which could cause exceptions on the transmission since some messages might not arrive in the order expected by Loki; to solve this problem, we used to force our Loki connector to just send one Chunk at a time, so the order was preserved. + +Since newer versions of Loki no longer have this restriction, we are removing the restriction too from our side and allowing concurrency again, which is a boost in performance. No changes are needed. + +**Note**: this improvement might be a breaking change if you are using an older version of Loki. If you are using **Loki \>= 2.4 ** you are good to go. + +To learn more about this Loki restriction and the enhancements, you can refer to the following blog post from Grafana team: + +[https://grafana.com/blog/2021/12/03/new-feature-in-loki-2.4-no-more-ordering-constraint/](https://grafana.com/blog/2021/12/03/new-feature-in-loki-2.4-no-more-ordering-constraint/) + +##### Splunk + Metrics (output) + +The current Splunk output connector now supports the delivery of Metrics. No changes in your configuration are needed, if you send metrics records to a Splunk output plugin, the data will be converted and delivered as expected by the Splunk HEC endpoint. + +
+ +#### Developers Experience + +There are many cases where a user would like to **extend** the functionality that Fluent Bit provides, usually for specific business reasons. + +For customizations, we currently support plugins that can be written in C, filters in Lua, or output plugins in Golang. But understanding our user’s needs and the restrictions of this limited functionality, we are going beyond that. + +**Fluent Bit v2** comes with support to integrate more plugin types with Golang and WebAssembly. The following table describes the supported languages by plugin type. + +| Language | Input / Source | Filter | Output / Destination | +| ----------- | -------------- | ------ | -------------------- | +| C | Yes | Yes | Yes | +| Golang | Yes | —- | Yes | +| WebAssembly | Yes | Yes | —- | + + +
+ +#### TAP + +One of the common questions from our users has been _“how can I see the data flowing through a pipeline?”_, usually, this comes up when there is a need to perform troubleshooting without stopping the agent. + +TAP is a very advanced functionality that allows inspecting the data that flows through a pipeline, it can be enabled on runtime by just using a simple HTTP call. + +More details about this functionality can be seen here in the official documentation: + +- [https://docs.fluentbit.io/manual/administration/troubleshooting#tap-functionality](https://docs.fluentbit.io/manual/administration/troubleshooting#tap-functionality) + +#### Internal Metrics + +Historically, a common way to extract internal metrics from Fluent Bit has been by enabling the built-in [HTTP service](https://docs.fluentbit.io/manual/administration/monitoring#http-server "Fluent Bit Monitoring - HTTP Server") which exposes the following endpoints in the [REST API](https://docs.fluentbit.io/manual/administration/monitoring#rest-api-interface "Fluent Bit Monitoring REST API"): + +| Endpoint | Description | +| -------------------------- | ------------------------------------------------------------ | +| /api/v1/metrics | JSON metrics for records and byte sizes associated. | +| /api/v1/metrics/prometheus | Same metrics as `/api/v1/metrics` but formatted in Prometheus Text format. | +| /api/v1/storage | JSON metrics for the storage component which expose metrics for Chunks. | + +One of the restrictions of this interface, is that those metrics are not part of the pipeline, which means, you can only access them remotely. But what about if you want to _send_ your metrics to a destination like Prometheus, InfluxDB or other ?. + +During the release cycle of the previous series of Fluent Bit, we introduced a new mechanism to collect and process internal metrics through a new input plugin plugin called `fluentbit_metrics`. This new mechanism allows to expose or send the metrics to a destination in many ways like: + +
+ +- Prometheus Exporter +- Prometheus Remote Write +- InfluxDB +- OpenTelemetry +- Standard output interface (stdout) + +
+ +#### New Storage Metrics + +As mentioned, the Storage metrics originally were exposed in a different endpoint which forced the user to scrape another endpoint and also faced the restriction that _storage_ metrics were only exposed in JSON. + +Starting from Fluent Bit v2, `fluentbit_metrics` exposes seven new metrics for the Storage layer: + +| Metrics name | Description | +| ------------------------------------------- | ------------------------------------------------------------ | +| `fluentbit_input_storage_overlimit` | It takes a value of 1 if the input plugin instance is over the limit imposed by a _mem\_buf\_limit_ configuration. Otherwise is just set to 0. | +| `fluentbit_input_storage_memory_bytes` | Ttotal number of bytes used by the chunks which are up in memory. | +| `fluentbit_input_storage_chunks` | Total number of chunks currently associated to the input plugin instance. This includes chunks in an 'up' and 'down' state. | +| `fluentbit_input_storage_chunks_up` | Total number of chunks that are 'up' in memory generated by the input plugin instance. | +| `fluentbit_input_storage_chunks_down` | Total number of chunks that are in a 'down' state. When the input plugin has enabled the filesystem storage type, not all chunks are loaded 'up' in memory, some of them will be on disk only and not loaded in memory until a later time. This metric represents the number of chunks on that state. | +| `fluentbit_input_storage_chunks_busy` | Total number of chunks that are in a _busy_ state. If a chunk is being used by an output plugin, likely under a flush operation, the chunk is marked as _busy_. | +| `fluentbit_input_storage_chunks_busy_bytes` | Total number of bytes being used by chunks in a _busy_ state. | + +
+ + +#### Memory Ring buffer + +The memory-ring-buffer mechanism is an optional buffering strategy for input plugins that relies only in memory. When this is enabled, the chunks management behaves as a ring buffer of a fixed size were setting up mem_buf_limit is mandatory to specify that limit. + +When the chunks are enqueued, and mem_buf_limit is reached, instead of pausing the input plugin as normally would happen with the default memory strategy, this new mechanism removes the oldest Chunks from the queue until to make sure there is enough space for the new incoming data. + +The configuration of this new strategy is as follows: + +``` + [INPUT] + name tail + path /var/log/* + # new configuration options for memory ring buffer + storage.type memrb + mem_buf_limit 20M +``` + +NOTE: by default Fluent Bit tries to enqueue Chunks of size around 2M, this is an internal size that cannot be changed, but there are scenarios where the Chunk size can be greater than that. So mem_buf_limit should be no less than 20M. + +__Metrics exposed__ + +In addition to the ring-buffer functionality, this PR implements an extension of metrics to count the number of dropped bytes and dropped chunks, e.g: + +``` + # HELP fluentbit_input_memrb_dropped_chunks Number of memrb dropped chunks. + # TYPE fluentbit_input_memrb_dropped_chunks counter + fluentbit_input_memrb_dropped_chunks{name="dummy.0"} 22 + # HELP fluentbit_input_memrb_dropped_bytes Number of memrb dropped bytes. + # TYPE fluentbit_input_memrb_dropped_bytes counter + fluentbit_input_memrb_dropped_bytes{name="dummy.0"} 572 +``` + +The metrics above are an example of how they are retrieved when exposed through the Prometheus Exporter output plugin. The following configuration was used: + +``` +[INPUT] + name dummy + # new configuration options for memory ring buffer + storage.type memrb + mem_buf_limit 20M + +[INPUT] + name fluentbit_metrics + tag metrics + +[OUTPUT] + name prometheus_exporter + match metrics + +[OUTPUT] + name http + match dummy* + host 127.0.0.1 + port 9999 +``` + +
+ +#### Deprecating td-agent-bit + +In the early days, Fluent Bit packages were called `td-agent-bit`, since it was the distribution of Fluent Bit by Treasure Data, the original sponsor of this project, but that’s sponsorship is no longer active. + +TD Agent Bit was widely adopted by the community and during the v1.9 release cycle, we informed the community that this package name will be deprecated, now with v2, we are finalizing that transition. + +From now on, all Fluent Bit packages and container images follow the project name `fluent-bit` as it should be. + +We encourage users to move to Fluent Bit v2 and adopt the new package name. Note that repositories keep being the same. + +{{< contributor-list >}} +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v2.0/v2.0.3.md b/content/announcements/v2.0/v2.0.3.md new file mode 100644 index 0000000..3dd9835 --- /dev/null +++ b/content/announcements/v2.0/v2.0.3.md @@ -0,0 +1,59 @@ +--- +title: 'v2.0.3' +description: 'We provides the means for the collection, organization and computerized retrieval of knowledgeand Lightweight Data Forwarder for Linux, BSD, macOS and Windows.' +url: "/announcements/v2.0.3/" +release_date: 2022-10-28 +publishdate: 2022-10-28 +ver: v2.0.3 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +> NOTE: This is a minor hotfix release that covers for corner cases for metrics handling. We keep same +release notes as part of v2 + +### Release Notes v2.0.3 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, +BSD and OSX. We are proud to announce the availability of **Fluent Bit v2.0.3**. + +#### Introduction + +Fluent Bit v2.0 is the start of the new stable series of the project. During the last months our primary focus has been around extending support for Metrics, Traces and improving performance, among many others. + +#### News + +[Fluent Bit](https://fluentbit.io) v2.0 is the stable release!, new changes on this version: + +
+ + - Core + - config_format: fluentbit: call stat again when resolve real path (#6281) + - tests: internal: cf_fluentbit: add test case for #6281 + + - Plugins + - [Winevtlog (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/winevtlog/) + - Fix ignoring non-existent channel handling + - [Forward (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/forward/) + - Put chunk key as the first key of the option map + +{{< contributor-list >}} +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +- [Hiroshi Hatake](https://github.com/cosmo0920) +- [Takahiro Yamashita](https://github.com/nokute78) +- [Eduardo Silva](https://github.com/edsiper) + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v2.0/v2.0.4.md b/content/announcements/v2.0/v2.0.4.md new file mode 100644 index 0000000..cfbbbba --- /dev/null +++ b/content/announcements/v2.0/v2.0.4.md @@ -0,0 +1,89 @@ +--- +title: 'v2.0.4' +description: 'We provides the means for the collection, organization and computerized retrieval of knowledgeand Lightweight Data Forwarder for Linux, BSD, macOS and Windows.' +url: "/announcements/v2.0.4/" +release_date: 2022-11-08 +publishdate: 2022-11-08 +ver: v2.0.4 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +> NOTE: This is a minor hotfix release that covers for corner cases for metrics handling. We keep same +release notes as part of v2 + +### Release Notes v2.0.4 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, +BSD and OSX. We are proud to announce the availability of **Fluent Bit v2.0.4**. + +#### Introduction + +Fluent Bit v2.0 is the start of the new stable series of the project. During the last months our primary focus has been around extending support for Metrics, Traces and improving performance, among many others. + +#### News + +[Fluent Bit](https://fluentbit.io) v2.0 is the stable release!, new changes on this version: + +
+ + - Core + - bin: fix return type for flb_cf_section_property_add. (#6299) + - network: added function to set socket receiving buffer size + - docs: update README and images + - strptime: use flb_tm instead of tm + - parser: use flb_tm in all backends and fix timezone support for Windows + - flb_network: freebsd: Don't use Linux-like structs on FreeBSD + - build: wasm: Disable WASM for *BSD + - env: increase temp buffer size(#6313) + - input: pthread: Ensure default values of initialization for mutexattr + - input: fixed how TLS certificate and private key availability is enforced + - input: added input plugin flag for plugins that expose listeners + - help: enabled networking help for input plugins that start listeners and enabled TLS help listing. + +
+ + - Libs: + - cmetrics: upgrade to v0.5.6 + - monkey: upgrade to 24e68bbf + +
+ + - Plugins + - [Opentelemetry (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/opentelemetry/) + - Upload log as object instead of json string + - Increase batch size for logs export + - [Forward (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/forward/) + - Introduce `fluentd_compat` option for sending Fluentd compatible payloads + - Make forwardable for msgpack payloads of ctraces and cmetrics + - [Opensearch (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/opensearch/) + - Support dynamic index with types + - Add record accessor support for `index` property + +{{< contributor-list >}} +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +- [Phillip Whelan](https://github.com/pwhelan) +- [Hiroshi Hatake](https://github.com/cosmo0920) +- [Benoît GARNIER](https://github.com/ChezBunch) +- [Leonardo Alminana](https://github.com/leonardo-albertovich) +- [Pat](https://github.com/patrick-stephens) +- [Takahiro Yamashita](https://github.com/nokute78) +- [Eduardo Silva](https://github.com/edsiper) +- [Aditya Prajapati](https://github.com/Syn3rman) +- [David Korczynski](https://github.com/DavidKorczynski) +- [Jeff Erbrecht](https://github.com/jefferbrecht) + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v2.0/v2.0.5.md b/content/announcements/v2.0/v2.0.5.md new file mode 100644 index 0000000..9352e13 --- /dev/null +++ b/content/announcements/v2.0/v2.0.5.md @@ -0,0 +1,91 @@ +--- +title: 'v2.0.5' +description: 'We provides the means for the collection, organization and computerized retrieval of knowledgeand Lightweight Data Forwarder for Linux, BSD, macOS and Windows.' +url: "/announcements/v2.0.5/" +release_date: 2022-11-11 +publishdate: 2022-11-11 +ver: v2.0.5 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +> NOTE: This is a minor hotfix release that covers for corner cases for metrics handling. We keep same +release notes as part of v2 + +### Release Notes v2.0.5 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, +BSD and OSX. We are proud to announce the availability of **Fluent Bit v2.0.5**. + +#### Introduction + +Fluent Bit v2.0 is the start of the new stable series of the project. During the last months our primary focus has been around extending support for Metrics, Traces and improving performance, among many others. + +#### News + +[Fluent Bit](https://fluentbit.io) v2.0 is the stable release!, new changes on this version: + +
+ + - Core + - engine: failed/expired retries is an error level event + - chunk-trace: add a warning to enable tracing when it is not enabled. + - multiline: do not permanently set ml group time to the first log time. + - upstream: fix the stuck issue happened in Windows Server 2016 and 2019. (#6288) + - input: fixed the argument verification code so it properly handles input plugins that establish outgoing connections. + - build: windows: enable ECS filter (#6268) + - build: Enable LTO for release builds (#6078) + +
+ + - Libs + - monkey: upgrade to 4b081b3 + - cmetrics: update to v0.5.7 + +
+ + - Plugins + - [Nginx_Exporter_Metrics (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/nginx_exporter_metrics/) + - Enable scraping of tls endpoints. + - Fix vmstat metrics as counter metrics (#6302) + - [Tail (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/tail/) + - Fix st_mtime timestamp format in Windows (#6291) + - [Health (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/health/) + - Enabled optional TLS. + - [Cpu (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/cpu/) + - Fix pointer to v_cpuid (#6238) + - [Prometheus_Scrape (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/prometheus_scrape/) + - Enable scraping of TLS endpoints. + - [ECS (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/ecs/) + - Reduce error/warn messages when the filter processes logs not from a task + - Make metadata api host and port configurable + +{{< contributor-list >}} +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +- [Tobias Nießen](https://github.com/tniessen) +- [Wesley Pettit](https://github.com/PettitWesley) +- Florian BEZANNIER +- [David Sandström](https://github.com/davidsandstrom) +- [Phillip Whelan](https://github.com/pwhelan) +- [Hiroshi Hatake](https://github.com/cosmo0920) +- [Patrick Stephens](https://github.com/patrick-stephens) +- [sudahang](https://github.com/sudahang) +- [Takahiro Yamashita](https://github.com/nokute78) +- [Eduardo Silva](https://github.com/edsiper) +- [Christian Menges](https://github.com/https://github.com/Garfield96) +- [Leonardo Alminana](https://github.com/leonardo-albertovich) + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v2.0/v2.0.6.md b/content/announcements/v2.0/v2.0.6.md new file mode 100644 index 0000000..740023f --- /dev/null +++ b/content/announcements/v2.0/v2.0.6.md @@ -0,0 +1,92 @@ +--- +title: 'v2.0.6' +description: 'We provides the means for the collection, organization and computerized retrieval of knowledgeand Lightweight Data Forwarder for Linux, BSD, macOS and Windows.' +url: "/announcements/v2.0.6/" +release_date: 2022-11-22 +publishdate: 2022-11-22 +ver: v2.0.6 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +> NOTE: This is a minor hotfix release that covers for corner cases for metrics handling. We keep same +release notes as part of v2 + +### Release Notes v2.0.6 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, +BSD and OSX. We are proud to announce the availability of **Fluent Bit v2.0.6**. + +#### Introduction + +Fluent Bit v2.0 is the start of the new stable series of the project. During the last months our primary focus has been around extending support for Metrics, Traces and improving performance, among many others. + +#### News + +[Fluent Bit](https://fluentbit.io) v2.0 is the stable release!, new changes on this version: + +
+ + - Core + - regex: add support for `/patttern/option` (#6257) + - tls: restore connection registration after async io operation + - tls: removed repeated SSL_shutdown call + - tls: properly set the result value on external error + - tls: fix when the TLS session is used after being disposed when a timeout occurs + - downstream: added a check to only set the current coroutine when the downstream is running in async mode. + - core: add minimal synchronous optional scheduler + - upstream_ha: Fixed extra parameters storage + - aws: different user agent on windows vs linux (#6326) + - packaging: add support for Debian "bookworm" (#6409) + - chunk_trace: use ring buffers from the main thread to the trace pipeline. (#6255) + - io: restore connection registration after async io operation + - cloudwatch_logs: enable synchronous scheduler and async networking on cloudwatch + - dockerfiles: fix typo (#6405) + +
+ + - Plugins + - [Tail (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/tail/) + - Add a lasterror to errno conversion layer. + - [Tcp (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/tcp/) + - User friendly warn message for skipping records (#6062) + - [Record_Modifier (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/record_modifier/) + - Support `uuid_key` + - [Forward (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/forward/) + - Fix concurrency issue when operating in Unix domain socket mode + - [Datadog (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/datadog/) + - Fix/add error handling for all flb_sds calls (#5930) + - [OpenTelemetry (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/opentelemetry/) + - Add support for gzip compression (#6232) + - [Elasticsearch (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/es/) + - Fix bulk buffer overrun + +{{< contributor-list >}} +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +- [Takahiro Yamashita](https://github.com/nokute78) +- [Matthew Fala](https://github.com/matthewfala) +- [Patrick Stephens](https://github.com/patrick-stephens) +- [Leonardo Alminana](https://github.com/leonardo-albertovich) +- [Sean Fausett](https://github.com/gitfool) +- [Pat](https://github.com/patrick-stephens) +- [bmarinov](https://github.com/bmarinov) +- [Wesley Pettit](https://github.com/PettitWesley) +- [Florian BEZANNIER](https://github.com/flobz) +- Benoit GARNIER +- [Phillip Whelan](https://github.com/pwhelan) +- [Eduardo Silva](https://github.com/edsiper) + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v2.0/v2.0.7.md b/content/announcements/v2.0/v2.0.7.md new file mode 100644 index 0000000..f4648ae --- /dev/null +++ b/content/announcements/v2.0/v2.0.7.md @@ -0,0 +1,119 @@ +--- +title: 'v2.0.7' +description: 'We provides the means for the collection, organization and computerized retrieval of knowledgeand Lightweight Data Forwarder for Linux, BSD, macOS and Windows.' +url: "/announcements/v2.0.7/" +release_date: 2022-12-19 +publishdate: 2022-12-19 +ver: v2.0.7 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +> NOTE: This is a minor hotfix release that covers for corner cases for metrics handling. We keep same +release notes as part of v2 + +### Release Notes v2.0.7 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, +BSD and OSX. We are proud to announce the availability of **Fluent Bit v2.0.7**. + +#### Introduction + +Fluent Bit v2.0 is the start of the new stable series of the project. During the last months our primary focus has been around extending support for Metrics, Traces and improving performance, among many others. + +#### News + +[Fluent Bit](https://fluentbit.io) v2.0 is the stable release!, new changes on this version: + +
+ + - Core + - bin: fix handling of buffer for text helper + - tls: openssl: add SSL_ERROR_SYSCALL case (#6489) + - stream: fixed an condition that caused concurrency issues + - parser: logfmt: optionally reject messages that have keys without values + - parser: fix potential overflow + - aws: util: extra_user_agent is always type flb_sds_t + - init: systemd: allow to set environment variables from file + - proxy: fix memory bugs when use same golang output plugin multiple times (#6469) + - security policy: update supported versions (#6505) + - wasm: release buffer if error occurred + - wasm: fix memory handling error + - config: cf_fluentbit: print config file load error (#6575) + - calyptia: fix segfault when concatenating custom configuration. (#6553) + - calyptia: concatenate configuration as raw strings. (#6529) + +
+ + - Libs + - cfl: upgrade to v0.2.0 + +
+ + - Plugins + - [Syslog (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/syslog/) + - Rename new option to 'raw_message_key' and other adjustments + - Added an option to append the raw message to the log record under a specific key. + - [Tail (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/tail/) + - Remove extra fstat() syscall and remove unused function + - [Windows_Exporter_Metrics (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/windows_exporter_metrics/) + - CPU: don't compare float values directly (#6568) + - [Wasm (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/wasm/) + - Fix error message + - Use calloc to initialize value + - [Throttle (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/throttle/) + - Fix print_status being partially uninitialized (#6500) + - [UDP (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/udp/) + - New output plugin added (#6450) + - [Opensearch (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/opensearch/) + - Fix '(null)' index when using record accessor and id_key/generate_id. (#6543) + - [S3 (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/s3/) + - Always use sync io mode (#6574) + - Add store_dir_limit_size to limit s3 disk usage + - [TCP (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/tcp/) + - New 'raw_message_key' option + - [Kafka (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/kafka/) + - Fix segmentation fault issue while produce_message + - [Opentelemetry (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/opentelemetry/) + - Move variables to heap and sanitize batch size + - Make log records batch size to configurable + - [Prometheus_Exporter (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/prometheus_exporter/) + - Add content-type (#6554) (#6572) + - [Cloudwatch_Logs (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/cloudwatch_logs/) + - AWS client extra_user_agent is sds string + +{{< contributor-list >}} +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +- [Eduardo Silva](https://github.com/edsiper) +- [Wesley Pettit](https://github.com/PettitWesley) +- [Aditya Prajapati](https://github.com/Syn3rman) +- [Pat](https://github.com/patrick-stephens) +- [Hiroshi Hatake](https://github.com/cosmo0920) +- [Takahiro Yamashita](https://github.com/nokute78) +- [DavidKorczynski](https://github.com/DavidKorczynski) +- [Phillip Whelan](https://github.com/pwhelan) +- [Leonardo Alminana](https://github.com/leonardo-albertovich) +- Dennis Kaarsemaker +- Nick Brown +- Alec Lofquist +- José Lecaros +- [David Korczynski](https://github.com/DavidKorczynski) +- Christian Menges +- Yop Lee +- Jan Zajic +- pioka + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v2.0/v2.0.8.md b/content/announcements/v2.0/v2.0.8.md new file mode 100644 index 0000000..94a3437 --- /dev/null +++ b/content/announcements/v2.0/v2.0.8.md @@ -0,0 +1,86 @@ +--- +title: 'v2.0.8' +description: 'We provides the means for the collection, organization and computerized retrieval of knowledgeand Lightweight Data Forwarder for Linux, BSD, macOS and Windows.' +url: "/announcements/v2.0.8/" +release_date: 2022-12-23 +publishdate: 2022-12-23 +ver: v2.0.8 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +> NOTE: This is a minor hotfix release that covers for corner cases for metrics handling. We keep same +release notes as part of v2 + +### Release Notes v2.0.8 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, +BSD and OSX. We are proud to announce the availability of **Fluent Bit v2.0.8**. + +#### Introduction + +Fluent Bit v2.0 is the start of the new stable series of the project. During the last months our primary focus has been around extending support for Metrics, Traces and improving performance, among many others. + +#### News + +[Fluent Bit](https://fluentbit.io) v2.0 is the stable release!, new changes on this version: + +
+ + - Core + - CMakeLists: use FLB_DEFINTION for FLB_HAVE_*(#6519) + - revert: packaging: provide more options to build packages with (#6550) + - parser: fix two memory leaks when handling exceptions/exit + - parser: free `time_fmt_full` during interim cleanup + - storage: added code to set the irrecoverable chunk deletion flag + - input_chunk: added a function to destroy corrupted chunks + - config: fix two memory leaks when handling exceptions + - config: bail if multiline failed on initialization + - config: add safety checks on exit + - config: added a new storage config to delete irrecoverable chunks + +- Libs + - ctraces: upgrade to v0.2.7 + - cmetrics: upgrade to v0.5.8 + - chunkio: upgrade to v1.4.0 + + - Plugins + - Storage Backlog (Input) + - Added code to discard irrecoverable chunks if desired + - [Windows_Exporter_Metrics (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/windows_exporter_metrics/) + - Implement allowing regex on net metrics to filter by name of nic + - Implement allow/deny regex for logical_disks + - Implement WMI system metrics + - Provide cs metrics + - Provide logical_disk metrics from perflib + - Proceed perflib operation at least 1 perf_object is existing + - Provide network interface metrics + - Implement OS metrics + - [Stackdriver (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/stackdriver/) + - Add Cache token expiration and use it when using cached tokens (#6453) + - [Kafka (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/kafka/) + - Support `iso8601_ns` format + +{{< contributor-list >}} +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +- [David Korczynski](https://github.com/DavidKorczynski) +- [Takahiro Yamashita](https://github.com/nokute78) +- [Eduardo Silva](https://github.com/edsiper) +- [Leonardo Alminana](https://github.com/leonardo-albertovich) +- FIXME: Ridwan Sharif <18472685+ridwanmsharif@users.noreply.github.com> +- [Hiroshi Hatake](https://github.com/cosmo0920) + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v2.0/v2.0.9.md b/content/announcements/v2.0/v2.0.9.md new file mode 100644 index 0000000..0860bff --- /dev/null +++ b/content/announcements/v2.0/v2.0.9.md @@ -0,0 +1,113 @@ +--- +title: 'v2.0.9' +description: 'We provide the means for the collection, organization and computerized retrieval of knowledgeand Lightweight Data Forwarder for Linux, BSD, macOS and Windows.' +url: "/announcements/v2.0.9/" +release_date: 2023-02-06 +publishdate: 2023-02-06 +ver: v2.0.9 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +> NOTE: This is a minor hotfix release that covers for corner cases for metrics handling. We keep same +release notes as part of v2 + +### Release Notes v2.0.9 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, +BSD and OSX. We are proud to announce the availability of **Fluent Bit v2.0.9**. + +#### Introduction + +Fluent Bit v2.0 is the start of the new stable series of the project. During the last months our primary focus has been around extending support for Metrics, Traces and improving performance, among many others. + +#### News + +[Fluent Bit](https://fluentbit.io) v2.0 is the stable release!, new changes on this version: + + - Core + - bin: fix handling of buffer for text helper + - tls: openssl: add SSL_ERROR_SYSCALL case (#6489) + - stream: fixed an condition that caused concurrency issues + - parser: logfmt: optionally reject messages that have keys without values + - parser: fix overflow + - aws: util: extra_user_agent is always type flb_sds_t + - init: systemd: allow to set environment variables from file + - proxy: fix memory bugs when use same golang output plugin multiple times (#6469) + - security policy: update supported versions (#6505) + - wasm: release buffer if error occurred + - wasm: fix memory handling error + - dockerfiles: update systemd to v251.3 (#6508) + - config: cf_fluentbit: print config file load error (#6575) + - calyptia: fix segfault when concatenating custom configuration. (#6553) + - calyptia: concatenate configuration as raw strings. (#6529) + + - Libs + - cfl: upgrade to v0.2.0 + + - Plugins + - [Syslog (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/syslog/) + - Added an option `raw_message_key` to append the raw message to the log record under a specific key. + - [Tail (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/tail/) + - Remove extra fstat() syscall and remove unused function + - [Windows_Exporter_Metrics (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/windows_exporter_metrics/) + - CPU: don't compare float values directly (#6568) + - [Wasm (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/wasm/) + - Fix error message + - Use calloc to initialize value + - [Throttle (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/throttle/) + - Fix print_status being partially uninitialized (#6500) + - [UDP (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/udp/) + - Add new UDP output plugin (#6450) + - [Opensearch (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/opensearch/) + - Fix '(null)' index when using record accessor and id_key/generate_id. (#6543) + - [S3 (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/s3/) + - Always use sync io mode (#6574) + - Add `store_dir_limit_size` option to limit s3 disk usage + - [TCP (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/tcp/) + - Add new `raw_message_key` option + - [Kafka (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/kafka/) + - Fix segmentation fault issue while produce_message + - [Opentelemetry (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/opentelemetry/) + - Move variables to heap and sanitize batch size + - Make log records batch size to configurable + - [Prometheus_Exporter (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/prometheus_exporter/) + - Add content-type header (#6554) (#6572) + - [Cloudwatch_Logs (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/cloudwatch_logs/) + - Aws client extra_user_agent is sds string + +{{< contributor-list >}} +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +- [Eduardo Silva](https://github.com/edsiper) +- [Wesley Pettit](https://github.com/PettitWesley) +- [Aditya Prajapati](https://github.com/Syn3rman) +- [Pat](https://github.com/patrick-stephens) +- [Hiroshi Hatake](https://github.com/cosmo0920) +- [Takahiro Yamashita](https://github.com/nokute78) +- [DavidKorczynski](https://github.com/DavidKorczynski) +- [Phillip Whelan](https://github.com/pwhelan) +- [Leonardo Alminana](https://github.com/leonardo-albertovich) +- Dennis Kaarsemaker +- Nick Brown +- Alec Lofquist +- [José Lecaros](https://github.com/lecaros) +- [David Korczynski](https://github.com/DavidKorczynski) +- Christian Menges +- Yop Lee +- Jan Zajic +- pioka + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v2.1/_index.md b/content/announcements/v2.1/_index.md new file mode 100644 index 0000000..9f13c70 --- /dev/null +++ b/content/announcements/v2.1/_index.md @@ -0,0 +1,14 @@ +--- +title: 'Fluent Bit v2.1 Series (EOL Feb 6, 2024)' +description: "[Fluent Bit v1.9](https://github.com/fluent/fluent-bit/tree/2.0) is the new **stable branch** for production usage. Based on bug reports or +specific minor feature requests, we do quick releases upon demand. Below is a list of the notes for each version." +url: "/announcements/v2.1/" +herobg: "/images/hero@2x.jpg" +latestVer: true +releaseNotes: + heading: "Release Notes v2.0.1" + version: "v2.1.0" + text: "Fluent Bit is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD and OSX. We are proud to announce the availability of Fluent Bit v1.9.9.
+ For people upgrading from previous versions you must read the Upgrading Notes section of our documentation: + https://docs.fluentbit.io/manual/installation/upgrade_notes" +--- diff --git a/content/announcements/v2.1/v2.1.0.md b/content/announcements/v2.1/v2.1.0.md new file mode 100644 index 0000000..2fc5b84 --- /dev/null +++ b/content/announcements/v2.1/v2.1.0.md @@ -0,0 +1,71 @@ +--- +title: 'v2.1.0' +description: 'We provide the means for the collection, organization and computerized retrieval of knowledge and Lightweight Data Forwarder for Linux, BSD, macOS and Windows.' +url: "/announcements/v2.1.0/" +release_date: 2023-04-19 +publishdate: 2023-04-19 +ver: v2.1.0 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +### Release Notes v2.1.0 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD and OSX. We are proud to announce the availability of **Fluent Bit v2.1.0**. + +For people upgrading from previous versions you **must read** the Upgrading Notes section of our documentation: + +[https://docs.fluentbit.io/manual/installation/upgrade_notes](https://docs.fluentbit.io/manual/installation/upgrade_notes) + +#### Introduction + +Fluent Bit v2.1 is the start of the new stable series of the project. During the last months our primary focus has been around extending support for Metrics, Traces and improving performance, among many others. + +#### News + +[Fluent Bit](https://fluentbit.io) v2.1.0 is the stable release!, new changes on this version: + +
+ +#### Hot Reload Support + +Fluent Bit's new hot reload support allows users to reload the configuration without stopping the service. This can be triggered through the SIGHUP signal or through the new /api/v2/reload HTTP endpoint. + +#### Convert your Logs to metrics + +The "logs to metrics" functionality in Fluent Bit allows users to convert logs into metrics. This feature enables users to extract information from log data and create metrics such as counters, gauges, and histograms. The logs are parsed, and the desired metrics are extracted based on defined patterns. Once extracted, the metrics are formatted into a native metric payload and can be sent to the desired destination, such as a monitoring system. This functionality provides users with greater flexibility in how they analyze and utilize log data, and can be particularly useful for monitoring and troubleshooting purposes. + +#### Metrics Collection Extended + +__Linux__ + +The node_exporter metrics plugin for Linux has been extended with new features and enhancements in Fluent Bit's latest release. These improvements include a more detailed filesystem metrics collector and the addition of a textfile collector, which can collect Prometheus text metrics from a file. Additionally, Fluent Bit 2.1 introduces a new feature that allows users to configure different scraping intervals for Linux collectors. With these new features, Fluent Bit provides users with a more customizable and comprehensive monitoring solution for Linux-based systems. + +__Windows__ + +Fluent Bit's Windows exporter metrics plugin has been updated with new features and improvements. This update includes extended support for WMI metrics, enabling users to collect more detailed information from Windows systems. In addition, users can now configure the scrape interval for collectors, allowing for greater flexibility in how often metrics are collected. Furthermore, Fluent Bit now offers general support for Windows on ARM64, enabling users to monitor a wider range of Windows systems. + +#### Log Metadata Support + +In this latest release we introduced metadata support for Logs, providing users with more enriched and customizable log data. Prior to this update, Fluent Bit's data model supported only basic content such as the tag, timestamp, and event/record. However, with the addition of metadata support, users can now attach additional information to their logs, such as context-specific details or additional tags. + +#### Podman Container Metrics + +[Podman](https://podman.io) is a container engine for Linux-based systems that provides a lightweight, non-daemon container environment. It is a popular alternative to Docker for container management and offers advantages such as better security and performance. With Fluent Bit's latest release, a new plugin has been added to collect Podman container metrics. This feature allows users to collect and monitor detailed information about their Podman containers, such as CPU and memory usage, network activity, and more. By providing support for Podman container metrics, Fluent Bit extends its monitoring capabilities to a wider range of container management systems, enabling users to more effectively monitor and manage their containerized applications. + +{{< contributor-list >}} +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v2.1/v2.1.1.md b/content/announcements/v2.1/v2.1.1.md new file mode 100644 index 0000000..44b11b0 --- /dev/null +++ b/content/announcements/v2.1/v2.1.1.md @@ -0,0 +1,75 @@ +--- +title: 'v2.1.1' +description: 'We provides the means for the collection, organization and computerized retrieval of knowledgeand Lightweight Data Forwarder for Linux, BSD, macOS and Windows.' +url: "/announcements/v2.1.1/" +release_date: 2023-04-20 +publishdate: 2023-04-20 +ver: v2.1.1 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +### Release Notes v2.1.1 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD and OSX. We are proud to announce the availability of **Fluent Bit v2.1.1**. + +For people upgrading from previous versions you **must read** the Upgrading Notes section of our documentation: + +[https://docs.fluentbit.io/manual/installation/upgrade_notes](https://docs.fluentbit.io/manual/installation/upgrade_notes) + +--- +> __Apology__ for missing Podman plugin in Fluent Bit v2.1.0. It has been added to the latest version. We apologize for any inconvenience caused. Please upgrade to enjoy the enhanced functionality. +--- + +#### Introduction + +Fluent Bit v2.1 is the start of the new stable series of the project. During the last months our primary focus has been around extending support for Metrics, Traces and improving performance, among many others. + +#### News + +[Fluent Bit](https://fluentbit.io) v2.1.1 is the stable release!, new changes on this version: + +
+ +#### Hot Reload Support + +Fluent Bit's new hot reload support allows users to reload the configuration without stopping the service. This can be triggered through the SIGHUP signal or through the new /api/v2/reload HTTP endpoint. + +#### Convert your Logs to metrics + +The "logs to metrics" functionality in Fluent Bit allows users to convert logs into metrics. This feature enables users to extract information from log data and create metrics such as counters, gauges, and histograms. The logs are parsed, and the desired metrics are extracted based on defined patterns. Once extracted, the metrics are formatted into a native metric payload and can be sent to the desired destination, such as a monitoring system. This functionality provides users with greater flexibility in how they analyze and utilize log data, and can be particularly useful for monitoring and troubleshooting purposes. + +#### Metrics Collection Extended + +__Linux__ + +The node_exporter metrics plugin for Linux has been extended with new features and enhancements in Fluent Bit's latest release. These improvements include a more detailed filesystem metrics collector and the addition of a textfile collector, which can collect Prometheus text metrics from a file. Additionally, Fluent Bit 2.1 introduces a new feature that allows users to configure different scraping intervals for Linux collectors. With these new features, Fluent Bit provides users with a more customizable and comprehensive monitoring solution for Linux-based systems. + +__Windows__ + +Fluent Bit's Windows exporter metrics plugin has been updated with new features and improvements. This update includes extended support for WMI metrics, enabling users to collect more detailed information from Windows systems. In addition, users can now configure the scrape interval for collectors, allowing for greater flexibility in how often metrics are collected. Furthermore, Fluent Bit now offers general support for Windows on ARM64, enabling users to monitor a wider range of Windows systems. + +#### Log Metadata Support + +In this latest release we introduced metadata support for Logs, providing users with more enriched and customizable log data. Prior to this update, Fluent Bit's data model supported only basic content such as the tag, timestamp, and event/record. However, with the addition of metadata support, users can now attach additional information to their logs, such as context-specific details or additional tags. + +#### Podman Container Metrics + +[Podman](https://podman.io) is a container engine for Linux-based systems that provides a lightweight, non-daemon container environment. It is a popular alternative to Docker for container management and offers advantages such as better security and performance. With Fluent Bit's latest release, a new plugin has been added to collect Podman container metrics. This feature allows users to collect and monitor detailed information about their Podman containers, such as CPU and memory usage, network activity, and more. By providing support for Podman container metrics, Fluent Bit extends its monitoring capabilities to a wider range of container management systems, enabling users to more effectively monitor and manage their containerized applications. + +{{< contributor-list >}} +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v2.1/v2.1.10.md b/content/announcements/v2.1/v2.1.10.md new file mode 100644 index 0000000..25056b3 --- /dev/null +++ b/content/announcements/v2.1/v2.1.10.md @@ -0,0 +1,108 @@ +--- +title: 'v2.1.10' +description: 'We provide the means for the collection, organization and computerized retrieval of knowledgeand Lightweight Data Forwarder for Linux, BSD, macOS and Windows.' +url: "/announcements/v2.1.10/" +release_date: 2023-09-28 +publishdate: 2023-09-28 +ver: v2.1.10 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +> NOTE: This is a minor hotfix release that covers for corner cases for metrics handling. We keep same +release notes as part of v2 + +### Release Notes v2.1.10 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, +BSD and OSX. We are proud to announce the availability of **Fluent Bit v2.1.10**. + +#### Introduction + +Fluent Bit v2.1 is the start of the new stable series of the project. During the last months our primary focus has been around extending support for Metrics, Traces and improving performance, among many others. + + - Core + - engine: add sleep to output logs on initialization (#7931) + - kernel: add release function + - plugin_proxy: handle no ingestion case (#7922) + - stream processor: prevent converting from string to num if config is set + - storage: new storage option to handle chunks truncation (new default=off). This is a performance improvement. + - uri: do not allow negative pos + - metrics: register hot-reload metrics + - lua: support metatable to save table type + - lua: push flb_null as a nil value + - http_server: v2: reload: Use more concrate key for storing reloaded times + - http_server: v2: reload: Provide GET endpoint to retrieve the count of hot-reloaded + - scheduler: use calloc instead of malloc (#7962) + - input_chunk: move info message to debug to reduce log noise (#7901) + - config_format: fix memory leak + - config: support add new option sp.convert_from_str_to_num + - log_event_encoder: fix potential issue on 64bit integer corner case + - log_event_encoder: switched data size type to overcome MSVC issue + - fluent-bit binary: follow PHANDLER_ROUTINE definition more closely. + - fluent-bit binary: use console ctrl handlers in win32 to simulate SIGHUP. + + - Libs + - chunkio: upgrade to v1.5.0 + - monkey: upgrade to v1.7.2 (#7951) + + - Connectors / Plugins + - [Oracle_Log_Analytics (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/oracle_log_analytics/) + - Add new Oracle Log Analytics connector (#7830) + - Calyptia_Fleet (Input) + - Add readlink equivalent for Windows. + - Add support for creating configuration files for win32. + - Use generateconsolectrlevent on win32 to execute reload. + - Reenable fleet on windows. + - Improve configuration reloading. (#7925) + - [Systemd (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/systemd/) + - Fix missing initialization to track `last_tag` (#7247) + - [Lua (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/lua/) + - Add new option `enable_flb_null` to deal with NULL values. + - [Log_To_Metrics (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/log_to_metrics/) + - Add new option `add_label` to add label to generated metrics (#7739) + - [Azure_Logs_Ingestion (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/azure_logs_ingestion/) + - Fix null dereference (#7624) + - [Logdna (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/logdna/) + - Remove unused store (#7629) + - [HTTP (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/http/) + - Fix bug where post_all_requests() always returns a value less than 0 + - [Forward (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/forward/) + - Add new advanced property 'add_option' to allow to send custom options in the Forward protocol (advance users only). + - [Loki (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/loki/) + - Add GZIP compression support through the `compress` option (#7949) + - [Azure (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/azure/) + - Add table prefix support for Azure Log Analytics connector (#7663) + - [Elasticsearch (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/es/) + - Check if host contains port number + +{{< contributor-list >}} + +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +- [Aditya Bharadwaj](https://github.com/adiforluls) +- [Leonardo Alminana](https://github.com/leonardo-albertovich) +- [David Korczynski](https://github.com/DavidKorczynski) +- [Phillip Whelan](https://github.com/pwhelan) +- [Hiroshi Hatake](https://github.com/cosmo0920) +- [Takahiro Yamashita](https://github.com/nokute78) +- [DavidKorczynski](https://github.com/DavidKorczynski) +- [Kushal Azim Ekram](https://github.com/https://github.com/kforeverisback) +- [Eduardo Silva](https://github.com/edsiper) +- [Markus Meyer](https://github.com/https://github.com/wndhydrnt) +- Baek Choong deok +- [Kazuhiro Suzuki](https://github.com/https://github.com/ksauzz) + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v2.1/v2.1.2.md b/content/announcements/v2.1/v2.1.2.md new file mode 100644 index 0000000..4956bc8 --- /dev/null +++ b/content/announcements/v2.1/v2.1.2.md @@ -0,0 +1,68 @@ +--- +title: 'v2.1.2' +description: 'We provide the means for the collection, organization and computerized retrieval of knowledgeand Lightweight Data Forwarder for Linux, BSD, macOS and Windows.' +url: "/announcements/v2.1.2/" +release_date: 2023-04-26 +publishdate: 2023-04-26 +ver: v2.1.2 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +> NOTE: This is a minor hotfix release that covers for corner cases for metrics handling. We keep same +release notes as part of v2 + +### Release Notes v2.1.2 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, +BSD and OSX. We are proud to announce the availability of **Fluent Bit v2.1.2**. + +#### Introduction + +Fluent Bit v2.1 is the start of the new stable series of the project. During the last months our primary focus has been around extending support for Metrics, Traces and improving performance, among many others. + + - Core + - developer guide: document the option `-DFLB_VALGRIND=on` + - build: stream processor don't require c++ + - build: macOS: Don't override OPENSSL_ROOT_DIR cmake option if specified + + +- Libs + - CFL: upgrade to v0.2.3 + + - Plugins + - [Opentelemetry (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/opentelemetry/) + - Fix warning at build time + - [Forward (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/forward/) + - Allow to override events tag with 'tag' property + - [Podman_Metrics (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/podman_metrics/) + - Fixed unit tests on cgroupsv2 + - [Forward (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/forward/) + - Added code to drop metadata by default + +{{< contributor-list >}} +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + + +- [Eduardo Silva](https://github.com/edsiper) +- [Leonardo Alminana](https://github.com/leonardo-albertovich) +- [Pat](https://github.com/patrick-stephens) +- [Thomas Devoogdt](https://github.com/ThomasDevoogdt) +- [Hiroshi Hatake](https://github.com/cosmo0920) +- [Pawel Cendrzak](https://github.com/pabloxxl) +- [Takahiro Yamashita](https://github.com/nokute78) +- [Wesley Pettit](https://github.com/PettitWesley) + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v2.1/v2.1.3.md b/content/announcements/v2.1/v2.1.3.md new file mode 100644 index 0000000..ec60965 --- /dev/null +++ b/content/announcements/v2.1/v2.1.3.md @@ -0,0 +1,122 @@ +--- +title: 'v2.1.3' +description: 'We provide the means for the collection, organization and computerized retrieval of knowledgeand Lightweight Data Forwarder for Linux, BSD, macOS and Windows.' +url: "/announcements/v2.1.3/" +release_date: 2023-05-18 +publishdate: 2023-05-18 +ver: v2.1.3 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +> NOTE: This is a minor hotfix release that covers for corner cases for metrics handling. We keep same +release notes as part of v2 + +### Release Notes v2.1.3 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, +BSD and OSX. We are proud to announce the availability of **Fluent Bit v2.1.3**. + +#### Introduction + +Fluent Bit v2.1 is the start of the new stable series of the project. During the last months our primary focus has been around extending support for Metrics, Traces and improving performance, among many others. + + - Core + - core: add new compression abstraction layer + - output_thread: added event memory intialization (#7303) + - cmake: sanitizers-cmake: bump package + - utils: parse quotes when splitting strings + - http_client_debug: fixed format string related compiler warning + - filter: remove unused variable + - build: windows: Handle libyaml directory + - install: update to fix AmazonLinux versions (#7354) + - record_accessor: allow single character input in flb_ra_translate + - config_format: yaml: Plug a lack of argument on Windows + - mp: updated the format check to accept the new format released with 2.1 + - processor: moved the processor stack initialization to the right stage + - parser_json: keep time key when parsing fails (#7323) + +- Libs + - ctraces: upgrade to v0.3.1 + + - Plugins + - [Otel (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/otel/) + - Do not send content-length at 204 + - [Windows_Exporter_Metrics (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/windows_exporter_metrics/) + - Improve value type conversion + - [Http (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/http/) + - Fix parser memory reinitialization + - Do not send content-length at 204 + - [Winevtlog (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/winevtlog/) + - Log event encoder in default case + - Restore the previous behavior of packing string inserts + - Properly pass the log encoder arg + - [Kafka (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/kafka/) + - Updated the plugin to use the log event abstraction layer + - [Syslog (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/syslog/) + - Enable in_syslog plugin on windows + - [Node_Exporter_Metrics (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/node_exporter_metrics/) + - Made diskstats ignore pattern configurable + - Made filesystem ignore patterns configurable + - Added a systemd metrics collector (#7319) + - [Elasticsearch (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/elasticsearch/) + - Refer the plugin provided tag + - [Podman_Metrics (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/podman_metrics/) + - Added image label + - [Geoip2 (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/geoip2/) + - Properly handle the end of chunk data condition + - [Lua (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/lua/) + - Fix buffer size calculation + - [Rewrite_Tag (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/rewrite_tag/) + - Emit using decoder parameters + - [Parser (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/parser/) + - Fix timestamp and metadata to be properly copied (#7293) + - [Modify (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/modify/) + - Fix clean up process + - [Opensearch (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/opensearch/) + - Add support for gzip compression + - [Prometheus_Remote_Write (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/prometheus_remote_write/) + - Added missing header inclusion + - Added configurable compression method + - [Bigquery (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/bigquery/) + - Initialize sig_len + - [Opentelemetry (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/opentelemetry/) + - Fixed ctraces decoder error handling + - [Stackdriver (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/stackdriver/) + - Process spanid special field + - [Vivo_Exporter (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/vivo_exporter/) + - Updated the code to use the log event decoder (#7302) + +{{< contributor-list >}} +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +- [Leonardo Alminana](https://github.com/leonardo-albertovich) +- Markus Meyer +- [Hiroshi Hatake](https://github.com/cosmo0920) +- [Aditya Prajapati](https://github.com/Syn3rman) +- tszshingt +- [Pat](https://github.com/patrick-stephens) +- Andre Silva +- [Takahiro Yamashita](https://github.com/nokute78) +- Braydon Kains +- [Eduardo Silva](https://github.com/edsiper) +- Thomas Devoogdt +- Christian Menges +- Tero Saarni +- Thomas Devoogdt +- Chitoku +- Paweł Cendrzak + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v2.1/v2.1.5.md b/content/announcements/v2.1/v2.1.5.md new file mode 100644 index 0000000..78ad81f --- /dev/null +++ b/content/announcements/v2.1/v2.1.5.md @@ -0,0 +1,94 @@ +--- +title: 'v2.1.5' +description: 'We provide the means for the collection, organization and computerized retrieval of knowledgeand Lightweight Data Forwarder for Linux, BSD, macOS and Windows.' +url: "/announcements/v2.1.5/" +release_date: 2023-06-16 +publishdate: 2023-06-16 +ver: v2.1.5 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +> NOTE: This is a minor hotfix release that covers for corner cases for metrics handling. We keep same +release notes as part of v2 + +### Release Notes v2.1.5 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, +BSD and OSX. We are proud to announce the availability of **Fluent Bit v2.1.5**. + +#### Introduction + +Fluent Bit v2.1 is the start of the new stable series of the project. During the last months our primary focus has been around extending support for Metrics, Traces and improving performance, among many others. + + - Core + - log_event_decoder: fix error description function + - log_event_decoder: add strerror function for decoder + - aws: protect credential providers with pthread_mutex + - aws: check for full xml end node in flb_aws_xml_get_val + - aws: always use calloc for creds + - aws: sts_credentials: make STS response parsing less brittle by checking for the full XML end node (#7357) + - multiline: remove incorrect flush + - signv4: always use calloc for memory allocation + - input_chunk: change a specifier to fix warning + - bin: Ensure thread safety on reloading by default + - bin: Add opt-in option to ensure thread safety on hot reloading + +- Libs + - cmetrics: upgrade to v0.6.2 + + - Plugins + - [Kubernetes_Events (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/kubernetes_events/) + - Fix error when lasttimestamp is missing from an item. + - New input plugin to retrieve kubernetes events (#7494) + - [Windows_Exporter_Metrics (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/windows_exporter_metrics/) + - Fix wrong metrics type of free megabytes + - [Node_Exporter_Metrics (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/node_exporter_metrics/) + - Update the timestamp for metrics of systemd version + - Update systemd gauge metrics on each interval + - Textfile: plug unhandled null case + - [Emitter (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/emitter/) + - Remove unused variable + - [Exec (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/exec/) + - Exit fluent-bit after oneshot and propagate exit code (#7207) + - [S3 (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/s3/) + - Use new flb_aws_xml_get_val + - Fix wrong decrementing of index when uploadpart fails (#7393) + - [Cloudwatch_Logs (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/cloudwatch_logs/) + - Fix logic in free-ing log streams on shutdown (#7159) + - [Azure_Logs_Ingestion (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/azure_logs_ingestion/) + - Implementing azure logs ingestion (with dcr, dce) output plugin (#7155) + - [Prometheus_Remote_Write (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/prometheus_remote_write/) + - Handle 400 status as error immediately + - [Stackdriver (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/stackdriver/) + - Process tracesampled special field + +{{< contributor-list >}} +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +- [Wesley Pettit](https://github.com/PettitWesley) +- [Eduardo Silva](https://github.com/edsiper) +- [Leonardo Alminana](https://github.com/leonardo-albertovich) +- Kushal Azim Ekram +- [Hiroshi Hatake](https://github.com/cosmo0920) +- Craig Ringer +- [Aditya Prajapati](https://github.com/Syn3rman) +- [Takahiro Yamashita](https://github.com/nokute78) +- [Pat](https://github.com/patrick-stephens) +- Andre Silva +- [Phillip Whelan](https://github.com/pwhelan) +- [Thiago Padilha](https://github.com/tarruda) + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v2.1/v2.1.6.md b/content/announcements/v2.1/v2.1.6.md new file mode 100644 index 0000000..69a437e --- /dev/null +++ b/content/announcements/v2.1/v2.1.6.md @@ -0,0 +1,47 @@ +--- +title: 'v2.1.6' +description: 'We provide the means for the collection, organization and computerized retrieval of knowledgeand Lightweight Data Forwarder for Linux, BSD, macOS and Windows.' +url: "/announcements/v2.1.6/" +release_date: 2023-06-16 +publishdate: 2023-06-16 +ver: v2.1.6 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +> NOTE: This is a minor hotfix release that covers for corner cases for metrics handling. We keep same +release notes as part of v2 + +### Release Notes v2.1.6 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, +BSD and OSX. We are proud to announce the availability of **Fluent Bit v2.1.6**. + +#### Introduction + +Fluent Bit v2.1 is the start of the new stable series of the project. During the last months our primary focus has been around extending support for Metrics, Traces and improving performance, among many others. + + - Core + - upstream: new property `net.max_worker_connections` that sets the limit of active TCP connections (#7586) + + +{{< contributor-list >}} + +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +- [Eduardo Silva](https://github.com/edsiper) +- [Celalettin Calis](https://github.com/celalettin1286) + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v2.1/v2.1.7.md b/content/announcements/v2.1/v2.1.7.md new file mode 100644 index 0000000..c9de244 --- /dev/null +++ b/content/announcements/v2.1/v2.1.7.md @@ -0,0 +1,104 @@ +--- +title: 'v2.1.7' +description: 'We provide the means for the collection, organization and computerized retrieval of knowledgeand Lightweight Data Forwarder for Linux, BSD, macOS and Windows.' +url: "/announcements/v2.1.7/" +release_date: 2023-07-13 +publishdate: 2023-07-13 +ver: v2.1.7 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +> NOTE: This is a minor hotfix release that covers for corner cases for metrics handling. We keep same +release notes as part of v2 + +### Release Notes v2.1.7 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, +BSD and OSX. We are proud to announce the availability of **Fluent Bit v2.1.7**. + +#### Introduction + +Fluent Bit v2.1 is the start of the new stable series of the project. During the last months our primary focus has been around extending support for Metrics, Traces and improving performance, among many others. + + - Core + - upstream: fix logic to restrict number of connections with `net.max_worker_connections` option + - core: fix up boolean conversion logic (#7618) + - log_event_decoder: fix handling of results states and extend API + - custom_calyptia: use verbose code to pass TLS parameters to fleet plugin. + - custom_calyptia: set defaults for calyptia fleet. + - downstream: fixed the event loop registration detection mechanism + - utils: plugin: freeing properly for duplicated string shared properties (#7464) + - config_format: yaml: fix 'customs' section handling(#7559) + + - Libs: + - cmetrics: upgrade to v0.6.3 + - wamr: Upgrade the bundled WAMR version to 1.2.2 + + - Plugins + - [Kubernetes_Filter (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/kubernetes_filter/) + - Fix leak on exception + - [Windows_Exporter_Metrics (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/windows_exporter_metrics/) + - Handle `we.service.include` and `we.service.exclude` options to construct where clause + - Prevent to operate service metrics when terminating + - Display errored property name + - Implement windows service metrics + - [Splunk (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/splunk/) + - Implement new Splunk HEC input plugin (#7349) + - [Node_Exporter_Metrics (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/node_exporter_metrics/) + - Update metrics value only if succeeded + - Handle missing throttle metrics case such as vm + - [Elasticsearch (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/elasticsearch/) + - Provide version parameter for configurable server version (#7579) + - [Podman_Metrics (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/podman_metrics/) + - Fixed reading undefined PID + - [Kubernetes_Events (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/kubernetes_events/) + - Call destructor when init failed + - Remove unnecessary type check + - Allow setting `kube_token_file` as empty for use with kubectl proxy. + - Allow listening to a single namespace with the 'namespace' parameter. + - Increase lookup interval to 500 milliseconds. + - [Exec_Wasi (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/exec_wasi/) + - Add error message + - Fix possible null deref + - [Rewrite_Tag (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/rewrite_tag/) + - Fix potential null dereference + - [Stackdriver (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/stackdriver/) + - Support gzip compression with the new `compress` option (#7101) + - [Stdout (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/stdout/) + - Modify to check decoding result + - [Gelf (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/gelf/) + - Fix releasing function handling (#7640) + - [Chronicle (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/chronicle/) + - Implement new Google Chronicle output plugin (#7232) + - [Syslog (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/syslog/) + - Check error after flb_log_event_decoder_next (#7639) + +{{< contributor-list >}} + +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +- [Phillip Whelan](https://github.com/pwhelan) +- [Leonardo Alminana](https://github.com/leonardo-albertovich) +- [Eduardo Silva](https://github.com/edsiper) +- [Braydon Kains](https://github.com/braydonk) +- CatherineF-dev +- [Hiroshi Hatake](https://github.com/cosmo0920) +- [David Korczynski](https://github.com/DavidKorczynski) +- Paweł Cendrzak +- [Takahiro Yamashita](https://github.com/nokute78) +- [Pat](https://github.com/patrick-stephens) + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v2.1/v2.1.8.md b/content/announcements/v2.1/v2.1.8.md new file mode 100644 index 0000000..47f229f --- /dev/null +++ b/content/announcements/v2.1/v2.1.8.md @@ -0,0 +1,98 @@ +--- +title: 'v2.1.8' +description: 'We provide the means for the collection, organization and computerized retrieval of knowledgeand Lightweight Data Forwarder for Linux, BSD, macOS and Windows.' +url: "/announcements/v2.1.8/" +release_date: 2023-07-13 +publishdate: 2023-07-13 +ver: v2.1.8 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +> NOTE: This is a minor hotfix release that covers for corner cases for metrics handling. We keep same +release notes as part of v2 + +### Release Notes v2.1.8 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, +BSD and OSX. We are proud to announce the availability of **Fluent Bit v2.1.8**. + +#### Introduction + +Fluent Bit v2.1 is the start of the new stable series of the project. During the last months our primary focus has been around extending support for Metrics, Traces and improving performance, among many others. + + - Core + - chunk_traces: update the help for `--enable-chunk-trace` + - network: initialize result var for unix socket + - lib: monkey: upgrade to v1.7.1 (fix warnings) + - storage_backlog: improve the release strategy + - calyptia_fleet: fix to use custom calyptia plugin on reload. + - config: allow up to 32MB length of line buffer in read_config (#7691) + - bin: remove cleanup code from flb_signal_exit (#7735) + - custom_calyptia: do not register as agent before loading configuration. + - custom_calyptia: set calyptia_name after instancing the input plugin. + - trace: add new command line options for enabling tracing at startup. (#6726) + - aws: imds: set length when initializing token to dummy value + - upstream: decoupled socket shutdown from upstream connection release + - upstream: added parent upstream transversal to the metrics functions + - upstream: added metrics for upstream connections + - task: added locking and route dropping mechanisms + - connection: a shutdown guard flag was added + - output: added a flush abortion mechanism + - output: added metrics for upstream connections + - input_chunk: changed the chunk release strategy + + - Libs: + - monkey: upgrade to v1.7.1 + + - Plugins + - [HTTP (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/http/) + - Add support for form/url_encoded payloads. (#6960) + - [Tail (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/tail/) + - Fix `ignore_older` description. + - Don't append records if sl_log_event_encoder buffer is 0 (#7723) + - [Collectd (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/collectd/) + - Add missing flb_input_net_server flag. + - [Kafka (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/kafka/) + - Implement new `format` config option. + - Fix json packing + - [Calyptia_Fleet (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/calyptia_fleet/) + - Keep casing consistent in the generated configuration file. + - Maintain fleet_name parameter across reloads. + - Fix use after free and non-null terminated string. + - Add `fleet_name` parameter. + - [Syslog (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/syslog/) + - Provide appending source address parameter (#7651) + - [Statsd (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/statsd/) + - Add missing `flb_input_net_server` flag. + +{{< contributor-list >}} + +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +- [Takahiro Yamashita](https://github.com/nokute78) +- [Hiroshi Hatake](https://github.com/cosmo0920) +- [Thiago Padilha](https://github.com/tarruda) +- [Leonardo Alminana](https://github.com/leonardo-albertovich) +- [Phillip Whelan](https://github.com/pwhelan) +- [Eduardo Silva](https://github.com/edsiper) +- [Lecaros](https://github.com/lecaros) +- Daniel Lenar +- Prateek Mishra +- [Braydon Kains](https://github.com/braydonk) +- Josh Baird + + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v2.1/v2.1.9.md b/content/announcements/v2.1/v2.1.9.md new file mode 100644 index 0000000..6aacbff --- /dev/null +++ b/content/announcements/v2.1/v2.1.9.md @@ -0,0 +1,122 @@ +--- +title: 'v2.1.9' +description: 'We provide the means for the collection, organization and computerized retrieval of knowledgeand Lightweight Data Forwarder for Linux, BSD, macOS and Windows.' +url: "/announcements/v2.1.9/" +release_date: 2023-09-05 +publishdate: 2023-09-05 +ver: v2.1.9 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +> NOTE: This is a minor hotfix release that covers for corner cases for metrics handling. We keep same +release notes as part of v2 + +### Release Notes v2.1.9 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, +BSD and OSX. We are proud to announce the availability of **Fluent Bit v2.1.9**. + +#### Introduction + +Fluent Bit v2.1 is the start of the new stable series of the project. During the last months our primary focus has been around extending support for Metrics, Traces and improving performance, among many others. + + +- Core + - io: added code to detect non-recoverable connection errors + - tls: openssl: added code to propagate syscall errors + - config_format: Yaml + - support idiomatic Yaml (camelCase) + - use a stack based parser + - utils: use ferror and not math to check file handler for error. + - utils: suppress errno when file is empty but readable. + - utils: check existence of machine-id files before reading them. (#7806) + - emitter_create: exit and destroy instance on fail + - lua: pass index to lua_arraylength + - input: apply back pressure if input threaded plugin is paused (#7847) + - input: send resume signal to the input thread event loop if plugin is threaded + - config: throw an error if a configuration property is not applied. + - core: Expand pthread stack size on macOS + - aws: Allocate S3_KEY_SIZE +1 instead of S3_KEY_SIZE for allocating buffer for s3 key + - upstream: added code to discard failed connections + + - Libs + - c-ares: upgrade to v1.19.1 + + - Plugins + - [Nginx_Exporter_Metrics (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/nginx_exporter_metrics/) + - Add missing release function + - [Udp (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/udp/) + - Add a capability to inject source ip (#7673) + - [Windows_Exporter_Metrics (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/windows_exporter_metrics/) + - Implement wmi based process metrics (#7860) + - Implement wmi based memory and paging_file metrics (#7817) + - [Prometheus_Scrape (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/prometheus_scrape/) + - Added authorization support (#7785) + - [Lib (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/lib/) + - Fix error handling after flb_log_event_decoder_next + - [Winevtlog (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/winevtlog/) + - Support xml query parameter for filtering events + - [MQTT (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/mqtt/) + - Support payload_key + - [Event_Test (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/event_test/) + - Add +1 for acceptable time elapse + - [Calyptia_Fleet (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/calyptia_fleet/) + - Use the machine_id in the fleet configuration directory. + - [Tail (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/tail/) + - Removed ignore_older enforcement on files that are being ingested + - Refactored the progress check to signal the event based collector + - Fix event based pending data ingestion collector limits + - [Node_Exporter_Metrics (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/node_exporter_metrics/) + - Fix registering callback for Systemd + - [Exec_Wasi (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/exec_wasi/) + - Fix possible file descriptor leak + - [S3 (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/s3/) + - Fix file descriptor leak + - Fix potential file descriptor leak + - [Stackdriver (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/stackdriver/) + - Change default workers to 1 + - Fix access token caching + - [Kinesis_Streams (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/kinesis_streams/) + - Remove dead stores + - [Splunk (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/splunk/) + - Release flb_ra_translate result buffer on failure + - Calyptia (Custom) + - pass the fleet_id parameter and label to out_calyptia. + - generate the default machine_id in custom_calyptia. + - clarify code that conditionally activates metrics. + - submit metrics when not in fleet mode. + +{{< contributor-list >}} + +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + + +- [Phillip Whelan](https://github.com/pwhelan) +- Daniel Lenar +- [Takahiro Yamashita](https://github.com/nokute78) +- [Leonardo Alminana](https://github.com/leonardo-albertovich) +- [Hiroshi Hatake](https://github.com/cosmo0920) +- [David Korczynski](https://github.com/DavidKorczynski) +- [Eduardo Silva](https://github.com/edsiper) +- Jeff +- [DavidKorczynski](https://github.com/DavidKorczynski) +- [Celalettin Calis](https://https://github.com/celalettin1286) +- [Braydon Kains](https://github.com/braydonk) +- Chico Venancio +- [Pat](https://github.com/patrick-stephens) +- Ridwan Sharif + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v2.2/_index.md b/content/announcements/v2.2/_index.md new file mode 100644 index 0000000..4020120 --- /dev/null +++ b/content/announcements/v2.2/_index.md @@ -0,0 +1,14 @@ +--- +title: 'Fluent Bit v2.2 Series (EOL Jun 3, 2024)' +description: "[Fluent Bit v1.9](https://github.com/fluent/fluent-bit/tree/2.0) is the new **stable branch** for production usage. Based on bug reports or +specific minor feature requests, we do quick releases upon demand. Below is a list of the notes for each version." +url: "/announcements/v2.1/" +herobg: "/images/hero@2x.jpg" +latestVer: true +releaseNotes: + heading: "Release Notes v2.2.1" + version: "v2.2.1" + text: "Fluent Bit is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD and OSX. We are proud to announce the availability of Fluent Bit v1.9.9.
+ For people upgrading from previous versions you must read the Upgrading Notes section of our documentation: + https://docs.fluentbit.io/manual/installation/upgrade_notes" +--- diff --git a/content/announcements/v2.2/v2.2.0.md b/content/announcements/v2.2/v2.2.0.md new file mode 100644 index 0000000..2a4e407 --- /dev/null +++ b/content/announcements/v2.2/v2.2.0.md @@ -0,0 +1,155 @@ +--- +title: 'v2.2.0' +description: 'We provide the means for the collection, organization and computerized retrieval of knowledge and Lightweight Data Forwarder for Linux, BSD, macOS and Windows.' +url: "/announcements/v2.2.0/" +release_date: 2023-11-06 +publishdate: 2023-11-06 +ver: v2.2.0 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +### Release Notes v2.2.0 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD and OSX. We are proud to announce the availability of **Fluent Bit v2.2.0**. + +For people upgrading from previous versions you **must read** the Upgrading Notes section of our documentation: + +[https://docs.fluentbit.io/manual/installation/upgrade_notes](https://docs.fluentbit.io/manual/installation/upgrade_notes) + +#### Introduction + +Fluent Bit v2.2 is the start of the new stable series of the project. During the last months our primary focus has been around extending support for Metrics, Traces and improving performance, among many others. + +#### News + +[Fluent Bit](https://fluentbit.io) v2.2.0 is the stable release!, new changes on this version: + +
+ + - Core + - filter: improve performance by 6x when doing multiple filters (chain) + - http_server: v2: reload: windows: Enable hot reloading (#7984) + - output: set tcp keepalive flag when desired + - output: plugin_proxy: Migrate initialization of Golang output as cb_init based + - fluent-bit: improve hot reload handler (#8041) + - input: Add ingestion_paused metrics to confirm whether an input plugin is paused or not (#8044) + - http_client: set Connection header to keepalive to fix problem with HTTP proxies. + - config: plugin_proxy: Pass whether hot-reloading state or not into Golan plugins on pre_exit callbacks + - bin: reload: filter: Provide additional validator before loading filter plugins + - bin: Fix build on non-Windows + - bin: windows: Restore Ctrl-C behavior on windows + - custom_calyptia: correctly set fleet_id for out_calyptia when fleet_name is set. + - custom_calyptia: free machine_id when it is automatically set. + - containers: add support s390x architecture (#8095) + - plugin_proxy: Plug an occurrence of garbage character on hot-reloading + - plugin_proxy: Prevent to execute flush callback on Golang side during hot-reloading + - plugin_proxy: Tell whether hot-reloading is enabled or not for Golang plugins + - plugin_proxy: Expose pause/resume callbacks for Golang input plugins + - plugin_proxy: Expose pre_run callbacks into Golang plugins + - stream processor: fix missing key names when query contains GROUP BY (#8028) + - reload: bin: Handle hot-reloading is aborted case + - reload: add missing flb_cf_destroy (#8042) + - chunk_trace: abstract out the pipeline thread into its own API. + - chunk_trace: coordinate the stopping and starting of the pipeline thread. + - config_format: flb_cf_yaml: change log level in processing. Fixes #8130 + - config_format: yaml: throw an error when the array cannot be allocated. + - input_chunk: fixed a corner case when records are dropped + - input_chunk: adapted chunk filtering code to operate on a local buffer + - chunk_trace: add reporting for outputs. + - chunk_trace: use a separate thread for trace pipelines. + - utils: add code for getting the machineID on macOS. (#8096) + - utils: add code to get the machine-id on windows machines. (#7970) + + - Libs + - cmetrics: upgrade to v0.6.4 + + - Plugins + - [Fluentbit_Metrics (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/fluentbit_metrics/) + - Add missing release function on init + - [Process_Exporter_Metrics (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/process_exporter_metrics/) + - Implement process exporter metrics (#7943) + - [Dummy (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/dummy/) + - Use interval_sec/nsec instead of time_interval. + - Reducing tests maximum duration. + - Add unit tests for rate and time_interval. + - Rename time interval variables. + - Add time_interval setting. + - [Calyptia_Fleet (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/calyptia_fleet/) + - Get `__mkdir` to work on non-unix operating systems. (#7969) + - Free up filenames and data during collect callback. + - Free http client when retrieving fleet_id from fleet_name. + - Free fleet_url and fleet_id. + - Remove premature return which leads to a memory leak. + - Fix memory leaks when loading new configurations. + - Add missing release function on collect + - Add missing release function + - Check cfg parameter in is_old_fleet_config. + - Check return value of old_fleet_config_filename. + - Add old file name to list of valid configuration names. + - Remove info messsage using event_fd. + - Remove unused event_fd configuration property. + - [Thermal (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/thermal/) + - Support hwmon (#8016) + - [Node_Exporter_Metrics (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/node-exporter-metrics) + - Implement a more pluggable architecture for the collectors (#7944) + - Implement nvme metrics (#8049) + - Fix referencing wrong interval (#8019) + - Fix macos build (#8048) + - Implement processes metrics (#7880) + - Plug undefined value for netif name + - Support diskstats metrics on macos + - Add netdev metrics on macos + - Support meminfo metrics on macos + - Support uname metrics on macos + - Add loadavg metrics for macos + - Support cpu seconds collection on macos + - Make buildable with skeletons on macos + - [Splunk (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/splunk/) + - Ensure releasing tag_from_record after procressing it + - [Forward (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/forward/) + - Extract a function for appending logs + - Handle non log types of compressed events + - Fix checking return value of cmt_decode_msgpack_create(#8000) (#8015) + - [Docker (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/docker/) + - Implement a capability for cgroups v2 + - Add unit test and make configurable paths (mostly for testing) + - [Lua (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/lua/) + - Add index to arraylength call + - Provide additional script validator before running as cb_pre_run + - Plug a memory leak on loading for an invalid lua script + - [Sysinfo (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/sysinfo/) + - Add filter sysinfo plugin + - [Wasm (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/wasm/) + - Provide additional validator for checking existence of the file in wasm_path + - [Chronicle (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/chronicle/) + - Remove redundant code (#8004) + - [Syslog (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/syslog/) + - When using a message key, avoid prefixing raw messages + - [Loki (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/loki/) + - Add ability to add arbitrary http headers and adjust the uri path (#7979) + - Add new option `uri` to customize http uri endpoint + - [Stackdriver (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/stackdriver/) + - fix: don't overwrite ctx->client_email and ctx->private_key. (#8093) + - [Forward (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/forward/) + - Add compressed=gzip option for non log types of events + - [Cloudwatch_Logs (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/cloudwatch_logs/) + - Enhance api error response readability + - Remove sequence tokens from api calls (#7973) + +{{< contributor-list >}} +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v2.2/v2.2.1.md b/content/announcements/v2.2/v2.2.1.md new file mode 100644 index 0000000..794608f --- /dev/null +++ b/content/announcements/v2.2/v2.2.1.md @@ -0,0 +1,117 @@ +--- +title: 'v2.2.1' +description: 'We provide the means for the collection, organization and computerized retrieval of knowledgeand Lightweight Data Forwarder for Linux, BSD, macOS and Windows.' +url: "/announcements/v2.2.1/" +release_date: 2023-12-22 +publishdate: 2023-12-22 +ver: v2.2.1 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +> NOTE: This is a minor hotfix release that covers for corner cases for metrics handling. We keep same +release notes as part of v2 + +### Release Notes v2.2.1 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, +BSD and OSX. We are proud to announce the availability of **Fluent Bit v2.2.1**. + +#### Introduction + +Fluent Bit v2.1 is the start of the new stable series of the project. During the last months our primary focus has been around extending support for Metrics, Traces and improving performance, among many others. + +==> Changes between: v2.2.0 -> v2.2.1 + - Core + - wasm: Remove forcibly disabling AOT load on Apple Silicon macOS + - go: plugin_proxy: Handle FLBPluginInit(cb_init) error correctly and plug a SEGV case + - build: enable in_exec plugin on win32 (#7664) + - build: windows: Eagerly fail on missing openssl or flex + - test: internal: log: fix internal log test to use a clock instead of an interval. (#8148) + - engine: Handle zero size as no limit + - engine: Extract a function for calculating capacity of chunks + - engine: output: Add metrics for displaying the available capacity of chunks as percent + - task: remove guard around total_events + - filter: update total_records outside of ifdef + - config_format: yaml: test: Make runnable for yaml config on Windows + - config_format: yaml: Use \ instead of / for concatenating paths on Windows + - input_chunk: skip overlimit check in mem mode (#8199) + - input_chunk: handle filter_do edge case (#8229) + - input_chunk: add total_records test + - input_chunk: update records in the right place + + - Libs + - cmetrics: upgrade to v0.6.6 + - ctraces: upgrade to v0.4.0 + - cfl: upgrade to v0.3.0 + - mpack: upgrade to v1.1.1 + - librdkafka: upgrade to v2.3.0 + - sqlite: upgrade to v3.44.2 + - wasm: upgrade to v1.3.0 + - c-ares: upgrade from v1.19.1 to v1.24.0 + + - Plugins + - [Calyptia_Fleet (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/calyptia_fleet/) + - Fix double free when calyptia api returns with an empty response payload. + - Create configuration directory before the fleet header. + - Fix warnings in error handling code for get_calyptia_file. + - Use header with main configuration settings. (#8102) + - Support multiple files for configurations. (#8092) + - [Process_Exporter_Metrics (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/process_exporter_metrics/) + - Release str in error cases (#8163) + - [Winevtlog (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/winevtlog/) + - Pass correct value (#8179) + - [Kafka (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/kafka/) + - Append logs only for non zero length cases + - Rely on only buffer_max_size for preventing greedy consuming topics + - Implement restrictive polling by default + - Handle limit of buffer for fs chunks with buffer_chunk_size config parameter + - Suspend polling when reaching mem_limit + - Setup `fetch.max.bytes` properly + - Register pause/resume callbacks to handle back pressure correctly + - [Opentelemetry (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/opentelemetry/) + - Support new `tag_from_uri` property + - Fix memory leak + - [Node_Exporter_Metrics (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/node_exporter_metrics/) + - Add support for thermal_zone. (#7522) + - Suppress error log when nvme is not mounted in the node + - [Kubernetes_Events (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/kubernetes_events/) + - Set default timestamp_key to `$lasttimestamp` (record accessor pattern) + - Make timestamp from incoming record configurable + - [Stackdriver (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/stackdriver/) + - Add partialsuccess: true + - Test_log_entry_format will output log entries to stdout + - [Stdout (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/stdout/) + - Fix incomplete printing of traces + +{{< contributor-list >}} + +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +- [Türkalp Burak Kayrancıoğlu](https://github.com/bkayranci) +- [ryanohnemus](https://github.com/ryanohnemus) +- [Eduardo Silva](https://github.com/edsiper) +- [Takahiro Yamashita](https://github.com/nokute78) +- [Hiroshi Hatake](https://github.com/cosmo0920) +- [Phillip Whelan](https://github.com/pwhelan) +- [Srinivasan J](https://github.com/srini38) +- [Craig Ringer](https://github.com/ringerc) +- [Pat](https://github.com/patrick-stephens) +- [Braydon Kains](https://github.com/braydonk) +- [Alex Ellis](https://github.com/alexellis) +- [avilevy](https://github.com/avilevy) +- [Celalettin Calis](https://github.com/celalettin1286) + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v2.2/v2.2.2.md b/content/announcements/v2.2/v2.2.2.md new file mode 100644 index 0000000..2624184 --- /dev/null +++ b/content/announcements/v2.2/v2.2.2.md @@ -0,0 +1,84 @@ +--- +title: 'v2.2.2' +description: 'We provide the means for the collection, organization and computerized retrieval of knowledgeand Lightweight Data Forwarder for Linux, BSD, macOS and Windows.' +url: "/announcements/v2.2.2/" +release_date: 2023-12-22 +publishdate: 2023-12-22 +ver: v2.2.2 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +> NOTE: This is a minor hotfix release that covers for corner cases for metrics handling. We keep same +release notes as part of v2 + +### Release Notes v2.2.2 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, +BSD and OSX. We are proud to announce the availability of **Fluent Bit v2.2.2**. + +#### Introduction + +Fluent Bit v2.2 is the start of the new stable series of the project. During the last months our primary focus has been around extending support for Metrics, Traces and improving performance, among many others. + + - Core + - motd: new interface for banners (#8374) + - storage: recreate cio_stream if storage type is different(#8259) (#8290) + - build: Remove duplicated line FLB_IN_EVENT_TEST (#8268) + - config: add new `net.dns.prefer_ipv6`` bool option to prioritize IPv6 over IPv4 + - config: add default logfmt parser (#8325) + - tests: runtime: filter_nest: add test case for add_prefix (#8372) + + + - Libs + - lib: monkey: upgrade to v1.7.3 (#8363) + + - Plugins + - [Dummy (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/dummy/) + - Add `flush_on_startup` config. (#8222) + - [Http (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/http/) + - Fix payload parsing and error response + - [Storage_Backlog (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/storage_backlog/) + - If a chunk tag cannot be read, delete it + - [Calyptia_Fleet (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/calyptia_fleet/) + - Release upstream conn in collect. (#8366) + - [Node_Exporter_Metrics (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/node_exporter_metrics/) + - Collect metrics at startup (#8368) (#8370) + - [Kubernetes_Events (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/kubernetes_events/) + - Consolidate record timestamp logic (#8323) + - [Stackdriver (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/stackdriver/) + - Add `project_id_key` override (#8209) + - [Azure_Blob (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/azure_blob/) + - Add support for Azure blob SAS authentication (#8243) + +{{< contributor-list >}} + +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +- [Türkalp Burak Kayrancıoğlu](https://github.com/bkayranci) +- [ryanohnemus](https://github.com/ryanohnemus) +- [Eduardo Silva](https://github.com/edsiper) +- [Takahiro Yamashita](https://github.com/nokute78) +- [Hiroshi Hatake](https://github.com/cosmo0920) +- [Phillip Whelan](https://github.com/pwhelan) +- [Srinivasan J](https://github.com/srini38) +- [Craig Ringer](https://github.com/ringerc) +- [Pat](https://github.com/patrick-stephens) +- [Braydon Kains](https://github.com/braydonk) +- [Alex Ellis](https://github.com/alexellis) +- [avilevy](https://github.com/avilevy) +- [Celalettin Calis](https://github.com/celalettin1286) + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v2.2/v2.2.3.md b/content/announcements/v2.2/v2.2.3.md new file mode 100644 index 0000000..7e4494d --- /dev/null +++ b/content/announcements/v2.2/v2.2.3.md @@ -0,0 +1,84 @@ +--- +title: 'v2.2.3' +description: 'We provide the means for the collection, organization and computerized retrieval of knowledgeand Lightweight Data Forwarder for Linux, BSD, macOS and Windows.' +url: "/announcements/v2.2.3/" +release_date: 2024-05-21 +publishdate: 2024-05-21 +ver: v2.2.3 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +> NOTE: This is a minor hotfix release that covers for corner cases for metrics handling. We keep same +release notes as part of v2 + +### Release Notes v2.2.3 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, +BSD and OSX. We are proud to announce the availability of **Fluent Bit v2.2.3**. + +#### Introduction + +Fluent Bit v2.2 is currently is maintenance only with security fixes. This specific version is a one-off due to a security issue found in the HTTP endpoint (CVE-2024-4323). It also contains other fixes that were not released before v3.0 series: + +
+ + - __Core__ + + - http_server: api: v1: backport fix for CVE-2024-4323 + - engine_dispatch: remove chunks from memory if the task fails to be created + - log_cache: avoid double free by setting the log_cache pointer to NULL. + - task: expose map_get_task_id to add a check in engine_dispatch + - sched: use sched->evl when freeing the scheduler timer. + - sched: free the base timer used by the scheduler. + - network: Fix int-conversion error by using POSIX version of strerror_r + - utils: Remove a needless flb_free + - output_thread: destroy thread events channel on shutdown. + - engine: destroy self events channels on shutdown. + - reload: Mark hot reloading for the old context + - api: v2: reload: Distinguish the status of hot reloading + - input_chunk: added proper accounting of chunks that are rolled over + - config: release data pipe using flb_pipe_destroy. + - log: use flb_pipe_destroy to close worker log pipe on error. + - log: free log worker pipe when appropriate. + +
+ +- __Plugins__ + - Calyptia_Fleet (Input) + - Fix memory leak when setting conf_path_file. + - Code style cleanup + - Improve connection context handling + - Remove double free of cfl_array_destroy() + - [Kubernetes_Events (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/kubernetes_events/) + - Refactor time check to use struct flb_time. (#8845) + - [HTTP (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/http/) + - Adds support for content-encoding: gzip (#7667) + - [Forward (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/forward/) + - Fix memory leak during connection loss + +{{< contributor-list >}} + +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +- [Eduardo Silva](https://github.com/edsiper) +- [Phillip Whelan](https://github.com/pwhelan) +- seblaz +- Edmund Rhudy +- [Hiroshi Hatake](https://github.com/cosmo0920) +- [Leonardo Alminana](https://github.com/leonardo-albertovich) +- [Christian Norbert Menges](https://github.com/Garfield96) + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v3.0/_index.md b/content/announcements/v3.0/_index.md new file mode 100644 index 0000000..3c85f8f --- /dev/null +++ b/content/announcements/v3.0/_index.md @@ -0,0 +1,15 @@ +--- +title: 'Fluent Bit v3.0 Series (EOL Sep 11, 2024)' +description: "[Fluent Bit v1.9](https://github.com/fluent/fluent-bit/tree/master) is the new **stable branch** for production usage. Based on bug reports or +specific minor feature requests, we do quick releases upon demand. Below is a list of the notes for each version." +url: "/announcements/v3.0/" +herobg: "/images/hero@2x.jpg" +latestVer: true +releaseNotes: + heading: "Release Notes v3.0.0" + version: "v3.0.0" + text: "Fluent Bit is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD and OSX. We are proud to announce the availability of Fluent Bit v1.9.9.
+ For people upgrading from previous versions you must read the Upgrading Notes section of our documentation: + https://docs.fluentbit.io/manual/installation/upgrade_notes" +--- + diff --git a/content/announcements/v3.0/v3.0.0.md b/content/announcements/v3.0/v3.0.0.md new file mode 100644 index 0000000..e214f14 --- /dev/null +++ b/content/announcements/v3.0/v3.0.0.md @@ -0,0 +1,184 @@ +--- +title: 'v3.0.0' +description: 'Next generation Telemetry Agent for Logs, Metrics and Traces. ' +url: "/announcements/v3.0.0/" +release_date: 2024-03-20 +publishdate: 2024-03-03 +ver: v3.0.0 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +## Release Notes v3.0.0 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD and OSX. We are proud to announce the availability of **Fluent Bit v3.0**. + +For people upgrading from previous versions you **must read** the Upgrading Notes section of our documentation: + +[https://docs.fluentbit.io/manual/installation/upgrade_notes](https://docs.fluentbit.io/manual/installation/upgrade_notes) + +### Introduction + +Fluent Bit, a CNCF graduated project under the umbrella of Fluentd, announces the availability of v3.0. + +In every release, there are many improvements and fixes, on this notes we will refer to the major changes that will make your infrastructure happier ;) + +### HTTP/2 Support + +In today’s world, most modern applications prefer using HTTP/2 for communication with Web Services instead of HTTP/1.1, while compatibility exists, in Fluent Bit we want to deliver support for industry standards like this. Fluent Bit is not only used as a sender but also as an aggregator/collector to receive telemetry data for different types of signals, such as Logs, Metrics and Traces: + + +With the new addition of HTTP/2 in input/source/receivers that operate on top of HTTP protocol, such as OpenSearch, Splunk, and OpenTelemetry, they now can transparently operate in either HTTP/1.1 or HTTP/2 mode without any changes, but now with better performance due to the HTTP/2 concurrency main feature. +The new transport is enabled by default and is up to the client to use or prefer other mechanism, however it can be disabled with the option `http2 off` in your input plugin configuration section. + +### Processors + +Months ago, we introduced the concept of Processors, which, similar to Filters functionality, allows to enrich or transform the telemetry data that is being received, but they have the following differences: +

+ +- They are not routable. Instead, they are attached directly to an input plugin and run one after the other. +- If the plugin is running in threaded mode, meaning that it runs in a separate thread, the processor will run there too, significantly reducing the contention that Filters used to generate in the main event loop and thread, as a side effect: better performance. + +In this new version we are adding three new processors: + +#### Content Modifier + +Similar to the functionality exposed by filters, this new processor aims to unify the logic to manipulate content in Logs and Traces; it also can explicitly operate only on metadata/attributes or the content of the messages itself. + +The following example creates a simple pipeline that generates a dummy message with 2 keys, then the content\_modifier processor is loaded to work over the logs stream where it will insert a new key called `color` with value `blue`. Once the key has been inserted, the next action is to extract and decompose the value of the key `http.url` in multiple keys by using a regular expression pattern: + +```yaml +pipeline: + inputs: + - name: dummy + dummy: '{"key1": "123.4", "http.url": "https://www.google.com/search?q=example"}' + rate: 1 + + processors: + logs: + - name: content_modifier + action: insert + key: "color" + value: "blue" + + - name: content_modifier + action: extract + key: "http.url" + pattern: ^(?https?):\/\/(?[^\/\?]+)(?\/[^?]*)?(?:\?(?.*))? + outputs: + - name: stdout + match: '*' + format: json_lines +``` + +

+The output of that processing will be: + +```json +{ + "date": 1711036844.96304, + "key1": "123.4", + "http.url": "https://www.google.com/search?q=example", + "color": "blue", + "http_protocol": "https", + "http_domain": "www.google.com", + "http_path": "/search", + "http_query_params": "q=example" +} +``` + +

+ +#### Metrics Selector + +When collecting metrics from the host, receiving or scrapping them from a remote endpoint, there are many cases where you are not interested in everything that has been collected, metrics selector allows you to specify what metrics to include and what to exclude by using patterns on it name. + +In the following example data, generated in a macOS system where Fluent Bit is gathering metrics from the host by using the the `node_exporter_metrics` input plugin, we have attached a simple processor that will only include metrics that it name starts with `node_disk_w` (we are using a regular expression to perform further match): + +```yaml +pipeline: + inputs: + - name: node_exporter_metrics + processors: + metrics: + - name: metrics_selector + action: include + metric_name: /^node_disk_w.*/ + outputs: + - name: stdout + match: "*" + format: json_lines +``` + +
+Instead of getting the hundreds of metrics generated, the following will only available in the pipeline for processing and delivery: + +
+ +```txt +2024-03-21T21:45:39.028335270Z node_disk_writes_completed_total{device="disk0"} = 21667544 +2024-03-21T21:45:39.028342061Z node_disk_written_sectors_total{device="disk0"} = 5289.927734375 +2024-03-21T21:45:39.028362895Z node_disk_write_time_seconds_total{device="disk0"} = 1728.3985221549999 +2024-03-21T21:45:39.028352270Z node_disk_written_bytes_total{device="disk0"} = 404240666624 +2024-03-21T21:45:39.028373228Z node_disk_write_errors_total{device="disk0"} = 0 +2024-03-21T21:45:39.028383186Z node_disk_write_retries_total{device="disk0"} = 0 +``` + +
+ +#### SQL + +We don’t believe everybody needs to learn every single processor or filter provided by Fluent Bit to achieve data selection however many of you have some knowledge of SQL (Structured Query Language).The new SQL processor, allows you to define queries by using SQL expressions to select parts of your Logs and Traces which can also be used with conditionals. Here is a simple example of it: + + +```yaml +pipeline: + inputs: + - name: dummy + dummy: '{"key1": "123.4", "http.url": "https://fluentbit.io/search?q=docs"}' + + processors: + logs: + - name: content_modifier + action: extract + key: "http.url" + pattern: ^(?https?):\/\/(?[^\/\?]+)(?\/[^?]*)?(?:\?(?.*))? + + - name: sql + query: "SELECT new_key AS key, http_domain FROM STREAM;" + + outputs: + - name : stdout + match: '*' + format: json_lines +``` + +
+ +The dummy plugin, will generate a sample message with two keys `key1` and `http.url` , then the new processor content\_modifier will extract the content of the key `http.url` as a series of key/value pairs by using the regular expression defined in `pattern`, finally the SQL processor will only select the new key ingested with a new name and only the key `http_domain`as part of the result: + +
+ +```json +{ + "date": 1711059261.630668, + "http_domain": "fluentbit.io" +} +``` + +{{< contributor-list >}} +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v3.0/v3.0.1.md b/content/announcements/v3.0/v3.0.1.md new file mode 100644 index 0000000..c55e4c8 --- /dev/null +++ b/content/announcements/v3.0/v3.0.1.md @@ -0,0 +1,59 @@ +--- +title: 'v3.0.1' +description: 'Next generation Telemetry Agent for Logs, Metrics and Traces. ' +url: "/announcements/v3.0.1/" +release_date: 2024-03-29 +publishdate: 2024-03-29 +ver: v3.0.1 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +## Release Notes v3.0.1 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD, Windows and macOS. We are proud to announce the availability of **Fluent Bit v3.0.1**. + +For people upgrading from previous versions you **must read** the Upgrading Notes section of our documentation: + +[https://docs.fluentbit.io/manual/installation/upgrade_notes](https://docs.fluentbit.io/manual/installation/upgrade_notes) + +### What's new ? + +In this maintenance release, we are including a couple of fixes which are listed below: + + - Core + - http_server: added parser reset code to prevent use after free + + - Libraries: + - CMetrics: upgrade to v0.7.1 + + - Plugins + - [Tail (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/tail/) + - Add support for CRLF when `skip_empty_line` is `on`. + - [SQL (Processor)](https://docs.fluentbit.io/manual/pipeline/processors/sql) + - add missing handling of conditionals for unsigned values + - parser: add missing tokens for conditional operations + +{{< contributor-list >}} + +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + + +- [Eduardo Silva](https://github.com/edsiper) +- [Leonardo Alminana](https://github.com/leonardo-albertovich) +- [Hiroshi Hatake](https://github.com/cosmo0920) +- [Takahiro Yamashita](https://github.com/nokute78) + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v3.0/v3.0.2.md b/content/announcements/v3.0/v3.0.2.md new file mode 100644 index 0000000..8a360a7 --- /dev/null +++ b/content/announcements/v3.0/v3.0.2.md @@ -0,0 +1,107 @@ +--- +title: 'v3.0.2' +description: 'Next generation Telemetry Agent for Logs, Metrics and Traces. ' +url: "/announcements/v3.0.2/" +release_date: 2024-04-15 +publishdate: 2024-04-15 +ver: v3.0.2 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +## Release Notes v3.0.2 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD, Windows and macOS. We are proud to announce the availability of **Fluent Bit v3.0.2**. + +For people upgrading from previous versions you **must read** the Upgrading Notes section of our documentation: + +[https://docs.fluentbit.io/manual/installation/upgrade_notes](https://docs.fluentbit.io/manual/installation/upgrade_notes) + +### What's new ? + +In this maintenance release, we are including a couple of fixes which are listed below: + +
+ + - Core + - input: Add missing input resume message This commit will add a resume message, when a paused input plugin is resumed. + - engine_dispatch: remove chunks from memory if the task fails to be created + - task: on retry, only put chunk down if there are no users (fix #8691) + - input_metric: fixed a double free + - input_chunk: handle some chunk size debug messages as trace messages + - config: fix compilation with `FLB_HAVE_STATIC_CONF` (#7536) + +
+ + - Plugins + - [HTTP (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/http/) + - Fix buffer addresses on reallocation when processing headers (#8643) + - [Winevtlog (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/winevtlog/) + - Make configurable for the size of collecting threshold per a cycle (#8677) + - [Opentelemetry (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/opentelemetry/) + - Fixed the URIs used to detect gRPC requests + - [Tail (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/tail/) + - Introducing the `compare_filename` option to db_file_exists (#8025)(2/2) + - Delete unmanaged inodes from database file during startup (#8025) (1/2) + - [Forward (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/forward/) + - Handle multiply concatenated gzip payloads (#8665) + - [Emitter (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/emitter/) + - Pause source input plugins on in_emitter pause. This will pause all known inputs (sending to multiline) to not loose any in-flight records. in_emitter will keep track of all sending input plugins and actively pause/resume them in case in_emitter is paused/resumed. + - Fix to prevent single record chunks and do pause on `mem_buf_limit` + - [Rewrite_Tag (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/rewrite_tag/) + - Pause source input plugins on filter pause this commit will pause the inputs (sending to rewrite_tag) to not loose any in-flight records. + - [Multiline (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/multiline/) + - Pause source input plugins on filter pause this commit will pause the inputs (sending to multiline) to not loose any in-flight records. + - [Kubernetes (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/kubernetes/) + - Add new configuration property `namespace_metadata_only` (#8695) + - [Opentelemetry (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/opentelemetry/) + - Add support to set OpenTelemetry fields from the event body based on keys (#8644) + - [Websocket (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/websocket/) + - Add ability to provide additional headers in the HTTP request. + - [Loki (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/loki/) + - Change loglevel for non-existing label in record accessor. + - [Oracle_Log_Analytics (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/oracle_log_analytics/) + - Fix 400 bad request + +
+ + - Libraries: + - sqlite: upgrade to v3.45.2 + - cmetrics: upgrade to v0.7.3 + - chunkio: upgrade to v1.5.2 + - monkey: upgrade to v1.7.4 + - cmetrics: upgrade to v0.7.2 + - ctraces: upgrade to v0.4.1 + - cfl: upgrade to v0.4.1 + +{{< contributor-list >}} + +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +- [Richard Treu](https://github.com/drbugfinder-work) +- [Eduardo Silva](https://github.com/edsiper) +- [Phillip Whelan](https://github.com/pwhelan) +- [Leonardo Alminana](https://github.com/leonardo-albertovich) +- [Hiroshi Hatake](https://github.com/cosmo0920) +- maflcko +- [Ryan Ohnemus](https://github.com/ryanohnemus) +- [bglimepoint](https://github.com/bglimepoint) +- [cb645j](https://github.com/cb645j) +- hc +- jinyong.choi +- [Markus Thömmes](https://github.com/markusthoemmes) +- [seblaz](https://github.com/seblaz) + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v3.0/v3.0.3.md b/content/announcements/v3.0/v3.0.3.md new file mode 100644 index 0000000..e3eb5e1 --- /dev/null +++ b/content/announcements/v3.0/v3.0.3.md @@ -0,0 +1,80 @@ +--- +title: 'v3.0.3' +description: 'Next generation Telemetry Agent for Logs, Metrics and Traces. ' +url: "/announcements/v3.0.3/" +release_date: 2024-04-27 +publishdate: 2024-04-27 +ver: v3.0.3 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +## Release Notes v3.0.3 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD, Windows and macOS. We are proud to announce the availability of **Fluent Bit v3.0.3**. + +For people upgrading from previous versions you **must read** the Upgrading Notes section of our documentation: + +[https://docs.fluentbit.io/manual/installation/upgrade_notes](https://docs.fluentbit.io/manual/installation/upgrade_notes) + +### What's new ? + +In a few words, this release add fixes for OpenTelemetry handling for Traces and Metrics, provides better performance for processors and add enhance some connectors (listed below). + +Specific fixes per component: + +
+ + - Core + - mp: use referenced values when composing CFL object + - processor_content_modifier: use copied values for referenced values when converting + - processor_content_modifier: use new CFL API to reference strings and bytes + +
+ + - Plugins + - [HTTP (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/http/) + - Fix URL decoder by unifying normal interface with the one used in http/1.x + - [Splunk (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/splunk/) + - Store ingested HEC token as metadata + - [HTTP (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/http/) + - Use `x-ndjson` content type for `json_lines` + - [Chronicle (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/chronicle/) + - Allow to handle 1MB or larger chunks + - [Splunk (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/splunk/) + - Migrate to use record_accesor pattern for extracting token + - Handle HEC token from metadata + +
+ + - Libraries: + - [CMetrics](https://github.com/fluent/cmetrics): upgrade to v0.9.0 + - [CTraces](https://github.com/fluent/ctraces): upgrade to v0.5.1 + - [CFL](https://github.com/fluent/cfl): upgrade to v0.5.0 + +{{< contributor-list >}} + +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + + +- [Aidan Leuck](https://github.com/aidanleuck) +- [Manuel Rüger](https://github.com/mrueg) +- [Eduardo Silva](https://github.com/edsiper) +- [Leonardo Alminana](https://github.com/leonardo-albertovich) +- [lecaros](https://github.com/lecaros) +- [Pat](https://github.com/patrick-stephens) +- [Hiroshi Hatake](https://github.com/cosmo0920) + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v3.0/v3.0.4.md b/content/announcements/v3.0/v3.0.4.md new file mode 100644 index 0000000..f492a8b --- /dev/null +++ b/content/announcements/v3.0/v3.0.4.md @@ -0,0 +1,85 @@ +--- +title: 'v3.0.4' +description: 'Next generation Telemetry Agent for Logs, Metrics and Traces. ' +url: "/announcements/v3.0.4/" +release_date: 2024-05-20 +publishdate: 2024-05-20 +ver: v3.0.4 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +## Release Notes v3.0.4 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD, Windows and macOS. We are proud to announce the availability of **Fluent Bit v3.0.4**. + +
+ +### Security updates and other improvements + +__Important__: This release addresses a critical security vulnerability in the interface handling trace requests. The identified issue involves memory corruption, which could potentially be exploited to cause denial of service (DoS) conditions. Note that this issue is related to the internal tracing interface and __NOT__ Traces telemetry data handling. It is highly recommended to update to this version immediately to ensure the stability and security of your system. + + +
+ + - __Core__ + - http_server: api: v1: trace: fix memory corruption ([#9311b43a](https://github.com/fluent/fluent-bit/commit/9311b43a258352797af40749ab31a63c32acfd04)) + - kernel: fix incorrect assignment of Linux Kernel version between major and minor (#8797) + - http_server: fix keep-alive for HTTP/1.1 requests + - config: fix error message grammar (#8818) + +
+ + - __Pipeline Plugins__ + - [AWS (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/aws-metadata/) + - add support for `metadata_group` + - [Premetheus_Remote_Write (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/premetheus_remote_write/) + - New (EXPERIMENTAL) input plugin to receive Prometheus Remote Write prototol (#8725) + - [Opentelemetry (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/opentelemetry/) + - Enable keep-alive + - Add a function to store metadata + - Support to add resource of log + - [Elasticsearch (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/elasticsearch/) + - Enable keep-alive on HTTP/1.1 + - [HTTP (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/http/) + - Enable keep-alive on HTTP/1.1 + - [Splunk (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/splunk/) + - Enable keep-alive on HTTP/1.1 + - [Datadog (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/datadog/) + - Update the descriptions of special field options #8751 + - [Kafka (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/kafka/) + - Increase max kafka dynamic topic length to 249 characters + - [Splunk (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/splunk/) + - Fix record accessor key for `hec_token` (#8793) + +
+ +{{< contributor-list >}} + +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +- [Phillip Whelan](https://github.com/pwhelan) +- [Eduardo Silva](https://github.com/edsiper) +- [Hiroshi Hatake](https://github.com/cosmo0920) +- [Takahiro Yamashita](https://github.com/nokute78) +- [Braydon Kains](https://github.com/braydonk) +- [Jesse Szwedko](https://github.com/jszwedko) +- [Maneesh Singh](https://github.com/mannbiher) +- [xiaobaowen](https://github.com/thenamehasbeentake) +- Michael Meunier +- [Mateusz Warzyński](https://github.com/mwarzynski) +- [Andrew Titmuss](https://github.com/iandrewt) + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v3.0/v3.0.5.md b/content/announcements/v3.0/v3.0.5.md new file mode 100644 index 0000000..81db163 --- /dev/null +++ b/content/announcements/v3.0/v3.0.5.md @@ -0,0 +1,57 @@ +--- +title: 'v3.0.5' +description: 'Next generation Telemetry Agent for Logs, Metrics and Traces. ' +url: "/announcements/v3.0.5/" +release_date: 2024-05-24 +publishdate: 2024-05-24 +ver: v3.0.5 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +## Release Notes v3.0.5 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD, Windows and macOS. We are proud to announce the availability of **Fluent Bit v3.0.5**. + +
+ +### What's new ? + +
+ + - __Core__ + - engine: log worker context thread once it has been stopped + - output: Fix handling of metrics in output processor + +
+ + - __Pipeline Plugins__ + - [Winevtlog (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/winevtlog/) + - Change total_size_threshold to size_t (#8853) + - [Splunk (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/splunk/) + - Reduce noise and fix `hec_token` handling (fix #8859) +
+ +{{< contributor-list >}} + +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +- [Braydon Kains](https://github.com/braydonk) +- [Thiago Padilha](https://github.com/tarruda) +- [Eduardo Silva](https://github.com/edsiper) +- Hiroyuki Hasebe +- [Pat](https://github.com/patrick-stephens) + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v3.0/v3.0.6.md b/content/announcements/v3.0/v3.0.6.md new file mode 100644 index 0000000..7fe9bee --- /dev/null +++ b/content/announcements/v3.0/v3.0.6.md @@ -0,0 +1,58 @@ +--- +title: 'v3.0.6' +description: 'Next generation Telemetry Agent for Logs, Metrics and Traces. ' +url: "/announcements/v3.0.6/" +release_date: 2024-05-26 +publishdate: 2024-05-26 +ver: v3.0.6 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +## Release Notes v3.0.6 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD, Windows and macOS. We are proud to announce the availability of **Fluent Bit v3.0.6**. + +
+ +### What's new ? + +
+ + - __Core__ + - core: fix crash when running dry test mode (#8872) + +
+ + - __Pipeline Plugins__ + + - [Stackdriver (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/stackdriver/) + - support writing to textPayload field of Cloud Logging LogEntry. (#8850) + + - [Splunk (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/splunk/) + - Fix metadata `hec_token` bug that overrides default splunk_token behavior +
+ +{{< contributor-list >}} + +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +- [Holger Freyther](https://github.com/zecke) +- [Markus Bergholz](https://github.com/markuman) +- [Eduardo Silva](https://github.com/edsiper) +- [shuaichen](https://github.com/shuaich) + + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v3.0/v3.0.7.md b/content/announcements/v3.0/v3.0.7.md new file mode 100644 index 0000000..b8d3fdc --- /dev/null +++ b/content/announcements/v3.0/v3.0.7.md @@ -0,0 +1,91 @@ +--- +title: 'v3.0.7' +description: 'Next generation Telemetry Agent for Logs, Metrics and Traces. ' +url: "/announcements/v3.0.7/" +release_date: 2024-06-11 +publishdate: 2024-06-11 +ver: v3.0.7 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +## Release Notes v3.0.7 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD, Windows and macOS. We are proud to announce the availability of **Fluent Bit v3.0.7**. + +
+ +### What's new ? + +
+ + - __Core__ + - tls: fix virtual host verification when `tls.verify` is enabled + - build: fix cmake build with ninja (#8835) + - http_server: api: v1: traces: fix finding inputs by alias. + - dockerfiles: + - windows: Update to use vcpkg instead of chocolatey + - dockerfile: update to bookworm base image (#8916) + +
+ +- __Libraries__ + - CMetrics: upgrade to v0.9.1 + +
+ + - __Pipeline Plugins__ + + - [Systemd (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/systemd/) + - Move one record back after skipping to the end + - [HTTP (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/http/) + - Add tests for 400 responses to bad requests and buffer write errors. + - Return a 400 error when unable to receive a message. + - [Prometheus_Scrape (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/prometheus_scrape/) + - Add user-agent http header (#8880) + - [Winevtlog (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/winevtlog/) + - Fix threshold condition to avoid unnecessary warning + - [Splunk (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/splunk/) + - Add switch for storing Splunk token in metadata or records + - Store HEC token on enabled HTTP2 and raw endpoints + - [Kubernetes_Events (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/kubernetes_events/) + - Type cast explicitly + - Refactor time check to use struct flb_time. + - [Metrics Selector (Processor)](https://docs.fluentbit.io/manual/pipeline/processors/metrics-selector) + - Implement delete by label key-value pair feature + - [Cloudwatch_Logs (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/cloudwatch_logs/) + - Support `log_group_class` option + - [Calyptia (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/calyptia/) + - Move debug payload to debug level (less noise). + + +{{< contributor-list >}} + +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +- [Hiroshi Hatake](https://github.com/cosmo0920) +- Thomas Devoogdt +- [Phillip Whelan](https://github.com/pwhelan) +- [Eduardo Silva](https://github.com/edsiper) +- [Pat Stephens](https://github.com/patrick-stephens) +- Meet +- Ardavast Dayleryan +- javex +- [Phil Whelan](https://github.com/pwhelan) +- [lecaros](https://github.com/lecaros) +- [Wesley Pettit](https://github.com/PettitWesley) +- Enderson Maia + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v3.1/_index.md b/content/announcements/v3.1/_index.md new file mode 100644 index 0000000..9173bd0 --- /dev/null +++ b/content/announcements/v3.1/_index.md @@ -0,0 +1,15 @@ +--- +title: 'Fluent Bit v3.1 Series' +description: "[Fluent Bit v1.9](https://github.com/fluent/fluent-bit/tree/master) is the new **stable branch** for production usage. Based on bug reports or +specific minor feature requests, we do quick releases upon demand. Below is a list of the notes for each version." +url: "/announcements/v3.1/" +herobg: "/images/hero@2x.jpg" +latestVer: true +releaseNotes: + heading: "Release Notes v3.1.0" + version: "v3.1.0" + text: "Fluent Bit is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD and OSX. We are proud to announce the availability of Fluent Bit v1.9.9.
+ For people upgrading from previous versions you must read the Upgrading Notes section of our documentation: + https://docs.fluentbit.io/manual/installation/upgrade_notes" +--- + diff --git a/content/announcements/v3.1/v3.1.0.md b/content/announcements/v3.1/v3.1.0.md new file mode 100644 index 0000000..ec0b135 --- /dev/null +++ b/content/announcements/v3.1/v3.1.0.md @@ -0,0 +1,256 @@ +--- +title: 'v3.1.0' +description: 'Next generation Telemetry Agent for Logs, Metrics and Traces. ' +url: "/announcements/v3.1.0/" +release_date: 2024-07-08 +publishdate: 2024-07-08 +ver: v3.1.0 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +## Release Notes v3.1.0 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Telemetry Agent for Linux, BSD, MacOS and Windows. We are proud to announce the availability of **Fluent Bit v3.1**. + +``` + Fluent Bit v3.1.0 + * Copyright (C) 2015-2024 The Fluent Bit Authors + * Fluent Bit is a CNCF sub-project under the umbrella of Fluentd + * https://fluentbit.io + ______ _ _ ______ _ _ _____ __ + | ___| | | | | ___ (_) | |____ |/ | + | |_ | |_ _ ___ _ __ | |_ | |_/ /_| |_ __ __ / /`| | + | _| | | | | |/ _ \ '_ \| __| | ___ \ | __| \ \ / / \ \ | | + | | | | |_| | __/ | | | |_ | |_/ / | |_ \ V /.___/ /_| |_ + \_| |_|\__,_|\___|_| |_|\__| \____/|_|\__| \_/ \____(_)___/ + + [2024/07/09 12:55:40] [ info] [fluent bit] version=3.1.0, commit=8ed2e80a9e, pid=51082 + [2024/07/09 12:55:40] [ info] [storage] ver=1.5.2, type=memory, sync=normal, checksum=off, max_chunks_up=128 + [2024/07/09 12:55:40] [ info] [cmetrics] version=0.9.1 + [2024/07/09 12:55:40] [ info] [ctraces ] version=0.5.1 + [2024/07/09 12:55:40] [ info] [sp] stream processor started +``` + +For people upgrading from previous versions you **must read** the Upgrading Notes section of our documentation: + +[https://docs.fluentbit.io/manual/installation/upgrade_notes](https://docs.fluentbit.io/manual/installation/upgrade_notes) + +### Introduction + +Fluent Bit, a CNCF graduated project under the umbrella of Fluentd, announces the availability of v3.1. + +In every release, there are many improvements and fixes, on this notes we will refer to the major changes that will make your infrastructure happier ;) + +Below a list of the highlights of this release: + +### LuaJIT Upgrade (scripting) + +One of the powerful features of Fluent Bit, is the ability to provide the user an interface to control how to modify their telemetry data by using a very simple, but powerful, scripting language: [Lua](https://docs.fluentbit.io/manual/pipeline/filters/lua). Our Lua interface is powered by [LuaJIT](https://github.com/LuaJIT/LuaJIT) library, this has not been upgraded in the last two years, primary because it has been very stable for a long time. + +In this release, we are upgrading to the latest of LuaJIT which brings overall architecture improvements and extended support for Windows on ARM. + +If you are curious to learn how to use Lua, just copy the following configuration file and see how a simple dumy message can be modified inline in the config with some embedded Lua code: + +```yaml +service: + flush: 1 + log_level: info + +pipeline: + inputs: + - name: dummy + dummy: '{"project": "Fluent Bit", "version": "v3.1"}' + + processors: + logs: + - name: lua + call: cb_replace + code: | + function cb_replace(tag, timestamp, record) + record['tag'] = tag + record['new_key'] = 'new value' + + -- Record modified, so 'code' return value (first parameter) is 1 + return 1, timestamp, record + end + outputs: + - name: stdout + match: '*' +``` + +
+ +> You can also have your Lua code in a separate file, just read more about the [Lua](https://docs.fluentbit.io/manual/pipeline/filters/lua) in our documentation. + +
+ +### Add support for local/system timezone on parsing + +When parsing log record dates, there are cases when the date string don't contain any relevant information about the timezone it was generated, however is ideal to assume those records were generated in the local system so it parsing should assume the local timezone set in the operating system. + +This change introduces a new configuration flag for the parsers section called `time_system_timezone` (default: off) that allows to use system timezone (`mktime(3)`) for date representation. + +The following configuration snippet provides an example of it (note this is in classic config mode): + +```python +[PARSER] + Name no_timezone + Format json + Time_Key time + Time_Format %Y-%m-%d:%H:%M:%S + Time_System_Timezone true +``` + +

+ +### Networking + +In this release we did some small changes to our networking stack. + +#### Configurable KeepAlive for Downstream based input plugins + +Downstream is an internal interface to allow to recieve TCP connections. In this release we added a new configuration flag to control manually the keepalive functionality (this allows to have persistent TCP connection), the new option available is `net.keepalive` (default: on). + +Users of this interface are the __input__ plugins such as [HTTP](https://docs.fluentbit.io/manual/pipeline/inputs/http), [OpenTelemetry](https://docs.fluentbit.io/manual/pipeline/inputs/opentelemetry), [Elasticsearch](https://docs.fluentbit.io/manual/pipeline/inputs/elasticsearch), [Splunk](https://docs.fluentbit.io/manual/pipeline/inputs/splunk), etc. + +
+ +#### TLS Hostname verification + +When performing TLS connections, the default verification is for the certificates. In this new release, we added a new check to verify the hostname associated to the certificates through the new option `tls.verify_hostname` (default: off). + +Note that some plugins that uses TLS might have a different default for `tls.verify_hostname`. + +
+ +#### OpenSSL: reduce log noise + +A small change to reduce the noise in the TLS logger make sure you don't get false positives when doing secured sessions backed by OpenSSL. + +
+ +### Core: Log Groups + +When input plugins collects or generate log records, usually those are serialized one after each other, the only shared information that existed for records grouped inside a Chunk is the `Tag`. This simple metadata field is used for routing through the pipeline, however there are cases where we need to set metadata for a group of records, this is where we have implemented the concept of Groups. + +A Group is a special record definition that aims to share specific information for all records under that group. + +``` + :---+--------------+---------------------------------------------: + | 1 | Group | [ [Group Start, {METADATA}], {GROUP_INFO} ] | + :---+--------------+---------------------------------------------: + | 2 | Log Record 1 | [ [TIMESTAMP, {METADATA}], {RECORD} ] | + :---+--------------+---------------------------------------------: + | 3 | Log Record 2 | [ [TIMESTAMP, {METADATA}], {RECORD} ] | + :---+--------------+---------------------------------------------: + | 4 | Log Record N | [ [TIMESTAMP, {METADATA}], {RECORD} ] | + :---+--------------+---------------------------------------------: + | 5 | Group | [ [Group End, {}], {} ] | + '---'--------------'---------------------------------------------' +``` + +As of now, the only components of plugins generating or consuming those groups are [OpenTelemetry input](https://docs.fluentbit.io/manual/pipeline/inputs/opentelemetry) and [OpenTelemetry output](https://docs.fluentbit.io/manual/pipeline/outputs/opentelemetry). This aims to address some issues we had handling OTel Logs metadata. + +
+ +### Processors + +Processors are the new way to do data processing or filtering. In this release we are announcing the following changes for end users interested in OpenTelemetry log processing: + +
+ +- OpenTelemetry Envelope +- Content Modifier contexts for OpenTelemetry logs + +
+ +#### OpenTelemetry Envelope + +When the source of data generated comes from an OpenTelemetry SDK, it comes already formatted with the expected schema, however for majority of cases where the data __does not come from__ an OTel instrumented application and the destination is an OpenTelemetry endpoint, we need to format the data accordingly. + +OpenTelemetry Envelope is a new processor that puts in place the minimum data structure for Log records that are not coming from a native OTel instrumented application, making it easier to process it and deliver it to an OpenTelemetry endpoint. + +![Processor: OpenTelemetry Envelope](/images/processor_opentelemetry_envelope.png) + +Here is a simple example on how to use it: + +```yaml +pipeline: + inputs: + - name: dummy + samples: 1 + + processors: + logs: + - name: opentelemetry_envelope + + outputs: + - name : stdout + match: '*' +``` + +

+ +#### Content Modifier for OpenTelemetry + +This new processor introduced recently in v3 series, provides similar functionality exposed by filters to modify or alter the content of records. This time we have extended [Content Modifier](https://docs.fluentbit.io/manual/pipeline/processors/content-modifier) processor to allow to perform modifications on different parts of OpenTelemetry Logs schema, the following new contexts are supported + +| Context | Description | +|--|--| +| ```otel_resource_attributes```   | alter resource attributes | +| ```otel_scope_name``` | manipulate the scope name | +| ```otel_scope_version``` | manipulate the scope version | +| ```otel_scope_attributes``` | alter the scope attributes | + +
+ +In this example, we are using OpenTelemetry Envelope processor and Content Modifier, to alter the Log resource and Scope attributes: + +```yaml +service: + flush: 1 + log_level: info + +pipeline: + inputs: + - name: dummy + samples: 1 + + processors: + logs: + - name: opentelemetry_envelope + + - name: content_modifier + context: otel_resource_attributes + action: upsert + key: "new_key" + value: "some value" + + - name: content_modifier + context: otel_scope_attributes + action: upsert + key: "new_span_key" + value: "abc" + + outputs: + - name : stdout + match: '*' +``` + +{{< contributor-list >}} +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v3.1/v3.1.1.md b/content/announcements/v3.1/v3.1.1.md new file mode 100644 index 0000000..233a2df --- /dev/null +++ b/content/announcements/v3.1/v3.1.1.md @@ -0,0 +1,258 @@ +--- +title: 'v3.1.1' +description: 'Next generation Telemetry Agent for Logs, Metrics and Traces. ' +url: "/announcements/v3.1.1/" +release_date: 2024-07-10 +publishdate: 2024-07-10 +ver: v3.1.1 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +## Release Notes v3.1 + +> FYI: v3.1.1 is just a minor update that fixes some build system issues found in v3.1.0 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Telemetry Agent for Linux, BSD, MacOS and Windows. We are proud to announce the availability of **Fluent Bit v3.1**. + +``` + Fluent Bit v3.1.1 + * Copyright (C) 2015-2024 The Fluent Bit Authors + * Fluent Bit is a CNCF sub-project under the umbrella of Fluentd + * https://fluentbit.io + ______ _ _ ______ _ _ _____ __ + | ___| | | | | ___ (_) | |____ |/ | + | |_ | |_ _ ___ _ __ | |_ | |_/ /_| |_ __ __ / /`| | + | _| | | | | |/ _ \ '_ \| __| | ___ \ | __| \ \ / / \ \ | | + | | | | |_| | __/ | | | |_ | |_/ / | |_ \ V /.___/ /_| |_ + \_| |_|\__,_|\___|_| |_|\__| \____/|_|\__| \_/ \____(_)___/ + + [2024/07/09 12:55:40] [ info] [fluent bit] version=3.1.1, commit=8ed2e80a9e, pid=51082 + [2024/07/09 12:55:40] [ info] [storage] ver=1.5.2, type=memory, sync=normal, checksum=off, max_chunks_up=128 + [2024/07/09 12:55:40] [ info] [cmetrics] version=0.9.1 + [2024/07/09 12:55:40] [ info] [ctraces ] version=0.5.1 + [2024/07/09 12:55:40] [ info] [sp] stream processor started +``` + +For people upgrading from previous versions you **must read** the Upgrading Notes section of our documentation: + +[https://docs.fluentbit.io/manual/installation/upgrade_notes](https://docs.fluentbit.io/manual/installation/upgrade_notes) + +### Introduction + +Fluent Bit, a CNCF graduated project under the umbrella of Fluentd, announces the availability of v3.1. + +In every release, there are many improvements and fixes, on this notes we will refer to the major changes that will make your infrastructure happier ;) + +Below a list of the highlights of this release: + +### LuaJIT Upgrade (scripting) + +One of the powerful features of Fluent Bit, is the ability to provide the user an interface to control how to modify their telemetry data by using a very simple, but powerful, scripting language: [Lua](https://docs.fluentbit.io/manual/pipeline/filters/lua). Our Lua interface is powered by [LuaJIT](https://github.com/LuaJIT/LuaJIT) library, this has not been upgraded in the last two years, primary because it has been very stable for a long time. + +In this release, we are upgrading to the latest of LuaJIT which brings overall architecture improvements and extended support for Windows on ARM. + +If you are curious to learn how to use Lua, just copy the following configuration file and see how a simple dumy message can be modified inline in the config with some embedded Lua code: + +```yaml +service: + flush: 1 + log_level: info + +pipeline: + inputs: + - name: dummy + dummy: '{"project": "Fluent Bit", "version": "v3.1"}' + + processors: + logs: + - name: lua + call: cb_replace + code: | + function cb_replace(tag, timestamp, record) + record['tag'] = tag + record['new_key'] = 'new value' + + -- Record modified, so 'code' return value (first parameter) is 1 + return 1, timestamp, record + end + outputs: + - name: stdout + match: '*' +``` + +
+ +> You can also have your Lua code in a separate file, just read more about the [Lua](https://docs.fluentbit.io/manual/pipeline/filters/lua) in our documentation. + +
+ +### Add support for local/system timezone on parsing + +When parsing log record dates, there are cases when the date string don't contain any relevant information about the timezone it was generated, however is ideal to assume those records were generated in the local system so it parsing should assume the local timezone set in the operating system. + +This change introduces a new configuration flag for the parsers section called `time_system_timezone` (default: off) that allows to use system timezone (`mktime(3)`) for date representation. + +The following configuration snippet provides an example of it (note this is in classic config mode): + +```python +[PARSER] + Name no_timezone + Format json + Time_Key time + Time_Format %Y-%m-%d:%H:%M:%S + Time_System_Timezone true +``` + +

+ +### Networking + +In this release we did some small changes to our networking stack. + +#### Configurable KeepAlive for Downstream based input plugins + +Downstream is an internal interface to allow to recieve TCP connections. In this release we added a new configuration flag to control manually the keepalive functionality (this allows to have persistent TCP connection), the new option available is `net.keepalive` (default: on). + +Users of this interface are the __input__ plugins such as [HTTP](https://docs.fluentbit.io/manual/pipeline/inputs/http), [OpenTelemetry](https://docs.fluentbit.io/manual/pipeline/inputs/opentelemetry), [Elasticsearch](https://docs.fluentbit.io/manual/pipeline/inputs/elasticsearch), [Splunk](https://docs.fluentbit.io/manual/pipeline/inputs/splunk), etc. + +
+ +#### TLS Hostname verification + +When performing TLS connections, the default verification is for the certificates. In this new release, we added a new check to verify the hostname associated to the certificates through the new option `tls.verify_hostname` (default: off). + +Note that some plugins that uses TLS might have a different default for `tls.verify_hostname`. + +
+ +#### OpenSSL: reduce log noise + +A small change to reduce the noise in the TLS logger make sure you don't get false positives when doing secured sessions backed by OpenSSL. + +
+ +### Core: Log Groups + +When input plugins collects or generate log records, usually those are serialized one after each other, the only shared information that existed for records grouped inside a Chunk is the `Tag`. This simple metadata field is used for routing through the pipeline, however there are cases where we need to set metadata for a group of records, this is where we have implemented the concept of Groups. + +A Group is a special record definition that aims to share specific information for all records under that group. + +``` + :---+--------------+---------------------------------------------: + | 1 | Group | [ [Group Start, {METADATA}], {GROUP_INFO} ] | + :---+--------------+---------------------------------------------: + | 2 | Log Record 1 | [ [TIMESTAMP, {METADATA}], {RECORD} ] | + :---+--------------+---------------------------------------------: + | 3 | Log Record 2 | [ [TIMESTAMP, {METADATA}], {RECORD} ] | + :---+--------------+---------------------------------------------: + | 4 | Log Record N | [ [TIMESTAMP, {METADATA}], {RECORD} ] | + :---+--------------+---------------------------------------------: + | 5 | Group | [ [Group End, {}], {} ] | + '---'--------------'---------------------------------------------' +``` + +As of now, the only components of plugins generating or consuming those groups are [OpenTelemetry input](https://docs.fluentbit.io/manual/pipeline/inputs/opentelemetry) and [OpenTelemetry output](https://docs.fluentbit.io/manual/pipeline/outputs/opentelemetry). This aims to address some issues we had handling OTel Logs metadata. + +
+ +### Processors + +Processors are the new way to do data processing or filtering. In this release we are announcing the following changes for end users interested in OpenTelemetry log processing: + +
+ +- OpenTelemetry Envelope +- Content Modifier contexts for OpenTelemetry logs + +
+ +#### OpenTelemetry Envelope + +When the source of data generated comes from an OpenTelemetry SDK, it comes already formatted with the expected schema, however for majority of cases where the data __does not come from__ an OTel instrumented application and the destination is an OpenTelemetry endpoint, we need to format the data accordingly. + +OpenTelemetry Envelope is a new processor that puts in place the minimum data structure for Log records that are not coming from a native OTel instrumented application, making it easier to process it and deliver it to an OpenTelemetry endpoint. + +![Processor: OpenTelemetry Envelope](/images/processor_opentelemetry_envelope.png) + +Here is a simple example on how to use it: + +```yaml +pipeline: + inputs: + - name: dummy + samples: 1 + + processors: + logs: + - name: opentelemetry_envelope + + outputs: + - name : stdout + match: '*' +``` + +

+ +#### Content Modifier for OpenTelemetry + +This new processor introduced recently in v3 series, provides similar functionality exposed by filters to modify or alter the content of records. This time we have extended [Content Modifier](https://docs.fluentbit.io/manual/pipeline/processors/content-modifier) processor to allow to perform modifications on different parts of OpenTelemetry Logs schema, the following new contexts are supported + +| Context | Description | +|--|--| +| ```otel_resource_attributes```   | alter resource attributes | +| ```otel_scope_name``` | manipulate the scope name | +| ```otel_scope_version``` | manipulate the scope version | +| ```otel_scope_attributes``` | alter the scope attributes | + +
+ +In this example, we are using OpenTelemetry Envelope processor and Content Modifier, to alter the Log resource and Scope attributes: + +```yaml +service: + flush: 1 + log_level: info + +pipeline: + inputs: + - name: dummy + samples: 1 + + processors: + logs: + - name: opentelemetry_envelope + + - name: content_modifier + context: otel_resource_attributes + action: upsert + key: "new_key" + value: "some value" + + - name: content_modifier + context: otel_scope_attributes + action: upsert + key: "new_span_key" + value: "abc" + + outputs: + - name : stdout + match: '*' +``` + +{{< contributor-list >}} +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v3.1/v3.1.10.md b/content/announcements/v3.1/v3.1.10.md new file mode 100644 index 0000000..46e53ae --- /dev/null +++ b/content/announcements/v3.1/v3.1.10.md @@ -0,0 +1,60 @@ +--- +title: 'v3.1.10' +description: 'Next generation Telemetry Agent for Logs, Metrics and Traces. ' +url: "/announcements/v3.1.10/" +release_date: 2024-11-04 +publishdate: 2024-11-04 +ver: v3.1.10 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +## Release Notes v3.1.10 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD, Windows and macOS. We are proud to announce the availability of **Fluent Bit v3.1.10**. + +
+ +### What's new ? + +
+ + - __Libraries__ + - cmetrics: upgrade to v0.9.7 + +
+ + - __Pipeline Plugins__ + + - [Forward (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/forward/) + - Fixed incorrect shared key length in authentication (#9551) + - [Opentelemetry (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/opentelemetry/) + - Add `tag_from_uri` support to http2 codepath + - [Http (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/http/) + - Fix parsing of root json object + - [Opentelemetry (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/opentelemetry/) + - Fix check for uri sanitization when exiting + +{{< contributor-list >}} + +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + + +- [Patrick Stephens](https://github.com/patrick-stephens) +- [Leonardo Alminana](https://github.com/leonardo-albertovich) +- [Eduardo Silva](https://github.com/edsiper) +- [Stewart Webb](https://github.com/nuclearpidgeon) + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v3.1/v3.1.2.md b/content/announcements/v3.1/v3.1.2.md new file mode 100644 index 0000000..7ffdab1 --- /dev/null +++ b/content/announcements/v3.1/v3.1.2.md @@ -0,0 +1,61 @@ +--- +title: 'v3.1.2' +description: 'Next generation Telemetry Agent for Logs, Metrics and Traces. ' +url: "/announcements/v3.1.2/" +release_date: 2024-07-12 +publishdate: 2024-07-12 +ver: v3.1.2 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +## Release Notes v3.1.2 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD, Windows and macOS. We are proud to announce the availability of **Fluent Bit v3.1.2**. + +
+ +### What's new ? + +
+ + - __Core__ + - build: correctly fetch FLB_GIT_HASH from signed commit (#9070) + - input: Config parsing order fixed for FS storage pause parameter (#8720) + +
+ + - __Pipeline Plugins__ + + - [Forward (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/forward/) + - Add plugin pause/resume mechanism + - [Tail (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/tail/) + - Add option `file_cache_advise` (default: on) to reduce file cache usage (less memory) + - [Log_To_Metric (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/log_to_metric/) + - Add emitter alias and mem limit + - [Opentelemetry (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/opentelemetry/) + - Restore manual parsing for log record fields (fix #9071) + +{{< contributor-list >}} + +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +- [Richard Treu](https://github.com/drbugfinder-work) +- [Eduardo Silva](https://github.com/edsiper) +- [xl-sec](https://github.com/xl-sec) +- [Erik Bledsoe](https://github.com/erikbledsoe) +- [Hiroshi Hatake](https://github.com/cosmo0920) + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v3.1/v3.1.3.md b/content/announcements/v3.1/v3.1.3.md new file mode 100644 index 0000000..1cb1589 --- /dev/null +++ b/content/announcements/v3.1/v3.1.3.md @@ -0,0 +1,63 @@ +--- +title: 'v3.1.3' +description: 'Next generation Telemetry Agent for Logs, Metrics and Traces. ' +url: "/announcements/v3.1.3/" +release_date: 2024-07-17 +publishdate: 2024-07-17 +ver: v3.1.3 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +## Release Notes v3.1.3 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD, Windows and macOS. We are proud to announce the availability of **Fluent Bit v3.1.3**. + +
+ +### What's new ? + +
+ + - __Core__ + - metrics: fix dynamic composing of old metric names (fix #9086) + +
+ + - __Libraries__ + - [CTraces](https://github.com/fluent/ctraces): upgrade to v0.5.2 + +
+ + - __Pipeline Plugins__ + - [Winevtlog (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/winevtlog/) + - Retrieve messages of forwarded events correctly + - [Splunk (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/splunk/) + - Downgrade from error response to debug message when content-type is not set + - Fix issues when processing credentials and content-type + - [Datadog (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/datadog/) + - Fix default message key remapping (#9085) + - Fix typo for `dd_hostname` helper (#9032) + +{{< contributor-list >}} + +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +- [Philip Meier](https://github.com/pmeier) +- [Eduardo Silva](https://github.com/edsiper) +- [Hiroshi Hatake](https://github.com/cosmo0920) +- [Jesse Szwedko](https://github.com/jszwedko) + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v3.1/v3.1.4.md b/content/announcements/v3.1/v3.1.4.md new file mode 100644 index 0000000..0eb1316 --- /dev/null +++ b/content/announcements/v3.1/v3.1.4.md @@ -0,0 +1,52 @@ +--- +title: 'v3.1.4' +description: 'Next generation Telemetry Agent for Logs, Metrics and Traces. ' +url: "/announcements/v3.1.4/" +release_date: 2024-07-25 +publishdate: 2024-07-25 +ver: v3.1.4 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +## Release Notes v3.1.4 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD, Windows and macOS. We are proud to announce the availability of **Fluent Bit v3.1.4**. + +
+ +### What's new ? + +
+ + - __Core__ + - input_chunk: do not `exit(0)` if cannot release old chunks + - log_to_metrics: allow custom namespace and subsystem (#9101) + +
+ + - __Pipeline Plugins__ + - [Stdout (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/stdout/) + - Consume entire metrics type of buffers (#9118) + +{{< contributor-list >}} + +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +- [Eduardo Silva](https://github.com/edsiper) +- [Florian BEZANNIER](https://github.com/flobz) +- [Hiroshi Hatake](https://github.com/cosmo0920) + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v3.1/v3.1.5.md b/content/announcements/v3.1/v3.1.5.md new file mode 100644 index 0000000..3c3318d --- /dev/null +++ b/content/announcements/v3.1/v3.1.5.md @@ -0,0 +1,99 @@ +--- +title: 'v3.1.5' +description: 'Next generation Telemetry Agent for Logs, Metrics and Traces. ' +url: "/announcements/v3.1.5/" +release_date: 2024-08-09 +publishdate: 2024-08-09 +ver: v3.1.5 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +## Release Notes v3.1.5 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD, Windows and macOS. We are proud to announce the availability of **Fluent Bit v3.1.5**. + +
+ +### What's new ? + +
+ + - __Core__ + - log_event_decoder: updated code to use aligned memory reads + - gzip: add support for concatenated GZip payloads + - wasm: Make configurable heap and stack sizes with a struct type + - core: add new `FLB_ENFORCE_ALIGNMENT` cmake option for old targets + - core: add new option `FLB_PREFER_SYSTEM_LIB_LUAJIT` to use system provided (external) LuaJIT + +
+ + - __Libraries__ + - ctraces: upgrade to v0.5.4 + - cmetrics: upgrade to v0.9.3 + - cfl: upgrade to v0.5.2 + - c-ares: ugprade to v1.32.3 + +
+ + - __Pipeline Plugins__ + - [MQTT (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/mqtt/) + - Add `buffer_size` setting and fixed a leak (#9163) + - [Tail (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/tail/) + - Fix double-free on exception (cid 507963) + - [Node_Exporter_Metrics (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/node_exporter_metrics/) + - Align the collecting metrics of unit statuses (#9134) + - [Forward (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/forward/) + - Fix leak on exception (cid 507786) + - Fix leak on exception (cid 508219) + - Add support for concatenated GZip payloads + - [Kubernetes_Events (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/kubernetes_events/) + - Fix end of chunked stream deadlock + - Add chunked streaming test + - [Exec_Wasi (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/exec_wasi/) + - Provide configurable stack and heap sizes for wasm + - [Emitter (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/emitter/) + - Fix use-after-free on exception (cid 507860) + - [Wasm (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/wasm/) + - Provide configurable heap and stack sizes for WASM + - [Log_To_Metrics (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/log_to_metrics/) + - Add new option `discard_logs` + - [Opensearch (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/opensearch/) + - Fixed wrong payload buffer usage for traces + - [S3 (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/s3/) + - Plug memory leaks on gzipped buffer during the swapping contents + - [Oracle_Log_Analytics (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/oracle_log_analytics/) + - Fix to prevent double free + - [Kafka (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/kafka/) + - Add missing initialization (cid 507783) + - [Prometheus_Exporter (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/prometheus_exporter/) + - Handle multiply concatenated metrics type of events (#9122) + - [Stackdriver (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/stackdriver/) + - Fix leak on exception (cid 508239) + +{{< contributor-list >}} + +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +- [Ryan Ohnemus](https://github.com/ryanohnemus) +- [Hiroshi Hatake](https://github.com/cosmo0920) +- [Leonardo Alminana](https://github.com/leonardo-albertovich) +- [Eduardo Silva](https://github.com/edsiper) +- [Thomas Devoogdt](https://github.com/ThomasDevoogdt) +- [kangaechu](https://github.com/kangaechu) +- [Pat](https://github.com/patrick-stephens) +- [Takahiro Yamashita](https://github.com/nokute78) + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v3.1/v3.1.6.md b/content/announcements/v3.1/v3.1.6.md new file mode 100644 index 0000000..2fcb0c7 --- /dev/null +++ b/content/announcements/v3.1/v3.1.6.md @@ -0,0 +1,87 @@ +--- +title: 'v3.1.6' +description: 'Next generation Telemetry Agent for Logs, Metrics and Traces. ' +url: "/announcements/v3.1.6/" +release_date: 2024-08-09 +publishdate: 2024-08-09 +ver: v3.1.6 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +## Release Notes v3.1.6 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD, Windows and macOS. We are proud to announce the availability of **Fluent Bit v3.1.6**. + +
+ +### What's new ? + +
+ + - __Core__ + - parser: declare ambiguous DST + - upstream_ha: fix memory leaks + - upstream_node: process `tls.verify_hostname` parameter + - azure_blob_http: use sds_cat variant (CID 306764 306765) + - pack: do not multiply out_size by realloc_size (#9193) + +
+ + - __Libraries__ + - ctraces: upgrade to v0.5.5 + - cmetrics: upgrade to v0.9.4 + +
+ + - __Pipeline Plugins__ + - [StatsD (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/statsd/) + - Implement statsd decoder to translate metrics type of events + - [HTTP (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/http/) + - Add flexible handling of content-type for application/json* (#9190) + - Use cat_safe on tag handling + - [Splunk (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/splunk/) + - Release uri on exception (cid 507867) + - [Forward (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/forward/) + - Fix resource leak on exception (cid 508219) (#9209) + - Fix leaks when processing metrics and traces (#9199) + - [Kubernetes_Events (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/kubernetes_events/) + - Use time_t for time(null) returned value (cid 507947) + - [Exec_Wasi (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/exec_wasi/) + - Fix resource leak on exception (cid 508347) (#9208) + - [Cloudwatch_Logs (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/cloudwatch_logs/) + - Fix disabling of `auto_create_group` config option + - [Datadog (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/datadog/) + - Use cat_safe for remapped tags (cid 253605) + - [Azure_Kusto (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/azure_kusto/) + - Follow the creating node function change + - [Stackdriver (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/stackdriver/) + - Release `log_name` on exception (cid 313112) + - Add latency metric for write log entries http request. (#9182) + +{{< contributor-list >}} + +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +- [Ryan Ohnemus](https://github.com/ryanohnemus) +- [Hiroshi Hatake](https://github.com/cosmo0920) +- [Leonardo Alminana](https://github.com/leonardo-albertovich) +- [Eduardo Silva](https://github.com/edsiper) +- [Thomas Devoogdt](https://github.com/ThomasDevoogdt) +- [kangaechu](https://github.com/kangaechu) +- [Pat](https://github.com/patrick-stephens) +- [Takahiro Yamashita](https://github.com/nokute78) + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v3.1/v3.1.7.md b/content/announcements/v3.1/v3.1.7.md new file mode 100644 index 0000000..4527929 --- /dev/null +++ b/content/announcements/v3.1/v3.1.7.md @@ -0,0 +1,164 @@ +--- +title: 'v3.1.7' +description: 'Next generation Telemetry Agent for Logs, Metrics and Traces. ' +url: "/announcements/v3.1.7/" +release_date: 2024-08-09 +publishdate: 2024-08-09 +ver: v3.1.7 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +## Release Notes v3.1.7 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD, Windows and macOS. We are proud to announce the availability of **Fluent Bit v3.1.7**. + +
+ +### What's new ? + +
+ + - __Core__ + - config_map: fix configuration type cast issue found on s390x (#8904) + - utils: + - fix compatibility for macOS < 12 (#9217) + - use MachineGUID correctly on Windows + - output_thread: add simple conditional on event handling + - examples: filter_wasm_go: Upgrade to support Go 1.22.5 and TinyGo 0.32 + - network: fixed a wrong data type used for an option + - build: use the system provided lib luajit for s390x (#9172) + - build: added cpu byte order detection (same as msgpack-c) + - build: add build time event loop selection + - upstream_ht: added support for inherited property lookup + - upstream_ht: added environment variable support + - bin: fix memory leaks (#9310) + - scheduler: new wrapper for timers to consume notification byte + - sp: changed property type to prevent lossy cast + - static analysis related fixes: + - regex: check return of strchr (CID 507956) + - aws: release resource on exception (CID 50 8048) + - record_accessor: Plug a resource leak on exception (CID 508119) + - processor_content_modifier: logs: do not remove object on type excepcion (CID 507615) + - aws_util: release resource on exception (CID 507694 507588) + - reload: added appropriate type cast (CID 508209) + - output: Remove a needless calling of flb_output_instance_destroy() (CID 507809) + - log_event_encoder: log_event_decoder: enhanced byte order handling for timestamps (#9196) + +
+ + - __Libraries__ + - lib: cmetrics: upgrade to v0.9.5 + - lib: c-ares: upgrade to v1.33.1 + - lib: monkey: sync poll(2) new backend support + +
+ + - __Pipeline Plugins__ + - [Nginx_Exporter_Metrics (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/nginx_exporter_metrics/) + - Fixed a wrong data type used for an option + - [Dummy (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/dummy/) + - Fix the timestamp issue on s390x + - [Head (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/head/) + - Fix cannot allocate memory on big endian system + - [Process_Exporter_Metrics (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/process_exporter_metrics/) + - Initialize with null for flb_sds_t pointer variables (cid 507975) + - [Event_Type (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/event_type/) + - Fix a resource leak on exception (cid 507968) (#9235) + - [Prometheus_Remote_Write (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/prometheus_remote_write/) + - Use sds_cat_safe (cid 508087) + - [Winevtlog (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/winevtlog/) + - Remove a needless third argument + - Add a describing link for capability sids + - Skip to translate sid for capability sids + - [Podman_Metrics (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/podman_metrics/) + - Plug a resource leak on exception (cid 507805) + - [Collectd (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/collectd/) + - Fixed out of bounds read + - [Opentelemetry (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/opentelemetry/) + - Fixed a wrong data type used for an option + - [Node_Exporter_Metrics (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/node_exporter_metrics/) + - Fixed possible invalid dereference + - [Elasticsearch (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/elasticsearch/) + - Fix string in http response code 400 (#8977) + - Store log_encoder pointer (cid 508245) + - [Splunk (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/splunk/) + - handle 1.0 suffixed endpoints + - [Forward (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/forward/) + - Release resource on exception (cid 508219) + - Rel resources on exception (cid 507596 507786 507954 508064 508123) + - [Docker (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/docker/) + - Release resource on exception (cid 507574) + - [Emitter (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/emitter/) + - Plug a possible resource leak on exception (cid 508193) + - [Kubernetes (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/kubernetes/) + - Fix data type for token creation (cid 507947) + - [Aws (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/aws/) + - Fixed group metadata fetch issue and addressed (cid 507941) + - [Logs_To_Metrics (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/logs_to_metrics/) + - Fixed memory corruption issue + - [Multiline (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/multiline/) + - Fix conditional after allocation (cid 507750) + - [Tensorflow (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/tensorflow/) + - Fixed a wrong data type used for a boolean option + - [Alter_Size (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/alter_size/) + - Fixed memory leak + - [Logdna (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/logdna/) + - Refactored logdna uri formation to support configurable endpoints (#8051) + - [Opensearch (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/opensearch/) + - Fix a use-after-free issue (cid 508013) (#9233) + - Fixed a wrong data type used for a boolean option (#9221) + - [Chronicle (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/chronicle/) + - Use flb_sds_cat_safe (cid 507603) + - Fix a resource leak on exception (cid 508008) (#9234) + - [S3 (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/s3/) + - Plug an error for too many argument for mkdir + - Fix double free (cid 508202) + - [Opentelemetry (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/opentelemetry/) + - Prevent null dereference on memcpy (cid 507910) + - Skip ra_trace_flags_metadata check (cid 508199) + - Fix record accessor pattern (cid 507960) + - Release buf on exception (cid 507777) + - logs: release scope on exception (cid 507600) + - On proxy config check status + - [Azure_Blob (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/azure_blob/) + - Fixed double free + - [Elasticsearch (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/elasticsearch/) + - Process error information properly + - [Cloudwatch_Logs (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/cloudwatch_logs/) + - Release resources on exception (cid 507707) + - [Stackdriver (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/stackdriver/) + - Fixed a wrong data type used for two boolean options + - [Loki (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/loki/) + - Use sds_cat_safe (cid 507977 311957) + - [Calyptia (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/calyptia/) + - Initialize return and http context (cid 507836) + +{{< contributor-list >}} + +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +- [Leonardo Alminana](https://github.com/leonardo-albertovich) +- [Eduardo Silva](https://github.com/edsiper) +- [Hiroshi Hatake](https://github.com/cosmo0920) +- Sergey Fedorov +- Mirko Lazarevic +- rightblank (IBM) +- José Lecaros +- JinXinWang +- [Phillip Whelan](https://github.com/pwhelan) +- YingJie Fu (IBM) + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v3.1/v3.1.8.md b/content/announcements/v3.1/v3.1.8.md new file mode 100644 index 0000000..07ab876 --- /dev/null +++ b/content/announcements/v3.1/v3.1.8.md @@ -0,0 +1,71 @@ +--- +title: 'v3.1.8' +description: 'Next generation Telemetry Agent for Logs, Metrics and Traces. ' +url: "/announcements/v3.1.8/" +release_date: 2024-09-16 +publishdate: 2024-09-16 +ver: v3.1.8 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +## Release Notes v3.1.8 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD, Windows and macOS. We are proud to announce the availability of **Fluent Bit v3.1.8**. + +
+ +### What's new ? + +
+ + - __Core__ + - tls: check async status before using coroutines in flb_tls_session_create. + - io: use connection flags when setting async in flb_io_net_connect. + - input: added missing initializer (CID 508207) (#9332) + +
+ + - __Libraries__ + - lib: cmetrics: upgrade to v0.9.6 + +
+ + - __Pipeline Plugins__ + - [Calyptia_Fleet (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/calyptia_fleet/) + - Fixed memory leaks (cid 508187) (#9347) + - [Elasticsearch (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/elasticsearch/) + - Fixed potential memory leak + - Improved buffer size check + - [Opentelemetry (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/opentelemetry/) + - Fixed memory leaks on exception + - [HTTP (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/http/) + - Request timed out(408) to be treated as retry (#9379) + - [Influxdb (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/influxdb/) + - Handle signed/unsigned integer as influx's integer of that representation (#9361) + - [Splunk (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/splunk/) + - Extend retrying http status code (#9380) + +{{< contributor-list >}} + +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + + +- [Eduardo Silva](https://github.com/edsiper) +- [Phillip Whelan](https://github.com/pwhelan) +- [Hiroshi Hatake](https://github.com/cosmo0920) +- [Leonardo Alminana](https://github.com/leonardo-albertovich) + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v3.1/v3.1.9.md b/content/announcements/v3.1/v3.1.9.md new file mode 100644 index 0000000..571b2b9 --- /dev/null +++ b/content/announcements/v3.1/v3.1.9.md @@ -0,0 +1,71 @@ +--- +title: 'v3.1.9' +description: 'Next generation Telemetry Agent for Logs, Metrics and Traces. ' +url: "/announcements/v3.1.9/" +release_date: 2024-09-27 +publishdate: 2024-09-27 +ver: v3.1.9 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +## Release Notes v3.1.9 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD, Windows and macOS. We are proud to announce the availability of **Fluent Bit v3.1.9**. + +
+ +### What's new ? + +
+ + - __Core__ + - multiline: reset group metadata buffer on flush (fix high CPU issue) + - reload: do not call `flb_stop()` when `flb_start()` fails (fix crash on RHEL) + - utils: improve error messaging inside the `flb_utils_proxy_url_split()` + - config_format: yaml: cleanup variants only if occurred an exception + - config_format: yaml: Plug memory leaks on exception for variants on plugin elements + - upstream: remove call to `flb_errno()` when is not needed. + +
+ + - __Libraries__ + - lib: ctraces: upgrade to v0.5.6 + +
+ + - __Pipeline Plugins__ + - [Calyptia_Fleet (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/calyptia_fleet/) + - Fix 'unable to find fleet by name' (backport of #9410). + - [Elasticsearch (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/elasticsearch/) + - Fix potential issue caused by ingored reallocation + - [Multiline (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/multiline/) + - Reset group metadata buffer on flush + - [Calyptia (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/calyptia/) + - Add missing parameter for msgpack_str for windows os field. + - Register os and architecture for agents. + +{{< contributor-list >}} + +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +- [Richard Treu](https://github.com/drbugfinder-work) +- [Jose Lecaros](https://github.com/lecaros) +- [Phillip Whelan](https://github.com/pwhelan) +- [Hiroshi Hatake](https://github.com/cosmo0920) +- [Leonardo Alminana](https://github.com/leonardo-albertovich) +- [Eduardo Silva](https://github.com/edsiper) + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v3.2/_index.md b/content/announcements/v3.2/_index.md new file mode 100644 index 0000000..e438caa --- /dev/null +++ b/content/announcements/v3.2/_index.md @@ -0,0 +1,15 @@ +--- +title: 'Fluent Bit v3.2 Series' +description: "[Fluent Bit v3.2](https://github.com/fluent/fluent-bit/tree/master) is the new **stable branch** for production usage. Based on bug reports or +specific minor feature requests, we do quick releases upon demand. Below is a list of the notes for each version." +url: "/announcements/v3.2/" +herobg: "/images/hero@2x.jpg" +latestVer: true +releaseNotes: + heading: "Release Notes v3.2.0" + version: "v3.2.0" + text: "Fluent Bit is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD and OSX. We are proud to announce the availability of Fluent Bit v1.9.9.
+ For people upgrading from previous versions you must read the Upgrading Notes section of our documentation: + https://docs.fluentbit.io/manual/installation/upgrade_notes" +--- + diff --git a/content/announcements/v3.2/v3.2.0.md b/content/announcements/v3.2/v3.2.0.md new file mode 100644 index 0000000..96d5f55 --- /dev/null +++ b/content/announcements/v3.2/v3.2.0.md @@ -0,0 +1,182 @@ +--- +title: 'v3.2.0' +description: 'Next generation Telemetry Agent for Logs, Metrics and Traces. ' +url: "/announcements/v3.2.0/" +release_date: 2024-11-12 +publishdate: 2024-11-12 +ver: v3.2.0 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +## Release Notes v3.2.0 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Telemetry Agent for Linux, BSD, MacOS and Windows. We are proud to announce the availability of **Fluent Bit v3.1**. + +``` +Fluent Bit v3.2.0 +* Copyright (C) 2015-2024 The Fluent Bit Authors +* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd +* https://fluentbit.io + +______ _ _ ______ _ _ _____ _____ +| ___| | | | | ___ (_) | |____ |/ __ \ +| |_ | |_ _ ___ _ __ | |_ | |_/ /_| |_ __ __ / /`' / /' +| _| | | | | |/ _ \ '_ \| __| | ___ \ | __| \ \ / / \ \ / / +| | | | |_| | __/ | | | |_ | |_/ / | |_ \ V /.___/ /./ /___ +\_| |_|\__,_|\___|_| |_|\__| \____/|_|\__| \_/ \____(_)_____/ + + +[2024/11/11 20:40:02] [ info] [fluent bit] version=3.2.0, commit=58cc2d5944, pid=3050130 +[2024/11/11 20:40:02] [ info] [storage] ver=1.5.2, type=memory, sync=normal, checksum=off, max_chunks_up=128 +[2024/11/11 20:40:02] [ info] [simd ] SSE2 +[2024/11/11 20:40:02] [ info] [cmetrics] version=0.9.9 +[2024/11/11 20:40:02] [ info] [ctraces ] version=0.5.7 +[2024/11/11 20:40:02] [ info] [sp] stream processor started +``` + +For people upgrading from previous versions you **must read** the Upgrading Notes section of our documentation: + +[https://docs.fluentbit.io/manual/installation/upgrade_notes](https://docs.fluentbit.io/manual/installation/upgrade_notes) + +### Introduction + +Fluent Bit, a CNCF graduated project under the umbrella of Fluentd, announces the availability of v3.2. + +In every release, there are many improvements and fixes, on this notes we will refer to the major changes that will make your infrastructure happier ;) + +Below a list of the highlights of this release: + +### Complete YAML Support + +Fluent Bit is moving away from its classic configuration mode, which only supports basic key-value pairs and lacks advanced features. YAML, now fully supported in Fluent Bit v3.2, offers a more intuitive and flexible configuration experience, essential for modern cloud environments. We encourage users to transition to YAML to take advantage of features like processors. + +For more details refer to the official documentation: +

+ +- [YAML Configuration](https://docs.fluentbit.io/manual/administration/configuring-fluent-bit/yaml) + + +### SIMD Support + +The JSON encoder now supports SIMD operations which helps to improve the performance of JSON encoding. SIMD is a set of instructions that can be executed in parallel on multiple data points. This feature is disabled by default, but can be enabled by setting the CMake `FLB_SIMD` option to `on`. +

+ +- [SIMD for JSON encoding](https://docs.fluentbit.io/manual/3.2-dev/administration/performance#fluent-bit-and-simd-for-json-encoding) + +### Blob Support + +Fluent Bit now supports the `blob` data type, which is a binary large object that can store large amounts of data. This feature is useful for storing large log messages, binary data, or other large data types that needs to be delivered in order. + +For this implementation the following plugins are supported: + +- Blob input: read data from a file or a stream and send it to the pipeline. +- Azure Blob: extended support to handle Blob data type. + +### Experimental Features + +The following experimental features, are still under active development and are not recommended for production use. However, we encourage you to try them out and provide feedback to help us improve them. + +Note that experimental features are disabled by default, but can be enabled at build time. + +#### eBPF + +Fluent Bit now supports eBPF, a powerful technology that allows you to run custom code in the Linux kernel. The plugin, allows to load eBPF programs and attach them to kernel probes, which can be used to collect data from the kernel. + +This feature is experimental and disabled by default, but can be enabled by setting the CMake `FLB_EBPF` option to `on`. + +#### Profiles (signal types) + +Following the development version of OpenTelemetry Profile specification, Fluent Bit now supports profiles signal types. + +### Libraries + - cmetrics: upgrade to v0.9.9 + - fluent-otel-proto: update files for latest profiles support + - ctraces: upgrade to v0.5.7 + - monkey: upgrade to v1.8.1 + +### Plugins + + - Plugins + - [Opentelemetry (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/opentelemetry/) + - Adjust HTTP/1.1 buffer based on the request size + - Add support for HTTP/1.1 chunked transfer encoding + - Add `tag_from_uri` support to HTTP/2 code path + - [HTTP (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/http/) + - Adjust HTTP/1.1 buffer based on the request size + - Add support for HTTP/1.1 chunked transfer encoding + - [eBPF (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/ebpf/) + - Initial version of the plugin (experimental) + - [Calyptia_Fleet (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/calyptia_fleet/) + - Fix registering agent after first invocation. + - Increase max http buffer size and make it configurable. + - Fleet id search by name must use exact name. (#9561) + - Fix `unable to find fleet by name`. + - Fixed memory leaks (cid 508187) (#9346) + - [Elasticsearch (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/elasticsearch/) + - Add support for chunked transfer encoding + - Fixed potential issue caused by ingored reallocation + - Improved buffer size check + - [Blob (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/blob/) + - New plugin to read binary/large files and process them as Blob data type + - [Splunk (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/splunk/) + - Small fixes on return values (cid 508015 508060 510619) + - Adjust HTTP/1.1 buffer based on the request size + - Add support for HTTP/1.1 chunked transfer encoding + - [Forward (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/forward/) + - Fixed incorrect `shared key` length in authentication (#9550) + - Fix leaks on exception (cid 507786 507954 508064) + - Validate return value when ingesting data (cid 507683) + - Release shared salt on exception (cid 507596) + - [Multiline (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/multiline/) + - Add support for Processors + - Reset group metadata buffer on flush + - [Log_To_Metrics (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/log_to_metrics/) + - Add timer callback for emitting metrics this commit will change the log_to_metrics filter to use a timer based metric inject and not directly inject metrics on every incoming log record anymore. this will lower the overall load and memory consumption especially in high-volume and high-cardinality situations. + - [Datadog (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/datadog/) + - Custom added ability to set custom HTTP headers + - [HTTP (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/http/) + - Request timed out(408) to be treated as retry + - [Stdout (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/stdout/) + - Support blob type (print targe file name) + - [S3 (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/s3/) + - Release context on mock client exception (cid 310831) + - Fix buffered data on exception (cid 309438) + - [Influxdb (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/influxdb/) + - Test: follow the change of parameter name + - Use `add_integer_suffix` name for adding integer suffix on the line protocol + - Provide a way to turn on to use integer type on influxdb's line protocol + - Handle signed/unsigned integer as influx's integer of that representation + - [Kafka (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/kafka/) + - Fix invalid free of the message_key + - Introduce the raw_log_key option raw format + - [Opentelemetry (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/opentelemetry/) + - Added HTTP/2 and gRPC support + - Fix check for uri sanitization when exiting + - [Azure_Blob (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/azure_blob/) + - Add support for Blob data type + - [Splunk (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/splunk/) + - Extend retrying http status code + - [Calyptia (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/calyptia/) + - Add missing parameter for msgpack_str for windows os field. + - Register os and architecture for agents. + - [Kinesis_Streams (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/kinesis_streams/) + - Add custom port support for kinesis output plugin + +{{< contributor-list >}} + +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v3.2/v3.2.1.md b/content/announcements/v3.2/v3.2.1.md new file mode 100644 index 0000000..921b87a --- /dev/null +++ b/content/announcements/v3.2/v3.2.1.md @@ -0,0 +1,61 @@ +--- +title: 'v3.2.1' +description: 'Next generation Telemetry Agent for Logs, Metrics and Traces. ' +url: "/announcements/v3.2.1/" +release_date: 2024-11-16 +publishdate: 2024-11-16 +ver: v3.2.1 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +## Release Notes v3.2.1 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD, Windows and macOS. We are proud to announce the availability of **Fluent Bit v3.2.1**. + +
+ +### What's new ? + +
+ + - __Core__ + - tls: removed alpn selector requirement + - config_format: cf_yaml: Follow the change of variable name + - config_format: cf_fluentbit: Handle static config correctly + +
+ + - __Pipeline Plugins__ + - [Splunk (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/splunk/) + - Splunk token validation must be case-insensitive, since that's the behavior from splunk hec + - [eBPF (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/ebpf/) + - Handle debian like systems' signal probes (#9596) + - Prepare build directory for building ebpf objects (#9592) + - [Kubernetes (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/kubernetes/) + - Retrieve kubernetes pod ip address if it is set in status.podip (#2783) + +{{< contributor-list >}} + +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +- [Richard Treu](https://github.com/drbugfinder-work) +- [Jose Lecaros](https://github.com/lecaros) +- [Phillip Whelan](https://github.com/pwhelan) +- [Hiroshi Hatake](https://github.com/cosmo0920) +- [Leonardo Alminana](https://github.com/leonardo-albertovich) +- [Eduardo Silva](https://github.com/edsiper) + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v3.2/v3.2.2.md b/content/announcements/v3.2/v3.2.2.md new file mode 100644 index 0000000..ed441eb --- /dev/null +++ b/content/announcements/v3.2/v3.2.2.md @@ -0,0 +1,108 @@ +--- +title: 'v3.2.2' +description: 'Next generation Telemetry Agent for Logs, Metrics and Traces. ' +url: "/announcements/v3.2.2/" +release_date: 2024-11-27 +publishdate: 2024-11-27 +ver: v3.2.2 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +## Release Notes v3.2.2 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD, Windows and macOS. We are proud to announce the availability of **Fluent Bit v3.2.2**. + +
+ +### What's new ? + +
+ +#### Core +- http_client + - added missing exit path ([PR #9607](https://github.com/fluent/fluent-bit/pull/9607)) + - fixed potential memory corruption ([PR #9611](https://github.com/fluent/fluent-bit/pull/9611)) + - added missing exit path ([PR #9612](https://github.com/fluent/fluent-bit/pull/9612)) + - added per client temporary buffer ([PR #9645](https://github.com/fluent/fluent-bit/pull/9645)) + - added per session temporary buffer ([PR #9645](https://github.com/fluent/fluent-bit/pull/9645)) + - Implement response testing framework ([PR #9401](https://github.com/fluent/fluent-bit/pull/9401)) +- http_common + - added missing initializers ([PR #9608](https://github.com/fluent/fluent-bit/pull/9608)) + - moved initializers to correct a bug introduced in PR 9608 ([PR #9619](https://github.com/fluent/fluent-bit/pull/9619)) + - added guards to prevent leaks ([PR #9645](https://github.com/fluent/fluent-bit/pull/9645)) + - added pre-generated authority field ([PR #9645](https://github.com/fluent/fluent-bit/pull/9645)) +- signv4: added missing length calculation ([PR #9606](https://github.com/fluent/fluent-bit/pull/9606)) +- plugin: Plug a use-after-free issue (CID 514582) ([PR #9639](https://github.com/fluent/fluent-bit/pull/9639)) +- build: made FLB_DEV detectable from code ([PR #9645](https://github.com/fluent/fluent-bit/pull/9645)) +- tls: openssl: added support for SSLKEYLOGFILE on DEV builds ([PR #9645](https://github.com/fluent/fluent-bit/pull/9645)) +- http_client_http2 + - improved protocol compliance ([PR #9645](https://github.com/fluent/fluent-bit/pull/9645)) + - added missing header ([PR #9645](https://github.com/fluent/fluent-bit/pull/9645)) +- output: Add a capability to inject HTTP response testing environment ([PR #9401](https://github.com/fluent/fluent-bit/pull/9401)) +- lib: Implement injecting HTTP response mechanism ([PR #9401](https://github.com/fluent/fluent-bit/pull/9401)) +- network + - Update struct type for sock_addr ([PR #9404](https://github.com/fluent/fluent-bit/pull/9404)) + - Update struct type being passed into accept ([PR #9404](https://github.com/fluent/fluent-bit/pull/9404)) + + +
+ + #### Pipeline Plugins + +- **[Forward (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/forward)** + - Plug a resource leak on exception (CID 508064) ([PR #9616](https://github.com/fluent/fluent-bit/pull/9616)) +- **[Opentelemetry (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/opentelemetry)** + - Propogate tag in http2 metrics and trace handlers ([PR #9635](https://github.com/fluent/fluent-bit/pull/9635)) +- **[Calyptia_fleet (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/calyptia_fleet)** + - improved interval handling ([PR #9648](https://github.com/fluent/fluent-bit/pull/9648)) +- **[HTTP (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/http)** + - use 'tag_key' option when json array is received ([PR #9590](https://github.com/fluent/fluent-bit/pull/9590)) +- **[Lua (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/lua)** + - expose env variables in FLB_ENV Lua table ([PR #9617](https://github.com/fluent/fluent-bit/pull/9617)) +- **[Labels (Processor)](https://docs.fluentbit.io/manual/pipeline/processors/labels)** + - Process operations for output purposed contexts of metrics instead of the original one ([PR #9632](https://github.com/fluent/fluent-bit/pull/9632)) +- **[Opentelemetry (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/opentelemetry)** + - added missing result checks and fixed leaks ([PR #9609](https://github.com/fluent/fluent-bit/pull/9609)) + - decoupled HTTP/2 and gRPC ([PR #9645](https://github.com/fluent/fluent-bit/pull/9645)) +- **[Azure_kusto (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/azure_kusto )** + - fix multiple files tail issue and timeout issue (#8430) ([PR #8430](https://github.com/fluent/fluent-bit/pull/8430)) +- **[Stackdriver (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/stackdriver)** + - return cached token when current_timestamp is less than cached_expiration (#9652) ([PR #9652](https://github.com/fluent/fluent-bit/pull/9652)) +- **[Elasticsearch (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/es)** + - tests: Add HTTP response testing ([PR #9401](https://github.com/fluent/fluent-bit/pull/9401)) +- **[Calyptia (Custom)](https://docs.fluentbit.io/manual/pipeline/customs/calyptia)** + - added interval handling and tests ([PR #9648](https://github.com/fluent/fluent-bit/pull/9648)) + +- **[Node Exporter Metrics](https://docs.fluentbit.io/manual/pipeline/inputs/node_exporter_metrics)** : + - Use real_path for complaining on glob(3) error ([PR #9618](https://github.com/fluent/fluent-bit/pull/9618)) + + +{{< contributor-list >}} + +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + + +- [leonardo-albertovich](https://github.com/leonardo-albertovich) +- [Marat Abrarov](https://github.com/mabrarov) +- [Hiroshi Hatake](https://github.com/cosmo0920) +- [Tanmaya Panda](https://github.com/tanmaya-panda1) +- [Stewart Webb](https://github.com/nuclearpidgeon) +- [Eduardo Silva Pereira](https://github.com/edsiper) +- [shuaichen](https://github.com/shuaich) +- [Ankur Patel](https://github.com/imankurpatel000) +- [jomillerOpen](https://github.com/jomillerOpen) + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/announcements/v3.2/v3.2.3.md b/content/announcements/v3.2/v3.2.3.md new file mode 100644 index 0000000..fcf38a4 --- /dev/null +++ b/content/announcements/v3.2/v3.2.3.md @@ -0,0 +1,108 @@ +--- +title: 'v3.2.3' +description: 'Next generation Telemetry Agent for Logs, Metrics and Traces. ' +url: "/announcements/v3.2.3/" +release_date: 2024-12-19 +publishdate: 2024-12-19 +ver: v3.2.3 +herobg: "/images/hero@2x.jpg" +latestVer: true +--- + +###### KNOWLEDGE BASE + +## Release Notes v3.2.3 + +[Fluent Bit](https://fluentbit.io) is a Fast and Lightweight Data Processor and Forwarder for Linux, BSD, Windows and macOS. We are proud to announce the availability of **Fluent Bit v3.2.3**. + +
+ +### What's new ? + +
+ +#### Core + +- http_server: hs: Handle IPv4 and IPv6 addresses ([PR #9449](https://github.com/fluent/fluent-bit/pull/9449)) +- utf8 + - enhance handling of multibyte sequences ([PR #9687](https://github.com/fluent/fluent-bit/pull/9687)) + - remove lookup table ([PR #9687](https://github.com/fluent/fluent-bit/pull/9687)) +- build + - cmake: fix UNICODE-escaped characters on aarch64 (#8851) ([PR #8851](https://github.com/fluent/fluent-bit/pull/8851)) + - Use signed char in RISC-V 64bit (#9524) ([PR #9524](https://github.com/fluent/fluent-bit/pull/9524)) + - add flb_utf8.c ([PR #9687](https://github.com/fluent/fluent-bit/pull/9687)) + - added include path for the headers cprofiles generates ([PR #9747](https://github.com/fluent/fluent-bit/pull/9747)) +- wasm: Plug a resource leak on exception (CID 508177) (#9615) ([PR #9615](https://github.com/fluent/fluent-bit/pull/9615)) +- input: Add callback member for testing input format ([PR #9291](https://github.com/fluent/fluent-bit/pull/9291)) +- cfl_ra_key: Implement generic CFL based RA key functionality ([PR #9566](https://github.com/fluent/fluent-bit/pull/9566)) +- cfl_record_accessor + - Implement generic CFL based record accessor ([PR #9566](https://github.com/fluent/fluent-bit/pull/9566)) + - tests: internal: Add internal test cases for generic CFL based RA ([PR #9566](https://github.com/fluent/fluent-bit/pull/9566)) + - Plug resource leaks (CID 516097) ([PR #9728](https://github.com/fluent/fluent-bit/pull/9728)) +- utils + - Detect machine_id corruption and fill out a dummy value ([PR #9690](https://github.com/fluent/fluent-bit/pull/9690)) + - use new utf8 decoder API ([PR #9687](https://github.com/fluent/fluent-bit/pull/9687)) + - add utf8 encoding for codepoints beyond BMP ([PR #9687](https://github.com/fluent/fluent-bit/pull/9687)) +- sds + - use new utf8 decoder for sds_cat_utf8 ([PR #9687](https://github.com/fluent/fluent-bit/pull/9687)) +- plugin: Detect config format and use a proper loader ([PR #9738](https://github.com/fluent/fluent-bit/pull/9738)) +- plugin_proxy: Allow to execute flush callback on Golang side during hot-reloading (#9734) ([PR #9734](https://github.com/fluent/fluent-bit/pull/9734)) + +#### Libraries + - c-ares: upgrade to v1.34.4 ([PR #9747](https://github.com/fluent/fluent-bit/pull/9747)) + - cprofiles: upgrade to v0.1.0 ([PR #9747](https://github.com/fluent/fluent-bit/pull/9747)) + - wasm-micro-runtime: Bump up wasm-micro-runtime to v1.3.3 ([PR #9643](https://github.com/fluent/fluent-bit/pull/9643)) + +#### Plugins + +- **[Winevtlog (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/winevtlog)** + - Use correct flag for EvtRender ([PR #9659](https://github.com/fluent/fluent-bit/pull/9659)) +- **[Process_exporter_metrics (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/process_exporter_metrics)** + - fix error logic when directory cannot be opened (#9549) ([PR #9549](https://github.com/fluent/fluent-bit/pull/9549)) +- **[Systemd (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/systemd)** + - Process enumerated data as cfl_kvlist(s) at first ([PR #9291](https://github.com/fluent/fluent-bit/pull/9291)) + - test: Add a format test case for duplicated keys ([PR #9291](https://github.com/fluent/fluent-bit/pull/9291)) +- **[Forward (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/forward)** + - Support `empty_shared_key` parameter (#9681) ([PR #9681](https://github.com/fluent/fluent-bit/pull/9681)) + - Recreate connection when resumed ([PR #9605](https://github.com/fluent/fluent-bit/pull/9605)) +- **[Opentelemetry (Input)](https://docs.fluentbit.io/manual/pipeline/inputs/opentelemetry)** + - logs: add support for OTel resource/scope metadata in JSON payload ([PR #9724](https://github.com/fluent/fluent-bit/pull/9724)) +- **[Parser (Filter)](https://docs.fluentbit.io/manual/pipeline/filters/parser)** + - fix reserve data and preserve key handling (#9675) ([PR #9675](https://github.com/fluent/fluent-bit/pull/9675)) +- **[Calyptia (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/calyptia)** + - retry agent registration on flush callback (#9656) ([PR #9656](https://github.com/fluent/fluent-bit/pull/9656)) +- **[Opentelemetry (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/opentelemetry)** + - remove strict check on severityText value ([PR #9724](https://github.com/fluent/fluent-bit/pull/9724)) +- **[Lib (Output)](https://docs.fluentbit.io/manual/pipeline/outputs/lib)** + - add new option `data_mode` ([PR #9724](https://github.com/fluent/fluent-bit/pull/9724)) + +{{< contributor-list >}} + +#### Contributors + +On every release, there are many people involved doing contributions on different areas like bug reporting, troubleshooting, documentation and coding, without these contributions from the community, the project won’t be the same and won’t be in the good shape that it is now. So THANK YOU! to everyone who takes part of this journey! + +## Contributors + +- [Jorge Niedbalski](https://github.com/niedbalski) +- [Rama Malladi](https://github.com/RamaMalladiAWS) +- [quocbao](https://github.com/baonq-me) +- [Hiroshi Hatake](https://github.com/cosmo0920) +- [Pat](https://github.com/patrick-stephens) +- [Markus Frey](https://github.com/mkfrey) +- [Lars](https://github.com/PromyLOPh) +- [Erik Bledsoe](https://github.com/erikbledsoe) +- [Eduardo Silva Pereira](https://github.com/edsiper) +- [leonardo-albertovich](https://github.com/leonardo-albertovich) +- [Ankur Patel](https://github.com/imankurpatel000) + + +{{< /contributor-list >}} + +#### Join us + +We want to hear about you, our community is growing and you can be part of it!, you can contact us at: + +* Github: [http://github.com/fluent/fluent-bit](https://github.com/fluent/fluent-bit) +* Slack: [http://slack.fluentd.org](http://slack.fluentd.org) +* Twitter: [@fluentbit](https://twitter.com/fluentbit) diff --git a/content/enterprise.md b/content/enterprise.md index 0be6079..8c9ad31 100644 --- a/content/enterprise.md +++ b/content/enterprise.md @@ -10,16 +10,24 @@ enterprise: enabled: true position: 0 list: - - title: Calyptia - description: Support services and Enterprise features from the creators and maintainers of Fluentd and Fluent Bit. - logo: /images/calyptia.svg - buttonUrl: "https://calyptia.com/" - buttonText: "read More" + - title: Chronosphere + description: Enterprise telemetry pipeline solution and support services from the creators of Fluent Bit and Calyptia. Recognized as a leader by major analyst firms, Chronosphere also offers a full observability platform. + logo: /images/chronosphere-logo-vertical.png + buttonUrl: "https://chronosphere.io/platform/telemetry-pipeline/?utm_source=fluent-bit&utm_medium=referral&utm_content=enterprise-tile" + buttonText: "Learn more" tabOpen: "_blank" - title: ITOCHU Techno-Solutions America, Inc. description: "Enterprise services from a global IT solution provider. Services include: consulting, technology enablement, and a global service desk." logo: /images/fluentd.svg buttonUrl: "https://fluentd.ctc-america.com/" - buttonText: "read More" + buttonText: "Learn more" tabOpen: "_blank" ---- \ No newline at end of file + - title: ClearCode + description: > + ClearCode is a core maintainer of Fluentd since 2014, based in Japan. + クリアコードは、日本国内において航空会社、官公庁、通信事業者をはじめとした大規模ユーザーに対する支援を行ってきました。 ご相談にはFluentd/Fluent Bitのコアメンテナが対応いたしますので、まずはお気軽にお問い合わせください。 + logo: /images/ClearCode.svg + buttonUrl: "https://www.clear-code.com/services/fluentd.html" + buttonText: "Learn more" + tabOpen: "_blank" +--- diff --git a/content/how-it-works.md b/content/how-it-works.md index 343c469..aee5fe8 100644 --- a/content/how-it-works.md +++ b/content/how-it-works.md @@ -1,10 +1,10 @@ --- title: 'How It Works' -description: 'Works for Logs and Metrics' +description: 'Works for Logs, Metrics and Traces' type: how-it-works hero: heading: | - Works for Logs and Metrics + Works for Logs, Metrics & Traces desc: | Fluent Bit was designed for speed, scale, and flexibility in a very lightweight, efficient package. image: "/images/overview-new.svg" diff --git a/content/opentelemetry.md b/content/opentelemetry.md new file mode 100644 index 0000000..e667b59 --- /dev/null +++ b/content/opentelemetry.md @@ -0,0 +1,81 @@ +--- +title: 'OpenTelemetry with Fluent Bit' +description: > + Fluent Bit is a vendor-agnostic Telemetry agent with full support for + OpenTelemetry Protocol (OTLP) and overall processing capabilities. +date: 2024-11-12 +headerTheme: light +#herobg: "/images/hero@2x.jpg" +herobg: "/images/hero@2x.jpg" +--- + +## OpenTelemetry + +
+ +OpenTelemetry provides an open-source, vendor-neutral standard for Logs, Metrics, and Traces. It defines an open source schema for these telemetry data types, collectively known as signals, ensuring consistency and compatibility across diverse implementations. Because of its vendor-neutral approach, OpenTelemetry has been widely adopted and supported by numerous platforms and tools in the observability ecosystem. __Fluent Bit__ is one such implementation, offering a __lightweight__ and efficient solution for __processing__ and forwarding telemetry data in line with OpenTelemetry standards. + +Each telemetry type in OpenTelemetry has a specific schema that defines the structure and format of the data. This schema ensures that telemetry data is consistent and interoperable across different systems, enabling seamless integration: + +![/images/opentelemetry/otel_schema.png](/images/opentelemetry/otel_schema.png) + +While schemas ensure consistency and interoperability of telemetry data, the OTLP (OpenTelemetry Protocol) handles the transport. + +#### OTLP - Transport Protocol + +A key component of OpenTelemetry is __OTLP__ (OpenTelemetry Protocol), a gRPC-based transport protocol designed for the structured transmission and reception of telemetry data like logs, metrics, and traces. OTLP ensures data consistency and interoperability across different systems: + +![/images/opentelemetry/otel_otlp.png](/images/opentelemetry/otel_otlp.png) + +
+ +#### Instrumentation Libraries + +OpenTelemetry offers instrumentation libraries for a wide range of programming languages and frameworks. These libraries make it easy for developers to instrument their applications, collecting telemetry data in the OpenTelemetry format and seamlessly sending it to the desired backend using the OTLP protocol: + +![/images/opentelemetry/otel_sdk.png](/images/opentelemetry/otel_sdk.png) + +While instrumentation libraries are essential for collecting telemetry data within applications, a telemetry agent is needed to process and forward this data to the desired backend: Fluent Bit. + +## Fluent Bit + +In the OpenTelemetry ecosystem Fluent Bit is a high-performance, vendor-agnostic telemetry agent that fully supports the OpenTelemetry Protocol (OTLP). As an alternative to the OpenTelemetry Collector, Fluent Bit offers robust processing capabilities, efficient resource utilization, and seamless integration with various backends. + +![/images/opentelemetry/otel_fluentbit.png](/images/opentelemetry/otel_fluentbit.png) + +
+ +### Why Choose Fluent Bit for OpenTelemetry Collector ? + +
+ +- **High Performance**: Designed for high throughput and low resource usage, Fluent Bit can handle large volumes of data with minimal overhead. + +- **Extensive Plugin Ecosystem**: With a rich set of input, processors and output plugins, Fluent Bit can easily integrate with diverse data sources and destinations. + +- **Flexible and Scalable**: Fluent Bit is lightweight and can be deployed in various environments, from edge devices to cloud infrastructures. + +
+ +### Getting Started with Fluent Bit and OpenTelemetry + +To get started with Fluent Bit and OpenTelemetry, follow these steps: + +1. **Install Fluent Bit**: Follow the [installation guide](https://docs.fluentbit.io/manual/installation) for your platform. + +2. **Configure OpenTelemetry**: Set up Fluent Bit to use the OpenTelemetry Protocol by configuring the appropriate [input (source)](https://docs.fluentbit.io/manual/pipeline/inputs/opentelemetry) and [output (destination)](https://docs.fluentbit.io/manual/pipeline/outputs/opentelemetry) plugins. + +3. **Run Fluent Bit**: Start Fluent Bit and verify that it's collecting and forwarding telemetry data as expected. + +For detailed configuration instructions, visit the [Fluent Bit documentation](https://docs.fluentbit.io/manual/pipeline/outputs/opentelemetry). + +### Community and Support + +Join our community to stay updated, share your experiences, and get support from other Fluent Bit users: + +- **[Slack](https://launchpass.com/fluent-all)**: Engage with experts and other community members. +- **[Twitter](https://twitter.com/fluentbit)**: Follow us for the latest news and announcements. +- **[Newsletter](https://www.fluentd.org/newsletter)**: Sign up for updates on product information, events, and contributions. + +--- + diff --git a/content/posts/_index.md b/content/posts/_index.md index 61f4ad4..da1046c 100644 --- a/content/posts/_index.md +++ b/content/posts/_index.md @@ -1,7 +1,7 @@ --- title: 'News & insights' description: 'Explore the latest news from Fluent Bit.
Research analysis, and perspectives for our latest projects' -date: 2021-02-10 +# date: 2021-02-10 url: 'blog' herobg: "/images/hero@2x.jpg" diff --git a/content/posts/avoiding-data-loss-and-backpressure-problems-with-fluent-bit.md b/content/posts/avoiding-data-loss-and-backpressure-problems-with-fluent-bit.md new file mode 100644 index 0000000..af81583 --- /dev/null +++ b/content/posts/avoiding-data-loss-and-backpressure-problems-with-fluent-bit.md @@ -0,0 +1,311 @@ +--- +title: "A practical guide for avoiding data loss and backpressure problems with +Fluent Bit" +date: "2024-01-29" +description: "A practical guide on how to detect and avoid backpressure problems +with Fluent Bit by balancing memory-based and filesystem-based buffering." +image: "/images/blog/1701353456-general-fluent-bit-preview-card.png" +author: "Sharad Regoti" +canonicalUrl: "https://chronosphere.io/learn/avoiding-data-loss-and-backpressure-problems-with-fluent-bit/" +herobg: "/images/blog/1689182792-background-fluent-bit.png" +--- +*This post is [republished from the Chronosphere blog](https://chronosphere.io/learn/avoiding-data-loss-and-backpressure-problems-with-fluent-bit/). With [Chronosphere’s acquisition of Calyptia](https://chronosphere.io/news/chronosphere-acquires-calyptia/) in 2024, Chronosphere became the [primary corporate sponsor of Fluent Bit](https://chronosphere.io/fluent-bit/). Eduardo Silva — the original creator of Fluent Bit and co-founder of Calyptia — leads a team of Chronosphere engineers dedicated full-time to the project, ensuring its continuous development and improvement.* + +## Introduction + +Fluent Bit is a widely used open-source data collection agent, processor, and forwarder +that enables you to collect logs, metrics, and traces from various sources, filter and +transform them, and then forward them to multiple destinations. With over +[ten billion Docker](https://chronosphere.io/learn/fluent-bit-surpasses-10-billion-docker-pulls/) +pulls, Fluent Bit has established itself as a preferred choice for log processing, +collecting, and shipping. + +At its core, Fluent Bit is a simple data pipeline consisting of various stages, +as depicted below. + +![Illustration of a Fluent Bit pipeline](/images/blog/1706536426-fluent-bit-pipeline.png)Most data pipelines eventually suffer from backpressure, which is when data is ingested at a higher rate than the ability to flush it. Backpressure causes problems such as high memory usage, service downtime, and data loss. Network failures, latency, or third-party service failures are common scenarios where backpressure occurs. + +Fluent Bit offers special strategies to deal with [backpressure](https://www.notion.so/manual/administration/backpressure) to help ensure data safety and reduce downtime. Recognizing when Fluent Bit is experiencing backpressure and knowing how to address it is crucial for maintaining a healthy data pipeline. + +This post provides a practical guide on how to detect and avoid backpressure +problems with Fluent Bit. + +## Prerequisites + +* **Docker**: Installed on your local machine. +* **Familiarity with Fluent Bit concepts:** Such as, inputs, outputs, parsers, +and filters. If you’re not familiar with these concepts, please refer to the +[official documentation](https://docs.fluentbit.io/manual/concepts/data-pipeline). + +## The default: memory-based buffering + +In Fluent Bit, [records](https://docs.fluentbit.io/manual/concepts/key-concepts#event-or-record) +are emitted by an [input plugin](https://docs.fluentbit.io/manual/concepts/data-pipeline/input). +These records are then grouped together by the engine into a unit called a **Chunk**, +which typically has a size of around 2MB. Based on the configuration, the engine determines +where to store these **Chunks**. By default, all **Chunks** are created in memory. + +With the default mechanism, Fluent Bit will store data in memory as much as +possible. This is the fastest mechanism with the least system overhead, but in +certain scenarios, data can be ingested at a higher rate than the ability to +flush it to some destinations. This generates backpressure, leading to high +memory consumption in the service. + +Network failures, latency, or unresponsive third-party services are common +scenarios where we cannot deliver data fast enough as we receive new data to +process, which leads to backpressure. + +In a high-load environment with backpressure, there's a risk of increased memory +usage, which leads to the termination of the Fluent Bit process by the Kernel. + +Let’s see a demo where Fluent Bit is running in a constrained environment, which +causes backpressure, leading to getting killed by **Kernel (OOM)**. + +### **Instructions** + +**Clone the Samples Git Repository:** + +This repository contains all the required configuration files. Use the command +below to clone the repository. + + +```bash +git clone git@github.com:calyptia/blog-posts.git +cd blog-posts +``` + +**Evaluating Default RAM Consumption:** + +```bash +docker run -v $(pwd)/fluent-bit-empty.conf:/fluent-bit/etc/fluent-bit.conf:ro -ti cr.fluentbit.io/fluent/fluent-bit:2.2 +``` +The above command runs Fluent Bit in a docker container with an empty +configuration file; this lets us evaluate what the default memory consumption +and memory limits for the container. + +Use the `docker ps` command to get the name or container ID of the newly created +container and use that value in the below command to get its stats. + + +```bash +docker stats +``` + +The image below shows that by default, the container consumes 10MB of RAM and +can take up to 8GB of system RAM. + +![screen capture of console showing default memory settings for container](/images/blog/1706539594-blog-backpressure-container-default.png)**Fluent Bit Configuration** + + +```yaml +[INPUT] + Name dummy + copies 1500 + dummy {"host":"31.163.219.152"...} # A large JSON object, refer git repository + tag dummy-a + +[OUTPUT] + name stdout + match * +``` + +The above configuration uses the [dummy](https://docs.fluentbit.io/manual/pipeline/inputs/dummy) +input plugin to generate large amounts of data for test purposes. We have configured it +to generate 1500 records per second. This data is then printed to the console using the +[stdout](https://docs.fluentbit.io/manual/pipeline/outputs/standard-output) output plugin. + +**Simulating Backpressure** + +To simulate OOM kill behavior caused due to backpressure, we will generate data +at a higher rate while restricting the container RAM to just 20MB. With this +configuration as Fluent Bit tries to buffer more data in memory it will +eventually hit the imposed RAM limit of the container and the service will +crash. + +Execute the below command and observe the result. + + +```bash +docker run --memory 20MB -v $(pwd)/fluent-bit-oom.conf:/fluent-bit/etc/fluent-bit.conf:ro -ti cr.fluentbit.io/fluent/fluent-bit:2.2 +``` + +After a few seconds, the container will stop automatically. Once it stops, grab +the container ID using `docker ps -a` and inspect the container—you should +observe that it was killed due to a container `Out Of Memory` error. + +![Screen shot of command line showing contain memory limit](/images/blog/1706537578-blog-backpressure-memory-limit.png) +![screen shot of command line showing the container was killed due to out of memory error](/images/blog/1706537718-blog-backpressure-container-killed.png)This demonstrates how backpressure in Fluent Bit leads to increased memory usage, resulting in the Kernel terminating the application upon reaching memory limits, which causes downtime and data loss. + +If your Fluent Bit process is continuously getting killed, it is likely an +indication that Fluent Bit is experiencing backpressure. In the following +section, we'll explore a solution to this problem. + +## A quick fix: limiting memory-based buffering + +A workaround for backpressure scenarios like the above is to limit the amount of +memory in records that an input plugin can register. This can be configured +using the `mem_buf_limit` property. If a plugin has enqueued more than the +`mem_buf_limit`, it won't be able to ingest more until that buffered data is +delivered. + +When the set limit is reached, the specific input plugin gets paused, halting +record ingestion until resumed, which inevitably leads to data loss. + +When an input plugin is paused, Fluent Bit logs the information on the console +with an example shown below: + + +```bash +[input] tail.1 paused (mem buf overlimit) +[input] tail.1 resume (mem buf overlimit) +``` + +The workaround of `mem_buf_limit` is good for certain scenarios and +environments, it helps to control the memory usage of the service, but at the +cost of data loss. This can happen with any input plugin. + +The goal of `mem_buf_limit` is memory control and survival of the service. Let’s +see what happens when we modify our Fluent Bit configuration by adding the +`mem_buf_limit` property to our input plugin. + + +```yaml +[INPUT] + Name dummy + copies 1500 + dummy {"host":"31.163.219.152"...} # A large JSON object, refer git repository + tag dummy-a + Mem_Buf_Limit 10MB + +[OUTPUT] + name stdout + match * +``` + +We've set a 20MB memory limit for the container. With Fluent Bit using 10MB in +its default configuration, we allocated an additional 10MB as mem\_buf\_limit. + +**Note:** The combined memory limit assigned to each input plugin, must be lower +than the resource restriction placed on the container. + +Execute the below command and observe the result. + + +```bash +docker run --memory 20MB -v $(pwd)/fluent-bit-memory-limit.conf:/fluent-bit/etc/fluent-bit.conf:ro -ti cr.fluentbit.io/fluent/fluent-bit:2.2 +``` + +Unlike the previous scenario, the container is not killed and continues to emit +dummy records on the console. Grab the container ID using `docker ps` and +execute the below command. + + +```bash +docker logs | grep -i "pausing\\|resume" +``` + +![screen shot of console showing impact of backpressure](/images/blog/1706538342-blog-backpressure-example.png)The above image indicates that as Fluent Bit reaches the 10MB buffer limit of the input plugin, **it pauses ingesting new records, potentially leading to data loss, but this pause prevents the service from getting terminated due to high memory usage**. Upon buffer clearance, the ingestion of new records resumes. + +If you are observing the above logs in Fluent Bit, it is a sign of Fluent Bit hitting the +configured memory limits at input plugins due to backpressure. Check out this blog post +on [how to configure alerts from logs using Fluent Bit](https://chronosphere.io/learn/fluent-bit-alerting-slack/). + +In the upcoming section, we will see how to achieve both data safety and memory +safety. + +## A permanent fix: filesystem-based buffering + +[`Filesystem buffering`](https://docs.fluentbit.io/manual/administration/buffering-and-storage#filesystem-buffering-to-the-rescue) provides control over backpressure and can help guarantee data safety. Memory and filesystem buffering approaches are **not** mutually exclusive. When filesystem buffering is enabled for your input plugin, you are getting the best of both worlds: performance and data safety. + +When Filesystem buffering is enabled, the behavior of the engine is different. Upon +Chunk creation, the engine stores the content in memory and also maps a copy on disk +(through [`mmap(2)`](https://man7.org/linux/man-pages/man2/mmap.2.html)`).` The newly +created Chunk is (1) active in memory, (2) backed up on disk. + +**How does the Filesystem buffering mechanism deal with high memory usage and +backpressure?** Fluent Bit controls the number of Chunks that are `up` in +memory. By default, the engine allows us to have 128 Chunks `up` in memory, this +value is controlled by service property `storage.max_chunks_up`. The active +Chunks that are up are ready for delivery. + +Any other remaining Chunk is in a `down` state, which means that it is only in +the filesystem and won't be up in memory unless it is ready to be delivered. +Remember, chunks are never much larger than 2 MB, thus the default +`storage.max_chunks_up` value of 128, each input is limited to roughly 256 MB of +memory. + +If the input plugin has enabled `storage.type` as `filesystem`, when reaching +the `storage.max_chunks_up` threshold, instead of the plugin being paused, all +new data will go to Chunks that are `down` in the filesystem. This allows us to +control the memory usage of the service and also provides a guarantee that the +service won't lose any data. + +Let’s modify our Fluent Bit configuration by enabling filesystem buffering. + + +```yaml +[SERVICE] + flush 1 + log_Level info + storage.path /var/log/flb-storage/ + storage.sync normal + storage.checksum off + storage.storage.max_chunks_up 5 + +[INPUT] + Name dummy + copies 1500 + dummy {"host":"31.163.219.152"...} # A large JSON object, refer git repository + tag dummy-a + storage.type filesystem + +[OUTPUT] + name stdout + match * +``` + +In the above configuration, we have added `storage.type` as `filesystem` in our +input plugin and a `[SERVICE]` block to configure +`storage.storage.max_chunks_up` to `5` (~10MB) + +Execute the below command and observe the result. + + +```bash +docker run --memory 20MB -v $(pwd)/fluent-bit-filesystem.conf:/fluent-bit/etc/fluent-bit.conf:ro -ti cr.fluentbit.io/fluent/fluent-bit:2.2 +``` + +Unlike the default scenario, the container does not crash and continues to emit +dummy records on the console. When the `storage.storage.max_chunks_up` limit is +reached, the chunks are backed up in the filesystem and delivered once the +memory is free. + +**Note:** While file system-based buffering helps prevent container crashes due +to backpressure, it introduces new considerations related to filesystem limits. +It's important to know that just as memory can be exhausted, so can filesystem +storage. When opting for filesystem-based buffering, it is essential to +incorporate a plan that addresses potential filesystem-related challenges. + +## Conclusion + +This guide has explored effective strategies to manage backpressure and prevent +data loss in Fluent Bit. We've highlighted the limitations of default +memory-based buffering and how `Mem_Buf_Limit` is a quick fix to balance memory +usage. The ultimate solution, filesystem-based buffering, offers a comprehensive +approach, ensuring data safety and efficient memory management. These techniques +are essential for optimizing Fluent Bit in high-throughput environments, +ensuring robust and reliable log processing. + +![Meme: Thanos perfectly balanced as all things should be](/images/blog/1706539804-blog-backpressure-meme.jpg)For more information on this topic, refer to the documentation below + +* [Administrating Backpressure In Fluent Bit](https://docs.fluentbit.io/manual/administration/backpressure) +* [How to configure buffering & storage in Fluent Bit](https://docs.fluentbit.io/manual/administration/buffering-and-storage) + +## You may also be interested in + +To continue expanding your Fluent Bit knowledge, check out [Fluent Bit Academy](https://chronosphere.io/fluent-bit-academy/). +It’s filled with on-demand videos guiding you through all things Fluent Bit — best +practices and how-to’s on advanced processing rules, routing to multiple destinations, +and much more. + +![Fluent Bit Academy](/images/blog/1702491671-fluent-bit-academy.png) \ No newline at end of file diff --git a/content/posts/common-architecture-patterns-with-fluentd.md b/content/posts/common-architecture-patterns-with-fluentd.md index 2fb4427..5fa0d8c 100644 --- a/content/posts/common-architecture-patterns-with-fluentd.md +++ b/content/posts/common-architecture-patterns-with-fluentd.md @@ -1,5 +1,5 @@ --- -title: '3 Common Architecture Patterns with Fluentd and Fluent Bit' +title: 'Common Architecture Patterns with Fluentd and Fluent Bit' date: "2020-12-03" description: "Learn common ways to deploy Fluent Bit and Fluentd. Including forwarder-aggregator, side-car/agent, and network device aggregator pattern" image: "/images/3 Common Architecture Patterns with Fluentd and Fluent Bit.jpg" diff --git a/content/posts/convert-logs-to-metrics-in-legacy-applications-using-fluent-bit.md b/content/posts/convert-logs-to-metrics-in-legacy-applications-using-fluent-bit.md new file mode 100644 index 0000000..3acc7ee --- /dev/null +++ b/content/posts/convert-logs-to-metrics-in-legacy-applications-using-fluent-bit.md @@ -0,0 +1,404 @@ +--- +title: "Convert Logs To Metrics in Legacy Applications Using Fluent Bit" +date: "2023-11-13" +description: "Learn how Fluent Bit can derive metrics from application logs, helping to bridge the gap between limited telemetry sources and comprehensive observability." +image: "/images/blog/1699633863-logs-to-metrics-social.png" +author: "Sharad Regoti" +canonicalUrl: "https://chronosphere.io/learn/convert-logs-to-metrics-using-fluent-bit/" +herobg: "/images/blog/1689182792-background-fluent-bit.png" +--- +*This post is [republished from the Chronosphere blog](https://chronosphere.io/learn/convert-logs-to-metrics-using-fluent-bit/). With [Chronosphere’s acquisition of Calyptia](https://chronosphere.io/news/chronosphere-acquires-calyptia/) in 2024, Chronosphere became the [primary corporate sponsor of Fluent Bit](https://chronosphere.io/fluent-bit/). Eduardo Silva — the original creator of Fluent Bit and co-founder of Calyptia — leads a team of Chronosphere engineers dedicated full-time to the project, ensuring its continuous development and improvement.* + +In software development, observability allows us to understand a system from the outside, by asking questions about the system without knowing its inner workings. Furthermore, it allows us to easily troubleshoot and helps answer the question, “Why is this happening?” + +To be able to ask those questions to a system, the application must be instrumented. That is, the application code must emit signals such as traces, metrics, and logs. + +Most modern applications are instrumented to output telemetry signals. However, at times you may find yourself working with systems that may not provide all the telemetry data you need. This is often the case when working with legacy applications, which may offer only a subset of the essential telemetry signals and whose codebase cannot be instrumented to provide more. + +In such challenging situations, innovative solutions become imperative. You may have to derive telemetry data from other means.  + +This is where Fluent Bit comes into play. Fluent Bit is a widely used open-source data collection agent, processor, and forwarder that enables you to collect logs, metrics, and traces from various sources, filter and transform them, and then forward them to multiple destinations. It also addresses the aforementioned issues ingeniously. + +In this blog , we will demonstrate how Fluent Bit can  derive metrics from application logs, helping to bridge the gap between limited telemetry sources and comprehensive system observability. + +## **Use Case: Getting Metrics from Legacy Applications** + +Consider a legacy application that logs HTTP requests and their corresponding status codes during processing. Here is an example log: + + +```json +{ + "host": "31.163.219.152", + "user-identifier": "-", + "datetime": "31/Oct/2023:04:30:04 +0000", + "method": "GET", + "request": "/optimize/exploit/architect", + "protocol": "HTTP/2.0", + "status": 201, + "bytes": 21985, + "referer": "" +} +``` + +The challenge is to identify the **HTTP Request Rate** and **Error Rate.** However, instrumenting the application to generate metrics is not feasible, as it was developed using an outdated programming language version, and the existing instrumentation libraries do not support this older version. + +However, the application log already contains status and request information. We can extract this data and convert it into metrics. + +## **Prerequisites** + +* **Kubernetes Cluster:** We will deploy Fluent Bit in a Kubernetes cluster. We will be using an EKS cluster, but any cluster will suffice. +* **Kubectl and Helm CLI:** Installed on your local machine. +* **Familiarity with Fluent Bit concepts:** Such as, inputs, outputs, parsers, and filters. If you’re not familiar with these concepts, please refer to the [official documentation](https://docs.fluentbit.io/manual/concepts/data-pipeline). + +## **Deploying a Sample Legacy Application** + +Let’s deploy our legacy application into Kubernetes. For demonstration purposes, we’ll use [a fake log generator](https://hub.docker.com/r/mingrammer/flog).  + +### **Instructions:** + +1) Save the below deployment configuration in a file called `application.yaml`. + + + +```yaml + +apiVersion: apps/v1 +kind: Deployment +metadata: + labels: + app: legacy-application + name: legacy-application + namespace: default +spec: + replicas: 1 + selector: + matchLabels: + app: legacy-application + template: + metadata: + labels: + app: legacy-application + spec: + containers: + - image: mingrammer/flog:0.4.3 + name: legacy-application + args: ["--loop", "--delay", "2", "--format", "json"] +``` + +2) Now, apply the configuration using the below command: + + +```bash +kubectl apply -f application.yaml +``` + +3) View the Application Logs + + +```bash +kubectl logs deploy/legacy-application +``` + +![Screenshots of log data in terminal](/images/blog/1699628377-blog_logs_to_metrics_image2.png)## **Converting Logs To Metrics** + +Metrics are stored in a [Time Series Database](https://en.wikipedia.org/wiki/Time_series_database) such as Prometheus, InfluxDB, VictoriaMetrics etc. These databases require metrics to be stored in particular data types such as counter, gauge and histogram. Converting raw logs into structured metrics involves a systematic approach, the below rules give a brief outline. + +### **Rules for Converting Logs to Metrics** + +1. **List the Required Metrics:** + +As discussed above, We want two metrics HTTP request rate and HTTP error rate. +2. **Identify Corresponding Log Values:** + +From application logs, the request field contains the endpoint path for HTTP requests which can be mapped to the HTTP request rate metric while the status field indicates the HTTP status code. It ranges from 200 to 599. We are particularly interested in monitoring status codes greater than 500, indicating server errors that can be mapped to the HTTP error rate. +3. **Determine the Metric Type:** +* **HTTP Request Rate (e.g., Requests Per Second):** + + + + **Appropriate Metric Type:** Counter + + **Explanation:** Counters are ideal for measuring rates, such as the total number of HTTP requests served. To calculate the request rate, take the derivative of the counter value over time. This yields metrics like requests per second (RPS) or requests per minute (RPM), based on our preference. +* **Error Rate (Based on HTTP Status Codes):** + + + + **Appropriate Metric Type:** Counter or Gauge + + **Explanation:** For error rates derived from HTTP status codes, counters are commonly used. You can create counters like `http_errors_total{status="4xx"}` to tally 4xx client errors or `http_errors_total{status="5xx"}` for 5xx server errors. Counters are suitable because they steadily increase, providing a cumulative count of errors over time. + +With the prerequisites for metrics now elucidated, let’s implement it in practice. + +![Flow cart showing how to convert logs to metrics](/images/blog/1699629283-blog_logs_to_metrics_image4.png)Converting Logs to Metrics with Fluent BitHere’s the Fluent Bit configuration that enables the log processing pipeline depicted above: + + +```yaml +config: + customParsers: | + [PARSER] + Name crijsonparser + Format regex + Regex ^(?