Example creates hanging container

[bluekitedreamer]

Describe the bug

The docker container web in the docker-compose example will zombie/hang and cause strange docker engine issues because of the dependency on the logging driver not existing if the stack decides to stop the fluentd container first.

If the fluentd container gets removed before the web container, the web containers gets zombified and can be stopped/killed by normal docker commands.

If anyone gets caught in this mess the easiest solution is to do the following which is generally not recommended but works:. Please note the last command will prune ALL docker containers, select the web containers image specifically to have other containers unaffected.

systemctl restart docker.socket docker.service
docker rm -f <web container name>
docker container prune

I've previously not been very fond of docker's logging driver functionality, but I'm coming back around and giving it another try. If the logging driver doesn't exist the containers seem to hang. Is this typical? Is there something I'm missing?

Link to the problematic documentation

https://docs.fluentd.org/container-deployment/docker-compose

Expected explanation

compose up/down removes the stack properly

Additional context

No response

[bluekitedreamer]

This ended up being something that was highly customized in my environment.

Never the less, below are updated configs for using the latest and greatest versions of EFK. All versions compatibilities provided are confirmed working. There are some minor different like volume mappings from the original documentation to my environment. Also the web container was switched to nginx.

Dockerfile

FROM fluent/fluentd:v1.16.2-debian-1.1
USER root
RUN ["gem", "install", "elasticsearch", "--no-document", "--version", "8.12.2"]
RUN ["gem", "install", "fluent-plugin-elasticsearch", "--no-document", "--version", "5.4.3"]
USER fluent

fluent.conf

# fluentd/conf/fluent.conf
<source>
  @type forward
  port 24224
  bind 0.0.0.0
</source>
<match *.**>
  @type copy
  <store>
    @type elasticsearch
    host elasticsearch
    scheme http
    port 9200
    logstash_format true
    logstash_prefix fluentd
    logstash_dateformat %Y%m%d
    include_tag_key true
    #type_name access_log
    tag_key @log_name
    flush_interval 1s
  </store>
  <store>
    @type stdout
  </store>
</match>

elasticsearch.yml

cluster.name: "<your cluster name>"
network.host: 0.0.0.0

docker-compose.yaml

version: "3"
services:
  web:
    depends_on: 
      - fluentd
    image: nginx
    ports:
      - "80:80"
    links:
      - fluentd
    logging:
      driver: "fluentd"
      options:
        fluentd-address: localhost:24224
        tag: nginx.access
  fluentd:
    restart: unless-stopped
    build: ./fluentd
    volumes:
      - <dir location>/fluentd/conf/fluentd.conf:/fluentd/etc/fluent.conf
      - <dir location>/fluentd/data:/fluentd/log
    links:
      - "elasticsearch"
    ports:
      - "5140:5140"
      - "24224:24224"
      - "24224:24224/udp"
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:8.12.2
    volumes:
      - <dir location>/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
      - <dir location>/elasticsearch/data:/usr/share/elasticsearch/data
    environment:
      - "discovery.type=single-node"
      - xpack.security.enabled=false
    expose:
      - "9200"
    ports:
      - "9200:9200"
  kibana:
    image: docker.elastic.co/kibana/kibana:8.12.2
    links:
      - "elasticsearch"
    ports:
      - "5601:5601"