Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 7 additions & 6 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,21 +38,21 @@ go mod vendor

Single-binary Go application in `app/` package (uses `package main`, not a library).

**Control flow**: `main()` → `do()` → `runEventLoop()` which listens on `EventNotif.Channel()` for container events and creates/destroys `LogStreamer` instances.
**Control flow**: `main()` → `do()` → `runEventLoop()` which listens on `EventNotif.Channel()` for container events and creates/destroys `LogStreamer` instances. Exits on context cancellation or when events channel is closed (e.g., docker event listener failure). Uses `closeStreamer` helper for resource cleanup.

### Packages

- **`app/`** — entry point, CLI options (`go-flags`), event loop, log writer factory (`makeLogWriters`). Wires discovery events to log streamers. Creates `MultiWriter` combining file (lumberjack) and syslog destinations.
- **`app/discovery/`** — `EventNotif` watches Docker daemon for container start/stop events via `go-dockerclient`. Emits `Event` structs on a channel. Handles include/exclude filtering by name lists and regex patterns. Extracts group name from image path.
- **`app/`** — entry point, CLI options (`go-flags`), event loop, log writer factory (`makeLogWriters` returns `logWriter, errWriter, error`). Wires discovery events to log streamers. Creates `MultiWriter` combining file (lumberjack) and syslog destinations. Uses `writeNopCloser` wrapper to prevent double-close when syslog writer is shared between log and err MultiWriters.
- **`app/discovery/`** — `EventNotif` watches Docker daemon for container start/stop events via `go-dockerclient`. `NewEventNotif(client, EventNotifOpts)` accepts filtering options via struct. Emits `Event` structs on a channel; closes the channel on listener failure. Handles include/exclude filtering by name lists and regex patterns. Extracts group name from image path.
- **`app/logger/`** — `LogStreamer` attaches to a container's log stream (blocking `Logs()` call in a goroutine). `MultiWriter` fans out writes to multiple `io.WriteCloser` destinations, optionally wrapping in JSON envelope.
- **`app/syslog/`** — platform-specific syslog writer. Build-tagged: real implementation on unix, stub on windows.

### Key Patterns

- **Docker client interface**: `discovery.DockerClient` and `logger.LogClient` are consumer-side interfaces wrapping `go-dockerclient`. Mocks generated with `moq` into `mocks/` subdirectories.
- **LogStreamer lifecycle**: `Go(ctx)` starts streaming in a goroutine, `Close()` cancels context and waits, `Wait()` blocks on `ctx.Done()`. Has retry logic for Docker EOF errors.
- **LogStreamer lifecycle**: `Go(ctx)` starts streaming in a goroutine, `Close()` cancels context and waits, `Wait()` blocks on `ctx.Done()`, `Err()` retrieves the error (if any) after `Wait()` returns. Has retry logic for Docker EOF errors.
- **MultiWriter**: ignores individual write errors unless all writers fail. `Close()` collects errors via `go-multierror`.
- **Container filtering**: supports name lists (`--exclude`/`--include`) and regex patterns (`--exclude-pattern`/`--include-pattern`), mutually exclusive within each group.
- **Container filtering**: supports name lists (`--exclude`/`--include`) and regex patterns (`--exclude-pattern`/`--include-pattern`), mutually exclusive within each group and across groups (e.g., `--include` + `--exclude-pattern` is also invalid).

## Dependencies

Expand All @@ -68,5 +68,6 @@ Single-binary Go application in `app/` package (uses `package main`, not a libra

- Tests use `moq`-generated mocks in `mocks/` subdirectories
- Most `app/` tests use mocks; no live Docker needed. `Test_Do` requires a live Docker daemon but is skipped unless `TEST_DOCKER` env var is set
- Channel-based synchronization preferred over `time.Sleep` for race-free tests
- Channel-based synchronization preferred over `time.Sleep` for race-free tests; use `require.Eventually` with condition checks for async operations
- Uses `t.TempDir()` for temporary files and `t.Context()` for test contexts
- Sentinel event technique: send a known event after the one being tested, wait for it via `require.Eventually` to confirm both events were processed
11 changes: 8 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
the `-t` option and configured with a logging driver that works with docker logs (journald and json-file).
It can forward both stdout and stderr of containers to local, rotated files and/or to remote syslog.

_note: [dkll](https://github.com/umputun/dkll) inlcudes all functionality of docker-logger, but adds server and cli client_
_note: [dkll](https://github.com/umputun/dkll) includes all functionality of docker-logger, but adds server and cli client_

## Install

Expand All @@ -30,13 +30,18 @@ All changes can be done via container's environment in `docker-compose.yml` or w
| `--include-pattern` | `INCLUDE_PATTERN` | | only include container names matching a regex |
| `--exclude-pattern` | `EXCLUDE_PATTERN` | | only exclude container names matching a regex |
| | `TIME_ZONE` | UTC | time zone for container |
| `--loc` | `LOG_FILES_LOC` | logs | log files location |
| `--syslog-prefix` | `SYSLOG_PREFIX` | docker/ | syslog prefix |
| `--json`, `-j` | `JSON` | false | output formatted as JSON |
| `--dbg` | `DEBUG` | false | debug mode |


- at least one of destinations (`files` or `syslog`) should be allowed
- location of log files can be mapped to host via `volume`, ex: `- ./logs:/srv/logs` (see `docker-compose.yml`)
- both `--exclude` and `--include` flags are optional and mutually exclusive, i.e. if `--exclude` defined `--include` not allowed, and vise versa.
- both `--include` and `--include-pattern` flags are optional and mutually exclusive, i.e. if `--include` defined `--include-pattern` not allowed, and vise versa.
- both `--exclude` and `--include` flags are optional and mutually exclusive, i.e. if `--exclude` defined `--include` not allowed, and vice versa.
- both `--include` and `--include-pattern` flags are optional and mutually exclusive, i.e. if `--include` defined `--include-pattern` not allowed, and vice versa.
- both `--exclude` and `--exclude-pattern` flags are optional and mutually exclusive, i.e. if `--exclude` defined `--exclude-pattern` not allowed, and vice versa.
- cross-kind combinations are also mutually exclusive: `--include` + `--exclude-pattern`, `--include-pattern` + `--exclude`, and `--include-pattern` + `--exclude-pattern` are not allowed.

## Build from the source

Expand Down
56 changes: 43 additions & 13 deletions app/discovery/events.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ type EventNotif struct {
includesRegexp *regexp.Regexp
excludesRegexp *regexp.Regexp
eventsCh chan Event
listenerErr chan error // communicates activate() failure back to the caller
}

// Event is simplified docker.APIEvents for containers only, exposed to caller
Expand All @@ -40,35 +41,44 @@ type DockerClient interface {

var reGroup = regexp.MustCompile(`/(.*?)/`)

// EventNotifOpts contains options for NewEventNotif
type EventNotifOpts struct {
Excludes []string
Includes []string
IncludesPattern string
ExcludesPattern string
}

// NewEventNotif makes EventNotif publishing all changes to eventsCh
func NewEventNotif(dockerClient DockerClient, excludes, includes []string, includesPattern, excludesPattern string) (*EventNotif, error) {
func NewEventNotif(dockerClient DockerClient, opts EventNotifOpts) (*EventNotif, error) {
log.Printf("[DEBUG] create events notif, excludes: %+v, includes: %+v, includesPattern: %+v, excludesPattern: %+v",
excludes, includes, includesPattern, excludesPattern)
opts.Excludes, opts.Includes, opts.IncludesPattern, opts.ExcludesPattern)

var err error
var includesRe *regexp.Regexp
if includesPattern != "" {
includesRe, err = regexp.Compile(includesPattern)
if opts.IncludesPattern != "" {
includesRe, err = regexp.Compile(opts.IncludesPattern)
if err != nil {
return nil, errors.Wrap(err, "failed to compile includesPattern")
}
}

var excludesRe *regexp.Regexp
if excludesPattern != "" {
excludesRe, err = regexp.Compile(excludesPattern)
if opts.ExcludesPattern != "" {
excludesRe, err = regexp.Compile(opts.ExcludesPattern)
if err != nil {
return nil, errors.Wrap(err, "failed to compile excludesPattern")
}
}

res := EventNotif{
dockerClient: dockerClient,
excludes: excludes,
includes: includes,
excludes: opts.Excludes,
includes: opts.Includes,
includesRegexp: includesRe,
excludesRegexp: excludesRe,
eventsCh: make(chan Event, 100),
listenerErr: make(chan error, 1),
}

// first get all currently running containers
Expand All @@ -88,12 +98,22 @@ func (e *EventNotif) Channel() (res <-chan Event) {
return e.eventsCh
}

// Err returns a channel that receives an error if the event listener fails to start.
// the channel is buffered (size 1) and will receive at most one error.
func (e *EventNotif) Err() <-chan error {
return e.listenerErr
}

// activate starts blocking listener for all docker events
// filters everything except "container" type, detects stop/start events and publishes to eventsCh
// filters everything except "container" type, detects stop/start events and publishes to eventsCh.
// on failure or channel close, it closes eventsCh to signal consumers.
func (e *EventNotif) activate(client DockerClient) {
dockerEventsCh := make(chan *docker.APIEvents)
if err := client.AddEventListener(dockerEventsCh); err != nil {
log.Fatalf("[ERROR] can't add even listener, %v", err)
log.Printf("[ERROR] can't add event listener, %v", err)
e.listenerErr <- errors.Wrap(err, "can't add event listener")
close(e.eventsCh)
return
}
Comment on lines 110 to 117
Copy link

Copilot AI Feb 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

activate still uses log.Fatalf when AddEventListener fails, which will terminate the whole process from inside the discovery component and bypass caller error handling/cleanup. Since similar paths were changed to return errors elsewhere in this PR, consider propagating this failure back to the caller (e.g., have NewEventNotif/activate return an error or send an error event) instead of exiting.

Copilot uses AI. Check for mistakes.

upStatuses := []string{"start", "restart"}
Expand All @@ -116,17 +136,23 @@ func (e *EventNotif) activate(client DockerClient) {
continue
}

ts := time.Unix(0, dockerEvent.TimeNano)
if dockerEvent.TimeNano == 0 {
ts = time.Unix(dockerEvent.Time, 0)
}

event := Event{
ContainerID: dockerEvent.Actor.ID,
ContainerName: containerName,
Status: slices.Contains(upStatuses, dockerEvent.Status),
TS: time.Unix(dockerEvent.Time/1000, dockerEvent.TimeNano),
TS: ts,
Group: e.group(dockerEvent.From),
}
log.Printf("[INFO] new event %+v", event)
e.eventsCh <- event
}
log.Fatalf("[ERROR] event listener failed")
log.Printf("[WARN] event listener closed")
close(e.eventsCh)
}

// emitRunningContainers gets all currently running containers and publishes them as "Status=true" (started) events
Expand All @@ -138,6 +164,10 @@ func (e *EventNotif) emitRunningContainers() error {
log.Printf("[DEBUG] total containers = %d", len(containers))

for _, c := range containers {
if len(c.Names) == 0 {
log.Printf("[WARN] container %s has no names, skipped", c.ID)
continue
}
containerName := strings.TrimPrefix(c.Names[0], "/")
if !e.isAllowed(containerName) {
log.Printf("[INFO] container %s excluded", containerName)
Expand All @@ -147,7 +177,7 @@ func (e *EventNotif) emitRunningContainers() error {
Status: true,
ContainerName: containerName,
ContainerID: c.ID,
TS: time.Unix(c.Created/1000, 0),
TS: time.Unix(c.Created, 0),
Group: e.group(c.Image),
}
log.Printf("[DEBUG] running container added, %+v", event)
Expand Down
Loading