Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: limit i/o related events #153

Merged
merged 6 commits into from
Dec 13, 2024
Merged

feat: limit i/o related events #153

merged 6 commits into from
Dec 13, 2024

Conversation

qjerome
Copy link
Member

@qjerome qjerome commented Dec 10, 2024

Summary of Changes in This PR

This PR addresses the need to throttle I/O events to prevent event loss, which can disrupt Kunai's eBPF-based correlation mechanisms.

Event loss is problematic because Kunai relies on eBPF events to allocate and free structures used for correlation. Missing crucial events may lead to incomplete correlations or memory leaks.

I/O events, being among the fastest, can easily generate tens of thousands of events per second, quickly saturating the event buffer.

Implemented Filtering Strategies

To mitigate event loss, this PR introduces the following filtering mechanisms:

  • De-duplication of write_close events: Prevents redundant events from overwhelming the buffer.
  • Per-task I/O throttling: Limits I/O events on a per-task basis, ensuring that excessive I/O from one task doesn't affect others.
  • Global I/O throttling: Introduces random throttling of I/O events when they consume excessive buffer space.

These strategies aim to maintain efficient event processing while preserving crucial correlations.

 - ebpf path resolution simplified
 - compute a hash for path during resolution
 - introduce path::MapKey struct to generate a unique key to be used in ebpf HashMap
 - better error handling in ebpf
 - change Path::eq implementation to be more correct
 - small changes in shim to access qstr hash and len members from struct
@qjerome qjerome merged commit 5291274 into main Dec 13, 2024
2 checks passed
@qjerome qjerome deleted the feat-limit-io branch December 13, 2024 08:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant