Skip to content

internal/appsec: rework appsec telemetry #3345

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Apr 10, 2025
Merged

Conversation

eliottness
Copy link
Contributor

What does this PR do?

  • Fix truncation metrics count that were not using a bitfield before
  • Add more tags to the telemetry rasp.timeout metric.
  • Change the RASPRuleType type from string to uint8 to make arrays of static size and use them as a dictionnary
  • in waf.HandleMetrics replace a lot of makes with keys being RASPRuleType with an array
  • Order metrics by their frequency of access and using this, replace some of the map generated ahead of time by lazily generated maps using xsync.MapOf
  • Rework the appsec start telemetry by simplyfing it
  • Remove the config.Origin type and replace it by the more general purpose telemetry.Origin (That I am thinking of promoting to the package internal/globalconfig 🤔 )

Motivation

Reviewer's Checklist

  • Changed code has unit tests for its functionality at or near 100% coverage.
  • System-Tests covering this feature have been added and enabled with the va.b.c-dev version tag.
  • There is a benchmark for any new code, or changes to existing code.
  • If this interacts with the agent in a new way, a system test has been added.
  • Add an appropriate team label so this PR gets put in the right place for the release notes.
  • Non-trivial go.mod changes, e.g. adding new modules, are reviewed by @DataDog/dd-trace-go-guild.
  • For internal contributors, a matching PR should be created to the v2-dev branch and reviewed by @DataDog/apm-go.

Unsure? Have a question? Request a review!

@datadog-datadog-prod-us1
Copy link

datadog-datadog-prod-us1 bot commented Mar 25, 2025

Datadog Report

Branch report: eliottness/more-lazy-metrics
Commit report: 5fd8be2
Test service: dd-trace-go

✅ 0 Failed, 4472 Passed, 65 Skipped, 3m 37.58s Total Time

@pr-commenter
Copy link

pr-commenter bot commented Mar 25, 2025

Benchmarks

Benchmark execution time: 2025-04-10 09:03:21

Comparing candidate commit 83a543f in PR branch eliottness/more-lazy-metrics with baseline commit ebe8cfe in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 55 metrics, 1 unstable metrics.

@eliottness eliottness force-pushed the eliottness/more-lazy-metrics branch 3 times, most recently from 5c6ec98 to 57ea557 Compare April 2, 2025 13:15
Signed-off-by: Eliott Bouhana <[email protected]>
Signed-off-by: Eliott Bouhana <[email protected]>
@eliottness eliottness changed the base branch from v2-dev to main April 9, 2025 13:28
@eliottness eliottness force-pushed the eliottness/more-lazy-metrics branch from 57ea557 to a3b5339 Compare April 9, 2025 13:29
@eliottness eliottness marked this pull request as ready for review April 9, 2025 14:41
@eliottness eliottness requested review from a team as code owners April 9, 2025 14:41
RASPRuleTypeSSRF RASPRuleType = "ssrf"
RASPRuleTypeSQLI RASPRuleType = "sql_injection"
RASPRuleTypeCMDI RASPRuleType = "command_injection"
RASPRuleTypeLFI RASPRuleType = iota
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggest making the 0-value invalid:

Suggested change
RASPRuleTypeLFI RASPRuleType = iota
_ RASPRuleType = iota
RASPRuleTypeLFI

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would make my for loops wrong because I loop on them like a dictionnary but they are arrays 😭

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Like I am doing here with explicit index array placement
image

}

// RASPRuleTypeFromAddressSet returns the RASPRuleType for the given address set if it has a RASP address.
func RASPRuleTypeFromAddressSet(addressSet waf.RunAddressData) (RASPRuleType, bool) {
if addressSet.Scope != waf.RASPScope {
return "", false
return math.MaxUint8, false
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the 0-value is invalid, you could return 0 instead, which looks less weird?

@@ -46,5 +60,5 @@ func RASPRuleTypeFromAddressSet(addressSet waf.RunAddressData) (RASPRuleType, bo
}
}

return "", false
return math.MaxUint8, false
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the 0-value is invalid, you could return 0 instead, which looks less weird?

@@ -24,6 +24,8 @@ import (
"github.com/DataDog/appsec-internal-go/apisec"
"github.com/stretchr/testify/assert"

"github.com/stretchr/testify/mock"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moving this seems unrelated... And the blank spacing there is odd... I'd normalize it or not touch it...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was indeed wrong regarding import order so I fixed it even more

@eliottness eliottness enabled auto-merge (squash) April 10, 2025 15:22
@eliottness eliottness merged commit 4539024 into main Apr 10, 2025
199 of 201 checks passed
@eliottness eliottness deleted the eliottness/more-lazy-metrics branch April 10, 2025 15:35
@darccio darccio added the v2.0 label Jun 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants