Added mapping for actor and target entity id, related.entity #224483

kfirpeled · 2025-06-18T18:03:18Z

Summary

Adds new fields mapping: actor.entity.id, target.entity.id, related.entity of type keyword to the security's alert index.
These fields are necessary for querying the alerts index as part of the graph feature (#221037).

related.entity is currently waiting for ECS approval - elastic/ecs#2360
actor.entity.id, target.entity.id are added only to security's alerts

How this PR effects:

Allows the user filter by actor.entity.id, target.entity.id, related.entity

Video 🎥

**Before:**

Screen.Recording.2025-06-18.at.19.19.18.mov

After:

Screen.Recording.2025-06-18.at.19.21.51.mov

Besides the immediate effect of the ability to fetch data using these fields, this PR is a requirement to move forward with upcoming features like: #224783

Example video of upcoming use 🎥

Screen.Recording.2025-06-29.at.19.42.23.mov

How to test locally

Start elasticsearch and kibana locally
To add mock data run the following:

node scripts/es_archiver load x-pack/solutions/security/test/cloud_security_posture_functional/es_archives/logs_gcp_audit \
 --es-url http://elastic:changeme@localhost:9200 \
 --kibana-url http://elastic:changeme@localhost:5601

node scripts/es_archiver load x-pack/solutions/security/test/cloud_security_posture_functional/es_archives/security_alerts \
 --es-url http://elastic:changeme@localhost:9200 \
 --kibana-url http://elastic:changeme@localhost:5601

Open Alerts page in kibana. Update the date-picker to include data from a year ago. Then check one of the alerts details opening the right-side flyout and find the "Graph preview" section in it.
Filter by one of these fields

Checklist

Unit or functional tests were updated or added to match the most common scenarios

Identify risks

In case the new mapping is not applied on existing documents, it means they won't be retrieved using the new fields

elasticmachine · 2025-06-18T18:05:29Z

Pinging @elastic/kibana-cloud-security-posture (Team:Cloud Security)

src/platform/packages/shared/kbn-alerts-as-data-utils/src/field_maps/alert_field_map.ts

...gration/spaces_only/tests/alerting/group4/alerts_as_data/alerts_as_data_dynamic_templates.ts

…s/alerting/group4/alerts_as_data/alerts_as_data_dynamic_templates.ts

kfirpeled · 2025-06-26T15:35:00Z

x-pack/solutions/security/plugins/security_solution/common/field_maps/9.2.0/alerts.ts

+ * 2.0.
+ */
+
+import { alertsFieldMap8190 } from '../8.19.0';


Is that correct to import from 8.19? or should I've imported from earlier release?

I think this is fine, given that the new mappings are always based on those of the previous release.

src/platform/packages/shared/kbn-alerts-as-data-utils/src/field_maps/alert_field_map.ts

ymao1

response ops changes lgtm

denar50

LGTM! I have ran the PR locally and tested that indeed I am able to filter by the added fields following the instructions provided.

elasticmachine · 2025-07-07T16:57:46Z

💚 Build Succeeded

Buildkite Build
Commit: 1e689a8

Metrics [docs]

✅ unchanged

History

💛 Build #313793 was flaky 091b849
💚 Build #313146 succeeded 106635f
💔 Build #312692 failed 74420bf
💔 Build #312613 failed 21cc8f2
💚 Build #311150 succeeded ae8ff27
💚 Build #310608 succeeded 318db3a

elasticmachine · 2025-07-13T08:09:00Z

🤖 Jobs for this PR can be triggered through checkboxes. 🚧

ℹ️ To trigger the CI, please tick the checkbox below 👇

Click to trigger kibana-pull-request for this PR!
Click to trigger kibana-deploy-project-from-pr for this PR!
Click to trigger kibana-deploy-cloud-from-pr for this PR!

## Summary Closes #221037 , shows alerts on the graph by querying both the logs and the alerts indices - [x] Graph API - new optional `indexPatterns` parameters to switch data views (not in use in the UI atm). Defaults to `.alerts-security.alerts-<spaceId>, logs-*` - [x] Visualize loaded alerts that are identified with alerts in graph preview and graph investigation ~Depends on #224483 `actor` and `target` are not part of ECS yet. And to ease our development process we wish to push forward with this feature in mind. This feature supports both cases when alert's index mappings contains definition for `actor` and `target`, and also when its not. In this PR, we add mappings of `actor` and `target` to the es_archive of the alerts. This way we are able to test the functionality of this feature instead of being blocked by #224483. <details> <summary>Video 🎥 </summary> https://github.com/user-attachments/assets/bcc86214-6e88-46f3-a990-300bbdc28125 </details> <details> <summary>Screenshots 📸 </summary> **Before (ignore label alignments - screenshot is from a local environment)** ![Screenshot 2025-06-29 at 19 33 00](https://github.com/user-attachments/assets/39b014ce-6b70-44cc-a486-906d39c205fe) **After (another event is identified with alert - marking it as such and expands the _alert_ details)** ![Screenshot 2025-06-29 at 19 32 30](https://github.com/user-attachments/assets/824d1d6f-9c17-4c4a-a8a7-18e65b89dbb2) **Before network page - preview** ![Screenshot 2025-06-29 at 19 40 59](https://github.com/user-attachments/assets/50716acc-b2bd-4d42-93e0-eb31cfa6fe9c) **After network page - preview identifies if event contains alert** ![Screenshot 2025-06-29 at 19 40 29](https://github.com/user-attachments/assets/531cec9f-2fb3-4a90-9cc1-1a73684f3612) </details> ### How to test locally 1. Edit `kibana.dev.yml` and add: ```yml uiSettings.overrides.securitySolution:enableGraphVisualization: true ``` 2. Start elasticsearch and kibana locally 3. To add mock data run the following: ```bash node scripts/es_archiver load x-pack/solutions/security/test/cloud_security_posture_functional/es_archives/logs_gcp_audit \ --es-url http://elastic:changeme@localhost:9200 \ --kibana-url http://elastic:changeme@localhost:5601 node scripts/es_archiver load x-pack/solutions/security/test/cloud_security_posture_functional/es_archives/security_alerts_modified_mappings \ --es-url http://elastic:changeme@localhost:9200 \ --kibana-url http://elastic:changeme@localhost:5601 ``` 3. Open `Alerts` page in kibana. Update the date-picker to include data from a year ago. Then check one of the alerts details opening the right-side flyout and find the "Graph preview" section in it. 4. Expand graph to show related alerts 5. Enable Asset Inventory in the `Inventory` page (if you don't see the page enable the feature flag in the advanced settings) 6. Add entities mock data ``` node scripts/es_archiver load x-pack/solutions/security/test/cloud_security_posture_api/es_archives/entity_store \ --es-url http://elastic:changeme@localhost:9200 \ --kibana-url http://elastic:changeme@localhost:5601 ``` 7. Open `Alerts` page in kibana. Check that the graph shows the admin entity with it's label ### Checklist - [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [x] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

kfirpeled · 2025-08-10T13:23:18Z

An agreement on the fields yet to come, closing the PR for now

…c#224783) ## Summary Closes elastic#221037 , shows alerts on the graph by querying both the logs and the alerts indices - [x] Graph API - new optional `indexPatterns` parameters to switch data views (not in use in the UI atm). Defaults to `.alerts-security.alerts-<spaceId>, logs-*` - [x] Visualize loaded alerts that are identified with alerts in graph preview and graph investigation ~Depends on elastic#224483 `actor` and `target` are not part of ECS yet. And to ease our development process we wish to push forward with this feature in mind. This feature supports both cases when alert's index mappings contains definition for `actor` and `target`, and also when its not. In this PR, we add mappings of `actor` and `target` to the es_archive of the alerts. This way we are able to test the functionality of this feature instead of being blocked by elastic#224483. <details> <summary>Video 🎥 </summary> https://github.com/user-attachments/assets/bcc86214-6e88-46f3-a990-300bbdc28125 </details> <details> <summary>Screenshots 📸 </summary> **Before (ignore label alignments - screenshot is from a local environment)** ![Screenshot 2025-06-29 at 19 33 00](https://github.com/user-attachments/assets/39b014ce-6b70-44cc-a486-906d39c205fe) **After (another event is identified with alert - marking it as such and expands the _alert_ details)** ![Screenshot 2025-06-29 at 19 32 30](https://github.com/user-attachments/assets/824d1d6f-9c17-4c4a-a8a7-18e65b89dbb2) **Before network page - preview** ![Screenshot 2025-06-29 at 19 40 59](https://github.com/user-attachments/assets/50716acc-b2bd-4d42-93e0-eb31cfa6fe9c) **After network page - preview identifies if event contains alert** ![Screenshot 2025-06-29 at 19 40 29](https://github.com/user-attachments/assets/531cec9f-2fb3-4a90-9cc1-1a73684f3612) </details> ### How to test locally 1. Edit `kibana.dev.yml` and add: ```yml uiSettings.overrides.securitySolution:enableGraphVisualization: true ``` 2. Start elasticsearch and kibana locally 3. To add mock data run the following: ```bash node scripts/es_archiver load x-pack/solutions/security/test/cloud_security_posture_functional/es_archives/logs_gcp_audit \ --es-url http://elastic:changeme@localhost:9200 \ --kibana-url http://elastic:changeme@localhost:5601 node scripts/es_archiver load x-pack/solutions/security/test/cloud_security_posture_functional/es_archives/security_alerts_modified_mappings \ --es-url http://elastic:changeme@localhost:9200 \ --kibana-url http://elastic:changeme@localhost:5601 ``` 3. Open `Alerts` page in kibana. Update the date-picker to include data from a year ago. Then check one of the alerts details opening the right-side flyout and find the "Graph preview" section in it. 4. Expand graph to show related alerts 5. Enable Asset Inventory in the `Inventory` page (if you don't see the page enable the feature flag in the advanced settings) 6. Add entities mock data ``` node scripts/es_archiver load x-pack/solutions/security/test/cloud_security_posture_api/es_archives/entity_store \ --es-url http://elastic:changeme@localhost:9200 \ --kibana-url http://elastic:changeme@localhost:5601 ``` 7. Open `Alerts` page in kibana. Check that the graph shows the admin entity with it's label ### Checklist - [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [x] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

Added mapping for actor and target entity id, related.entity

8a24d97

kfirpeled added release_note:skip Skip the PR/issue when compiling release notes Team:Cloud Security Cloud Security team related backport:prev-major labels Jun 18, 2025

kfirpeled marked this pull request as ready for review June 18, 2025 18:05

kfirpeled requested review from a team as code owners June 18, 2025 18:05

kfirpeled commented Jun 18, 2025

View reviewed changes

src/platform/packages/shared/kbn-alerts-as-data-utils/src/field_maps/alert_field_map.ts Outdated Show resolved Hide resolved

kfirpeled added 2 commits June 18, 2025 23:30

fix jest tests

98f2ceb

fix tests

a3db648

kfirpeled commented Jun 19, 2025

View reviewed changes

...gration/spaces_only/tests/alerting/group4/alerts_as_data/alerts_as_data_dynamic_templates.ts Outdated Show resolved Hide resolved

kfirpeled and others added 3 commits June 19, 2025 20:09

Update x-pack/platform/test/alerting_api_integration/spaces_only/test…

7254e95

…s/alerting/group4/alerts_as_data/alerts_as_data_dynamic_templates.ts

add auto generated updates

318db3a

Merge branch 'main' into cspm/alerts-mapping-update

ae8ff27

kfirpeled mentioned this pull request Jun 22, 2025

[Cloud Security] Show related alert's when fetching CDR graph #224783

Merged

6 tasks

Merge branch 'main' into cspm/alerts-mapping-update

21cc8f2

kfirpeled marked this pull request as draft June 25, 2025 12:33

kfirpeled added 2 commits June 25, 2025 17:34

Moved actor and target fields to be only part of security's alert

74420bf

fix tests

adc10bd

kfirpeled marked this pull request as ready for review June 26, 2025 15:30

kfirpeled requested a review from a team as a code owner June 26, 2025 15:30

kfirpeled requested a review from denar50 June 26, 2025 15:30

Merge branch 'main' into cspm/alerts-mapping-update

9a8d4b6

kfirpeled commented Jun 26, 2025

View reviewed changes

updated generated schemas

106635f

ymao1 reviewed Jun 26, 2025

View reviewed changes

src/platform/packages/shared/kbn-alerts-as-data-utils/src/field_maps/alert_field_map.ts Outdated Show resolved Hide resolved

kfirpeled added backport:skip This PR does not require backporting and removed backport:prev-major labels Jun 26, 2025

Merge branch 'main' into cspm/alerts-mapping-update

1ef6253

kfirpeled requested a review from a team as a code owner June 27, 2025 10:05

Moved related.entity to security's alerts only

ffc9f2b

kfirpeled force-pushed the cspm/alerts-mapping-update branch from f0269b4 to ffc9f2b Compare June 27, 2025 10:06

updated generated files

cfbb11e

kfirpeled requested review from ymao1 and removed request for a team June 27, 2025 10:07

ymao1 approved these changes Jun 27, 2025

View reviewed changes

Merge branch 'main' into cspm/alerts-mapping-update

091b849

denar50 approved these changes Jun 30, 2025

View reviewed changes

kfirpeled added 2 commits July 6, 2025 01:22

Merge branch 'main' into cspm/alerts-mapping-update

4936cfe

Merge branch 'main' into cspm/alerts-mapping-update

1e689a8

kfirpeled marked this pull request as draft July 13, 2025 08:08

kfirpeled added 3 commits July 13, 2025 11:09

Merge branch 'main' into cspm/alerts-mapping-update

e5dbff0

Merge branch 'main' into cspm/alerts-mapping-update

13d408c

Merge branch 'main' into cspm/alerts-mapping-update

56f1066

kfirpeled closed this Aug 10, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Added mapping for actor and target entity id, related.entity #224483

Added mapping for actor and target entity id, related.entity #224483

Uh oh!

kfirpeled commented Jun 18, 2025 •

edited

Loading

Uh oh!

elasticmachine commented Jun 18, 2025

Uh oh!

Uh oh!

Uh oh!

kfirpeled Jun 26, 2025

Uh oh!

denar50 Jun 30, 2025

Uh oh!

Uh oh!

ymao1 left a comment

Uh oh!

denar50 left a comment

Uh oh!

elasticmachine commented Jul 7, 2025

Uh oh!

elasticmachine commented Jul 13, 2025

Uh oh!

kfirpeled commented Aug 10, 2025

Uh oh!

Uh oh!

Added mapping for actor and target entity id, related.entity #224483

Added mapping for actor and target entity id, related.entity #224483

Uh oh!

Conversation

kfirpeled commented Jun 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

How to test locally

Checklist

Identify risks

Uh oh!

elasticmachine commented Jun 18, 2025

Uh oh!

Uh oh!

Uh oh!

kfirpeled Jun 26, 2025

Choose a reason for hiding this comment

Uh oh!

denar50 Jun 30, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ymao1 left a comment

Choose a reason for hiding this comment

Uh oh!

denar50 left a comment

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Jul 7, 2025

💚 Build Succeeded

Metrics [docs]

History

Uh oh!

elasticmachine commented Jul 13, 2025

Uh oh!

kfirpeled commented Aug 10, 2025

Uh oh!

Uh oh!

kfirpeled commented Jun 18, 2025 •

edited

Loading