Skip to content

Clarify ECH node bootlooping documentation #1093

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Apr 14, 2025
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 16 additions & 2 deletions troubleshoot/monitoring/node-bootlooping.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,9 +48,17 @@

The most frequent cause of a failed deployment configuration change is due to invalid or mislocated [secure settings](/deploy-manage/security/secure-settings.md). This can frequently be discovered by searching {{es}} logs for one of the following error messages:

```sh
```log

Check warning on line 51 in troubleshoot/monitoring/node-bootlooping.md

View workflow job for this annotation

GitHub Actions / preview / build

Unknown language: log
# Typical Error Message
#------------------------------
[ERROR][org.elasticsearch.bootstrap.Elasticsearch] ... fatal exception while booting Elasticsearch
IllegalStateException: security initialization failed

java.lang.IllegalArgumentException: unknown secure setting

org.elasticsearch.common.settings.SettingsException:
The configuration setting [xpack.security.authc.realms.foobar.foobar1.foobar2.client_secret] is required
#------------------------------
```

These are settings typically added to the keystore for the purpose of:
Expand All @@ -62,7 +70,6 @@

The following sections detail some secure settings problems that can result in a configuration change error that can prevent a deployment from restarting. You might diagnose these plan failures via the logs or via their [related exit codes](/deploy-manage/maintenance/start-stop-services/start-stop-elasticsearch.md#fatal-errors) `1`, `3`, and `78`.


### Invalid or outdated values [ec-config-change-errors-old-values]

The keystore does not validate any settings that you add, so invalid or outdated values are a common source of errors when you apply a configuration change to a deployment.
Expand Down Expand Up @@ -91,6 +98,13 @@

In some cases, settings may accidentally be added to the keystore that should have been added to the [{{es}} user settings file](/deploy-manage/deploy/elastic-cloud/edit-stack-settings.md). It’s always a good idea to check the [lists of reloadable settings](/deploy-manage/security/secure-settings.md) to determine if a setting can be stored in the keystore. Settings that can safely be added to the keystore are flagged as `Secure`.

### Missing or improperly configured

The error message `The configuration setting [...] is required` indicates that the corresponding setting is configured and present in the Elasticsearch instance via [Elasticsearch user settings](/deploy-manage/deploy/elastic-cloud/edit-stack-settings.md#ec-add-user-settings), but is either missing or improperly configured in [secure settings](/deploy-manage/security/secure-settings.md). Please review your [secure settings](/deploy-manage/security/secure-settings.md) to ensure they are configured correctly.

Additionally, if you configure these settings via a client tool or API and encounter the error, try configuring the settings directly in the Cloud UI to isolate the cause. If configuring in the Cloud UI does not result in the same error, it suggests that the keystore setting is valid, and the method of configuration should be examined. Conversely, if the same error is reported, it suggests that the keystore setting may be invalid and should be reviewed.



## Expired custom plugins or bundles [ec-config-change-errors-expired-bundle-extension]

Expand Down
Loading