Puppet for elastic stack 7.1

[tamama]

Greetings,

May I ask if this puppet module is compatibile with Elastic stack 7 (.1) ?

Many thanks.

Kind regards,
Tama Ma

[bryanpearson]

Also waiting on a compatible ES 7.* version.

[devinmatte]

Is this currently compatible with 7.x?
Has anyone tried setting the version to => 7.0 with any success?

[edupr91]

I tried with 7.2 but the puppet module is focused on setting up multiple elastic search instances.
But the last package (at least after installing from RPM) only works with one instance. Or at least the script that runs elastic won't retrieve correctly the specific conf path that the module setup.

[bryanpearson]

The problem we had is the module is outputting some config options that aren't compatible with the Kibana 7.0 Upgrade Assistant, https://www.elastic.co/guide/en/elasticsearch/reference/7.0/breaking-changes-7.0.html#tls-setting-fallback

Specifically in the instance.pp file:
'xpack.ssl.keystore.path' => $_keystore_path,
'xpack.ssl.keystore.password' => $keystore_password,

[sferry]

I managed to deploy one cluster 7.1.1 with 2 nodes (i.e. 2instances) without xpack nor searchguard.
I had to override pretty much every default lines using jvm_options => [ ], with entries provided by the rpm.
One of the 2 nodes is also a kibana endpoint and was deployed with elastic/puppet-kibana

[sticky-note]

@sferry Thanks for sharing. Which vars had you yo change ? I've seen already datadir environment var name not correct but had no time to dig into.
I'll test that tomorrow on three 7.2 nodes, 3 master including 2 data and will edit this messages after my experience. Maybe we can PR some early stage modification ?

[sferry]

@sticky-note
from the rpm see below [1]
I tried modifying one by one variable, looking at errors; pretty much every '*GC*' variables and "Print*"
But I gave up ( I don't know exactly all problematic variable). So in the end I just copied all of them.
For a PR it would be in template/etc/elasticsearch/jvm.options.erb
But it should be backward compatible.

[1]
-XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=75
-XX:+UseCMSInitiatingOccupancyOnly
-Des.networkaddress.cache.ttl=60
-Des.networkaddress.cache.negative.ttl=10
-XX:+AlwaysPreTouch
-Xss1m
-Djava.awt.headless=true
-Dfile.encoding=UTF-8
-Djna.nosys=true
-XX:-OmitStackTraceInFastThrow
-Dio.netty.noUnsafe=true
-Dio.netty.noKeySetOptimization=true
-Dio.netty.recycler.maxCapacityPerThread=0
-Dlog4j.shutdownHookEnabled=false
-Dlog4j2.disable.jmx=true
-Djava.io.tmpdir=${ES_TMPDIR}
-XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=data
-XX:ErrorFile=logs/hs_err_pid%p.log
8:-XX:+PrintGCDetails
8:-XX:+PrintGCDateStamps
8:-XX:+PrintTenuringDistribution
8:-XX:+PrintGCApplicationStoppedTime
8:-Xloggc:logs/gc.log
8:-XX:+UseGCLogFileRotation
8:-XX:NumberOfGCLogFiles=32
8:-XX:GCLogFileSize=64m
9-:-Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m
9-:-Djava.locale.providers=COMPAT

[smasa90]

Thanks for you help, I ended up with this PR to provision the bare minimum to have a cluster running with ES 7.x and not break backward compatibility, I Think..
#1040

[initharrington]

Any update on this or if #1040 will be merged?

[pw64]

Please, a working puppet module for ES7 (just one; not multiple ES instances) would be great, together with an working SSL implementation. Without SSL, the current version it works.

[rwaffen]

Any updates here? puppet-elastic-stack is already at 7.x ... so what's going on? :)
I'm eagerly waiting to use 7.x, but not without puppet support.

[uberjew666]

I have managed to get ELK 7.4.1 working using this module with xpack security running. I've updated my fork with all the requirements.
Due to xpack requiring encryption between all instances in a cluster, I've added a new parameter called elasticsearch::xpack. This will apply all xpack.security settings but requires ca_certificate, certificate and private_key parameters set.

[rwaffen]

@uberjew666 thanks for the great work! When will you do a PR?

[uberjew666]

@rwaffen Soon, just tidying my commits and confirming all the tests pass. Hopefully this week.