Will older logstash forward to newer elasticsearch?

Elastic Stack Logstash
1

I'm a newbie here, so apologies if I breach etiquette or protocol.
I did search this category and Logstash-forwarder for a similar topic and didn't find one, so apologies again if this question has been asked before...

We're rolling out v5.4. It's going OK so far, but I'm wondering how to transition our existing v2.4 stack into the new environment. A full restart upgrade is inappropriate as the new v5.4 elasticsearch will contain newer documents already. Furthermore, the two stacks will run in parallel during the transition; hence, my question...

During the transition, we would like to configure Logstash in the v2.4 environment to output to both Elasticsearch versions: existing 2.4 and new 5.4. Is that possible? The documents are sent as JSON payloads over HTTPS, so it should be feasible. If so, what needs to be configured on either side to enable this capability?

I know that the v2.4 index templates must be imported into new v5.4 Elasticsearch, possibly after some tweaking. I suspect that the 2.4 document [fields] must also be tweaked in Logstash 2.4 before sending to Elasticsearch 5.4, but what? Should Logstash 2.4 send to Logstash 5.4 so that the latter could apply the requisite changes?

I should mention that Logstash 5.4 pulls from Kafka queues fed by v5.4 Beats.
Thanks in advance for any assistance that you can offer.
.../kevOut

2

During the transition, we would like to configure Logstash in the v2.4 environment to output to both Elasticsearch versions: existing 2.4 and new 5.4. Is that possible?

Sure, just put two elasticsearch entries in your output section. Logstash always ships all events to all outputs (unless conditionals say otherwise).

I know that the v2.4 index templates must be imported into new v5.4 Elasticsearch, possibly after some tweaking. I suspect that the 2.4 document [fields] must also be tweaked in Logstash 2.4 before sending to Elasticsearch 5.4, but what?

Recent Logstash releases ship with different index templates for each ES release (and I believe they automatically pick the right template based on the server they connect to). You can find the files in https://github.com/logstash-plugins/logstash-output-elasticsearch/tree/master/lib/logstash/outputs/elasticsearch.

Should Logstash 2.4 send to Logstash 5.4 so that the latter could apply the requisite changes?

No, there's no point in that.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.