APM indexes throught logstash

[Alexkl (Alexandre Klein)]

Hello,

I am using ELK 6.4.0

I was using apm with elasticsearch output but i would like to replace it by logstash output.

In my output.elasticsearch i set multiples index depending on processor.event:

indices:
    - index: "apm-%{[beat.version]}-sourcemap"
      when.contains:
        processor.event: "sourcemap"
    - index: "apm-%{[beat.version]}-error-%{+yyyy.MM.dd}"
      when.contains:
        processor.event: "error"
    - index: "apm-%{[beat.version]}-transaction-%{+yyyy.MM.dd}"
      when.contains:
        processor.event: "transaction"
    - index: "apm-%{[beat.version]}-span-%{+yyyy.MM.dd}"
      when.contains:
        processor.event: "span"
    - index: "apm-%{[beat.version]}-metric-%{+yyyy.MM.dd}"
      when.contains:
        processor.event: "metric"
    - index: "apm-%{[beat.version]}-onboarding-%{+yyyy.MM.dd}"
      when.contains:
        processor.event: "onboarding"

So i tried to do the same in my logstash output :

else if "apm-server" in [@metadata][beat] {
elasticsearch {
hosts => ["logs-es1:9200","logs-es2:9200"]
index => "apm-%{[processor][event]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
}
}

However when i do that, i have no available logs showing in the discover part. But a lot of shards failed appeared (if i change my output to elasticsearch again, all my problems disappear).

Do you have an idea of what going on ?

Thank you very much

[gil (Gil Raphaelli)]

Hello @Alexkl,

You'll need to switch your logstash index directive a bit to match up with the output.elasticsearch setting provided - the event and version are reversed:

index => "apm-%{[@metadata][version]}-%{[processor][event]}-%{+YYYY.MM.dd}"

Once that's done, you'll be writing to an index with the correct template to power the APM UI.

[Alexkl (Alexandre Klein)]

Thanks... i am ashamed

[gil (Gil Raphaelli)]

Glad to hear that resolved it.