Index pattern routing

Hi,

We have setup our cluster and we have some logs and metrics flowing in to Elastic.
Now we have tried to add Gerrit logs and metrics.

The problem I have is the metrics-data is in both of the Kibana patterns.

I have configured Gerrit to send logs to our Logstash.

[remote "elasticsearch"]
  url = http://elk-ls01.foobar.se:5046/gerrit

And Metrics using Graphite-plugin in Gerrit

[graphite]
  host = elk-ls01.foobar.se
  port = 5049
  prefix = gerritmetrics

In Logstash I have following two pipelines:
Gerrit-logs pipeline file.

input {
  http {
    port => 5046
    codec => json
    id => gerrit
  }
}

output {
  elasticsearch {
    hosts => ["https://elk-es01.foobar.se:9200", "https://elk-es02.foobar.se:9200", "https://elk-es03.foobar.se:9200"]
    index => "gerrit-logs-%{+YYYY.MM.dd}"
    user => "logstash_internal"
    password => "XXXXXXXX"
  }
}

Gerrit Metrics pipeline file.

input {
  graphite {
    id => "gerritmetrics"
    port => 5049
  }
}

output {
  elasticsearch {
    hosts => ["https://elk-es01.foobar.se:9200", "https://elk-es02.foobar.se:9200", "https://elk-es03.foobar.se:9200"]
    index => "logstash-gerrit-metrics-%{+YYYY.MM.dd}"
    user => "logstash_internal"
    password => "XXXXXXXX"
  }
}

In Elasic I have two Kibana pattern
gerrit-logs-*
logstash-gerrit-metrics-*

Where should I look to get this separated?

I want Gerrit log to go to gerrit-logs pattern and Gerrit metrics go to logstash-gerrit-metrics-* pattern.

Thanks for you help!

The config looks right to me. But maybe gerrit also sends the metrics from their remote? If that's the case, you could use a filter in logstash to drop the metrics messages from the gerrit-logs indices.

I added gerrit-logs first and that ran for a cupule of days. There was no metrics in the gerrit-logs index.

On my first try to add Graphite I used JSON as input in Logstash, it didn't work. After changing it to Graphite input I got entry's in the indexes.
With that in mind I think I messed it up in Logstash or somewhere down the road to Kibana index.

Can I done something wrong in some ingress settings in the cluster?

The config looks correct to me. But do ask in the Logstash part of this forum, since this is really not a Kibana issue. Kibana only shows what data you have in Elasticsearch, nothing less, nothing more.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.