Index pattern routing

Elastic Stack Kibana
1

Hi,

We have setup our cluster and we have some logs and metrics flowing in to Elastic.
Now we have tried to add Gerrit logs and metrics.

The problem I have is the metrics-data is in both of the Kibana patterns.

I have configured Gerrit to send logs to our Logstash.

[remote "elasticsearch"]
  url = http://elk-ls01.foobar.se:5046/gerrit

And Metrics using Graphite-plugin in Gerrit

[graphite]
  host = elk-ls01.foobar.se
  port = 5049
  prefix = gerritmetrics

In Logstash I have following two pipelines:
Gerrit-logs pipeline file.

input {
  http {
    port => 5046
    codec => json
    id => gerrit
  }
}

output {
  elasticsearch {
    hosts => ["https://elk-es01.foobar.se:9200", "https://elk-es02.foobar.se:9200", "https://elk-es03.foobar.se:9200"]
    index => "gerrit-logs-%{+YYYY.MM.dd}"
    user => "logstash_internal"
    password => "XXXXXXXX"
  }
}

Gerrit Metrics pipeline file.

input {
  graphite {
    id => "gerritmetrics"
    port => 5049
  }
}

output {
  elasticsearch {
    hosts => ["https://elk-es01.foobar.se:9200", "https://elk-es02.foobar.se:9200", "https://elk-es03.foobar.se:9200"]
    index => "logstash-gerrit-metrics-%{+YYYY.MM.dd}"
    user => "logstash_internal"
    password => "XXXXXXXX"
  }
}

In Elasic I have two Kibana pattern
gerrit-logs-*
logstash-gerrit-metrics-*

Where should I look to get this separated?

I want Gerrit log to go to gerrit-logs pattern and Gerrit metrics go to logstash-gerrit-metrics-* pattern.

Thanks for you help!

2

The config looks right to me. But maybe gerrit also sends the metrics from their remote? If that's the case, you could use a filter in logstash to drop the metrics messages from the gerrit-logs indices.

3

I added gerrit-logs first and that ran for a cupule of days. There was no metrics in the gerrit-logs index.

On my first try to add Graphite I used JSON as input in Logstash, it didn't work. After changing it to Graphite input I got entry's in the indexes.
With that in mind I think I messed it up in Logstash or somewhere down the road to Kibana index.

Can I done something wrong in some ingress settings in the cluster?

4

The config looks correct to me. But do ask in the Logstash part of this forum, since this is really not a Kibana issue. Kibana only shows what data you have in Elasticsearch, nothing less, nothing more.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.