ES 5.0 new installation question

Elastic Stack Elasticsearch
1

After getting less-than-desired results trying to upgrade my ELK stack to version 5.0, I thought I'd start fresh, and create a brand new ELK 5.0 node. I have reused some config files (e.g. elasticsearch.yml, kibana.yml, logstash config files) from my older setup. So far, so good. ES, Logstash, and Kibana start OK, and I can access the Kibana Web UI. However, I can't configure an index pattern. For example, the pattern logstash-* returns the dreaded "Unable to fetch mapping. ... ". When I look in /var/lib/elasticsearch/nodes/0/indices, I see idices with names like Lk8wl5UtRASVyldRHspZQg, instead of e.g. logstash-2016.11.08. When I go to http://localhost:9200/_cat/indices, I see the following:

yellow open .monitoring-data-2 s147bGxoQ5WgDgsP4XlECA 1 1 4 0 13.5kb 13.5kb
yellow open .monitoring-es-2-2016.11.08 PcZAvTv8TjmTJgZ6e-rm0Q 1 1 2900 40 2.4mb 2.4mb
yellow open .monitoring-kibana-2-2016.11.08 m1MyHbtXRCiNPdeDr97aZA 1 1 550 0 335kb 335kb
yellow open .kibana Lk8wl5UtRASVyldRHspZQg 1 1 1 0 3.2kb 3.2kb

What happened to my logstash, filebeat, winlogbeat, and topbeat indices?

Thanks

2

Did you push data from beats etc into ES? That might sound obvious, but it doesn't look like you have.

3

warkolm, I take it you mean from all of the hosts that are forwarding logs? I would think so. I'm using logstash as output. Here's my output configuration (30-elasticsearch-output.conf):

output {
if [@metadata][beat] {
elasticsearch {
hosts => ["http://10.0.101.101:9200"]
manage_template => false
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
} else {
elasticsearch { hosts => ["10.0.101.101:9200"] }
stdout { codec => rubydebug }
}
}

4

And do you know that data is reaching the output? Do things show if you use a stdout?

5

Actually, I don't know. If I've followed installation instructions to-the-letter, but used various config files from the old installation, maybe that's where the problem lies. I've posted the various configs on Pastebin (http://pastebin.com/dcEbc32k). I would most appreciate it if someone could take a look. That way, either a misconfig will be uncovered, or it'll rule out that that's where the problem lies.

With thanks.

6

You should add a stdout to check :slight_smile:

7

OK. But, er (shuffling feet, looking sheepish), how do I do that?

8

It's totally ok to ask that :smiley:

output {
  stdout {
    codec => rubydebug
  }
}
9

Thanks so much. I'm taking it I add this to my logstash output config? And I should access, or see, output where?

10

Yeah add it to your config, it'll show the output of your pipeline to the console, aka stdout.

11

OK. Once added, and services restarted, am I supposed to see output flowing to stdout (I don't, as of now), or must I do something else?

12

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.