How to fix mapping conflict?

Elastic Stack Kibana
1

I have a few fields (bytesSent, bytesReceived, timetaken) that show mapping conflicts, and thus I cannot visualize them in Kibana.

All of these log entries are going to an index "logstash-*"

I am pretty green at this, so how would I go about finding all of the conflicting indexes and fixing or removing them?

Here is part of the grok expression used for these conflicting fields in Logstash:
%{NUMBER:bytesSent} %{NUMBER:bytesReceived} %{NUMBER:timetaken}

And then a mutate (is this even necessary?)

mutate {
  convert => ["bytesSent", "integer"]
  convert => ["bytesReceived", "integer"]
  convert => ["timetaken", "integer"]
}
2

You need to reindex the data so that it's all the same.

3

and how can this be accomplished if it is all under logstash-*?

again, I am very unfamiliar with the process... I tried reading the article from Elastic.co but was lost. Is it something I would have to do through Putty or another tool?

4

and how can this be accomplished if it is all under logstash-*?

The principle is that you copy each document in each index into one or more new index. When you're done, delete the old indexes.

again, I am very unfamiliar with the process... I tried reading the article from Elastic.co but was lost. Is it something I would have to do through Putty or another tool?

Reindexing can be done from any language with an Elasticsearch client library. Well, strictly speaking it can be done in any language where it's reasonably convenient to perform HTTP requests.

If you can ask more specific questions it'll be easier to help.

5

These two should give you some hints

http://david.pilato.fr/blog/2015/05/20/reindex-elasticsearch-with-logstash/

2 Likes