Topbeat beat.hostname analyzed

Elastic Stack Beats
1

Hi there,
I noticed that after I load in the topbeat template, and create an index pattern for topbeat, the 'beat.hostname' field is analyzed. I'm kind of a noob with beats/elasticsearch... could someone help me to change that field to non_analyzed? I've read through the Mapping docs, but i'm still unsure how to do it.
Having this field analyzed causes problems with hostnames that have a dash (-) in them because it sees it as 2 separate entities. What's odd is that in filebeat, this same field is not analyzed.

Thanks!
-James

2

Topbeat and Filebeat have similar templates and the result should be the same. I have tried it on my computer and it works fine for me.
What version of Elasticsearch and Topbeat are you using?

3

I had the topbeat 1.0.0-rc1 template installed, but I just updated the template to 1.0.0 yesterday to see if that would fix the problem, but it didn't.
I'm using elasticsearch 2.0.0.

topbeat.png1583×700 48.2 KB

4

How did you install the template? Did you follow the steps from: https://www.elastic.co/guide/en/beats/topbeat/current/topbeat-getting-started.html#topbeat-template ? Thanks!

5

Can you please send me what you loaded already under the template?

curl -XGET 'http://127.0.0.1:9200/_template/topbeat?pretty'

where 127.0.0.1:9200 is the URL where Elasticsearch is available.

6

Yup, I followed that doc exactly. Here's the output of that command:

{
"topbeat" : {
"order" : 0,
"template" : "topbeat-",
"settings" : {
"index" : {
"refresh_interval" : "5s"
}
},
"mappings" : {
"default" : {
"dynamic_templates" : [ {
"template1" : {
"mapping" : {
"ignore_above" : 1024,
"index" : "not_analyzed",
"type" : "{dynamic_type}",
"doc_values" : true
},
"match" : ""
}
} ],
"_all" : {
"norms" : {
"enabled" : false
},
"enabled" : true
},
"properties" : {
"proc" : {
"properties" : {
"mem" : {
"properties" : {
"rss_p" : {
"type" : "float",
"doc_values" : "true"
}
}
},
"cpu" : {
"properties" : {
"user_p" : {
"type" : "float",
"doc_values" : "true"
}
}
}
}
},
"@timestamp" : {
"type" : "date"
},
"load" : {
"properties" : {
"load5" : {
"type" : "float",
"doc_values" : "true"
},
"load1" : {
"type" : "float",
"doc_values" : "true"
},
"load15" : {
"type" : "float",
"doc_values" : "true"
}
}
},
"mem" : {
"properties" : {
"used_p" : {
"type" : "float",
"doc_values" : "true"
},
"actual_used_p" : {
"type" : "float",
"doc_values" : "true"
}
}
},
"swap" : {
"properties" : {
"used_p" : {
"type" : "float",
"doc_values" : "true"
},
"actual_used_p" : {
"type" : "float",
"doc_values" : "true"
}
}
},
"cpu" : {
"properties" : {
"user_p" : {
"type" : "float",
"doc_values" : "true"
},
"system_p" : {
"type" : "float",
"doc_values" : "true"
}
}
},
"fs" : {
"properties" : {
"used_p" : {
"type" : "float",
"doc_values" : "true"
}
}
}
}
}
},
"aliases" : { }
}
}

7

Can you please delete the old topbeat indices to make sure the "not_analyzed" is coming from the new indices created after you applied the template?

curl -XDELETE 'http://127.0.0.1:9200/topbeat-*'

After that you can refresh the topbeat index pattern in Kibana (from the Settings page) to see if it's set to "not_analyzed".

1 Like
8

Ah, that did it! Thanks for your help. So was it just holding onto the settings from the rc1 implementation?

9

Hi @monica ,

I've been noticing the same problem, and I'm using topbeat-dashboards-1.1.0.

I'm trying to understand the issue a little better, I'm hoping you might be able to provide some advice.

I have a few questions:
-The import template sets the analyzed flag?
-Is deleting all the indices the only method to remove the analyzed flag on the field?
-If I'm clear on this, your advice above is to:
1) delete all topbeat-* indices,
2) import the template from the load.sh/load.ps1 script and
3) refresh the topbeat-* index pattern in Kibana before loading more topbeat data?

10

I'm not sure if I should start a new thread on this subject:

I've managed to refresh the topbeat-* index pattern by deleting the pattern and importing via the load.ps1 script.

Is there some way I can update the existing indices to match the updated index pattern without deleting the data?

UPDATE: Started a new thread in ES forum here

11

@plonka2000 You would need to re-index your existing data. It takes a bit of work and the way you accomplish the task varies by your technology. Basically you read the data out of ES then re-write it to a new index. See the links below.

https://www.elastic.co/guide/en/elasticsearch/guide/current/reindex.html
http://david.pilato.fr/blog/2015/05/20/reindex-elasticsearch-with-logstash/

In the future this will be easier to do: https://github.com/elastic/elasticsearch/issues/15201