Custom fields or a way to analyze tags

Elastic Stack Beats
1

Hi,

We're evaluating topbeat and have a need for a double hostname setup, where both the 'real' hostnames (12349187324.somecloud.com) and nice hostname (elastic08.somecloud.com) can be used to search/filter and graph upon in kibana based on the topbeat indexes.
Since I could not find a way to add custom fields to topbeat I was hoping to use tags (configureable through topbeat.yml). Unfortunately the tags appear in kibana discoverer as fields that are not indexed and therefore not available for visualization and search.
In the kibana settings for that index I don't even see the field.

What options are there to get to the goal we'd like to achieve?

  • for instance, can we get the field indexed somehow?
  • is there a way to add custom fields?
  • some other way?

thanks,

Max

2

One way to could work is to load your own template with your required mapping for the tags. Like this they should get available. Best would be to extend the topbeat mapping: https://github.com/elastic/beats/blob/master/topbeat/etc/topbeat.template.json Please make sure to you the template for your version of topbeat as it changed recently. I didn't test the above, but should work in theory.

3

Hi Ruflin,

Thank you for the suggestion. I've added the following mapping to the template I took from /etc/topbeat/topbeat.template.json :
"tags": {
"properties": {
"nice_name": {
"doc_values": "true",
"type": "string"
}
}
},
unfortunately this causes the index to no longer be updated with new entries from topbeat clients. After deleting the index, no new index is created. Only when I restore the previous mapping template, everything starts to work again.
I've also tried this addition:
"tags": {
"properties": {
"nice_name": {
"doc_values": "true",
"type": "string",
"index": "not_analyzed"
}
}
},

and that works, but only for topbeat clients that are not sending tags. Those that sends tags don't end up in the index.

Am I doing it wrong? Maybe some essential thinking error from my side by not understanding this stuff well enough?

4

Can you post the full template code you put in? You can use three backticks to post is as code here with indentation.