Fields in Kibana 5 are non-searchable and non-aggregatable

Elastic Stack Kibana
1

Topbeat 1.3
elasticsearch 5.1
kibana 5.1
Red Hat Enterprise Linux Server
Version : 7.2 (Maipo)

Installing Topbeat
curl -L -O https://download.elastic.co/beats/topbeat/topbeat-1.3.1-x86_64.rpm
sudo rpm -vi topbeat-1.3.1-x86_64.rpm

Loading the Index Template In Elasticsearch
curl -XPUT 'http://IP:9200/_template/topbeat' -d@/etc/topbeat/topbeat.template.json

I have not started Topbeat on linux

I have configured topbeat 1.2 topbeat.yml to point to linux server
topbeat 1.2 is installed on windows 7 machine.

Now the issue begins================
When I configure the index pattern topbeat-* on kibana 5.1 there are some fields which are both non-searchable and non-aggregatable.

fields-1.png1006×101 4.23 KB
fields-2.png1016×282 13.9 KB

As you can see these fields are both non-searchable and non-aggregatable.
I am not able to create charts on these fields as they do not appear

How to make them searchable and aggregatable ??

1 Like
2

Do you mean that Elasticsearch is running on your linux server, and that you are pointing the topbeat instance on your windows machine to it?

Are you using Logstash to index the data from topbeat, or did you configure it to dump directly into Elasticsearch?

If you are dumping directly into elasticsearch, I don't think that you need to manually load the index template.

3

This looks like it might be related to Saved "field" parameter is now invalid. Please select a new field. .... Visualize: "field" is a required parameter

1 Like
4

Related to my earlier question. Are you sure that there are actual documents indexed with data for these fields?

5

@BigFunger

Do you mean that Elasticsearch is running on your linux server, and that you are pointing the topbeat instance on your windows machine to it?
Yes, elasticsearch 5.1 is running on my linux server and topbeat1.2 instance on my windows machine is pointing to elasticsearch 5.1 on linux machine.

Are you using Logstash to index the data from topbeat, or did you configure it to dump directly into Elasticsearch?
No, I am not using logstash to index the data from topbeat, I configured it to dump directly into elasticsearch 5.1 on linux machine

If you are dumping directly into elasticsearch, I don't think that you need to manually load the index template.
If you don't manually load the index template, how would you ensure the datatypes?

Related to my earlier question. Are you sure that there are actual documents indexed with data for these fields?
Yes

@Stacey_Gammon
Thanks, I tried your link and it worked.
I read the documentation and it says like that

fields-4.png788×325 25.1 KB

Going by the documentation,
You can reload the index fields list to pick up any newly-added fields.
I manually loaded the index template for topbeat-* index.
I found that in topbeat.template.json all the fields which were non-searchable and non-aggregatable were present in it. So, these fields were not newly-added, they were already present.

So, what happended when I pressed the Reload button, that made kibana to recognise these fields as searchable and aggregatable??

1 Like
6

Awesome, glad it worked!

The reload button goes back to the field_stats api to retrieve information about the fields such as searchacble, aggregateble, etc. The field_stats api doesn't return any information for fields that don't have any data indexed for them, so if Kibana generated the field list while there was no data for that field, it will display the information incorrectly.

You can read more about this issue at these related tickets:


7

Thanks, @Stacey_Gammon.
For the reply.

Will look into it.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.