Beat not creating index on start

[tostasqb (Hugo Bento)]

Just started using Beats and Version 5.

After installing filebeat and metricbeat on my mac, when starting either one I'm getting the following systematic error:

2016/11/10 15:50:37.490503 client.go:420: WARN Can not index event (status=404): {"type":"index_not_found_exception","reason":"no such index","resource.type":"index_expression","resource.id":"filebeat-2016.11.10","index_uuid":"_na_","index":"filebeat-2016.11.10"}

So, from this I'm getting that indexes are created daily and in this case filebeat-2016.11.10 was not yet created. The workaround I've found was straightforward, I created the index in ES and the tried again. That was a success but I guess there's something missing here since this is clearly not the expected use...

[ruflin]

What is your setup? Do you use Shield? Normally beats create the indices automatically. After creating the index manually, adding data from the beats side just worked?

[tostasqb (Hugo Bento)]

I used a fresh new installation of version 5 on a folder of my Mac (/opt/elastic/).
I installed Elasticsearch, kibana and X-Pack and finally beats and yes, after creating the indexes manually and running Metricbeat or Filebeat they run fine...the data is on elasticsearch and I'm able to search it on Kibana as well.

[ruflin]

As you have Shield installed, is it possible that the filebeat user does not have the rights to create indices?

[tostasqb (Hugo Bento)]

Got it!

I began by changing my folder to have all rights (777) but it didn't work so it got me thinking about X-Pack, so I got another look into the installation guides and found my mistake... I introduced the auto_create_index in the elasticsearch configs. Seems pretty obvious now...

After commenting that all is good again.

Thanks for your help.