When i try to create time based index for filebeats,it says "unable to fetch mapping , do you have indices matching that pattern". However the normal index is getting created when i type in filebeat-*.
Can anybody please tell what's wrong here!
Could be really helpful if you can post the answer .
Your topic title doesn't really match the question, you may want to edit it to make it clear what you are looking for?
I have updated the topic.
Request to to please provide a quick solution for this issue .I am new to ES & Its a critical use case for me , and i have been unable to find an answer for this in ES documentation.
If you list the indices in the cluster, has winlogbeat indexed any data?
yes on hitting this url
http://localhost:9200/_cat/indices?v
i get lot of indices related to winlogbeats.
What should i do next
Which version of Elasticsearch and Kibana are you using?
I am using 6.5.1 in local system/dev environment(above screenshot is from this)
But in Production we are using ES/Kibana 4.5
One additional ques(in production we are thinking of upgrading to Kibina 6.5.1 . Is is also necessary to update ES
OR
the old 4.5 ES which we have will work fine with new kibana)
Thanks
Elasticsearch and Kibana need to be the same version, so upgrading to version 6.5.3 across the board would be the way to go. I also do not think recent versions of Beats are compatible with such old Elasticsearch and Kibana versions. Please see the support matrix for further details.
Christian
Sure in production we will upgrade
But can you please suggest what to do next for kibana6.5.1 for deletion of documents in indices that are more than 30 days old
(since i have latest ES and Kibana-6.5.1 in my local system)
I would recommend using Curator to delete indices older than a threshold.
Currently i have 3 indices metricbeat,filebeat,winlogbeat
Curator deletes the entire index , but i don't want to delete the entire index.
Only the records/documents older than 30 days in each index needs to be deleted.
Or can we create monthy indices and delete the index themselves every month(need help how can we do this)
Also to install curator we need pip commands .. are they supported in windows ?
The you need to use the delete by query API, which is a lot less efficient. I would recommend switching to time-based (not necessarily daily) indices and use Curator instead.
Did the above screenshot which i pasted contain time-based indices ?
I am new to ES . Would be really thankful if you can explain in detail how to do that via
time-based indices and curator .
Yes, it does. There is an index per day as you can see in the index name. They are however very small, so you may benefit from switching to monthly indices instead.
Can you tell How can i create monthly indices
I cant find any option to make monthly time based index
You do that by specifying a monthly pattern in your Beats/Logstash config. It is the clients that determine which index they write to.
Read this and specifically the "Also be patient" part.
It's fine to answer on your own thread after 2 or 3 days (not including weekends) if you don't have an answer.
Can to please guide what config changes to be made to yml file of beats to specify the monthly index patten
Have a look at the example in the docs. Just change index: "filebeat-%{[beat.version]}-%{+yyyy.MM.dd}" to index: "filebeat-%{[beat.version]}-%{+yyyy.MM}".