Yes,I did same , but my index is not getting created with above filebeat.yml in which I've added below also.
index: "%{[fields.index]}-%{+yyyy.MM.dd}-000001".
Yeah, but I don't think this is correct, if you are using rollover, you need to point to the rollover alias, if it is not working while pointing to the rollover alias than it may be another issue.
Unfortunatelly I can not help further as I do not use filebeat directly to elasticsearch and I also do not use rollover, my only recommendation would be to check the documentation and validate each step of the configuration.
Hi @PRASHANT_MEHTA,
I suppose, I understand your intention. If I'm not wrong you are trying to achieve from filebeat.yml send two types of logs in two separate index and you have defined ILM and rollover alias in index template. Now you want this data to be written to their respective index with their rollover pattern.
I suppose this won't be possible through filebeat and you need to use logstash to achieve this. I am curious to know if you have achieved this through filebeat. Through documentation it seem not possible as you've self declared indexes.
Yes, you got it correct and I'm still looking out ways to do without logstash.I can't use logstash.
@legoguy1000 @Marius_Iversen you've any advice what I'm doing wrong ?
Thanx
Big warning: I only have experience with Filebeat -> Logstash -> Elasticsearch.
I think what you can do is to just have Filebeat send the data to Elasticsearch. On/In Elasticsearch you can define/update something called an Ingest Pipeline. I think you should already have a simple Ingest Pipeline called main defined there (it should be visible via Kibana -> Stack Management). I think you can modify this pipleline to do some manipulation of the data coming in - like separating the data into different indices. But I have never tried it.
Hello @jba ,
Thanx for your suggestion on this and time to look into this.The way you recommend didn't helped.
I have simple requirement :
1)different log paths
2)send the data of diffrent log path to diffrent indices using filebeat.
3)for every log path its index template with ilm and rollover is defined and log ingest pipeleine also written[grok pattern]
4)Somehow the data don't go to elastic index and dosent follow the ilm policy
I'm not sure how it can be done by using only filebeat and not using logstash.
Thanx
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.