Question Regarding `index` in logstash elasticsearch output plugin

Elastic Stack Logstash
1

I am using logstash elasticsearch output plugin to create time based indices.
I create index patterns in the following way currently:

output {
# other settings
index => "server-netlogs-%{+YYYY.MM.dd}" 
}

The events which I want to index have @timestamp field on them which represents the source of truth. But the %{+YYYY.MM.dd} used in creating the index pattern uses the current timestamp of the instance logstash is running on.

I wanted to know if it is possible to use @timestamp for the index as in my case real timestamp != timestamp on the event and I want the timestamp in the event to be used to index the data in ES.

Any pointers would be greatly appreciated.

2

The events which I want to index have @timestamp field on them which represents the source of truth. But the %{+YYYY.MM.dd} used in creating the index pattern uses the current timestamp of the instance logstash is running on.

No, %{+YYYY.MM.dd} uses the @timestamp value of each event.

1 Like
3

Thanks for the clarification. @magnusbaeck I have a mutation filter which mutated @timestamp so I had to change the ordering of a couple of filters to make sure @timestamp is primed and it does create indices based on that. Really appreciate the swift reply.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.