How to create Index Every five 5 Minute

Elastic Stack Logstash
1

Hi All,

Basically I am pulling data from SQL Server into ES.I am able to create Index daily, hourly, minute, second etc ..but i want to create every 5 minute new index ..anybody knows how i can do this?

Is there any way to give elasticsearch output plugin index name into other output plugin lets say exec, pip?

Thanks,
Sumit

2

Are you sure you want to do that? You might overload on shard count per node with an approach like that.

3

Here is how you can create one every minute:
index => "logstash-%{+YYYY-MM-dd_hh:mm}"

4

Hi Aaron,

Thanks for your quick response.
That's OK for me i am going to keep only 5 minute Data.

Thanks,
Sumit

5

Hi Hans,

Basically if you are going to pull from RDBMS and if logstash taking more then 59 Seconds then it will create new Index that is on minute basis..So using this you can create minute wise index not abjectly 5 Minutes.

Thanks,
Sumit

6

Hi Aaron,

Anything Can we do something like this:
Is there any way to give elasticsearch output plugin index name into other output plugin lets say exec, pip?

Thanks,
Sumit

7

Is there any way to give elasticsearch output plugin index name into other output plugin lets say exec, pip?

I'm not sure I get what you mean, but you can of course reuse the same value that you pass to the elasticsearch output's index option in other places. If you explain what you want to accomplish it might be easier to help out.

8

Hi,

It is simple lets say for example:

output {
elasticsearch {
index => "hdap_%{+YYYY-MM-dd_hh:mm}"
}
exec {
command => "echo %{index}"
}
}

So basically i want to pass index name in my shell script.

Thanks,
Sumit

9 
command => "echo hdap_%{+YYYY-MM-dd_hh:mm}"