Can hour and minute be appended to index name?

[danibe (Daniel)]

I am trying to create index names with minute resolution since I am still experimenting with ELK and would like to see the effect of various changes in my logstash filter, without having to delete an existing filter (and without overwriting the previous one that was created minutes ago). So I have been trying the following index naming schema, but it seems that Elastic/logstash doesn't like it:

    output {
      elasticsearch {
        hosts => ["http://localhost:9200"]
        index => "CPU-over-time-%{+YYYY.MM.dd}-%{+HH.mm}"
      }
    }

It only accepts -%{+YYYY.MM.dd}.

That is, for index => "CPU-over-time-%{+YYYY.MM.dd}-%{+HH.mm}" it will generate an index named CPU-over-time-. i.e. without the timestamp appended to it.

Why? Is there a way around this to accomplish what I want?

Thanks.

[Badger]

sprintf references can contain hours and minutes

input { generator { count => 1 lines => [ '' ] } }
filter { mutate { add_field => { "someField" => "CPU-over-time-%{+YYYY.MM.dd}-%{+HH.mm}" } } }
output  { stdout { codec => rubydebug { metadata => false } } }

gets me

 "someField" => "CPU-over-time-2020.12.16-16.45",

What does elasticsearch object to?

[danibe (Daniel)]

@Badger Thanks for your reply. My problem is not with 'filter' but rather with 'output'. More precisely, the index name part of 'output'. I edited my question above to clarify what I am getting for index name.

That is, for index => "CPU-over-time-%{+YYYY.MM.dd}-%{+HH.mm}" it will generate an index named CPU-over-time- . i.e. without the timestamp appended to it.

[Badger]

@danibe

My problem is not with 'filter' but rather with 'output'.

Exactly the same interpolation evaluation function is used in both situtations, so if it works in one it will work in the other. If the entire sprintf refence is missing that suggests that you have removed the [@timestamp] field from the message before sending it to the output.

[danibe (Daniel)]

@Badger You are right, I did remove the [@timestamp] field from the message before sending it to the output. Thanks for the amazing pinpointing diagnosis.