How to get part of the path name and add it to the index

Elastic Stack Logstash
#1

0
down vote
favorite
I currently have a file name like this. [SERIALNUMBER][2014_12_04][00_45_22][141204T014214]AB_DEF.log

i basically want to extract the year from the file (2014) and add it to the index name in logstash conf file.logstash.conf Below is my conf file.

input {
file {
path => "C:/ABC/DEF/HJK/LOGS/**/*"
start_position => beginning
type => syslog
}
}
filter {

grok {
    match => { "message" => "%{COMBINEDAPACHELOG}"}
}

}

output {
elasticsearch {
index => "type1logs"
}
stdout {}
}

Please help. thanks

#2

I'm not sure there is a way to do this. There is no timestamp in the file?

#3

the time stamp is saved in different format "[2014_12_04][00_45_22]".
but is there a way to capture the current indexing time stamp and extract only the year , add it to the index?

#4

Yes, use the date filter - https://www.elastic.co/guide/en/logstash/current/plugins-filters-date.html

#5

thanks a lot for your help.

in the output section, i used like below and it is giving results as expected. i didnt add the date filter though.

output {
elasticsearch {
index => "%{+YYYY}_lsclient08"
}
stdout {}
}

YYYY is getting replaced with the current year 2016

#6

You need a date filter if you want the "%{+YYYY}" to be replaced by the event's year field rather than the current year.

#7

ok thanks a lot, it works.

#8