Help with file name to date logstash grok

Hi, the date isn't included in my log files. But the filename itself has the date. So I'm trying to extract the year month and day from the filename and then put that into a field. But logstash can't parse my filename, I'm not sure what the issue is, here is my logstash config. Any help is appreciated!

I've used the grok debugger in kibana, and it's able to parse my filenames correctly so it's not an issue with the grok expression itself

input {
  file {
    path => "C:/Users/nwilc/AppData/Roaming/MetaQuotes/Terminal/73B7A2420D6397DFF9014A20F1201F97/Logs/UTF8/20230505.log"
    exclude => ["*metaeditor.log"]
    start_position => "beginning"
    sincedb_path => "NUL"
  }
}

filter {
  grok {
    match => { "log.file.path" => "./(?<file_year>[0-9]{4})(?<file_month>[0-9]{2})(?<file_day>[0-9]{2})\.log$" }
    tag_on_failure => ["grok_fail_filename"]
  }
  grok {
    match => {
      "message" => "^%{WORD:code}\t%{INT:number}\t%{TIME:timestamp}\t%{NOTSPACE:category}\t%{GREEDYDATA:log_message}"
    }
  }
  mutate {
    add_field => { "full_timestamp" => "%{file_year}-%{file_month}-%{file_day} %{timestamp}" }
  }
  date {
    match => [ "full_timestamp", "YYYY-MM-dd HH:mm:ss.SSS" ]
    target => "@timestamp"
    remove_field => [ "full_timestamp", "file_year", "file_month", "file_day" ]
  }
}

output {
  elasticsearch {
    hosts => ["http://localhost:9200"]
    index => "metatrader-5-logs"
  }
}

Can you try [log][file][path] instead of log.file.path?

1 Like

ohh perfect thank you so much, that's sorted it!

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.