Replace @timestamp to time from actual log file

Elastic Stack Logstash
1

Hello it is possible to Change @timestamp to time from actual log file

I use this config

filter {
  date {
    match => ["timestamp", "ISO8601"]
  }
}

But it does not work

2

What does not work?
Could you give a concrete example of the event before and after?

3

My index names is logstash-2020.06.09 but the @timestamp is

image

4

logstash not parsing @timestamp correctly, it is possible to change @timestamp to time from actual log file

5

Most of the people who might try to help you are not psychic. If you complain about the wrong output, you need to give us your input. If you cannot solve your problem while having all the information, you cannot expect us to solve it with only the half of it.

1 Like
6

my config logstash is

input {
beats  {
port => "5044"
}
}  

filter {

csv {
separator => ","
columns => ["chaine", "job", "date_plan", "statut", "date_debut", "date_fin", "serveur", "numero_passage", "application", "sous_application"]
skip_header => "true"
}

date {
match => [ "timestamp", "YYYY-MM-dd;HH:mm:ss.SSS"]
target => "@timestamp"
}

date {
match => [ "date_plan" , "YYYY-MM-dd" ]
timezone => "Europe/Paris"
}

date {
match => [ "date_debut" , "YYYY-MM-dd HH:mm:ss" ]
timezone => "Europe/Paris"
}

date {
match => [ "date_fin" , "YYYY-MM-dd HH:mm:ss" ]
timezone => "Europe/Paris"
}

mutate {
convert => { "numero_passage" => "integer" }
}
}

output {
elasticsearch {
hosts => ["http://elasticsearch:9200"]
index => "<logstash-{now/d}>"

}
stdout {
   codec => rubydebug
}
}
7

I hope you realize that this configuration would overwrite @timestamp again and again and again … That field is the default target of the date filter.

1 Like
8

I know but @timestamps is not parsing correctly with today's date

9

I just meant to say that, even if it would parse the date correctly, the value wouldn't be kept. (And you still haven't posted your input.)

10

my config filebeat is

filebeat.inputs:
- type: log
  enabled: true
  paths:  
     - /data/volumes/monitoring/logstash/logCtrlM/poc_prode.csv

exclude_lines: ['^chaine']

output.logstash:
  hosts: ["logstash:5044"]
11

Could you please indent your code correctly to make it easier to read?
Also post the first lines of your CSV file.

12

Adding to @Jenni's answer the link to the documentation. Date filter plugin | Logstash Reference [8.11] | Elastic

It says:

Store the matching timestamp into the given target field. If not provided, default to updating the @timestamp field of the event.

Which means that date_fin will at the end overwrite any other parsed values for @timestamp field.

13

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.