Is @timestamp get value automatically from field timestamp?

Elastic Stack Logstash
1

When we use this configure:

filter {
    grok {
        match => {
            "message" => [
                "(?<timestamp>%{YEAR}-%{MONTHNUM}-%{MONTHDAY} %{HOUR}:%{MINUTE}:%{SECOND}\.\d{3}) %{LOGLEVEL:level} \[%{DATA:feature}\] (?<body>.*$)",
                "(?<timestamp>%{YEAR}-%{MONTHNUM}-%{MONTHDAY} %{HOUR}:%{MINUTE}:%{SECOND}\.\d{3}) %{LOGLEVEL:level} (?<body>.*$)",
                "(?<timestamp>%{YEAR}-%{MONTHNUM}-%{MONTHDAY} %{HOUR}:%{MINUTE}:%{SECOND}\.\d{3}) (?<body>.*$)",
                "(?<body>.*$)"
            ]
        }
    }
    if [timestamp] != "" {
        date {
            match => [
                "timestamp",
                "yyyy-MM-dd HH:mm:ss.SSS"
            ]
        }
    } else {
        ruby {
            code => "event.set('timestamp', event.get('@timestamp').time.localtime + 8*60*60)"
        }
    }
    ruby {
        code => "event.set('@timestamp', event.get('timestamp'))"
    }
...

We can get many error log:

2023-08-24 13:21:53.120 ERROR [logstash.filters.ruby] Ruby exception occurred: wrong argument type String (expected LogStash::Timestamp) {:class=>"TypeError", :backtrace=>["org/logstash/ext/JrubyEventExtLibrary.java:95:in `set'", "(ruby filter code):2:in `block in filter_method'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-ruby-3.1.7/lib/logstash/filters/ruby.rb:93:in `inline_script'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-ruby-3.1.7/lib/logstash/filters/ruby.rb:86:in `filter'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:143:in `do_filter'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:162:in `block in multi_filter'", "org/jruby/RubyArray.java:1792:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:159:in `multi_filter'", "org/logstash/config/ir/compiler/AbstractFilterDelegatorExt.java:115:in `multi_filter'", "(eval):172:in `block in filter_func'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:358:in `filter_batch'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:337:in `worker_loop'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:304:in `block in start_workers'"]}

Error log is out from:

    ruby {
        code => "event.set('@timestamp', event.get('timestamp'))"
    }

Is this configure can not be used in new Logstash version?

2

I find answer for my question:

date {
    match => [
        "timestamp",
        "yyyy-MM-dd HH:mm:ss.SSS"
    ]
}

date.match would analyze filed timestamp with style yyyy-MM-dd HH:mm:ss.SSS, and return result to @timestamp.
:rofl: :rofl: :rofl:

3

Infact, this conf:

date {
    match => [
        "timestamp",
        "yyyy-MM-dd HH:mm:ss.SSS"
    ]
}

equal:

date {
    match => [
        "timestamp",
        "yyyy-MM-dd HH:mm:ss.SSS"
    ]
    target => "@timestamp"
}
4

Why don't you use %{TIMESTAMP_ISO8601:timestamp} like this:

    grok {
        match => {
            "message" => ["%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:level} %{GREEDYDATA:data}"] }
}

?
You can put GREEDYDATA or any other type of regex as you like.

Yes, it's equal to target => "@timestamp" because it's default field, if you don't specify conversion will be copied to @timestamp.

1 Like
5

%{TIMSTAMP_ISO8601} use %{YEAR}-%{MONTHNUM}-%{MONTHDAY}[T ]%{HOUR}:?%{MINUTE}(?::?%{SECOND})?%{ISO8601_TIMEZONE}? reg rule, not %{YEAR}-%{MONTHNUM}-%{MONTHDAY} %{HOUR}:?%{MINUTE}:%{SECOND}.%d{3}.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.