How to replace @timestamp with logtime?

Elastic Stack Logstash
1

I've read a lot of topics about this but I did not manage with this task. Please, help.
I'm using filebeat, logstash, elasticsearch and kibana to show my logs. Client and server run on Ubuntu 16.04. When I try to replace @timestamp with logtime all new logs disappear but everything works good without "date" filter or if the date-pattern is wrong (in last case @timestamp shows time when message was received).

Log example:

2018-02-07 18:08:15.717 DEBUG 2341 --- [nio-8080-exec-9] company.controller.MyController : some log message : hello

nput config file 02-beats-input.conf:

input {
  beats {
    port => 5044
  }
}

filter config file: 10-java-filter.conf

filter {
  if [type] == "java" {
    grok {
      match => { "message" => "%{TIMESTAMP_ISO8601:logtime} %{LOGLEVEL:level} \[%{DATA:thread}\] %{JAVACLASS:class} %{GREEDYDATA:message}" }
      add_field => [ "received_at", "%{@timestamp}" ]
      overwrite => [ "message" ]
      remove_field => [ "host", "count", "fields", "@version", "input_type", "offset", "source", "tags", "type" ]
  }

  date {
    match => [ "logtime", "yyyy-MM-dd HH:mm:ss.SSS" ]
    target => "@timestamp"
    locale => "en"
    timezone => "UTC"
  }
}

30-elasticsearch-output.conf:

output {
  elasticsearch {
    hosts => ["localhost:9200"]
    index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
  }
}
2

Do all logs contain the logtime field?

3

Try this -

filter {
    mutate {
	add_field => { "message" => "%{typenameru}" }
        convert => [ "datetime", "string" ]
}
date {
    timezone => "Etc/UTC"
    match => ["datetime" , "ISO8601", "yyyy-MM-dd HH:mm:ss.SSS"]
    target => "@timestamp"
    remove_field => [ "datetime", "timestamp" ]
}
}

For me it is helped.

4

yes. All my logs have the same pattern as the first example.
2018-02-07 18:08:15.717 DEBUG 2341 --- [nio-8080-exec-9] company.controller.MyController : some log message : hello

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.