Time zone in the field?

Good day :slightly_smiling_face:
(I probably have a simple question, but I haven't been able to figure it out on my own)

Please tell me how to properly configure the date display in the fields?

I have a service that, by a script, sends data to a node with Logstash.
Here's a sample:

MOVIE125|24ad515d4|0009|01/09/2020 19:53:25|01/09/2020 19:53:35|STAR555|007

Here is the Logstash config file:

input {
  file {
    path => "/elk/data/service*"
#    start_position => "beginning"
  }
}
 filter {
    csv {
        separator => "|"
            columns   => ["SERVICE_NAME", "RECORD_TYPE", "SERVICE_ID", "SHIPMENT_TIME", "DELIVERY_TIME", "DEST_OPERATOR", "TOTAL_PARTS"]
    }
    grok {
        match => { "time" => "%{COMBINEDAPACHELOG}"}
    }
    date {
    match => [ "timestamp", "dd/MM/YY HH:mm" ]
    }
    }
output {
    elasticsearch {
        hosts => [ "localhost:9200" ]
        user => "elastic"
        password => "******************"
        cacert => '/usr/share/logstash/elastic-certificates.pem'
        index => "service25"
        }
}

As you already understood :grinning:, with Logstash data is sent to Elasticsearch.
Here is the index mapping diagram:

PUT service25
{
  "mappings": {
    "properties": {
      "SERVICE_NAME": {
        "type": "text",
        "fields": {
          "exact": {
            "type": "keyword"
          }
        }
      },
     "RECORD_TYPE": {
       "type": "text",
        "fields": {
          "exact": {
            "type": "keyword"
          }
        }
      },
      "SERVICE_ID": {
        "type": "text",
        "fields": {
          "exact": {
            "type": "keyword"
          }
        }
      },
      "SHIPMENT_TIME": {
        "type": "date",
        "format": "dd/MM/yyyy HH:mm:ss"
      },
      "DELIVERY_TIME": {
        "type": "date",
        "format": "dd/MM/yyyy HH:mm:ss"
      },
      "DEST_OPERATOR": {
        "type": "text",
        "fields": {
          "exact": {
            "type": "keyword"
          }
        }
      },
      "TOTAL_PARTS": {
        "type": "text",
        "fields": {
          "exact": {
            "type": "keyword"
          }
        }
      }
      }
  }
}

At the exit in Kibana, I get the time in the "SHIPMENT_TIME" and "DELIVERY_TIME" fields for three hours more.

Annotation 2020-09-01 210643

Time zone on my server and on pc is GMT + 3

I changed the "Timezone for date formatting" parameter in Advanced Settings Kibana from "Browser" to Етс / GMT + 0, but then I get the wrong time (minus three hours) in the @timestamp field

Please tell me where to make the change to get the correct time in the fields?
Thanks in advance to everyone!

You are using a mapping template to tell elasticsearch how to parse the SHIPMENT_TIME and DELIVERY_TIME fields, but I am not aware of a way to tell elasticsearch what timezone to use, so it assumes the times are UTC.

Instead of having elasticsearch parse them you can have logstash parse them

date { match => [ "DELIVERY_TIME", "dd/MM/yyyy HH:mm:ss" ] timezone => "Etc/GMT-3" }

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.