How to convert time typestring into date type

Elastic Stack Logstash
1

HI,
I want to convert on string time field into date type. My log is json format and its getting index into this field,

bridgeNemoJenkinsData.time taken.keyword string

I have implemented date filter to convert this as shown below,

date{
                        match => ["bridgeNemoJenkinsData.time taken","HH:mm:SSS"]
                        target => ["bridgeNemoJenkinsData.time taken"]
                 }

and get index like this ,

bridgeNemoJenkinsData.time taken 00:08:027

2

Can you please post your full config and example input lines and any errors you have? You should also not have a space in the field name, change it to bridgeNemoJenkinsData.time_taken or similar.

3

This is my filter section

 mutate{
                                add_field => {
                        "ci_cd" => "bridgeNemoDataType"
                        }
                        gsub => [
                               "message", "time taken", "time_taken"
                        ]
                 }
                 date{
                        match => ["bridgeNemoJenkinsData.time_taken","HH:mm:SSS"]
                        target => ["bridgeNemoJenkinsData.time_taken"]
                 }
                ruby {
                        init => "require 'socket'"
                        code => "event.set('logstash-hostname',Socket.gethostname)"
                }
                json{
                        source => "message"
                }

I dont have any error

4

Can you show some input lines and also what ends up being in the field?

5 
  kafka {
                type => "doc"
                bootstrap_servers => "xxx:443"
                topics => ["PaymentNet_500000846_xx"]
                security_protocol => "SSL"
                ssl_key_password => "kafkae1"
                ssl_keystore_location => "/tls/kafkae1-signed.keystore"
                ssl_keystore_password => "kafkae1"
                ssl_truststore_location => "/tls/kafka01-client-truststore.jks"
                ssl_truststore_password => "kafka123"
                key_deserializer_class => "org.apache.kafka.common.serialization.StringDeserializer"
                value_deserializer_class => "org.apache.kafka.common.serialization.StringDeserializer"
                group_id => "PaymentNet_group-id_500000846"
                auto_offset_reset => "earliest"
        }

this is my message

{"bridgeNemoDataType":"nemo-jenkins-data","AIM_ID":"AIM","build_url":"https://cxx","stash_Url":"https://xx/tikr.git","bridgeNemoJenkinsData":{"build_Id":"22","current_branch":"develop","user":"Rohit Mohta","node_name":"jenkins-slave-8ncrq","stage_name":"DockerPublish","stage_result":"SUCCESS","pipeline_function":"Docker_Publish","time_taken":"00:21:448"}}

6

any update on my below input

7

What does the end result look like?

8

end result is look like that still in string type instead of data

9

in Kibana? Did you change the mapping to date instead of string?

10

How can i change the mapping in kibana

11

Delete your indexes if you don't care about the data and it should be automatically picked up, otherwise you will have to put a template:

https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-templates.html

12

Actually i tried to parsed that value into new index but even though it shows as string type

13

What is the index name? Can you go to the developer console in Kibana and do

GET index-name

And post the results?

14

please find my result

{
  "test-000001": {
    "aliases": {},
    "mappings": {
      "doc": {
        "properties": {
          "@timestamp": {
            "type": "date"
          },
          "@version": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "ci_cd": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "logstash-hostname": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "message": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "tags": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "type": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          }
        }
      }
    },
    "settings": {
      "index": {
        "creation_date": "1542959163031",
        "number_of_shards": "3",
        "number_of_replicas": "1",
        "uuid": "K7fNcr1eQaaFPDyHbg_wKg",
        "version": {
          "created": "6020499"
        },
        "provided_name": "test-000001"
      }
    }
  }
}
15

There is no field in there called "bridgeNemoJenkinsData.time_taken"

16

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.