Différence entre le Grok Debugger et le grok dans le filter

[martel (fabrice)]

Bonjour,
Je ne trouve pas la bonne manière de faire remplacer ce @timestamp par celui qui se trouve dans mon message.

Le principe est simple 1 log = 1 ligne, c'est au format JSON
{"startTime":"2023-01-17 14:17:50.238","endTime":"2023-01-17 14:17:50.534","transactionId":"G4D7Y4W0G5S3A1O54H5",etc.....

je fais ceci dans mon filer :

  grok {
    match => [ "message", ".\"startTime\":\"%{TIMESTAMP_ISO8601:a}%{GREEDYDATA:message}" ]
  }
  date {
    match => [ "a" , "yyyy-MM-dd HH:mm:ss.SSS" ]
    target => "@timestamp"
    timezone => "UTC"
    add_field => { "tmp_datetime" => "%{a}" }
  }

  json {
        source => "message"
      }
}

J'ai tenté beaucoup de chose, avec ou sans grok, uniquement avec date etc...

par contre quand je copie colle le pattern du grok, dans le grok debugger de kibana, il me retourne bien deux éléments

{
  "message": "\",\"endTime\":\"2023-01-17 14:17:50.534\",\"transactionId\"etc..........}",
  "a": "2023-01-17 14:17:50.238"
}

Si vous avez une idée, je suis preneur.

merci d'avance

[Rios]

Welcome to the Elastic community/Bienvenue dans la communauté Elastic

I'm not very good at French/ Je ne suis pas vraiment bon en français

I assume you want @timestamp became startTime. First parse JSON, then convert to date.

filter {

  json {
        source => "message"
      }
	  
  date {
    match => [ "startTime" , "yyyy-MM-dd HH:mm:ss.SSS" ]
    target => "@timestamp"
    timezone => "UTC"
    add_field => { "tmp_datetime" => "%{startTime}" }
  }

}

Result:

{
       "@timestamp" => 2023-01-17T14:17:50.238Z,
        "startTime" => "2023-01-17 14:17:50.238",
          "endTime" => "2023-01-17 14:17:50.534",
     "tmp_datetime" => "2023-01-17 14:17:50.238",
    "transactionId" => "G4D7Y4W0G5S3A1O54H5"
}

[martel (fabrice)]

I had done this before, but it didn't work.
then I deleted the previous indexes then recreated the data_view based on the filebeat-* pattern once saved and restarted the filebeats, it worked again....

anyway, thanks for confirming

good day

[Rios]

Yes, you have right. The first time was created as text field as default. You have to delete data mapping every time when you change a data type.