Cannot parse empty date

Elastic Stack Logstash
1

Hi, I have a log that has a json field inside that can have empty fields, specifically I have a date field, the log can be like this

"2022-11-28 09:24:46:705"|"+0100"|"transId: xxxxxx"|"resId: xxxxxx"|"1.1.1.1"|"https://example.com/xxxxxx"|"HTTP/1.1"|"EXE"|"GET"|"404"|"{"codCpi":"","codFis": "","codSap": "","codState": "","datBirth": "", "xml": ""}"|"token : xxxx"

Have this rule for match:

filter {
  grok {
    match => { "message" => "\"%{TIMESTAMP_ISO8601:timestamp}\"\|\"%{DATA:tz}\"\|\"transId: %{GREEDYDATA:transactionId}\"\|\"resId: %{GREEDYDATA:responseId}\"\|\"%{IP:ip}\"\|\"%{GREEDYDATA:url}\"\|\"%{DATA:httpver}\"\|\"%{DATA:exe}\"\|\"%{WORD:httpverb}\"\|\"%{GREEDYDATA:httpCodeResponse}\"\|\"%{DATA:contentRequest}\"\|\"%{GREEDYDATA:token}\"" }
  }
  json {
    source => "contentRequest"
  }
}

With this match i have this error:

"error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse field [datBirth] of type [date] in document with id 'VSFI54QBv1ziCbBLadWr'. Preview of field's value: ''", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"cannot parse empty date"}}}}

Instead, there is no error if the field datBirth is filled in

2

To confirm, are you only receiving this error for events where datBirth is empty, and parsing is successful when it's populated?

If so you could try specifying zero or one occurrences using the ? operator, as per the below:

3

I confirm that if the field is filled in, it works.
Unfortunately it is a service of which I have no control, I don't think it can be changed

4

You can remove the field if it is empty.

if [dateBirth] == "" {
    mutate {
        remove_field => ["dateBirth"]
    }
}
1 Like
5

Sorry I did not understand :grinning:
Seems to work with this rule

filter {
  grok {
    match => { "message" => "\"%{TIMESTAMP_ISO8601:timestamp}\"\|\"%{DATA:tz}\"\|\"transId: %{GREEDYDATA:transactionId}\"\|\"resId: %{GREEDYDATA:responseId}\"\|\"%{IP:ip}\"\|\"%{GREEDYDATA:url}sap/API/SAP/codFiscale\/%{GREEDYDATA:codicefiscale}\"\|\"%{DATA:httpver}\"\|\"%{DATA:gruppo}\"\|\"%{WORD:httpverb}\"\|\"%{GREEDYDATA:httpCodeResponse}\"\|\"(?<contentRequest>{.*})\"\|\"%{GREEDYDATA:token}\"" }
  }
  json {
    source => "contentRequest"
  }
}

Thanks!

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.