Syslog-ng logstash date_time_parse_exception

Elastic Stack Logstash
1

Hello!
I am trying to send logs from Syslog-ng to logstash and no logs are indexed in elasticsearch because of date time parse exception. I cannot use Beats on the devices, so the only option is Syslog-ng. Could anyone help me solve this issue?

Elasticsearch and Logstash version: 7.10.2.

Syslog-ng config:

destination d_server1 {
    network("X.X.X.X"
        transport("TCP")
        port(5514)
    );
};

filter f_server1 { level(debug..emerg); };

log { source(s_system); filter(f_server1); destination(d_server1); };

Logstash log:

[2021-02-03T12:38:49,898][WARN ][logstash.outputs.elasticsearch][main][84d3957836db029279ac38c116ae41eb95956720602d0201eac13ff144d7ef7a] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"logstash", :routing=>nil, :_type=>"_doc"}, #<LogStash::Event:0x709bc6cc>], :response=>{"index"=>{"_index"=>"logstash-2020.10.27-000005", "_type"=>"_doc", "_id"=>"7JR5Z3cBGVvnz4yVKG3a", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse field [timestamp] of type [date] in document with id '7JR5Z3cBGVvnz4yVKG3a'. Preview of field's value: 'Feb  3 12:38:49.000'", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"failed to parse date field [Feb  3 12:38:49.000] with format [strict_date_optional_time||epoch_millis]", "caused_by"=>{"type"=>"date_time_parse_exception", "reason"=>"date_time_parse_exception: Failed to parse with all enclosed parsers"}}}}}}

Input config:

input {
  syslog {
    id => "logstash-syslog"
    host => "X.X.X.X"
    port => 5514
    proxy_protocol => true
  }
2

Tried to change syslog-ng forwarding driver from network() - RFC3614 to syslog() - RFC5424, now at least the events are indexed into elasticsearch, but now all messages are not properly parsed and all documents have error - _grokparsefailure_sysloginput. Not sure why logstash can't parse any syslog-ng logs correctly ...

{
  "_index": "logstash-2020.10.27-000005",
  "_type": "_doc",
  "_id": "pmEncXcB3OwTgPcv-nqC",
  "_version": 1,
  "_score": null,
  "_source": {
    "message": "126 <85>1 2021-02-05T09:46:21.000+02:00 host-x httpd 6723 - - 127.0.0.1 200 anonymous GET /cgi/example.cgi\n",
    "severity": 0,
    "host": "X.X.X.X",
    "facility_label": "kernel",
    "facility": 0,
    "priority": 0,
    "@version": "1",
    "tags": [
      "_grokparsefailure_sysloginput"
    ],
    "@timestamp": "2021-02-05T07:46:21.340Z",
    "severity_label": "Emergency"
  },
  "fields": {
    "@timestamp": [
      "2021-02-05T07:46:21.340Z"
    ]
  },
  "sort": [
    1612511181340
  ]
}
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.