Malformed Error

romspacenut · January 10, 2017, 9:02pm

Can't figure out these errors I've been seeing. Been seeing a lot of these kind errors in my logstash.log file:

"caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Invalid format: \"2017-01-10T12:36:22-08:00\" is malformed at \"17-01-10T12:36:22-08:00\""}}

Here's my config:

filter {
  if [source] == "/var/log/test.log" {
    multiline {
        pattern => "^(%{SYSLOGTIMESTAMP:timestamp}|%{TIMESTAMP_ISO8601:timestamp})"
        what => "previous"
        negate=> true
    }

    grok {
      break_on_match => false
      match => [ "message", "(%{SYSLOGTIMESTAMP:timestamp}|%{TIMESTAMP_ISO8601:timestamp})%{SPACE}%{WORD:severity} \(%{INT:severity_level}\)\:\s(?<vx_tid>[0-9a-zA-Z-]*)\s(HAR = )(?:(?<har_log>[^\t]*))" ]
    }

    date {
      match => [ "timestamp", "yyyy-MM-dd'T'HH:mm:ssZZ" ]
      timezone => "America/Los_Angeles"
    }
  }
}

Currently using logstash 2.3.4.

magnusbaeck · January 10, 2017, 9:04pm

Please show the full log message. I suspect the error message comes from ES rather than Logstash.

romspacenut · January 10, 2017, 9:33pm

{:timestamp=>"2017-01-10T21:19:19.084000+0000", :message=>"Failed action. ", :status=>400, :action=>["index", {:_id=>nil, :_index=>"filebeat-smith-nginx-2017.01", :_type=>"log", :_routing=>nil}, #<LogStash::Event:0x7e5e344d @metadata_accessors=#<LogStash::Util::Accessors:0x5f6c2c26 @store={"beat"=>"filebeat-smith-nginx", "type"=>"log"}, @lut={"[type]"=>[{"beat"=>"filebeat-smith-nginx", "type"=>"log"}, "type"], "[beat]"=>[{"beat"=>"filebeat-smith-nginx", "type"=>"log"}, "beat"]}>, @cancelled=false, @data={"@timestamp"=>"2017-01-10T21:19:10.000Z", "beat"=>{"hostname"=>"HOST-123", "name"=>"HOST-123"}, "count"=>1, "input_type"=>"log", "message"=>"2017-01-10T13:19:10-08:00 HEAVYINFO (8): tId-e3789f2da5313c1847d560f7013b3e0f HAR = {\n "log": {\n "version": "1.2",\n "creator": {\n "name": "Sample Harchive",\n "version": "0.1",\n "comment": "SERVER_NAME: api.sample.local"\n },\n "entries": [\n\n ]\n }\n}", "offset"=>224089479, "source"=>"/var/log/request.log", "type"=>"log", "@version"=>"1", "mid"=>"smith-nginx.stage", "timestamp"=>"2017-01-10T13:19:10-08:00", "severity"=>"HEAVYINFO", "severity_level"=>"8", "tid"=>"tId-e3789f2da5313c1847d560f7013b3e0f", "har_log"=>"{\n "log": {\n "version": "1.2",\n "creator": {\n "name": "Sample Harchive",\n "version": "0.1",\n "comment": "SERVER_NAME: api.sample.local"\n },\n "entries": [\n\n ]\n }\n}"}, @metadata={"beat"=>"filebeat-smith-nginx", "type"=>"log"}, @accessors=#<LogStash::Util::Accessors:0x4643fbe0 @store={"@timestamp"=>"2017-01-10T21:19:10.000Z", "beat"=>{"hostname"=>"HOST-123", "name"=>"HOST-123"}, "count"=>1, "input_type"=>"log", "message"=>"2017-01-10T13:19:10-08:00 HEAVYINFO (8): tId-e3789f2da5313c1847d560f7013b3e0f HAR = {\n "log": {\n "version": "1.2",\n "creator": {\n "name": "Sample Harchive",\n "version": "0.1",\n "comment": "SERVER_NAME: api.sample.local"\n },\n "entries": [\n\n ]\n }\n}", "offset"=>224089479, "source"=>"/var/log/request.log", "type"=>"log", "@version"=>"1", "mid"=>"smith-nginx.stage", "timestamp"=>"2017-01-10T13:19:10-08:00", "severity"=>"HEAVYINFO", "severity_level"=>"8", "tid"=>"tId-e3789f2da5313c1847d560f7013b3e0f",

romspacenut · January 10, 2017, 9:34pm

"har_log"=>"{\n "log": {\n "version": "1.2",\n "creator": {\n "name": "Sample Harchive",\n "version": "0.1",\n "comment": "SERVER_NAME: api.sample.local"\n },\n "entries": [\n\n ]\n }\n}"}, @lut={"message"=>[{"@timestamp"=>"2017-01-10T21:19:10.000Z", "beat"=>{"hostname"=>"HOST-123", "name"=>"HOST-123"}, "count"=>1, "input_type"=>"log", "message"=>"2017-01-10T13:19:10-08:00 HEAVYINFO (8): tId-e3789f2da5313c1847d560f7013b3e0f HAR = {\n "log": {\n "version": "1.2",\n "creator": {\n "name": "Sample Harchive",\n "version": "0.1",\n "comment": "SERVER_NAME: api.sample.local"\n },\n "entries": [\n\n ]\n }\n}", "offset"=>224089479, "source"=>"/var/log/request.log", "type"=>"log", "@version"=>"1", "mid"=>"smith-nginx.stage", "timestamp"=>"2017-01-10T13:19:10-08:00", "severity"=>"HEAVYINFO", "severity_level"=>"8", "tid"=>"tId-e3789f2da5313c1847d560f7013b3e0f", "har_log"=>"{\n "log": {\n "version": "1.2",\n "creator": {\n "name": "Sample Harchive",\n "version": "0.1",\n "comment": "SERVER_NAME: api.sample.local"\n },\n "entries": [\n\n ]\n }\n}"}, "message"], "timestamp"=>[{"@timestamp"=>"2017-01-10T21:19:10.000Z", "beat"=>{"hostname"=>"HOST-123", "name"=>"HOST-123"}, "count"=>1, "input_type"=>"log", "message"=>"2017-01-10T13:19:10-08:00 HEAVYINFO (8): tId-e3789f2da5313c1847d560f7013b3e0f HAR = {\n "log": {\n "version": "1.2",\n "creator": {\n "name": "Sample Harchive",\n "version": "0.1",\n "comment": "SERVER_NAME: api.sample.local"\n },\n "entries": [\n\n ]\n }\n}", "offset"=>224089479, "source"=>"/var/log/request.log", "type"=>"log", "@version"=>"1", "mid"=>"smith-nginx.stage", "timestamp"=>"2017-01-10T13:19:10-08:00", "severity"=>"HEAVYINFO", "severity_level"=>"8", "tid"=>"tId-e3789f2da5313c1847d560f7013b3e0f", "har_log"=>"{\n "log": {\n "version": "1.2",\n "creator": {\n "name": "Sample Harchive",\n "version": "0.1",\n "comment": "SERVER_NAME: api.sample.local"\n },\n "entries": [\n\n ]\n }\n}"}, "timestamp"], "severity"=>[{"@timestamp"=>"2017-01-10T21:19:10.000Z", "beat"=>{"hostname"=>"HOST-123", "name"=>"HOST-123"}, "count"=>1, "input_type"=>"log", "message"=>"2017-01-10T13:19:10-08:00 HEAVYINFO (8): tId-e3789f2da5313c1847d560f7013b3e0f HAR = {\n "log": {\n "version": "1.2",\n "creator": {\n "name": "Sample Harchive",\n "version": "0.1",\n "comment": "SERVER_NAME: api.sample.local"\n },\n "entries": [\n\n ]\n }\n}", "offset"=>224089479, "source"=>"/var/log/request.log", "type"=>"log", "@version"=>"1", "mid"=>"smith-nginx.stage", "timestamp"=>"2017-01-10T13:19:10-08:00", "severity"=>"HEAVYINFO", "severity_level"=>"8", "tid"=>"tId-e3789f2da5313c1847d560f7013b3e0f", "har_log"=>"{\n "log": {\n "version": "1.2",\n "creator": {\n "name": "Sample Harchive",\n "version": "0.1",\n "comment": "SERVER_NAME: api.sample.local"\n },\n "entries": [\n\n ]\n }\n}"}, "severity"], "severity_level"=>[{"@timestamp"=>"2017-01-10T21:19:10.000Z", "beat"=>{"hostname"=>"HOST-123", "name"=>"HOST-123"}, "count"=>1, "input_type"=>"log", "message"=>"2017-01-10T13:19:10-08:00 HEAVYINFO (8): tId-e3789f2da5313c1847d560f7013b3e0f HAR = {\n "log": {\n "version": "1.2",\n "creator": {\n "name": "Sample Harchive",\n "version": "0.1",\n "comment": "SERVER_NAME: api.sample.local"\n },\n "entries": [\n\n ]\n }\n}", "offset"=>224089479, "source"=>"/var/log/request.log", "type"=>"log", "@version"=>"1", "mid"=>"smith-nginx.stage", "timestamp"=>"2017-01-10T13:19:10-08:00", "severity"=>"HEAVYINFO", "severity_level"=>"8", "tid"=>"tId-e3789f2da5313c1847d560f7013b3e0f", "har_log"=>"{\n "log": {\n "version": "1.2",\n "creator": {\n "name": "Sample Harchive",\n "version": "0.1",\n "comment": "SERVER_NAME: api.sample.local"\n },\n "entries": [\n\n ]\n }\n}"}, "severity_level"], "tid"=>[{"@timestamp"=>"2017-01-10T21:19:10.000Z", "beat"=>{"hostname"=>"HOST-123", "name"=>"HOST-123"}, "count"=>1, "input_type"=>"log", "message"=>"2017-01-10T13:19:10-08:00 HEAVYINFO (8): tId-e3789f2da5313c1847d560f7013b3e0f HAR = {\n "log": {\n "version": "1.2",\n "creator": {\n "name": "Sample Harchive",\n "version": "0.1",\n "comment": "SERVER_NAME: api.sample.local"\n },\n "entries": [\n\n ]\n }\n}",

romspacenut · January 10, 2017, 9:34pm

"offset"=>224089479, "source"=>"/var/log/request.log", "type"=>"log", "@version"=>"1", "mid"=>"smith-nginx.stage", "timestamp"=>"2017-01-10T13:19:10-08:00", "severity"=>"HEAVYINFO", "severity_level"=>"8", "tid"=>"tId-e3789f2da5313c1847d560f7013b3e0f", "har_log"=>"{\n "log": {\n "version": "1.2",\n "creator": {\n "name": "Sample Harchive",\n "version": "0.1",\n "comment": "SERVER_NAME: api.sample.local"\n },\n "entries": [\n\n ]\n }\n}"}, "tid"], "har_log"=>[{"@timestamp"=>"2017-01-10T21:19:10.000Z", "beat"=>{"hostname"=>"HOST-123", "name"=>"HOST-123"}, "count"=>1, "input_type"=>"log", "message"=>"2017-01-10T13:19:10-08:00 HEAVYINFO (8): tId-e3789f2da5313c1847d560f7013b3e0f HAR = {\n "log": {\n "version": "1.2",\n "creator": {\n "name": "Sample Harchive",\n "version": "0.1",\n "comment": "SERVER_NAME: api.sample.local"\n },\n "entries": [\n\n ]\n }\n}", "offset"=>224089479, "source"=>"/var/log/request.log", "type"=>"log", "@version"=>"1", "mid"=>"smith-nginx.stage", "timestamp"=>"2017-01-10T13:19:10-08:00", "severity"=>"HEAVYINFO", "severity_level"=>"8", "tid"=>"tId-e3789f2da5313c1847d560f7013b3e0f", "har_log"=>"{\n "log": {\n "version": "1.2",\n "creator": {\n "name": "Sample Harchive",\n "version": "0.1",\n "comment": "SERVER_NAME: api.sample.local"\n },\n "entries": [\n\n ]\n }\n}"}, "har_log"], "@timestamp"=>[{"@timestamp"=>"2017-01-10T21:19:10.000Z", "beat"=>{"hostname"=>"HOST-123", "name"=>"HOST-123"}, "count"=>1, "input_type"=>"log", "message"=>"2017-01-10T13:19:10-08:00 HEAVYINFO (8): tId-e3789f2da5313c1847d560f7013b3e0f HAR = {\n "log": {\n "version": "1.2",\n "creator": {\n "name": "Sample Harchive",\n "version": "0.1",\n "comment": "SERVER_NAME: api.sample.local"\n },\n "entries": [\n\n ]\n }\n}", "offset"=>224089479, "source"=>"/var/log/request.log", "type"=>"log", "@version"=>"1", "mid"=>"smith-nginx.stage", "timestamp"=>"2017-01-10T13:19:10-08:00", "severity"=>"HEAVYINFO", "severity_level"=>"8", "tid"=>"tId-e3789f2da5313c1847d560f7013b3e0f", "har_log"=>"{\n "log": {\n "version": "1.2",\n "creator": {\n "name": "Sample Harchive",\n "version": "0.1",\n "comment": "SERVER_NAME: api.sample.local"\n },\n "entries": [\n\n ]\n }\n}"}, "@timestamp"], "[source]"=>[{"@timestamp"=>"2017-01-10T21:19:10.000Z", "beat"=>{"hostname"=>"HOST-123", "name"=>"HOST-123"}, "count"=>1, "input_type"=>"log", "message"=>"2017-01-10T13:19:10-08:00 HEAVYINFO (8): tId-e3789f2da5313c1847d560f7013b3e0f HAR = {\n "log": {\n "version": "1.2",\n "creator": {\n "name": "Sample Harchive",\n "version": "0.1",\n "comment": "SERVER_NAME: api.sample.local"\n },\n "entries": [\n\n ]\n }\n}", "offset"=>224089479, "source"=>"/var/log/request.log", "type"=>"log", "@version"=>"1", "mid"=>"smith-nginx.stage", "timestamp"=>"2017-01-10T13:19:10-08:00", "severity"=>"HEAVYINFO", "severity_level"=>"8", "tid"=>"tId-e3789f2da5313c1847d560f7013b3e0f", "har_log"=>"{\n "log": {\n "version": "1.2",\n "creator": {\n "name": "Sample Harchive",\n "version": "0.1",\n "comment": "SERVER_NAME: api.sample.local"\n },\n "entries": [\n\n ]\n }\n}"}, "source"]}>>], :response=>{"create"=>{"_index"=>"filebeat-smith-nginx-2017.01", "_type"=>"log", "_id"=>"AVmKP-Ai3MMKPhA1Iw0h", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [timestamp]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Invalid format: "2017-01-10T13:19:10-08:00" is malformed at "17-01-10T13:19:10-08:00""}}}}, :level=>:warn}

romspacenut · January 10, 2017, 9:35pm

Sorry for the multiple posts, that was one single entry.

magnusbaeck · January 11, 2017, 6:39am

Okay, it's ES that's complaining. What's the mapping of the timestamp field? Use ES's get mapping API.

system · February 8, 2017, 6:39am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Logstash failed to parse timestamp field, Invalid format, malformed (grok, date filter, add_field) Logstash	4	4076	February 23, 2018
Malformed dateformat Logstash	3	1037	July 6, 2017
Failed to parse [timestamp] invalid formate Logstash	2	1189	July 6, 2017
Date Format is Malformed Logstash	5	710	October 31, 2018
Issue parsing date Logstash	3	688	July 13, 2017

Malformed Error

Related topics