Malformed Error

romspacenut · January 10, 2017, 9:02pm

Can't figure out these errors I've been seeing. Been seeing a lot of these kind errors in my logstash.log file:

"caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Invalid format: \"2017-01-10T12:36:22-08:00\" is malformed at \"17-01-10T12:36:22-08:00\""}}

Here's my config:

filter {
  if [source] == "/var/log/test.log" {
    multiline {
        pattern => "^(%{SYSLOGTIMESTAMP:timestamp}|%{TIMESTAMP_ISO8601:timestamp})"
        what => "previous"
        negate=> true
    }

    grok {
      break_on_match => false
      match => [ "message", "(%{SYSLOGTIMESTAMP:timestamp}|%{TIMESTAMP_ISO8601:timestamp})%{SPACE}%{WORD:severity} \(%{INT:severity_level}\)\:\s(?<vx_tid>[0-9a-zA-Z-]*)\s(HAR = )(?:(?<har_log>[^\t]*))" ]
    }

    date {
      match => [ "timestamp", "yyyy-MM-dd'T'HH:mm:ssZZ" ]
      timezone => "America/Los_Angeles"
    }
  }
}

Currently using logstash 2.3.4.

magnusbaeck · January 10, 2017, 9:04pm

Please show the full log message. I suspect the error message comes from ES rather than Logstash.

romspacenut · January 10, 2017, 9:33pm

{:timestamp=>"2017-01-10T21:19:19.084000+0000", :message=>"Failed action. ", :status=>400, :action=>["index", {:_id=>nil, :_index=>"filebeat-smith-nginx-2017.01", :_type=>"log", :_routing=>nil}, #<LogStash::Event:0x7e5e344d @metadata_accessors=#<LogStash::Util::Accessors:0x5f6c2c26 @store={"beat"=>"filebeat-smith-nginx", "type"=>"log"}, @lut={"[type]"=>[{"beat"=>"filebeat-smith-nginx", "type"=>"log"}, "type"], "[beat]"=>[{"beat"=>"filebeat-smith-nginx", "type"=>"log"}, "beat"]}>, @cancelled=false, @data={"@timestamp"=>"2017-01-10T21:19:10.000Z", "beat"=>{"hostname"=>"HOST-123", "name"=>"HOST-123"}, "count"=>1, "input_type"=>"log", "message"=>"2017-01-10T13:19:10-08:00 HEAVYINFO (8): tId-e3789f2da5313c1847d560f7013b3e0f HAR = {\n "log": {\n "version": "1.2",\n "creator": {\n "name": "Sample Harchive",\n "version": "0.1",\n "comment": "SERVER_NAME: api.sample.local"\n },\n "entries": [\n\n ]\n }\n}", "offset"=>224089479, "source"=>"/var/log/request.log", "type"=>"log", "@version"=>"1", "mid"=>"smith-nginx.stage", "timestamp"=>"2017-01-10T13:19:10-08:00", "severity"=>"HEAVYINFO", "severity_level"=>"8", "tid"=>"tId-e3789f2da5313c1847d560f7013b3e0f", "har_log"=>"{\n "log": {\n "version": "1.2",\n "creator": {\n "name": "Sample Harchive",\n "version": "0.1",\n "comment": "SERVER_NAME: api.sample.local"\n },\n "entries": [\n\n ]\n }\n}"}, @metadata={"beat"=>"filebeat-smith-nginx", "type"=>"log"}, @accessors=#<LogStash::Util::Accessors:0x4643fbe0 @store={"@timestamp"=>"2017-01-10T21:19:10.000Z", "beat"=>{"hostname"=>"HOST-123", "name"=>"HOST-123"}, "count"=>1, "input_type"=>"log", "message"=>"2017-01-10T13:19:10-08:00 HEAVYINFO (8): tId-e3789f2da5313c1847d560f7013b3e0f HAR = {\n "log": {\n "version": "1.2",\n "creator": {\n "name": "Sample Harchive",\n "version": "0.1",\n "comment": "SERVER_NAME: api.sample.local"\n },\n "entries": [\n\n ]\n }\n}", "offset"=>224089479, "source"=>"/var/log/request.log", "type"=>"log", "@version"=>"1", "mid"=>"smith-nginx.stage", "timestamp"=>"2017-01-10T13:19:10-08:00", "severity"=>"HEAVYINFO", "severity_level"=>"8", "tid"=>"tId-e3789f2da5313c1847d560f7013b3e0f",

romspacenut · January 10, 2017, 9:34pm

"har_log"=>"{\n "log": {\n "version": "1.2",\n "creator": {\n "name": "Sample Harchive",\n "version": "0.1",\n "comment": "SERVER_NAME: api.sample.local"\n },\n "entries": [\n\n ]\n }\n}"}, @lut={"message"=>[{"@timestamp"=>"2017-01-10T21:19:10.000Z", "beat"=>{"hostname"=>"HOST-123", "name"=>"HOST-123"}, "count"=>1, "input_type"=>"log", "message"=>"2017-01-10T13:19:10-08:00 HEAVYINFO (8): tId-e3789f2da5313c1847d560f7013b3e0f HAR = {\n "log": {\n "version": "1.2",\n "creator": {\n "name": "Sample Harchive",\n "version": "0.1",\n "comment": "SERVER_NAME: api.sample.local"\n },\n "entries": [\n\n ]\n }\n}", "offset"=>224089479, "source"=>"/var/log/request.log", "type"=>"log", "@version"=>"1", "mid"=>"smith-nginx.stage", "timestamp"=>"2017-01-10T13:19:10-08:00", "severity"=>"HEAVYINFO", "severity_level"=>"8", "tid"=>"tId-e3789f2da5313c1847d560f7013b3e0f", "har_log"=>"{\n "log": {\n "version": "1.2",\n "creator": {\n "name": "Sample Harchive",\n "version": "0.1",\n "comment": "SERVER_NAME: api.sample.local"\n },\n "entries": [\n\n ]\n }\n}"}, "message"], "timestamp"=>[{"@timestamp"=>"2017-01-10T21:19:10.000Z", "beat"=>{"hostname"=>"HOST-123", "name"=>"HOST-123"}, "count"=>1, "input_type"=>"log", "message"=>"2017-01-10T13:19:10-08:00 HEAVYINFO (8): tId-e3789f2da5313c1847d560f7013b3e0f HAR = {\n "log": {\n "version": "1.2",\n "creator": {\n "name": "Sample Harchive",\n "version": "0.1",\n "comment": "SERVER_NAME: api.sample.local"\n },\n "entries": [\n\n ]\n }\n}", "offset"=>224089479, "source"=>"/var/log/request.log", "type"=>"log", "@version"=>"1", "mid"=>"smith-nginx.stage", "timestamp"=>"2017-01-10T13:19:10-08:00", "severity"=>"HEAVYINFO", "severity_level"=>"8", "tid"=>"tId-e3789f2da5313c1847d560f7013b3e0f", "har_log"=>"{\n "log": {\n "version": "1.2",\n "creator": {\n "name": "Sample Harchive",\n "version": "0.1",\n "comment": "SERVER_NAME: api.sample.local"\n },\n "entries": [\n\n ]\n }\n}"}, "timestamp"], "severity"=>[{"@timestamp"=>"2017-01-10T21:19:10.000Z", "beat"=>{"hostname"=>"HOST-123", "name"=>"HOST-123"}, "count"=>1, "input_type"=>"log", "message"=>"2017-01-10T13:19:10-08:00 HEAVYINFO (8): tId-e3789f2da5313c1847d560f7013b3e0f HAR = {\n "log": {\n "version": "1.2",\n "creator": {\n "name": "Sample Harchive",\n "version": "0.1",\n "comment": "SERVER_NAME: api.sample.local"\n },\n "entries": [\n\n ]\n }\n}", "offset"=>224089479, "source"=>"/var/log/request.log", "type"=>"log", "@version"=>"1", "mid"=>"smith-nginx.stage", "timestamp"=>"2017-01-10T13:19:10-08:00", "severity"=>"HEAVYINFO", "severity_level"=>"8", "tid"=>"tId-e3789f2da5313c1847d560f7013b3e0f", "har_log"=>"{\n "log": {\n "version": "1.2",\n "creator": {\n "name": "Sample Harchive",\n "version": "0.1",\n "comment": "SERVER_NAME: api.sample.local"\n },\n "entries": [\n\n ]\n }\n}"}, "severity"], "severity_level"=>[{"@timestamp"=>"2017-01-10T21:19:10.000Z", "beat"=>{"hostname"=>"HOST-123", "name"=>"HOST-123"}, "count"=>1, "input_type"=>"log", "message"=>"2017-01-10T13:19:10-08:00 HEAVYINFO (8): tId-e3789f2da5313c1847d560f7013b3e0f HAR = {\n "log": {\n "version": "1.2",\n "creator": {\n "name": "Sample Harchive",\n "version": "0.1",\n "comment": "SERVER_NAME: api.sample.local"\n },\n "entries": [\n\n ]\n }\n}", "offset"=>224089479, "source"=>"/var/log/request.log", "type"=>"log", "@version"=>"1", "mid"=>"smith-nginx.stage", "timestamp"=>"2017-01-10T13:19:10-08:00", "severity"=>"HEAVYINFO", "severity_level"=>"8", "tid"=>"tId-e3789f2da5313c1847d560f7013b3e0f", "har_log"=>"{\n "log": {\n "version": "1.2",\n "creator": {\n "name": "Sample Harchive",\n "version": "0.1",\n "comment": "SERVER_NAME: api.sample.local"\n },\n "entries": [\n\n ]\n }\n}"}, "severity_level"], "tid"=>[{"@timestamp"=>"2017-01-10T21:19:10.000Z", "beat"=>{"hostname"=>"HOST-123", "name"=>"HOST-123"}, "count"=>1, "input_type"=>"log", "message"=>"2017-01-10T13:19:10-08:00 HEAVYINFO (8): tId-e3789f2da5313c1847d560f7013b3e0f HAR = {\n "log": {\n "version": "1.2",\n "creator": {\n "name": "Sample Harchive",\n "version": "0.1",\n "comment": "SERVER_NAME: api.sample.local"\n },\n "entries": [\n\n ]\n }\n}",

romspacenut · January 10, 2017, 9:34pm

"offset"=>224089479, "source"=>"/var/log/request.log", "type"=>"log", "@version"=>"1", "mid"=>"smith-nginx.stage", "timestamp"=>"2017-01-10T13:19:10-08:00", "severity"=>"HEAVYINFO", "severity_level"=>"8", "tid"=>"tId-e3789f2da5313c1847d560f7013b3e0f", "har_log"=>"{\n "log": {\n "version": "1.2",\n "creator": {\n "name": "Sample Harchive",\n "version": "0.1",\n "comment": "SERVER_NAME: api.sample.local"\n },\n "entries": [\n\n ]\n }\n}"}, "tid"], "har_log"=>[{"@timestamp"=>"2017-01-10T21:19:10.000Z", "beat"=>{"hostname"=>"HOST-123", "name"=>"HOST-123"}, "count"=>1, "input_type"=>"log", "message"=>"2017-01-10T13:19:10-08:00 HEAVYINFO (8): tId-e3789f2da5313c1847d560f7013b3e0f HAR = {\n "log": {\n "version": "1.2",\n "creator": {\n "name": "Sample Harchive",\n "version": "0.1",\n "comment": "SERVER_NAME: api.sample.local"\n },\n "entries": [\n\n ]\n }\n}", "offset"=>224089479, "source"=>"/var/log/request.log", "type"=>"log", "@version"=>"1", "mid"=>"smith-nginx.stage", "timestamp"=>"2017-01-10T13:19:10-08:00", "severity"=>"HEAVYINFO", "severity_level"=>"8", "tid"=>"tId-e3789f2da5313c1847d560f7013b3e0f", "har_log"=>"{\n "log": {\n "version": "1.2",\n "creator": {\n "name": "Sample Harchive",\n "version": "0.1",\n "comment": "SERVER_NAME: api.sample.local"\n },\n "entries": [\n\n ]\n }\n}"}, "har_log"], "@timestamp"=>[{"@timestamp"=>"2017-01-10T21:19:10.000Z", "beat"=>{"hostname"=>"HOST-123", "name"=>"HOST-123"}, "count"=>1, "input_type"=>"log", "message"=>"2017-01-10T13:19:10-08:00 HEAVYINFO (8): tId-e3789f2da5313c1847d560f7013b3e0f HAR = {\n "log": {\n "version": "1.2",\n "creator": {\n "name": "Sample Harchive",\n "version": "0.1",\n "comment": "SERVER_NAME: api.sample.local"\n },\n "entries": [\n\n ]\n }\n}", "offset"=>224089479, "source"=>"/var/log/request.log", "type"=>"log", "@version"=>"1", "mid"=>"smith-nginx.stage", "timestamp"=>"2017-01-10T13:19:10-08:00", "severity"=>"HEAVYINFO", "severity_level"=>"8", "tid"=>"tId-e3789f2da5313c1847d560f7013b3e0f", "har_log"=>"{\n "log": {\n "version": "1.2",\n "creator": {\n "name": "Sample Harchive",\n "version": "0.1",\n "comment": "SERVER_NAME: api.sample.local"\n },\n "entries": [\n\n ]\n }\n}"}, "@timestamp"], "[source]"=>[{"@timestamp"=>"2017-01-10T21:19:10.000Z", "beat"=>{"hostname"=>"HOST-123", "name"=>"HOST-123"}, "count"=>1, "input_type"=>"log", "message"=>"2017-01-10T13:19:10-08:00 HEAVYINFO (8): tId-e3789f2da5313c1847d560f7013b3e0f HAR = {\n "log": {\n "version": "1.2",\n "creator": {\n "name": "Sample Harchive",\n "version": "0.1",\n "comment": "SERVER_NAME: api.sample.local"\n },\n "entries": [\n\n ]\n }\n}", "offset"=>224089479, "source"=>"/var/log/request.log", "type"=>"log", "@version"=>"1", "mid"=>"smith-nginx.stage", "timestamp"=>"2017-01-10T13:19:10-08:00", "severity"=>"HEAVYINFO", "severity_level"=>"8", "tid"=>"tId-e3789f2da5313c1847d560f7013b3e0f", "har_log"=>"{\n "log": {\n "version": "1.2",\n "creator": {\n "name": "Sample Harchive",\n "version": "0.1",\n "comment": "SERVER_NAME: api.sample.local"\n },\n "entries": [\n\n ]\n }\n}"}, "source"]}>>], :response=>{"create"=>{"_index"=>"filebeat-smith-nginx-2017.01", "_type"=>"log", "_id"=>"AVmKP-Ai3MMKPhA1Iw0h", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [timestamp]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Invalid format: "2017-01-10T13:19:10-08:00" is malformed at "17-01-10T13:19:10-08:00""}}}}, :level=>:warn}

romspacenut · January 10, 2017, 9:35pm

Sorry for the multiple posts, that was one single entry.

magnusbaeck · January 11, 2017, 6:39am

Okay, it's ES that's complaining. What's the mapping of the timestamp field? Use ES's get mapping API.

system · February 8, 2017, 6:39am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Logstash failed to parse timestamp field, Invalid format, malformed (grok, date filter, add_field) Logstash	4	4076	February 23, 2018
Failed to parse [timestamp] invalid formate Logstash	2	1189	July 6, 2017
Logstash time parsing error Logstash	4	1629	July 6, 2017
ISO8601 date format : malformed error message in elasticsearch Logstash	9	9950	November 9, 2017
Syslog filter issue with timestamp Logstash	13	1844	January 3, 2022

Malformed Error

Related Topics