I can't parse my logs anymore

Elastic Stack Logstash
1 
[2018-07-09T10:56:45,779][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>"c5653e88eeea542ff562333cfea5db5e6bba4f3e", :_index=>"logstash-2018.07.09", :_type=>"doc", :_routing=>nil}, #<LogStash::Event:0x3b34e2b6>], :response=>{"index"=>{"_index"=>"logstash-2018.07.09", "_type"=>"doc", "_id"=>"c5653e88eeea542ff562333cfea5db5e6bba4f3e", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [Packets]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Invalid format: \"client-to-server\""}}}}}
[2018-07-09T10:56:56,257][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>"773f81c53b64840d1554957578c343a7cafb5666", :_index=>"logstash-2018.07.09", :_type=>"doc", :_routing=>nil}, #<LogStash::Event:0x78df84a2>], :response=>{"index"=>{"_index"=>"logstash-2018.07.09", "_type"=>"doc", "_id"=>"773f81c53b64840d1554957578c343a7cafb5666", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [Packets]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Invalid format: \"client-to-server\""}}}}}
[2018-07-09T10:56:57,856][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>"b68825125101462d447528e8e615296c0659b7eb", :_index=>"logstash-2018.07.09", :_type=>"doc", :_routing=>nil}, #<LogStash::Event:0x337070cb>], :response=>{"index"=>{"_index"=>"logstash-2018.07.09", "_type"=>"doc", "_id"=>"b68825125101462d447528e8e615296c0659b7eb", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [Packets]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Invalid format: \"client-to-server\""}}}}}
[2018-07-09T10:57:29,408][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>"bd351478bab25d5ee5bb9b15fab5ec610afa7473", :_index=>"logstash-2018.07.09", :_type=>"doc", :_routing=>nil}, #<LogStash::Event:0x3287cd36>], :response=>{"index"=>{"_index"=>"logstash-2018.07.09", "_type"=>"doc", "_id"=>"bd351478bab25d5ee5bb9b15fab5ec610afa7473", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [Packets]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Invalid format: \"client-to-server\""}}}}}
[2018-07-09T10:57:36,743][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>"b97f4b7558e762234d8ce54e1a18f6c36d0a7282", :_index=>"logstash-2018.07.09", :_type=>"doc", :_routing=>nil}, #<LogStash::Event:0x1fc29481>], :response=>{"index"=>{"_index"=>"logstash-2018.07.09", "_type"=>"doc", "_id"=>"b97f4b7558e762234d8ce54e1a18f6c36d0a7282", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [Packets]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Invalid format: \"client-to-server\""}}}}}
[2018-07-09T10:57:52,134][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>"62e5abb6f430534c6e3df55faaa919849f1febe8", :_index=>"logstash-2018.07.09", :_type=>"doc", :_routing=>nil}, #<LogStash::Event:0x7eec4c10>], :response=>{"index"=>{"_index"=>"logstash-2018.07.09", "_type"=>"doc", "_id"=>"62e5abb6f430534c6e3df55faaa919849f1febe8", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [Packets]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Invalid format: \"client-to-server\""}}}}}
[2018-07-09T10:58:13,795][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>"26be2832d32efc2d40139b37f57acd539f57ca5d", :_index=>"logstash-2018.07.09", :_type=>"doc", :_routing=>nil}, #<LogStash::Event:0x799e758>], :response=>{"index"=>{"_index"=>"logstash-2018.07.09", "_type"=>"doc", "_id"=>"26be2832d32efc2d40139b37f57acd539f57ca5d", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [Packets]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Invalid format: \"client-to-server\""}}}}}
[2018-07-09T10:59:39,135][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>"4201211bc068b4885a5be4b227bb6eaa0b421743", :_index=>"logstash-2018.07.09", :_type=>"doc", :_routing=>nil}, #<LogStash::Event:0x2f7a2a01>], :response=>{"index"=>{"_index"=>"logstash-2018.07.09", "_type"=>"doc", "_id"=>"4201211bc068b4885a5be4b227bb6eaa0b421743", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [Packets]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Invalid format: \"client-to-server\""}}}}}
[2018-07-09T11:00:57,085][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>404, :action=>["index", {:_id=>"7279916dd5852de4d3cf262c3ba12da77312257c", :_index=>"logstash-2018.07.09", :_type=>"doc", :_routing=>nil}, #<LogStash::Event:0x4866f874>], :response=>{"index"=>{"_index"=>"logstash-2018.07.09", "_type"=>"doc", "_id"=>"7279916dd5852de4d3cf262c3ba12da77312257c", "status"=>404, "error"=>{"type"=>"index_not_found_exception", "reason"=>"no such index", "index_uuid"=>"ue9jYagOTCG2S4QokKwF3g", "index"=>"logstash-2018.07.09"}}}}

Do you know why I have these errors ?

2

It looks like the Packets field suddenly has started to contain the string "client-to-server", which isn't compatible with the field's mapping. You can use ES's get mapping API to fetch the mapping.

3 
{
  "logstash-2018.07.09": {
    "mappings": {
      "_default_": {
        "dynamic_templates": [
          {
            "message_field": {
              "path_match": "message",
              "match_mapping_type": "string",
              "mapping": {
                "norms": false,
                "type": "text"
              }
            }
          },
          {
            "string_fields": {
              "match": "*",
              "match_mapping_type": "string",
              "mapping": {
                "fields": {
                  "keyword": {
                    "ignore_above": 256,
                    "type": "keyword"
                  }
                },
                "norms": false,
                "type": "text"
              }
            }
          }
        ],
        "properties": {
          "@timestamp": {
            "type": "date"
          },
          "@version": {
            "type": "keyword"
          },
          "geoip": {
            "dynamic": "true",
            "properties": {
              "ip": {
                "type": "ip"
              },
              "latitude": {
                "type": "half_float"
              },
              "location": {
                "type": "geo_point"
              },
              "longitude": {
                "type": "half_float"
              }
            }
          }
        }
      },
      "doc": {
        "dynamic_templates": [
          {
            "message_field": {
              "path_match": "message",
              "match_mapping_type": "string",
              "mapping": {
                "norms": false,
                "type": "text"
              }
            }
          },
          {
            "string_fields": {
              "match": "*",
              "match_mapping_type": "string",
              "mapping": {
                "fields": {
                  "keyword": {
                    "ignore_above": 256,
                    "type": "keyword"
                  }
                },
                "norms": false,
                "type": "text"
              }
            }
          }
        ],
        "properties": {
          "@timestamp": {
            "type": "date"
          },
          "@version": {
            "type": "keyword"
          },
          "beat": {
            "properties": {
              "hostname": {
                "type": "text",
                "norms": false,
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "name": {
                "type": "text",
                "norms": false,
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "version": {
                "type": "text",
                "norms": false,
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              }
            }
          },
          "cisco_tag": {
            "type": "text",
            "norms": false,
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
4 
          "dst_ip": {
            "type": "ip"
          },
          "fields": {
            "properties": {
              "log_type": {
                "type": "text",
                "norms": false,
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              }
            }
          },
          "fingerprint": {
            "type": "text",
            "norms": false,
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "geoip": {
            "dynamic": "true",
            "properties": {
              "ip": {
                "type": "ip"
              },
              "latitude": {
                "type": "half_float"
              },
              "location": {
                "type": "geo_point"
              },
              "longitude": {
                "type": "half_float"
              }
            }
          },
          "host": {
            "properties": {
              "name": {
                "type": "text",
                "norms": false,
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              }
            }
          },
          "hostname": {
            "type": "text",
            "norms": false,
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "input": {
            "properties": {
              "type": {
                "type": "text",
                "norms": false,
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              }
            }
          },
          "message": {
            "type": "text",
            "norms": false
          },
          "offset": {
            "type": "long"
          },
          "prospector": {
            "properties": {
              "type": {
                "type": "text",
                "norms": false,
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              }
            }
          },
          "source": {
            "type": "text",
            "norms": false,
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "src_ip": {
            "type": "ip"
          },
          "syslog_timestamp": {
            "type": "text",
            "norms": false,
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "tags": {
            "type": "text",
            "norms": false,
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "type": {
            "type": "text",
            "norms": false,
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          }
        }
      }
    }
  }
}
5

Can you help find a way to parse my logs agin, please !!

6

I don't understand. There's nothing in there related to a Packets field, yet that's what ES is complaining about.

7

What should I do ?

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.