Dynamic index mapping with pattern filter

Elastic Stack Elasticsearch
1

Hi,

I'm using my own custom analyzer with dynamic json mapping.Following substring pattern substring first 500 chars of string and send for analyzing.

"index": {
    "analysis": {
        "char_filter": {
            "punctuation_pattern": {
                "type": "pattern_replace",
                "pattern": "\\p{Punct}", 
                "replacement": " " 
            },
            "substring_pattern":{
                "type": "pattern_replace",
                "pattern": "^(.{1,500}(?<=\\S)\\b).*$",
                "replacement": "$1"
            }
        },
        "analyzer": {
            "log_analyzer": {
                "tokenizer": "standard",
                 "char_filter": [
                    "punctuation_pattern",
                    "substring_pattern"
                ]
            }
        }
    }
}

And Mapping is

 "mappings" : {
    "doc" : {
      "properties" : {
          "Message": {
            "type": "text",
            "analyzer": "log_analyzer",
            "index_options": "positions"
          }
         }
    }
}

This is working fine if "Message" is not empty , but if Message is empty then I'm getting following error on Logstash console output.

[2018-02-06T07:23:04,973][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"logstash-2018.02.06", :_type=>"doc", :_routing=>nil}, #<LogStash::Event:0x3fbdf907>], :response=>{"index"=>{"_index"=>"logstash-2018.02.06", "_type"=>"doc", "_id"=>"SmURa2EBGlSqJHLRzIOj", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [Message]", "caused_by"=>{"type"=>"illegal_state_exception", "reason"=>"Can't get text on a START_OBJECT at 1:29"}}}}}

And I don't receive that message in elasticsearch.

Any luck , How can I update substring pattern so that It only execute if given input is not empty otherwise skip it or How can do substring with some other method?

Thanks ,

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.