Limit to rawmessage length and search

Elastic Stack Kibana
1

Hello.

I'm trying to search within a rawmessage field for a specific string but i get no results if the rawmessage is rather long. Is there a limit? Example:

rawmessage:
    device_id=bla [Root]system-notification-00257(traffic): start_time="2016-05-17 14:29:21" duration=0 policy_id=163 service=dns proto=17 src zone=zone dst zone=Uplink action=Permit sent=0 rcvd=0 src=1.2.3.4 dst=1.2.3.4 src_port=50633 dst_port=53 src-xlated ip=1.2.3.4 port=43614 dst-xlated ip=1.2.3.4 port=53 session_id=1983453 reason=Creation

Now when i search for "rawmessage: Creation" i get no hit. If the rawmessage is much shorter this generally works.

any hints would be really appreciated.

thanks and cheers
t.

2

Hmmm, can you include your mapping for rawmessage? Thanks.

3

Hi Lukas.

Thanks for your answer. Here the mapping.

root@logserver01:/# curl -XGET http://localhost:9200/logs-2016.12.30/_mapping?pretty=true
{
  "logs-2016.12.30" : {
    "mappings" : {
      "_default_" : {
        "_all" : {
          "enabled" : false
        },
        "_ttl" : {
          "enabled" : true,
          "default" : 7776000000
        },
        "dynamic_templates" : [ {
          "disable_analyze" : {
            "mapping" : {
              "index" : "not_analyzed",
              "store" : "no",
              "type" : "string"
            },
            "match" : "*",
            "match_mapping_type" : "string"
          }
        } ]
      },
      "noJSON" : {
        "_all" : {
          "enabled" : false
        },
        "_ttl" : {
          "enabled" : true,
          "default" : 7776000000
        },
        "dynamic_templates" : [ {
          "disable_analyze" : {
            "mapping" : {
              "index" : "not_analyzed",
              "store" : "no",
              "type" : "string"
            },
            "match" : "*",
            "match_mapping_type" : "string"
          }
        } ],
        "properties" : {
          "@timestamp" : {
            "type" : "date",
            "format" : "strict_date_optional_time||epoch_millis"
          },
          "facility" : {
            "type" : "string",
            "index" : "not_analyzed"
          },
          "host" : {
            "type" : "string",
            "index" : "not_analyzed"
          },
          "rawmessage" : {
            "type" : "string",
            "index" : "not_analyzed"
          },
          "severity" : {
            "type" : "string",
            "index" : "not_analyzed"
          },
          "tag" : {
            "type" : "string",
            "index" : "not_analyzed"
          }
        }
      }
    }
  }
}
4

Looks to me like rawmessage is not_analyzed. I'm not sure why it is working at all when the string length is smaller, but if you'd like to be able to search the way you've suggested, you'd need to change your mapping to analyze rawmessage. Hope this helps!

5

Yeah. Seems to be the problem. Thanks.