Many fields not searchable after upgrade

Elastic Stack Kibana
1

After upgrading to the latest version of Elasticsearch and Kibana many of my fields are not searchable using Discover.

Elasticsearch 6.6.1
Kibana 6.6.1

Elasticsearch search indexes is managed by Graylog.

Example of a working field:

name: netflow_ipv4_src_addr
type: keyword

Example of a field that is not searchable:

name: asa_src_ip 
type: keyword

Even though the field is non searchable, when I enable "Turn on query features", it can see the field as well as field values for asa_src_ip

Both of these fields show up under the Index Patterns screen, and both of them are seen as type string, searchable and aggregatable.

Here is a stripped down Kibana query that works (netflow_ipv4_src_addr):

GET _search
{
  "version": true,
  "size": 5,
  "query": {
    "bool": {
      "must": [
        {
          "range": {
            "@timestamp": {
              "gte": 1551268505824,
              "lte": 1551269405824,
              "format": "epoch_millis"
            }
          }
        }
      ],
      "filter": [
        {
          "bool": {
            "should": [
              {
                "match_phrase": {
                  "netflow_ipv4_src_addr": "8.8.8.8"
                }
              }
            ],
            "minimum_should_match": 1
          }
        }
      ]
    }
  }
}

Here is a stripped down Kibana query that returns zero results (asa_src_ip):

GET _search
{
  "version": true,
  "size": 5,
  "query": {
    "bool": {
      "must": [
        {
          "range": {
            "@timestamp": {
              "gte": 1551268505824,
              "lte": 1551269405824,
              "format": "epoch_millis"
            }
          }
        }
      ],
      "filter": [
        {
          "bool": {
            "should": [
              {
                "match_phrase": {
                  "asa_src_ip": "8.8.8.8"
                }
              }
            ],
            "minimum_should_match": 1
          }
        }
      ]
    }
  }
}

Here is a stripped down Graylog query that works (asa_src_ip):

GET _search
{
  "from": 0,
  "size": 5,
  "query": {
    "bool": {
      "must": [
        {
          "query_string": {
            "query": "asa_src_ip:8.8.8.8"

          }
        }
      ],
      "filter": [
        {
          "bool": {
            "must": [
              {
                "range": {
                  "timestamp": {
                    "from": "2019-02-27 12:14:27.187",
                    "to": "2019-02-27 12:19:27.187"
                  }
                }
              }
            ]

          }
        }
      ]
    }
  }
}

The to/from times are slightly different, but there should be 100+ search results in each one.

Reloading indexes didn't help. Any idea what could be causing some fields not to be searchable?

2

What is the Elasticsearhc-mapping of your index for those fields, both are keywords after the upgrade?

Keyword fields do require an exact match to return with match_phrase. Is that IP-address stored exactly like the string 8.8.8.8?

3

Both are keywords.

IP is stored exactly like the string "8.8.8.8"

The "auto-complete" functionality in the search box "Turn on query features" can see both the field and the field value, but the search result comes up with "No results match your search criteria"

image

image

4

I would log this as a bug. If that query is not returning any results, the query-bar should not suggest this as a query option. https://github.com/elastic/kibana/issues/new

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.