Field value not found in aggregatable field

Elastic Stack Elasticsearch
1

Hi folks, I'm at a loss to understand this.
ELK stack 5.6.
Using filebeat i send json logs of nginx to logstash where they're parsed and fed into elasticsearch. No errors, all seems good.
In kibana I have a visualization data table to display counts of useragents, but I discovered today there's one useragent string that doesn't appear in the data table:

Expanse, a Palo Alto Networks company, searches across the global IPv4 space multiple times per day to identify customers' presences on the Internet. If you would like to be excluded from our scans, please send IP addresses/domains to: scaninfo@paloaltonetworks.com

I can search the field http_user_agent for the string just fine, but for some reason I get no results when aggregating on http_user_agent.keyword which is what the visualization uses.

Looking at each fields properties shows both as type string and searchable, and the .keyword field aggregatable.

Is there a char limit or something thats preventing the above string from being included in aggregate operations?

Thanks

2

Check the mapping for the field. It may be that only strings at 256 characters and below are indexed.

3 
{ - 
  "nginx-2023.01.14": { - 
    "mappings": { - 
      "log": { - 
        "http_user_agent.keyword": { - 
          "full_name": "http_user_agent.keyword",
          "mapping": { - 
            "keyword": { - 
              "type": "keyword",
              "ignore_above": 256
            }
          }
        }
      }
    }
  }
}

Sure enough, thats the culprit.

Thanks!

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.