Search without field name

Elastic Stack Kibana
1

Hi,

I'm with Kibana 7.10, and in discover, I try to just search without the field name. It seems to me for some fields, this works, for others, not.
For example, I have a string field fields.log_type: "log-type"

In Kibana, I search just enter "log-type" , it finds all entries with fields.log_type: set to "log-type". So far so good.

However, searching for a user.name without the field name, brings up some, where the username somehow matches a word in the message: field, but not where the username matches the user.name: field. When I do a search alike: user.name: "root" it will find all of them.

Similarly, searching for IPs, i.e. just entering an IP: "10.20.30.40" brings up nothing, but when I search for: source.ip: "10.20.30.40" it will find them all. Searching for IPs is even worse than usernames, as it might be any of source.ip, destination.ip client.ip etc....

I don't understand why searching without giving the field name works for the fields.log_type: but not for others?

cheers,
Sebastian

2

Hi @buzzdeee,

In case no field is provided Kibana builds the following es query:

multi_match: {
type: "best_fields", query: "10.20.30.40", lenient: true
}

Referring to Elasticsearch documentation:

If no fields are provided, the multi_match query defaults to the index.query.default_field index settings, which in turn defaults to * . * extracts all fields in the mapping that are eligible to term queries and filters the metadata fields. All extracted fields are then combined to build a query.

I think the answer to your question lies somewhere along those lines.

3

Hi @dosant

thanks for that pointer. Now I need to figure out, why the fields.log_type is eligible to term queries, but for example user.name not (:

cheers,
Sebastian

4

You can check in index pattern setting if user.name is aggregatable:
Stack management -> Index Patterns -> <your index pattern>.

If not, then to change a type, you'd have to use elasticsearch's field mapping api

5

I verified, the fields in question they're all aggregatable.

However, you pointed me to the right direction, reading up more on Multi-match query, I belive I run into the trouble the warning shows here:

There is a limit on the number of fields that can be queried at once. It is defined by the indices.query.bool.max_clause_count Search settings which defaults to 1024.

I search on my filebeat-* index, which has > 5000 fields. So I guess it only takes the first 1024 into account, and if the fields I'm interested in are not in that list, tough luck. Because of the warning, don't want to bump it right away.

Reading more on that page, I figured, it should be possible to give wildcard field names:

*ip: "10.20.30.40"

and that just seems to work, it finds all these *.ip fields.

However, for the .name it doesn't seem to help, half of the shards are failing, from the error message popping up in Kibana, from the response of one of the shards, all show same issue:

 {
         "shard": 0,
         "index": "filebeat-7.10.0-2020.11.19-000002",
         "node": "t7e3bbYgTSKW0lsrJK9udw",
         "reason": {
           "type": "query_shard_exception",
           "reason": "failed to create query: For input string: \"maggie\"",
           "index_uuid": "5v-geZN-R8KvoMm8hFhAvg",
           "index": "filebeat-7.10.0-2020.11.19-000002",
           "caused_by": {
             "type": "number_format_exception",
             "reason": "For input string: \"maggie\""
           }
         }
       },

and no event with "maggie" is shown, besides there are a number of them, as I can see with term search.

The worst problem, searching for IPs works, wondering why string fields throw this error.

Sebastian

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.