Search documents by unmapped fields for debugging

Elastic Stack Elasticsearch
1

We are using fixed (non dynamic) mapping in our indices in order to improve performance.
The mapping contains a subset of our document fields, and the rest of the fields are unmapped.
However, for debugging purposes we sometimes need to filter/count/sort documents in Kibana according to unmapped fields. It could be any unmapped field, and not specific ones.
For this debugging purpose performance is not important, we just need to be able to do it.

I thought it may be possible to achieve this by using scripts somehow, but haven't found any way to do this in the documentation or in anywhere else.
The only way we found is to write a client script that gets all documents and does the logic.

Is there anyway to do this using DSL ?

2

Can you please elaborate more on what you mean by unmapped here.

3

Can you please elaborate more on what you mean by unmapped here.

Meaning fields that weren't included when defining the index mapping, but exists in the document itself. I'm talking about a case of non-dynamic mapping.
In the following example we won't be able to search documents according to the field other_field. Again, we need this only for manually debugging our production database but our backend itself doesn't attempt to search by non-mapped fields.

PUT /test
{
  "mappings": {
    "dynamic" : "false",
    "properties": {
      "mapped_field": { "type": "text" }
    }
  }
}

POST test/_doc/
{
  "mapped_field": "this is mapped",
  "other_field": "this is unmapped since it is not included in the index  mapping"
}
4

There's no way to do what you want here unless you stop using "dynamic" : "false",.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.