Unexpected results when filtering docs in nested field

[OmarDacca]

Hi,

suppose we have the following docs: (index name is test_index)

{
    session_id: "123",
    "session_details":[
        {
            "page": "/abc"
        },
        {
            "page": "/xyz"
        }
    ]
}

{
    session_id: "456",
    "session_details":[
        {
            "page": "/abc"
        },
        {
            "page": "/qwe"
        }
    ]
}

mappings is like this:

{
    "session_id" : {
        "type" : "keyword"
    },
    "session_details" : {
        "type" : "nested",
        "properties" : {
          "page" : {
            "type" : "keyword"
          }
        }
    }
}

if I run the following query I expect to receive the doc with the session_id = 123
but I receive both docs, any explanation ?

GET test_index/_search?ignore_unavailable=true
{
  "track_total_hits": true,
  "size": 10,
  "query": {
    "bool": {
      "filter": [
        {
          "nested": {
            "path": "session_details",
            "query": {
              "bool": {
                "filter": [
                  {
                    "wildcard": {
                      "session_details.page": "*/abc*"
                    }
                  },
                  {
                    "bool": {
                      "must_not": [
                        {
                          "wildcard": {
                            "session_details.page": "*/xyz*"
                          }
                        }
                      ]
                    }
                  }
                ]
              }
            }
          }
        }
      ]
    }
  }
}

[RabBit_BR (andre.coelho)]

Hi!!

This post is a little similar:

Elasticsearch must_not on all elements in array

Hi, thanks! Abdon this would be ok, but I would need to reindex all the items (which are in millions of large items and takes days). Tamara yours looks easier but how to scripted filters affect performance? If it's a big performance hit it might still be better to reindex my items.

Perhaps the solution is to use a script.

[OmarDacca]

That post was 4 years ago, so many Elastic versions came out during this time.
painless scripts are not recommended in terms of performance.

I understand that each document in nested field is considered and indexed as separate document. The question is, given the above mappings, is it possible to query a nested field as a regular array or something like that ?

One solution is to change the field type from nested to a regular object like this:

PUT test_prop
{
  "mappings": {
    "properties": {
      "session_id": {
        "type": "keyword"
      },
      "session_details": {
        "properties": {
          "page": {
            "type": "keyword"
          }
        }
      }
    }
  }
}

But this will result in a huge data duplication, also suppose we have another field session_details: page, path:

session_details: [
     { page: "xyz", path: "qqq" },
     { page: "jkl", path: "aaa" }
]

a query like: page = "xyz" and path != "aaa" will still return the doc.