Must clause not working as expected

[jo_dev]

I am executing must clause query on a nested document. I am expecting that it will behave like logical AND, but observing inconsistent behavior.

Sample document mappings-

"mappings": {
            "properties": {
                "A": {
                    "type": "double"
                },
                "L1": {
                    "type": "nested",
                    "properties": {
                        "L2": {
                            "type": "nested",
                            "properties": {
                                "P": {
                                    "type": "double"
                                },
                                "Q": {
                                    "type": "double"
                                }
                            }
                        },
                        "X": {
                            "type": "double"
                        },
                        "Y": {
                            "type": "double"
                        }
                    }
                }
            }
        }

inserting one document -

{
    "A": 100,
    "L1": {
        "X": 50,
        "Y": 120,
        "L2": [
            {
                "P": 20,
                "Q": 40
            },
            {
                "P": 15,
                "Q": 65
            },
            {
                "P": 19,
                "Q": 40
            }
        ]
    }
}

First query - search document where value of P is greater than equal to 16 AND value of P is less than equal to 18

{
    "query": {
        "bool": {
            "filter": [
                {
                    "bool": {
                        "must": [
                            {
                                "nested": {
                                    "path": "L1",
                                    "query": {
                                        "bool": {
                                            "filter": [
                                                {
                                                    "nested": {
                                                        "path": "L1.L2",
                                                        "query": {
                                                            "bool": {
                                                                "filter": [
                                                                    {
                                                                        "range": {
                                                                            "L1.L2.P": {
                                                                                "gte": 16
                                                                            }
                                                                        }
                                                                    }
                                                                ]
                                                            }
                                                        }
                                                    }
                                                }
                                            ]
                                        }
                                    }
                                }
                            },
                            {
                                "nested": {
                                    "path": "L1",
                                    "query": {
                                        "bool": {
                                            "filter": [
                                                {
                                                    "nested": {
                                                        "path": "L1.L2",
                                                        "query": {
                                                            "bool": {
                                                                "filter": [
                                                                    {
                                                                        "range": {
                                                                            "L1.L2.P": {
                                                                                "lte": 18
                                                                            }
                                                                        }
                                                                    }
                                                                ]
                                                            }
                                                        }
                                                    }
                                                }
                                            ]
                                        }
                                    }
                                }
                            }
                        ]
                    }
                }
            ]
        }
    }
}

Document doesn't satisfy this query, but still document is getting returned.

Second query - search document where value of P is greater than equal to 16 AND value of Q is less than equal to 35

{
    "query": {
        "bool": {
            "filter": [
                {
                    "bool": {
                        "must": [
                            {
                                "nested": {
                                    "path": "L1",
                                    "query": {
                                        "bool": {
                                            "filter": [
                                                {
                                                    "nested": {
                                                        "path": "L1.L2",
                                                        "query": {
                                                            "bool": {
                                                                "filter": [
                                                                    {
                                                                        "range": {
                                                                            "L1.L2.P": {
                                                                                "gte": 16
                                                                            }
                                                                        }
                                                                    }
                                                                ]
                                                            }
                                                        }
                                                    }
                                                }
                                            ]
                                        }
                                    }
                                }
                            },
                            {
                                "nested": {
                                    "path": "L1",
                                    "query": {
                                        "bool": {
                                            "filter": [
                                                {
                                                    "nested": {
                                                        "path": "L1.L2",
                                                        "query": {
                                                            "bool": {
                                                                "filter": [
                                                                    {
                                                                        "range": {
                                                                            "L1.L2.Q": {
                                                                                "lte": 35
                                                                            }
                                                                        }
                                                                    }
                                                                ]
                                                            }
                                                        }
                                                    }
                                                }
                                            ]
                                        }
                                    }
                                }
                            }
                        ]
                    }
                }
            ]
        }
    }
}

This query works as expected and doesn't return any document

can someone explain what is wrong with the first query? What is the reason behind Elasticsearch returning the document though it doesn't match provided criteria?

[Christian_Dahlqvist]

As you have two separate nested clauses under the top must I believe these will be evaluated for all sub-documents independently. In the first query there are different nested documents that fulfil both clauses (15 < 18 and 19 > 16) and the indexed document therefore match. In the second example there is no sub-document that has Q less than 35 and all sub-documents therefore fail that clause.