"term" behave srangely in knn filter?

[Yoann_Buzenet]

Hello,

My range filters are working perfectly, but when I use a term that doesn't match anything, results are strange.

For example, I'm trying to get boardgames by age. I use a range filters that does work. If I use a "term" on a property, and it doesn't find anything, then the results become out of range.

Here is my query

{
  "from": 0,
  "size": 3,
  "query": {
    "bool": {
      "filter": [],
      "should": [],
      "must": [],
      "must_not": []
    }
  },
  "knn": {
    "field": "embedding",
    "query_vector": [
      -0.049409836530685425,
      0.004891033284366131,
      0.02810363657772541,
      0.05813274160027504,
      -0.02277800627052784,
      -0.08117515593767166,
      0.03672189638018608,
      -0.003742472967132926,
      0.09365282952785492,
      0.050613727420568466,
      0.013901573605835438,
      -0.020395124331116676,
      -0.007280553225427866,
      0.0022683285642415285,
      -0.0798516720533371,
      -0.020266443490982056,
      -0.027672559022903442,
      -0.0020338057074695826,
      -0.021304968744516373,
      -0.014463381841778755,
      0.013873877935111523,
      0.021126234903931618,
      -0.046334706246852875,
      0.019942348822951317,
      0.004430413246154785,
      0.07496818155050278,
      0.01780945248901844,
      0.027535229921340942,
      -0.16652490198612213,
      -0.05582409352064133,
      -0.024150878190994263,
      0.11059213429689407,
      -0.06544376164674759,
      -0.05248958244919777,
      -0.02786935865879059,
      0.012665773741900921,
      0.032295744866132736,
      0.006287253927439451,
      0.04858362302184105,
      0.004297235049307346,
      -0.037272483110427856,
      -0.13639354705810547,
      0.07986632734537125,
      0.05130545422434807,
      0.012659892439842224,
      0.004586723633110523,
      -0.016139421612024307,
      -0.03315325453877449,
      0.01820875145494938,
      0.06355420500040054,
      -0.07368788868188858,
      0.01303124614059925,
      -0.016112077981233597,
      0.01772332936525345,
      0.011353361420333385,
      -0.04840549826622009,
      0.005034640897065401,
      0.005646976176649332,
      0.0320860892534256,
      -0.11949650198221207,
      0.055256083607673645,
      -0.06408920139074326,
      -0.0685899630188942,
      0.05287231132388115,
      0.11058869957923889,
      0.013125206343829632,
      -0.029818184673786163,
      0.058537840843200684,
      -0.0017318298341706395,
      -0.05754252150654793,
      -0.0037360957358032465,
      0.0509151853621006,
      0.05079891160130501,
      0.06176655739545822,
      0.11007165163755417,
      -0.04189196601510048,
      0.05611620843410492,
      0.014012707397341728,
      0.11743518710136414,
      -0.059737835079431534,
      -0.007209470961242914,
      -0.05508410185575485,
      -0.10902736335992813,
      0.03140096738934517,
      -0.023651396855711937,
      0.02254081889986992,
      0.02056342363357544,
      0.0010580220259726048,
      -0.0675467923283577,
      0.03918212652206421,
      -0.08878873288631439,
      -0.06608942896127701,
      0.027788538485765457,
      0.0020840917713940144,
      -0.09848127514123917,
      0.04274114966392517,
      -0.05088504031300545,
      -0.08866234123706818,
      -0.011348066851496696,
      0.1275867074728012,
      -0.037042342126369476,
      0.0425901897251606,
      -0.05511794611811638,
      0.015774955973029137,
      0.09826533496379852,
      -0.09285881370306015,
      0.029072655364871025,
      -0.025634218007326126,
      0.030678875744342804,
      -0.000753039785195142,
      -0.005528176669031382,
      0.012077361345291138,
      -0.03817363828420639,
      0.012971200980246067,
      0.0861113965511322,
      0.07635430246591568,
      0.08410479873418808,
      0.041298262774944305,
      -0.0007305238978005946,
      -0.03311833366751671,
      -0.010606382042169571,
      0.004648401401937008,
      -0.11425226181745529,
      -0.02954685688018799,
      -0.0649784579873085,
      -0.03329478204250336,
      -0.0011585663305595517,
      -2.4508350081324278e-33,
      0.0316350944340229,
      -0.05173753574490547,
      0.020162351429462433,
      0.06597787886857986,
      0.051326408982276917,
      -0.032706670463085175,
      -0.024263229221105576,
      -0.023685527965426445,
      -0.0017541677225381136,
      0.0007936746114864945,
      -0.05776742100715637,
      0.06051736697554588,
      -0.06538413465023041,
      0.03647594526410103,
      0.08626258373260498,
      0.06327173113822937,
      -0.03350914269685745,
      0.017477653920650482,
      0.004297366365790367,
      -0.06454082578420639,
      -0.08032116293907166,
      0.021543487906455994,
      -0.0378723219037056,
      0.008934469893574715,
      0.02011488936841488,
      -0.01968550682067871,
      -0.035359419882297516,
      -0.05076120048761368,
      0.020596403628587723,
      0.029751403257250786,
      0.004540916066616774,
      0.11520950496196747,
      0.033320922404527664,
      -0.03450218588113785,
      0.057889603078365326,
      -0.062499918043613434,
      -0.03899291157722473,
      -0.02962207794189453,
      0.043336641043424606,
      0.06783860176801682,
      -0.04829634353518486,
      0.03092353790998459,
      -0.041549474000930786,
      0.0021326462738215923,
      0.018427260220050812,
      0.0223116185516119,
      0.09893038868904114,
      0.006302930414676666,
      -0.05646558851003647,
      0.088975690305233,
      -0.015586183406412601,
      -0.016343193128705025,
      0.02104244939982891,
      0.02683810330927372,
      0.01617836207151413,
      -0.01695675030350685,
      0.002295635873451829,
      -0.06167204678058624,
      -0.018849531188607216,
      0.004656618926674128,
      0.0322817824780941,
      0.13475686311721802,
      0.04503224045038223,
      -0.03957787901163101,
      0.04441124573349953,
      0.01955721527338028,
      0.04307923838496208,
      -0.021352173760533333,
      0.006152441259473562,
      -0.006302898749709129,
      -0.08052235096693039,
      0.08250440657138824,
      -0.01693098060786724,
      -0.0003445694746915251,
      0.005235031712800264,
      0.002937984187155962,
      0.08237919211387634,
      -0.012812864035367966,
      -0.09389758855104446,
      -0.006106135901063681,
      -0.024815933778882027,
      -0.03336304426193237,
      -0.02679702453315258,
      0.0192271638661623,
      0.02871078811585903,
      0.018955117091536522,
      -0.06065736338496208,
      -0.029904447495937347,
      -0.011691024526953697,
      -0.025439197197556496,
      -0.04619080573320389,
      -0.11162019520998001,
      0.0537695586681366,
      -0.05119529739022255,
      -0.09530068933963776,
      2.124256228851103e-33,
      -0.0758451297879219,
      -0.09364552050828934,
      -0.060507044196128845,
      0.046550050377845764,
      0.05473474785685539,
      -0.04278877377510071,
      -0.10247790068387985,
      -0.04446272924542427,
      -0.02499961294233799,
      0.0629647746682167,
      -0.10281739383935928,
      -0.006373060401529074,
      0.08628557622432709,
      0.010038319043815136,
      0.0498167984187603,
      -0.04578624665737152,
      -0.014049404300749302,
      -0.005578579381108284,
      -0.02636389434337616,
      0.04852703958749771,
      0.00042261360795237124,
      -0.008496541529893875,
      -0.015862051397562027,
      -0.040265291929244995,
      0.03237353637814522,
      0.06361255794763565,
      0.025288723409175873,
      0.019419299438595772,
      -0.08891844004392624,
      0.00151180534157902,
      0.02951311506330967,
      0.0008061358821578324,
      0.017357444390654564,
      0.007839015685021877,
      0.018047725781798363,
      0.1183367520570755,
      0.07228198647499084,
      -0.02813178487122059,
      0.027049094438552856,
      -0.07188857346773148,
      0.023648034781217575,
      0.028518086299300194,
      -0.08457920700311661,
      0.0813601016998291,
      -0.02336261235177517,
      -0.016507456079125404,
      0.06610347330570221,
      0.09352266788482666,
      -0.0036757360212504864,
      0.03184602037072182,
      -0.06957991421222687,
      0.017108725383877754,
      0.006004105322062969,
      -0.05464516207575798,
      0.030628317967057228,
      -0.029854673892259598,
      -0.09367018193006516,
      0.027028053998947144,
      -0.001451951451599598,
      0.05591083690524101,
      0.023848699405789375,
      0.021507469937205315,
      0.06981803476810455,
      0.06439214944839478,
      -0.07225064188241959,
      -0.05912204459309578,
      -0.03410935774445534,
      0.11967529356479645,
      -0.04684752970933914,
      0.003930406644940376,
      0.04005679115653038,
      0.02136135846376419,
      -0.04248519241809845,
      0.010312197729945183,
      -0.024079730734229088,
      0.010314052924513817,
      -0.022504057735204697,
      -0.03467293456196785,
      -0.04509783908724785,
      0.11609519273042679,
      -0.009770234115421772,
      -0.02358322963118553,
      0.023358827456831932,
      0.10350919514894485,
      -0.007483534049242735,
      0.09302595257759094,
      -0.002111472189426422,
      -0.03403764218091965,
      0.016529684886336327,
      -0.030004147440195084,
      0.016130944713950157,
      -0.03868066519498825,
      0.0104332584887743,
      0.05662151426076889,
      -0.04727974906563759,
      -1.357294454606972e-8,
      0.10642163455486298,
      -0.0013623692793771625,
      0.06026766076683998,
      -0.03951326385140419,
      0.024045240134000778,
      0.11666487157344818,
      0.0021126337815076113,
      0.022794438526034355,
      -0.04968615621328354,
      0.011026709340512753,
      -0.0664014220237732,
      0.02647843025624752,
      0.02762906812131405,
      0.06240236014127731,
      0.006782029755413532,
      -0.011187623254954815,
      0.03954063728451729,
      0.004930218681693077,
      -0.07038156688213348,
      0.06946661323308945,
      0.0376739539206028,
      0.044212184846401215,
      0.014575139619410038,
      -0.019005155190825462,
      -0.012388641014695168,
      -0.0029710594099014997,
      0.0694991946220398,
      -0.03681844845414162,
      0.01713174395263195,
      0.09619986265897751,
      -0.005801004823297262,
      0.024630054831504822,
      -0.11250384151935577,
      0.028666773810982704,
      0.006120818201452494,
      -0.042589426040649414,
      0.03790668398141861,
      -0.010988209396600723,
      -0.026690157130360603,
      -0.05288493633270264,
      0.013099487870931625,
      0.061343949288129807,
      -0.01793859899044037,
      0.05894609913229942,
      -0.0029380314517766237,
      -0.045418404042720795,
      0.07898731529712677,
      -0.03478022664785385,
      0.02704465575516224,
      -0.006895766593515873,
      0.016534635797142982,
      0.050051603466272354,
      0.006315434351563454,
      -0.010414527729153633,
      0.024025272578001022,
      0.061644546687603,
      -0.019397474825382233,
      0.08599943667650223,
      -0.08958672732114792,
      0.03130229935050011,
      0.054686710238456726,
      -0.03496289625763893,
      0.05289012938737869,
      -0.009723770432174206
    ],
    "k": 5,
    "num_candidates": 50,
    "boost": 0.5, 
    "filter": {
      "bool": {
        "filter": [
          {
            "range": {
              "tags.time.maxTime": {
                "lte": 30
              }
            }
          },
          { "term":  { "tags.age.minAge": 7 }}
        ],
        "should": [],
        "must": [],
        "must_not": []
      }
    }
  }
}

7 doesn't exist in my index.
This gives me a game that has age of 14 ! (see in red on the screenshot)

Even the "tags.time.maxTime" is not Lower Than or Equal to 30 like in the query. Result is 60.

But if I use terms : 6, which exists, then it finds result that stick to my query on all range fields.

Would you have an idea of what is happening ?

[dadoonet (David Pilato)]

Bienvenue @Yoann_Buzenet

Please don't post images of text as they are hard to read, may not display correctly for everyone, and are not searchable.

Instead, paste the text and format it with </> icon or pairs of triple backticks (```), and check the preview window to make sure it's properly formatted before posting it. This makes it more likely that your question will receive a useful answer.

It looks like a bug but could you provide a full recreation script as described in About the Elasticsearch category. It will help to better understand what you are doing. Please, try to keep the example as simple as possible.

A full reproduction script is something anyone can copy and paste in Kibana dev console, click on the run button to reproduce your use case. It will help readers to understand, reproduce and if needed fix your problem. It will also most likely help to get a faster answer.

Have a look at the Elastic Stack and Solutions Help · Forums and Slack | Elastic page. It contains also lot of useful information on how to ask for help.

[Yoann_Buzenet]

Hey David,

Here is a case.

PUT /bug_filters
{
  "mappings": {
    "properties": {
      "embedding": {
        "type": "dense_vector",
        "dims": 384
      }
    }
  }
}

Here is the data : gist:c7eccdb7ccecfc668873dc515f1fa50b · GitHub

And the query :

// http://localhost:5601/app/dev_tools#/console
// vector holds "science-fiction"
GET bug_filters/_search
{
  "from": 0,
  "size": 3,
  "query": {
    "bool": {
      "filter": [],
      "should": [],
      "must": [],
      "must_not": []
    }
  },
  "knn": {
    "field": "embedding",
    "query_vector": [
      -0.049409836530685425,
      0.004891033284366131,
      0.02810363657772541,
      0.05813274160027504,
      -0.02277800627052784,
      -0.08117515593767166,
      0.03672189638018608,
      -0.003742472967132926,
      0.09365282952785492,
      0.050613727420568466,
      0.013901573605835438,
      -0.020395124331116676,
      -0.007280553225427866,
      0.0022683285642415285,
      -0.0798516720533371,
      -0.020266443490982056,
      -0.027672559022903442,
      -0.0020338057074695826,
      -0.021304968744516373,
      -0.014463381841778755,
      0.013873877935111523,
      0.021126234903931618,
      -0.046334706246852875,
      0.019942348822951317,
      0.004430413246154785,
      0.07496818155050278,
      0.01780945248901844,
      0.027535229921340942,
      -0.16652490198612213,
      -0.05582409352064133,
      -0.024150878190994263,
      0.11059213429689407,
      -0.06544376164674759,
      -0.05248958244919777,
      -0.02786935865879059,
      0.012665773741900921,
      0.032295744866132736,
      0.006287253927439451,
      0.04858362302184105,
      0.004297235049307346,
      -0.037272483110427856,
      -0.13639354705810547,
      0.07986632734537125,
      0.05130545422434807,
      0.012659892439842224,
      0.004586723633110523,
      -0.016139421612024307,
      -0.03315325453877449,
      0.01820875145494938,
      0.06355420500040054,
      -0.07368788868188858,
      0.01303124614059925,
      -0.016112077981233597,
      0.01772332936525345,
      0.011353361420333385,
      -0.04840549826622009,
      0.005034640897065401,
      0.005646976176649332,
      0.0320860892534256,
      -0.11949650198221207,
      0.055256083607673645,
      -0.06408920139074326,
      -0.0685899630188942,
      0.05287231132388115,
      0.11058869957923889,
      0.013125206343829632,
      -0.029818184673786163,
      0.058537840843200684,
      -0.0017318298341706395,
      -0.05754252150654793,
      -0.0037360957358032465,
      0.0509151853621006,
      0.05079891160130501,
      0.06176655739545822,
      0.11007165163755417,
      -0.04189196601510048,
      0.05611620843410492,
      0.014012707397341728,
      0.11743518710136414,
      -0.059737835079431534,
      -0.007209470961242914,
      -0.05508410185575485,
      -0.10902736335992813,
      0.03140096738934517,
      -0.023651396855711937,
      0.02254081889986992,
      0.02056342363357544,
      0.0010580220259726048,
      -0.0675467923283577,
      0.03918212652206421,
      -0.08878873288631439,
      -0.06608942896127701,
      0.027788538485765457,
      0.0020840917713940144,
      -0.09848127514123917,
      0.04274114966392517,
      -0.05088504031300545,
      -0.08866234123706818,
      -0.011348066851496696,
      0.1275867074728012,
      -0.037042342126369476,
      0.0425901897251606,
      -0.05511794611811638,
      0.015774955973029137,
      0.09826533496379852,
      -0.09285881370306015,
      0.029072655364871025,
      -0.025634218007326126,
      0.030678875744342804,
      -0.000753039785195142,
      -0.005528176669031382,
      0.012077361345291138,
      -0.03817363828420639,
      0.012971200980246067,
      0.0861113965511322,
      0.07635430246591568,
      0.08410479873418808,
      0.041298262774944305,
      -0.0007305238978005946,
      -0.03311833366751671,
      -0.010606382042169571,
      0.004648401401937008,
      -0.11425226181745529,
      -0.02954685688018799,
      -0.0649784579873085,
      -0.03329478204250336,
      -0.0011585663305595517,
      -2.4508350081324278e-33,
      0.0316350944340229,
      -0.05173753574490547,
      0.020162351429462433,
      0.06597787886857986,
      0.051326408982276917,
      -0.032706670463085175,
      -0.024263229221105576,
      -0.023685527965426445,
      -0.0017541677225381136,
      0.0007936746114864945,
      -0.05776742100715637,
      0.06051736697554588,
      -0.06538413465023041,
      0.03647594526410103,
      0.08626258373260498,
      0.06327173113822937,
      -0.03350914269685745,
      0.017477653920650482,
      0.004297366365790367,
      -0.06454082578420639,
      -0.08032116293907166,
      0.021543487906455994,
      -0.0378723219037056,
      0.008934469893574715,
      0.02011488936841488,
      -0.01968550682067871,
      -0.035359419882297516,
      -0.05076120048761368,
      0.020596403628587723,
      0.029751403257250786,
      0.004540916066616774,
      0.11520950496196747,
      0.033320922404527664,
      -0.03450218588113785,
      0.057889603078365326,
      -0.062499918043613434,
      -0.03899291157722473,
      -0.02962207794189453,
      0.043336641043424606,
      0.06783860176801682,
      -0.04829634353518486,
      0.03092353790998459,
      -0.041549474000930786,
      0.0021326462738215923,
      0.018427260220050812,
      0.0223116185516119,
      0.09893038868904114,
      0.006302930414676666,
      -0.05646558851003647,
      0.088975690305233,
      -0.015586183406412601,
      -0.016343193128705025,
      0.02104244939982891,
      0.02683810330927372,
      0.01617836207151413,
      -0.01695675030350685,
      0.002295635873451829,
      -0.06167204678058624,
      -0.018849531188607216,
      0.004656618926674128,
      0.0322817824780941,
      0.13475686311721802,
      0.04503224045038223,
      -0.03957787901163101,
      0.04441124573349953,
      0.01955721527338028,
      0.04307923838496208,
      -0.021352173760533333,
      0.006152441259473562,
      -0.006302898749709129,
      -0.08052235096693039,
      0.08250440657138824,
      -0.01693098060786724,
      -0.0003445694746915251,
      0.005235031712800264,
      0.002937984187155962,
      0.08237919211387634,
      -0.012812864035367966,
      -0.09389758855104446,
      -0.006106135901063681,
      -0.024815933778882027,
      -0.03336304426193237,
      -0.02679702453315258,
      0.0192271638661623,
      0.02871078811585903,
      0.018955117091536522,
      -0.06065736338496208,
      -0.029904447495937347,
      -0.011691024526953697,
      -0.025439197197556496,
      -0.04619080573320389,
      -0.11162019520998001,
      0.0537695586681366,
      -0.05119529739022255,
      -0.09530068933963776,
      2.124256228851103e-33,
      -0.0758451297879219,
      -0.09364552050828934,
      -0.060507044196128845,
      0.046550050377845764,
      0.05473474785685539,
      -0.04278877377510071,
      -0.10247790068387985,
      -0.04446272924542427,
      -0.02499961294233799,
      0.0629647746682167,
      -0.10281739383935928,
      -0.006373060401529074,
      0.08628557622432709,
      0.010038319043815136,
      0.0498167984187603,
      -0.04578624665737152,
      -0.014049404300749302,
      -0.005578579381108284,
      -0.02636389434337616,
      0.04852703958749771,
      0.00042261360795237124,
      -0.008496541529893875,
      -0.015862051397562027,
      -0.040265291929244995,
      0.03237353637814522,
      0.06361255794763565,
      0.025288723409175873,
      0.019419299438595772,
      -0.08891844004392624,
      0.00151180534157902,
      0.02951311506330967,
      0.0008061358821578324,
      0.017357444390654564,
      0.007839015685021877,
      0.018047725781798363,
      0.1183367520570755,
      0.07228198647499084,
      -0.02813178487122059,
      0.027049094438552856,
      -0.07188857346773148,
      0.023648034781217575,
      0.028518086299300194,
      -0.08457920700311661,
      0.0813601016998291,
      -0.02336261235177517,
      -0.016507456079125404,
      0.06610347330570221,
      0.09352266788482666,
      -0.0036757360212504864,
      0.03184602037072182,
      -0.06957991421222687,
      0.017108725383877754,
      0.006004105322062969,
      -0.05464516207575798,
      0.030628317967057228,
      -0.029854673892259598,
      -0.09367018193006516,
      0.027028053998947144,
      -0.001451951451599598,
      0.05591083690524101,
      0.023848699405789375,
      0.021507469937205315,
      0.06981803476810455,
      0.06439214944839478,
      -0.07225064188241959,
      -0.05912204459309578,
      -0.03410935774445534,
      0.11967529356479645,
      -0.04684752970933914,
      0.003930406644940376,
      0.04005679115653038,
      0.02136135846376419,
      -0.04248519241809845,
      0.010312197729945183,
      -0.024079730734229088,
      0.010314052924513817,
      -0.022504057735204697,
      -0.03467293456196785,
      -0.04509783908724785,
      0.11609519273042679,
      -0.009770234115421772,
      -0.02358322963118553,
      0.023358827456831932,
      0.10350919514894485,
      -0.007483534049242735,
      0.09302595257759094,
      -0.002111472189426422,
      -0.03403764218091965,
      0.016529684886336327,
      -0.030004147440195084,
      0.016130944713950157,
      -0.03868066519498825,
      0.0104332584887743,
      0.05662151426076889,
      -0.04727974906563759,
      -1.357294454606972e-8,
      0.10642163455486298,
      -0.0013623692793771625,
      0.06026766076683998,
      -0.03951326385140419,
      0.024045240134000778,
      0.11666487157344818,
      0.0021126337815076113,
      0.022794438526034355,
      -0.04968615621328354,
      0.011026709340512753,
      -0.0664014220237732,
      0.02647843025624752,
      0.02762906812131405,
      0.06240236014127731,
      0.006782029755413532,
      -0.011187623254954815,
      0.03954063728451729,
      0.004930218681693077,
      -0.07038156688213348,
      0.06946661323308945,
      0.0376739539206028,
      0.044212184846401215,
      0.014575139619410038,
      -0.019005155190825462,
      -0.012388641014695168,
      -0.0029710594099014997,
      0.0694991946220398,
      -0.03681844845414162,
      0.01713174395263195,
      0.09619986265897751,
      -0.005801004823297262,
      0.024630054831504822,
      -0.11250384151935577,
      0.028666773810982704,
      0.006120818201452494,
      -0.042589426040649414,
      0.03790668398141861,
      -0.010988209396600723,
      -0.026690157130360603,
      -0.05288493633270264,
      0.013099487870931625,
      0.061343949288129807,
      -0.01793859899044037,
      0.05894609913229942,
      -0.0029380314517766237,
      -0.045418404042720795,
      0.07898731529712677,
      -0.03478022664785385,
      0.02704465575516224,
      -0.006895766593515873,
      0.016534635797142982,
      0.050051603466272354,
      0.006315434351563454,
      -0.010414527729153633,
      0.024025272578001022,
      0.061644546687603,
      -0.019397474825382233,
      0.08599943667650223,
      -0.08958672732114792,
      0.03130229935050011,
      0.054686710238456726,
      -0.03496289625763893,
      0.05289012938737869,
      -0.009723770432174206
    ],
    "k": 5,
    "num_candidates": 50,
    "boost": 0.5, 
    "filter": {
      "bool": {
        "filter": [
          {
            "range": {
              "tags.time.maxTime": {
                "lte": 30
              }
            }
          },
          {
            "range" : {
            "tags.age.minAge": {
              "gte": 7,
              "lte" : 7
            }
          }
          }
        ],
        "should": [],
        "must": [],
        "must_not": []
      }
    }
  }
}

If you launch that request, the third result gives "maxTime": 40" but I gave a range of Lower Than or Equal to 30. As I did put it in filters, I would expect it to remove it from the possibles solutions. Am I doing something wrong ?

[dadoonet (David Pilato)]

When you remove that part:

  "query": {
    "bool": {
      "filter": [],
      "should": [],
      "must": [],
      "must_not": []
    }
  },

I think this is giving the right results.

And actually you should not send an empty bool query. Not sure if it's a bug or not though.
I think that an empty bool query + knn request is seen as a match_all query + knn.
Whereas no query with a knn request is seen just as a knn request.

GET bug_filters/_search
{
  "from": 0,
  "size": 3,
  "query": {
    "bool": {
      "filter": [],
      "should": [],
      "must": [],
      "must_not": []
    }
  }
}

Is the same thing as:

GET bug_filters/_search
{
  "from": 0,
  "size": 3,
  "query": {
    "match_all": {}
  }
}

Also note that this works (on 8.13):

GET bug_filters/_search
{
  "from": 0,
  "size": 3,
  "query": {
    "bool": {
      "must": [
        {
          "knn": {
            "field": "embedding",
            "query_vector": [
              -0.049409836530685425,
              0.004891033284366131,
              0.02810363657772541,
              0.05813274160027504,
              -0.02277800627052784,
              -0.08117515593767166,
              0.03672189638018608,
              -0.003742472967132926,
              0.09365282952785492,
              0.050613727420568466,
              0.013901573605835438,
              -0.020395124331116676,
              -0.007280553225427866,
              0.0022683285642415285,
              -0.0798516720533371,
              -0.020266443490982056,
              -0.027672559022903442,
              -0.0020338057074695826,
              -0.021304968744516373,
              -0.014463381841778755,
              0.013873877935111523,
              0.021126234903931618,
              -0.046334706246852875,
              0.019942348822951317,
              0.004430413246154785,
              0.07496818155050278,
              0.01780945248901844,
              0.027535229921340942,
              -0.16652490198612213,
              -0.05582409352064133,
              -0.024150878190994263,
              0.11059213429689407,
              -0.06544376164674759,
              -0.05248958244919777,
              -0.02786935865879059,
              0.012665773741900921,
              0.032295744866132736,
              0.006287253927439451,
              0.04858362302184105,
              0.004297235049307346,
              -0.037272483110427856,
              -0.13639354705810547,
              0.07986632734537125,
              0.05130545422434807,
              0.012659892439842224,
              0.004586723633110523,
              -0.016139421612024307,
              -0.03315325453877449,
              0.01820875145494938,
              0.06355420500040054,
              -0.07368788868188858,
              0.01303124614059925,
              -0.016112077981233597,
              0.01772332936525345,
              0.011353361420333385,
              -0.04840549826622009,
              0.005034640897065401,
              0.005646976176649332,
              0.0320860892534256,
              -0.11949650198221207,
              0.055256083607673645,
              -0.06408920139074326,
              -0.0685899630188942,
              0.05287231132388115,
              0.11058869957923889,
              0.013125206343829632,
              -0.029818184673786163,
              0.058537840843200684,
              -0.0017318298341706395,
              -0.05754252150654793,
              -0.0037360957358032465,
              0.0509151853621006,
              0.05079891160130501,
              0.06176655739545822,
              0.11007165163755417,
              -0.04189196601510048,
              0.05611620843410492,
              0.014012707397341728,
              0.11743518710136414,
              -0.059737835079431534,
              -0.007209470961242914,
              -0.05508410185575485,
              -0.10902736335992813,
              0.03140096738934517,
              -0.023651396855711937,
              0.02254081889986992,
              0.02056342363357544,
              0.0010580220259726048,
              -0.0675467923283577,
              0.03918212652206421,
              -0.08878873288631439,
              -0.06608942896127701,
              0.027788538485765457,
              0.0020840917713940144,
              -0.09848127514123917,
              0.04274114966392517,
              -0.05088504031300545,
              -0.08866234123706818,
              -0.011348066851496696,
              0.1275867074728012,
              -0.037042342126369476,
              0.0425901897251606,
              -0.05511794611811638,
              0.015774955973029137,
              0.09826533496379852,
              -0.09285881370306015,
              0.029072655364871025,
              -0.025634218007326126,
              0.030678875744342804,
              -0.000753039785195142,
              -0.005528176669031382,
              0.012077361345291138,
              -0.03817363828420639,
              0.012971200980246067,
              0.0861113965511322,
              0.07635430246591568,
              0.08410479873418808,
              0.041298262774944305,
              -0.0007305238978005946,
              -0.03311833366751671,
              -0.010606382042169571,
              0.004648401401937008,
              -0.11425226181745529,
              -0.02954685688018799,
              -0.0649784579873085,
              -0.03329478204250336,
              -0.0011585663305595517,
              -2.4508350081324278e-33,
              0.0316350944340229,
              -0.05173753574490547,
              0.020162351429462433,
              0.06597787886857986,
              0.051326408982276917,
              -0.032706670463085175,
              -0.024263229221105576,
              -0.023685527965426445,
              -0.0017541677225381136,
              0.0007936746114864945,
              -0.05776742100715637,
              0.06051736697554588,
              -0.06538413465023041,
              0.03647594526410103,
              0.08626258373260498,
              0.06327173113822937,
              -0.03350914269685745,
              0.017477653920650482,
              0.004297366365790367,
              -0.06454082578420639,
              -0.08032116293907166,
              0.021543487906455994,
              -0.0378723219037056,
              0.008934469893574715,
              0.02011488936841488,
              -0.01968550682067871,
              -0.035359419882297516,
              -0.05076120048761368,
              0.020596403628587723,
              0.029751403257250786,
              0.004540916066616774,
              0.11520950496196747,
              0.033320922404527664,
              -0.03450218588113785,
              0.057889603078365326,
              -0.062499918043613434,
              -0.03899291157722473,
              -0.02962207794189453,
              0.043336641043424606,
              0.06783860176801682,
              -0.04829634353518486,
              0.03092353790998459,
              -0.041549474000930786,
              0.0021326462738215923,
              0.018427260220050812,
              0.0223116185516119,
              0.09893038868904114,
              0.006302930414676666,
              -0.05646558851003647,
              0.088975690305233,
              -0.015586183406412601,
              -0.016343193128705025,
              0.02104244939982891,
              0.02683810330927372,
              0.01617836207151413,
              -0.01695675030350685,
              0.002295635873451829,
              -0.06167204678058624,
              -0.018849531188607216,
              0.004656618926674128,
              0.0322817824780941,
              0.13475686311721802,
              0.04503224045038223,
              -0.03957787901163101,
              0.04441124573349953,
              0.01955721527338028,
              0.04307923838496208,
              -0.021352173760533333,
              0.006152441259473562,
              -0.006302898749709129,
              -0.08052235096693039,
              0.08250440657138824,
              -0.01693098060786724,
              -0.0003445694746915251,
              0.005235031712800264,
              0.002937984187155962,
              0.08237919211387634,
              -0.012812864035367966,
              -0.09389758855104446,
              -0.006106135901063681,
              -0.024815933778882027,
              -0.03336304426193237,
              -0.02679702453315258,
              0.0192271638661623,
              0.02871078811585903,
              0.018955117091536522,
              -0.06065736338496208,
              -0.029904447495937347,
              -0.011691024526953697,
              -0.025439197197556496,
              -0.04619080573320389,
              -0.11162019520998001,
              0.0537695586681366,
              -0.05119529739022255,
              -0.09530068933963776,
              2.124256228851103e-33,
              -0.0758451297879219,
              -0.09364552050828934,
              -0.060507044196128845,
              0.046550050377845764,
              0.05473474785685539,
              -0.04278877377510071,
              -0.10247790068387985,
              -0.04446272924542427,
              -0.02499961294233799,
              0.0629647746682167,
              -0.10281739383935928,
              -0.006373060401529074,
              0.08628557622432709,
              0.010038319043815136,
              0.0498167984187603,
              -0.04578624665737152,
              -0.014049404300749302,
              -0.005578579381108284,
              -0.02636389434337616,
              0.04852703958749771,
              0.00042261360795237124,
              -0.008496541529893875,
              -0.015862051397562027,
              -0.040265291929244995,
              0.03237353637814522,
              0.06361255794763565,
              0.025288723409175873,
              0.019419299438595772,
              -0.08891844004392624,
              0.00151180534157902,
              0.02951311506330967,
              0.0008061358821578324,
              0.017357444390654564,
              0.007839015685021877,
              0.018047725781798363,
              0.1183367520570755,
              0.07228198647499084,
              -0.02813178487122059,
              0.027049094438552856,
              -0.07188857346773148,
              0.023648034781217575,
              0.028518086299300194,
              -0.08457920700311661,
              0.0813601016998291,
              -0.02336261235177517,
              -0.016507456079125404,
              0.06610347330570221,
              0.09352266788482666,
              -0.0036757360212504864,
              0.03184602037072182,
              -0.06957991421222687,
              0.017108725383877754,
              0.006004105322062969,
              -0.05464516207575798,
              0.030628317967057228,
              -0.029854673892259598,
              -0.09367018193006516,
              0.027028053998947144,
              -0.001451951451599598,
              0.05591083690524101,
              0.023848699405789375,
              0.021507469937205315,
              0.06981803476810455,
              0.06439214944839478,
              -0.07225064188241959,
              -0.05912204459309578,
              -0.03410935774445534,
              0.11967529356479645,
              -0.04684752970933914,
              0.003930406644940376,
              0.04005679115653038,
              0.02136135846376419,
              -0.04248519241809845,
              0.010312197729945183,
              -0.024079730734229088,
              0.010314052924513817,
              -0.022504057735204697,
              -0.03467293456196785,
              -0.04509783908724785,
              0.11609519273042679,
              -0.009770234115421772,
              -0.02358322963118553,
              0.023358827456831932,
              0.10350919514894485,
              -0.007483534049242735,
              0.09302595257759094,
              -0.002111472189426422,
              -0.03403764218091965,
              0.016529684886336327,
              -0.030004147440195084,
              0.016130944713950157,
              -0.03868066519498825,
              0.0104332584887743,
              0.05662151426076889,
              -0.04727974906563759,
              -1.357294454606972e-8,
              0.10642163455486298,
              -0.0013623692793771625,
              0.06026766076683998,
              -0.03951326385140419,
              0.024045240134000778,
              0.11666487157344818,
              0.0021126337815076113,
              0.022794438526034355,
              -0.04968615621328354,
              0.011026709340512753,
              -0.0664014220237732,
              0.02647843025624752,
              0.02762906812131405,
              0.06240236014127731,
              0.006782029755413532,
              -0.011187623254954815,
              0.03954063728451729,
              0.004930218681693077,
              -0.07038156688213348,
              0.06946661323308945,
              0.0376739539206028,
              0.044212184846401215,
              0.014575139619410038,
              -0.019005155190825462,
              -0.012388641014695168,
              -0.0029710594099014997,
              0.0694991946220398,
              -0.03681844845414162,
              0.01713174395263195,
              0.09619986265897751,
              -0.005801004823297262,
              0.024630054831504822,
              -0.11250384151935577,
              0.028666773810982704,
              0.006120818201452494,
              -0.042589426040649414,
              0.03790668398141861,
              -0.010988209396600723,
              -0.026690157130360603,
              -0.05288493633270264,
              0.013099487870931625,
              0.061343949288129807,
              -0.01793859899044037,
              0.05894609913229942,
              -0.0029380314517766237,
              -0.045418404042720795,
              0.07898731529712677,
              -0.03478022664785385,
              0.02704465575516224,
              -0.006895766593515873,
              0.016534635797142982,
              0.050051603466272354,
              0.006315434351563454,
              -0.010414527729153633,
              0.024025272578001022,
              0.061644546687603,
              -0.019397474825382233,
              0.08599943667650223,
              -0.08958672732114792,
              0.03130229935050011,
              0.054686710238456726,
              -0.03496289625763893,
              0.05289012938737869,
              -0.009723770432174206
            ],
            "num_candidates": 50,
            "boost": 0.5,
            "filter": {
              "bool": {
                "filter": [
                  {
                    "range": {
                      "tags.time.maxTime": {
                        "lte": 30
                      }
                    }
                  },
                  {
                    "range": {
                      "tags.age.minAge": {
                        "gte": 7,
                        "lte": 7
                      }
                    }
                  }
                ]
              }
            }
          }
        }
      ]
    }
  }
}

And this can be simplified with:

GET bug_filters/_search
{
  "from": 0,
  "size": 3,
  "query": {
    "knn": {
      "field": "embedding",
      "query_vector": [
        -0.049409836530685425,
        0.004891033284366131,
        0.02810363657772541,
        0.05813274160027504,
        -0.02277800627052784,
        -0.08117515593767166,
        0.03672189638018608,
        -0.003742472967132926,
        0.09365282952785492,
        0.050613727420568466,
        0.013901573605835438,
        -0.020395124331116676,
        -0.007280553225427866,
        0.0022683285642415285,
        -0.0798516720533371,
        -0.020266443490982056,
        -0.027672559022903442,
        -0.0020338057074695826,
        -0.021304968744516373,
        -0.014463381841778755,
        0.013873877935111523,
        0.021126234903931618,
        -0.046334706246852875,
        0.019942348822951317,
        0.004430413246154785,
        0.07496818155050278,
        0.01780945248901844,
        0.027535229921340942,
        -0.16652490198612213,
        -0.05582409352064133,
        -0.024150878190994263,
        0.11059213429689407,
        -0.06544376164674759,
        -0.05248958244919777,
        -0.02786935865879059,
        0.012665773741900921,
        0.032295744866132736,
        0.006287253927439451,
        0.04858362302184105,
        0.004297235049307346,
        -0.037272483110427856,
        -0.13639354705810547,
        0.07986632734537125,
        0.05130545422434807,
        0.012659892439842224,
        0.004586723633110523,
        -0.016139421612024307,
        -0.03315325453877449,
        0.01820875145494938,
        0.06355420500040054,
        -0.07368788868188858,
        0.01303124614059925,
        -0.016112077981233597,
        0.01772332936525345,
        0.011353361420333385,
        -0.04840549826622009,
        0.005034640897065401,
        0.005646976176649332,
        0.0320860892534256,
        -0.11949650198221207,
        0.055256083607673645,
        -0.06408920139074326,
        -0.0685899630188942,
        0.05287231132388115,
        0.11058869957923889,
        0.013125206343829632,
        -0.029818184673786163,
        0.058537840843200684,
        -0.0017318298341706395,
        -0.05754252150654793,
        -0.0037360957358032465,
        0.0509151853621006,
        0.05079891160130501,
        0.06176655739545822,
        0.11007165163755417,
        -0.04189196601510048,
        0.05611620843410492,
        0.014012707397341728,
        0.11743518710136414,
        -0.059737835079431534,
        -0.007209470961242914,
        -0.05508410185575485,
        -0.10902736335992813,
        0.03140096738934517,
        -0.023651396855711937,
        0.02254081889986992,
        0.02056342363357544,
        0.0010580220259726048,
        -0.0675467923283577,
        0.03918212652206421,
        -0.08878873288631439,
        -0.06608942896127701,
        0.027788538485765457,
        0.0020840917713940144,
        -0.09848127514123917,
        0.04274114966392517,
        -0.05088504031300545,
        -0.08866234123706818,
        -0.011348066851496696,
        0.1275867074728012,
        -0.037042342126369476,
        0.0425901897251606,
        -0.05511794611811638,
        0.015774955973029137,
        0.09826533496379852,
        -0.09285881370306015,
        0.029072655364871025,
        -0.025634218007326126,
        0.030678875744342804,
        -0.000753039785195142,
        -0.005528176669031382,
        0.012077361345291138,
        -0.03817363828420639,
        0.012971200980246067,
        0.0861113965511322,
        0.07635430246591568,
        0.08410479873418808,
        0.041298262774944305,
        -0.0007305238978005946,
        -0.03311833366751671,
        -0.010606382042169571,
        0.004648401401937008,
        -0.11425226181745529,
        -0.02954685688018799,
        -0.0649784579873085,
        -0.03329478204250336,
        -0.0011585663305595517,
        -2.4508350081324278e-33,
        0.0316350944340229,
        -0.05173753574490547,
        0.020162351429462433,
        0.06597787886857986,
        0.051326408982276917,
        -0.032706670463085175,
        -0.024263229221105576,
        -0.023685527965426445,
        -0.0017541677225381136,
        0.0007936746114864945,
        -0.05776742100715637,
        0.06051736697554588,
        -0.06538413465023041,
        0.03647594526410103,
        0.08626258373260498,
        0.06327173113822937,
        -0.03350914269685745,
        0.017477653920650482,
        0.004297366365790367,
        -0.06454082578420639,
        -0.08032116293907166,
        0.021543487906455994,
        -0.0378723219037056,
        0.008934469893574715,
        0.02011488936841488,
        -0.01968550682067871,
        -0.035359419882297516,
        -0.05076120048761368,
        0.020596403628587723,
        0.029751403257250786,
        0.004540916066616774,
        0.11520950496196747,
        0.033320922404527664,
        -0.03450218588113785,
        0.057889603078365326,
        -0.062499918043613434,
        -0.03899291157722473,
        -0.02962207794189453,
        0.043336641043424606,
        0.06783860176801682,
        -0.04829634353518486,
        0.03092353790998459,
        -0.041549474000930786,
        0.0021326462738215923,
        0.018427260220050812,
        0.0223116185516119,
        0.09893038868904114,
        0.006302930414676666,
        -0.05646558851003647,
        0.088975690305233,
        -0.015586183406412601,
        -0.016343193128705025,
        0.02104244939982891,
        0.02683810330927372,
        0.01617836207151413,
        -0.01695675030350685,
        0.002295635873451829,
        -0.06167204678058624,
        -0.018849531188607216,
        0.004656618926674128,
        0.0322817824780941,
        0.13475686311721802,
        0.04503224045038223,
        -0.03957787901163101,
        0.04441124573349953,
        0.01955721527338028,
        0.04307923838496208,
        -0.021352173760533333,
        0.006152441259473562,
        -0.006302898749709129,
        -0.08052235096693039,
        0.08250440657138824,
        -0.01693098060786724,
        -0.0003445694746915251,
        0.005235031712800264,
        0.002937984187155962,
        0.08237919211387634,
        -0.012812864035367966,
        -0.09389758855104446,
        -0.006106135901063681,
        -0.024815933778882027,
        -0.03336304426193237,
        -0.02679702453315258,
        0.0192271638661623,
        0.02871078811585903,
        0.018955117091536522,
        -0.06065736338496208,
        -0.029904447495937347,
        -0.011691024526953697,
        -0.025439197197556496,
        -0.04619080573320389,
        -0.11162019520998001,
        0.0537695586681366,
        -0.05119529739022255,
        -0.09530068933963776,
        2.124256228851103e-33,
        -0.0758451297879219,
        -0.09364552050828934,
        -0.060507044196128845,
        0.046550050377845764,
        0.05473474785685539,
        -0.04278877377510071,
        -0.10247790068387985,
        -0.04446272924542427,
        -0.02499961294233799,
        0.0629647746682167,
        -0.10281739383935928,
        -0.006373060401529074,
        0.08628557622432709,
        0.010038319043815136,
        0.0498167984187603,
        -0.04578624665737152,
        -0.014049404300749302,
        -0.005578579381108284,
        -0.02636389434337616,
        0.04852703958749771,
        0.00042261360795237124,
        -0.008496541529893875,
        -0.015862051397562027,
        -0.040265291929244995,
        0.03237353637814522,
        0.06361255794763565,
        0.025288723409175873,
        0.019419299438595772,
        -0.08891844004392624,
        0.00151180534157902,
        0.02951311506330967,
        0.0008061358821578324,
        0.017357444390654564,
        0.007839015685021877,
        0.018047725781798363,
        0.1183367520570755,
        0.07228198647499084,
        -0.02813178487122059,
        0.027049094438552856,
        -0.07188857346773148,
        0.023648034781217575,
        0.028518086299300194,
        -0.08457920700311661,
        0.0813601016998291,
        -0.02336261235177517,
        -0.016507456079125404,
        0.06610347330570221,
        0.09352266788482666,
        -0.0036757360212504864,
        0.03184602037072182,
        -0.06957991421222687,
        0.017108725383877754,
        0.006004105322062969,
        -0.05464516207575798,
        0.030628317967057228,
        -0.029854673892259598,
        -0.09367018193006516,
        0.027028053998947144,
        -0.001451951451599598,
        0.05591083690524101,
        0.023848699405789375,
        0.021507469937205315,
        0.06981803476810455,
        0.06439214944839478,
        -0.07225064188241959,
        -0.05912204459309578,
        -0.03410935774445534,
        0.11967529356479645,
        -0.04684752970933914,
        0.003930406644940376,
        0.04005679115653038,
        0.02136135846376419,
        -0.04248519241809845,
        0.010312197729945183,
        -0.024079730734229088,
        0.010314052924513817,
        -0.022504057735204697,
        -0.03467293456196785,
        -0.04509783908724785,
        0.11609519273042679,
        -0.009770234115421772,
        -0.02358322963118553,
        0.023358827456831932,
        0.10350919514894485,
        -0.007483534049242735,
        0.09302595257759094,
        -0.002111472189426422,
        -0.03403764218091965,
        0.016529684886336327,
        -0.030004147440195084,
        0.016130944713950157,
        -0.03868066519498825,
        0.0104332584887743,
        0.05662151426076889,
        -0.04727974906563759,
        -1.357294454606972e-8,
        0.10642163455486298,
        -0.0013623692793771625,
        0.06026766076683998,
        -0.03951326385140419,
        0.024045240134000778,
        0.11666487157344818,
        0.0021126337815076113,
        0.022794438526034355,
        -0.04968615621328354,
        0.011026709340512753,
        -0.0664014220237732,
        0.02647843025624752,
        0.02762906812131405,
        0.06240236014127731,
        0.006782029755413532,
        -0.011187623254954815,
        0.03954063728451729,
        0.004930218681693077,
        -0.07038156688213348,
        0.06946661323308945,
        0.0376739539206028,
        0.044212184846401215,
        0.014575139619410038,
        -0.019005155190825462,
        -0.012388641014695168,
        -0.0029710594099014997,
        0.0694991946220398,
        -0.03681844845414162,
        0.01713174395263195,
        0.09619986265897751,
        -0.005801004823297262,
        0.024630054831504822,
        -0.11250384151935577,
        0.028666773810982704,
        0.006120818201452494,
        -0.042589426040649414,
        0.03790668398141861,
        -0.010988209396600723,
        -0.026690157130360603,
        -0.05288493633270264,
        0.013099487870931625,
        0.061343949288129807,
        -0.01793859899044037,
        0.05894609913229942,
        -0.0029380314517766237,
        -0.045418404042720795,
        0.07898731529712677,
        -0.03478022664785385,
        0.02704465575516224,
        -0.006895766593515873,
        0.016534635797142982,
        0.050051603466272354,
        0.006315434351563454,
        -0.010414527729153633,
        0.024025272578001022,
        0.061644546687603,
        -0.019397474825382233,
        0.08599943667650223,
        -0.08958672732114792,
        0.03130229935050011,
        0.054686710238456726,
        -0.03496289625763893,
        0.05289012938737869,
        -0.009723770432174206
      ],
      "num_candidates": 50,
      "boost": 0.5,
      "filter": {
        "bool": {
          "filter": [
            {
              "range": {
                "tags.time.maxTime": {
                  "lte": 30
                }
              }
            },
            {
              "range": {
                "tags.age.minAge": {
                  "gte": 7,
                  "lte": 7
                }
              }
            }
          ]
        }
      }
    }
  }
}

[Yoann_Buzenet]

Thanks !!! It is working !

I was using empty array of filters to programatically add some if I needed to, but I didn't know it was changing the behavior. I spent so many hours on this. Thank you!!

Merci David

[BenTrent]

The reason here is because when a top level query, we will do an OR. So, since the top-level query was effectively a match-all, it ended up matching everything.

The new Knn query | Elasticsearch Guide [8.13] | Elastic gives you a bit more flexibility.