Adjusting the numbers of docs retrieved via es python api

Elastic Search
1

I've been using the combination of lang chain retriever and ES python client API in searching, my code is like below:

 def embed_query(self,query_text,k =10,candidate_num = 20):
        if query_text is not None and len(query_text) == 0:
            print("query is empty")
            return None
        embedding_model,_ = self.embedding_model_local()
        query_vector = embedding_model.encode(query_text)
        return{
            "knn": {
                "field": "output_dense_vector",
                "query_vector": query_vector,
                "k": k,
                "num_candidates": candidate_num
            },
            "from":0,
            "size":20
        }


    def es_A_KNN(self,query):
        es_retriever = ElasticsearchRetriever(
        index_name=self.index_name,
        body_func=self.embed_query,
        content_field="text",
        es_client = self.es_client,
    )
        res =es_retriever.invoke(query)
        return {i.page_content: i.metadata["_score"] for i in res}

So, my question is that HOW can i increase the number of docs retrieved in both knn and bm25 methods? I am aware that by default ES return the top 10 docs in both method, but how can I increase that number to like 30? I've tried adding params like 'size’ : 30, but it won't work, I've also checked my elasticsearch.yml but I couldn't find any settings that can change this, so is there any other method I can try ?

2

For knn you need to make sure to adjust k and num_candidates to control how much content comes back.

3

Thanks！ But I've tried changing those two params earlier and still got only 10 docs returned, is there any other way?

4

Using size: 30 should work.
How many documents do you have? Could you share the json response?

Ideally, could you reproduce this in Kibana dev console? We can then know better if we should look at the Elasticsearch behavior or your Python code.

What is your elastic version?

5

Thanks!
In my case I've set up a test index where there are 1400+ docs, they are all of same field that I crawled earlier via arxiv loader.

I cannot perform that on kibana as I'm using a local embedding model.

My deployed ES version is 8.13.

Again I've tried integrating the size param in my body func yet it's not working.

My query is quite in general so there should be much more docs retrieved, is there any calling questions?

6

Yeah but may be the same request in Kibana with the array of floats computed by your model would work.

That's actually how I'm debugging those things.

7

ah yeah that rings a bell, I'll have a try, thanks!

8

I've tried copy the generated vector and paste on kibana dev tool by I received:

{
  "statusCode": 400,
  "error": "Bad Request",
  "message": """[request query.method]: Method must be one of, case insensitive ['HEAD', 'GET', 'POST', 'PUT', 'DELETE', 'PATCH']. Received '"query_vector"'."""
}

and my request body is :

GET /index_speed_test/_search
{"size": 30,
  "query": {
    "knn": {
      "field": "output_dense_vector",
      "vector":[0.014877190813422203, -0.00964432954788208, 0.047179367393255234, -0.037920862436294556, -0.03379930555820465, -0.01367100328207016, 0.002096517477184534, -0.01506971288472414, 0.009096572175621986, 0.06122267246246338, 0.03660641238093376, 0.01949719525873661, 0.02460334077477455, -0.03771049901843071, -0.0039428165182471275, 0.014637582004070282, -0.04660990834236145, -0.060452405363321304, -0.054135508835315704, 0.02593880519270897, 0.019100798293948174, 0.004387453198432922, -0.0704440176486969, -0.04110654070973396, 0.0063305930234491825, 0.035741619765758514, 0.030930209904909134, 0.04562607780098915, 0.05782672390341759, 0.09715761244297028, -0.006850224919617176, 0.0097988061606884, 0.01192520372569561, -0.050807129591703415, 0.012421529740095139, -0.02147330343723297, -0.0035631798673421144, -0.01659218780696392, -0.025850359350442886, -0.0753711611032486, -0.014150111004710197, -0.03203905001282692, 0.05386258661746979, -0.029379287734627724, -0.06714817881584167, 0.02972530946135521, -0.01359458826482296, -0.007314483635127544, 0.008689108304679394, -0.07705409824848175, -0.015831487253308296, 0.0042047868482768536, 0.04747992753982544, 0.0006109263631515205, 0.0045383828692138195, 0.0060368808917701244, -0.04277113452553749, 0.0021964279003441334, 0.008689725771546364, -0.008351512253284454, 0.049758635461330414, 0.004954839125275612, -0.015450707636773586, -0.035258352756500244, -0.02136588841676712, 0.055682405829429626, -0.023861469700932503, 0.012039057910442352, 0.02124408632516861, -0.05084434896707535, 0.008508061058819294, 0.021254602819681168, 0.030374988913536072, -0.07241565734148026, -0.04991608485579491, 0.009855658747255802, 0.00061379699036479, -0.026322923600673676, -0.015372933819890022, 0.03104882314801216, -0.02081126905977726, 0.04950472339987755, 0.010674667544662952, 0.02983057126402855, -0.057516537606716156, -0.04537239298224449, -0.025530124083161354, 0.026197390630841255, 0.016517959535121918, -0.04203428700566292, 0.015013390220701694, 0.05439336970448494, 0.011304570361971855, 0.002303289482370019, 0.008512003347277641, 0.01981200836598873, -0.030694739893078804, 0.06739850342273712, -0.019062569364905357, -0.01547209732234478, 0.028693068772554398, 0.025281554087996483, -0.01498508919030428, -0.0007564314873889089, -0.02178824134171009, 0.0010176399955525994, -0.0007770528318360448, 0.03209022805094719, -0.0021312644239515066, -0.016525670886039734, -0.019016867503523827, -0.014348332770168781, -0.005534196738153696, -0.022037969902157784, 0.012489063665270805, 0.03154178336262703, -0.011445309966802597, 0.01582528091967106, -0.026754213497042656, 0.02613447792828083, 0.014084339141845703, 0.011633286252617836, 0.02534499205648899, -0.04188947007060051, -0.016533832997083664, -0.029661880806088448, -0.029395692050457, 0.02065984345972538, -0.04887218773365021, 0.029428506270051003, -0.003748644143342972, -0.027951354160904884, 0.04932432621717453, 0.03535687178373337, 0.014974001795053482, 0.013417229987680912, -0.04726390913128853, 0.03848240524530411, 0.02708667889237404, -0.0475030280649662, 0.008707713335752487, -0.027086609974503517, -0.047661516815423965, 0.09585893899202347, -0.0017465401906520128, 0.04102129861712456, 0.0040794177912175655, -0.008011970669031143, 0.008968058042228222, 0.03539944440126419, -0.0006347823073156178, 0.016840990632772446, 0.016639187932014465, 0.03730950877070427, 0.011090746149420738, -0.02778654918074608, 0.016581429168581963, 0.027441222220659256, 0.006775369867682457, 0.01461210660636425, 0.011407832615077496, -0.003798933234065771, -0.042358819395303726, 0.045060474425554276, 0.0020587772596627474, 0.008091817609965801, 0.0082516735419631, -0.007052608300000429, 0.03097003512084484, -0.022625155746936798, -0.018050452694296837, -0.04118303582072258, -0.004030202981084585, -0.015697142109274864, 0.010962110944092274, -0.00029388043913058937, 0.03377857431769371, -0.01221536472439766, 0.07849838584661484, 0.041614603251218796, -0.013287404552102089, -0.008091814815998077, 0.03056575171649456, 0.01894361898303032, -0.013327793218195438, -0.0035150169860571623, -0.021552840247750282, 0.007485133595764637, -0.034491196274757385, -0.009606662206351757, 0.015628639608621597, 0.047993652522563934, -0.009929870255291462, 0.009387025609612465, 0.013294327072799206, 0.00984904170036316, -0.008695550262928009, -0.019797958433628082, -0.021533602848649025, -0.04703691601753235, -0.014251448214054108, 0.017406737431883812, -0.01542145200073719, 0.015258918516337872, 0.054813168942928314, -0.014865640550851822, 0.03617073968052864, 0.01840898022055626, -0.05854647606611252, -0.03851685672998428, -0.0005224772030487657, 0.024316642433404922, -0.031156303361058235, 0.015382654033601284, -0.010638105683028698, -0.017470436170697212, -0.02512843906879425, 0.013196117244660854, -0.017059147357940674, -0.0067016370594501495, -0.00306690507568419, 0.0408872589468956, 0.033851560205221176, 0.019343672320246696, 0.009189710021018982, -0.035170651972293854, -0.026040425524115562, 0.0642014667391777, -0.02860274165868759, 0.04395110532641411, -0.013058774173259735, 0.05331913009285927, 0.024571577087044716, 0.03140098601579666, 0.04982579126954079, 0.004676703363656998, 0.04305845871567726, 0.03269093856215477, 0.035080693662166595, 0.028683802112936974, 0.03137795254588127, 0.019999021664261818, 0.042326293885707855, -0.006882424931973219, -0.004304523579776287, 0.021970046684145927, 0.014457643032073975, -0.009079232811927795, -0.0031706904992461205, 0.04070373624563217, 0.0040634493343532085, 0.04007498547434807, 0.025965694338083267, 0.018168196082115173, -0.02938125841319561, -0.00040472435648553073, 0.04326288774609566, 0.04749785736203194, -0.015600323677062988, 0.003092097118496895, 0.0010756917763501406, 0.016279473900794983, 0.015196338295936584, 0.008512589149177074, 0.033914949744939804, 0.005462720990180969, -0.03702308610081673, 0.005741048138588667, -0.045935772359371185, -0.06160064414143562, -0.017154691740870476, -0.050090283155441284, -0.029467621818184853, 0.008402192033827305, -0.046515289694070816, -0.007276805583387613, 0.023676231503486633, -0.021975737065076828, -0.0005499374819919467, -0.020588697865605354, -0.008977622725069523, -0.012628233060240746, 0.013503565452992916, 0.010712054558098316, 0.025735510513186455, -0.004456554539501667, -0.00305060762912035, 0.016264518722891808, -0.035191211849451065, 0.020507492125034332, -0.006019191816449165, 0.009732008911669254, -0.04008232802152634, -0.0037818069104105234, 0.023059051483869553, 0.03779194876551628, -0.011069019325077534, -0.0320231169462204, -0.029160283505916595, -0.049275510013103485, -0.027990804985165596, -0.023243917152285576, 0.019629737362265587, -0.01938069798052311, -0.010218637995421886, 0.02536162920296192, 0.06290353834629059, -0.0457678847014904, 0.019540026783943176, 0.007013388443738222, -0.0364580899477005, 0.012586595490574837, 0.022190166637301445, -0.036218173801898956, -0.030835958197712898, 0.022498974576592445, 0.033217739313840866, 0.039072275161743164, -0.04443705081939697, 0.009495621547102928, -0.04275078698992729, -0.051260851323604584, 0.01821100525557995, -0.02116473764181137, -0.0074309539049863815, 0.03865576535463333, 0.015126237645745277, -0.06820692867040634, 0.0562957301735878, -0.020250454545021057, -0.05696530640125275, -0.0448654443025589, 0.011941072531044483, 0.033665090799331665, -0.0022611559834331274, 0.04049292951822281, -0.03327418863773346, -0.018623115494847298, 0.014067402109503746, 0.02683117426931858, -0.0019795240368694067, -0.024698669090867043, 0.0038174595683813095, 0.021183768287301064, -0.0012524803169071674, -0.026921015232801437, 0.011022011749446392, -0.0008797720656730235, 0.004446431063115597, 0.038386277854442596, -0.014341250993311405, 0.0031820409931242466, 0.029433850198984146, -0.010493448935449123, 0.051273442804813385, 0.053244322538375854, -0.03277934715151787, 0.06641846150159836, 0.03507956117391586, -0.0045494758524000645, 0.02436796762049198, 0.01830284669995308, -0.0012302716495469213, -0.04134947061538696, 0.0068635037168860435, -0.041446659713983536, -0.02294014021754265, -0.004244308453053236, 0.020084768533706665, -0.08435085415840149, 0.0842733234167099, -0.0224437415599823, 0.01542201079428196, 0.002220820402726531, -0.02025698497891426, -0.03293965384364128, 0.023586351424455643, 0.02440531551837921, 0.07371904700994492, -0.02519090846180916, -0.02080267295241356, -0.07135990262031555, 0.00026993610663339496, 0.04301485791802406, -0.00770069845020771, 0.0007907830877229571, -0.014863039366900921, -0.05179215595126152, -0.05206899344921112, -0.025341831147670746, -0.022118568420410156, 0.0021757709328085184, 0.02572280913591385, -0.029300184920430183, -0.020244918763637543, -0.05762265622615814, 0.04739172384142876, 0.03070664219558239, 0.01399970892816782, 0.021605301648378372, -0.007653075270354748, 0.061095524579286575, 0.05057588219642639, 0.04716305807232857, -0.02149246074259281, -0.0097361383959651, -0.018621347844600677, 0.03942466154694557, 0.0021744335535913706, -0.015722526237368584, -0.022294873371720314, -0.010331541299819946, -0.05081785470247269, 0.022285763174295425, -0.020602190867066383, 0.008169610053300858, -0.056171566247940063, 0.05158349499106407, -0.0003291048342362046, -0.02212579734623432, 0.011896569281816483, -0.023465709760785103, 0.021756934002041817, 0.003358700545504689, 0.04593992605805397, -0.04608972370624542, 0.025619668886065483, -0.0627284049987793, 0.029372068122029305, 0.050796590745449066, -0.0076603214256465435, -0.053464051336050034, 0.00011787407856900245, -0.05453422665596008, -0.012346716597676277, -0.012963722459971905, 0.05504828691482544, -0.01546421553939581, 0.05273667722940445, -0.06731195747852325, 0.04411652684211731, 0.005051721353083849, 0.0035566261503845453, 0.03235335648059845, -0.02454611286520958, 0.019648080691695213, -0.03052893839776516, 0.01952703855931759, -0.01990598998963833, -0.026784686371684074, 0.001002997043542564, -0.03951827809214592, 0.010304730385541916, -0.031815797090530396, 0.0018653759034350514, 0.028611911460757256, -0.006716581992805004, 0.029448458924889565, 0.040567245334386826, 0.012901149690151215, 0.01870628260076046, 0.00794687308371067, 0.022527514025568962, 0.023480819538235664, -0.022946244105696678, 0.044297728687524796, -0.022503720596432686, -0.021142572164535522, 0.04710077866911888, 0.04098281264305115, 0.005055297631770372, -0.015423904173076153, 0.04737192764878273, -0.00047135225031524897, -0.03090381808578968, -0.03163216635584831, 0.046671438962221146, 0.014409023337066174, -0.017144503071904182, -0.04663970693945885, -0.037801578640937805, 0.02367851324379444, -0.019476192072033882, -0.0036412090994417667, -0.044882334768772125, -0.05810846760869026, -0.035738226026296616, -0.027330584824085236, -0.01149086281657219, 0.0015045074978843331, 0.01721487008035183, -0.03824247047305107, -0.010506204329431057, -0.01633973978459835, -0.00871849711984396, -0.04408405348658562, -0.03931216895580292, 0.008783834986388683, 0.03238968923687935, 0.020786477252840996, 0.033686235547065735, -0.025454092770814896, -0.02098853327333927, 0.0032821206841617823, 0.020576145499944687, 0.019913651049137115, -0.018832601606845856, 0.03915763273835182, -0.05060019716620445, 0.020953945815563202, 0.007078374736011028, -0.00204140180721879, -0.02257845550775528, 0.028566595166921616, 0.031247563660144806, 0.02690303511917591, -0.015467433258891106, -0.01796361804008484, -0.031350165605545044, 0.028625279664993286, 0.018012164160609245, -0.042073510587215424, -0.026839040219783783, 0.0437338761985302, -0.0022371220402419567, 0.02065621316432953, 0.03802661970257759, -0.000279232335742563, -0.03531745448708534, -0.04975235462188721, 0.05360827222466469, -0.05404920130968094, -0.025734541937708855, -0.08127585798501968, 0.01336598303169012, -0.003947422839701176, 0.016089487820863724, 0.0366726778447628, -0.03245110064744949, 0.0005982230650261045, -0.01610167883336544, 0.05380721390247345, -0.019303057342767715, 0.014595158398151398, -0.042338646948337555, -0.024515077471733093, 0.01778712123632431, 0.05457732826471329, 0.0023582293651998043, 0.06408410519361496, 0.004353031981736422, 0.022893987596035004, 0.006734668277204037, -0.04450275003910065, -0.025955447927117348, -0.03131900727748871, 0.004246486350893974, -0.021738210693001747, 0.026606962084770203, 0.030088555067777634, -0.02656504511833191, 0.04135451465845108, -0.04038405790925026, -0.006336127873510122, -0.057346951216459274, -0.03877798467874527, -0.025782259181141853, 0.014183794148266315, 0.05593379586935043, -0.013800335116684437, -0.06013575941324234, 0.0496777780354023, 0.041082561016082764, 0.024477524682879448, -0.012450768612325191, -0.014069421216845512, -0.04529688507318497, -0.04662732034921646, -0.021376119926571846, 0.02800050750374794, 0.003465194022282958, 0.0002973472001031041, -0.03646054491400719, 0.006513099651783705, 0.049110740423202515, -0.022590357810258865, 0.003175935009494424, 0.06959319114685059, -0.013020130805671215, -0.0007741303998045623, -0.05536818876862526, 0.02138538286089897, 0.009915867820382118, -0.018130719661712646, -0.004312918521463871, -0.002968129236251116, -0.04653565585613251, 0.03067956119775772, -0.01531294360756874, -0.04791760444641113, -0.034668389707803726, 0.0018994787242263556, 0.03586926683783531, -0.0124830212444067, -0.012975642457604408, 0.037954479455947876, -0.019920714199543, -0.0563679076731205, 0.040415938943624496, 0.002396076451987028, -0.003413091180846095, 0.03189728409051895, -0.03258630633354187, 0.013738786801695824, -0.028058761730790138, -0.0013944559032097459, 0.009567836299538612, 0.0070859952829778194, 0.07241746783256531, -0.010103894397616386, -0.05062355846166611, -0.0014447805006057024, 0.011196957901120186, -0.040619075298309326, -0.02310745045542717, -0.051416315138339996, -0.06282549351453781, -0.020971722900867462, -0.037091586738824844, 0.03638594225049019, 0.01337733305990696, -0.005913778208196163, 0.005296691320836544, 0.029324021190404892, -0.027547433972358704, 0.025823799893260002, 0.02589205838739872, 0.011660928837954998, -0.02575068175792694, 0.03362514078617096, 0.0013422846095636487, 0.010340060107409954, -0.008632013574242592, -0.021705392748117447, -0.010115060955286026, -0.020479386672377586, 0.021255044266581535, 0.029144536703824997, 0.004750912543386221, 0.022912899032235146, 0.008807786740362644, 0.0307097677141428, 0.019021740183234215, -0.03632801026105881, -0.012210841290652752, 0.03846926614642143, -0.05831955000758171, -0.014528660103678703, -0.005520176142454147, -0.012482739984989166, 0.030765945091843605, 0.015514987520873547, -0.046948861330747604, -0.0029043618123978376, 0.04758819192647934, 0.03103654272854328, 0.013001436367630959, -0.07426807284355164, -0.02806241810321808, -0.0739947184920311, -0.02033238112926483, 0.0012462681625038385, -0.011234956793487072, -0.004795664921402931, 0.019882865250110626, 0.001973475329577923, -0.02618069387972355, -0.019767455756664276, 0.054507527500391006, 0.011022345162928104, 0.031361717730760574, -0.03623136132955551, 0.04141423478722572, -0.03087158128619194, -0.0266877431422472, 0.0005772734875790775, -0.004235364962369204, -0.0087025947868824, -0.0063502490520477295, -0.012379142455756664, 0.004848089534789324, 0.022540656849741936, 0.03401479870080948, 0.025882171466946602, 0.009674934670329094, -0.02605799026787281, -0.009755532257258892, 0.007013839669525623, 0.01957785338163376, 0.015247918665409088, 0.010196477174758911, 0.007159324362874031, -0.01824735291302204, 0.04811076074838638, 0.028145667165517807, -0.06024622544646263, -0.016154717653989792, -0.017861245200037956, 0.04034890606999397, 0.004611301235854626, -0.022826354950666428, -0.05375576391816139, 0.009675242006778717, -0.024830374866724014, 0.05476880818605423, -0.00788688100874424, -0.0532805360853672, -0.005320553667843342, 0.00041916442569345236, 0.005807976704090834, 0.0543099045753479, 0.0255923829972744, 0.03986520320177078, 0.02493361569941044, 0.005165687762200832, -0.00571638997644186, 0.017759133130311966, 0.025824349373579025, 0.029609911143779755, 0.012049637734889984, -0.005607698578387499, 0.031372614204883575, 0.019424062222242355, -0.011483492329716682, 0.015998318791389465, -0.02903958037495613, -0.0008431806927546859, -0.007486965972930193, 0.021048778668045998, 0.035216670483350754, 0.04268008843064308, -0.0016613128827884793, -0.005128379445523024, 0.00843129213899374, -0.06633921712636948, -0.04005887731909752, -0.013219715096056461, -0.07007990032434464, -0.00621415488421917, 0.040030840784311295, -0.01893097721040249, -0.00423947861418128, -0.0431206040084362, -0.03203153237700462, -0.034488193690776825, -0.002637356985360384, 0.0019196189241483808, -0.04172821715474129, 0.047330938279628754, -0.030305689200758934, 0.03450203314423561, -0.014994543045759201, 0.008091413415968418, 0.004283797461539507, 0.006265851203352213, 0.006155990529805422, -0.02933504804968834, 0.03093528188765049, 0.005879746749997139, -0.022586792707443237, -0.005479645915329456, 0.004598088096827269, 0.010248151607811451, -0.06868427991867065, 0.04690646007657051, 0.024371962994337082, -0.016566269099712372, -0.01812489703297615, 0.005327638704329729, 0.003787643276154995, -0.005616848357021809, -0.049463991075754166, 0.048050425946712494, 0.013784226961433887, -0.022046679630875587, -0.014384360052645206, 0.040461696684360504, -0.011091590858995914, 0.017112424597144127, 0.035586971789598465, 0.026428258046507835, 0.03630095347762108, -0.019594062119722366, -0.02045856975018978, 0.008805994875729084, -0.00042503990698605776, 0.021759474650025368, -0.005389260593801737, -0.03638923913240433, 0.01638391800224781, 0.022850709035992622, 0.010830811224877834, 0.013784686103463173, 0.004017945379018784, 0.00926978513598442, -0.005811302922666073, 0.009726356714963913, 0.040096864104270935, -0.026854287832975388, 0.026861172169446945, -0.038692086935043335, 0.03167109936475754, 0.009756878949701786, -0.04372242093086243, -0.0174575075507164, 0.03521124646067619, 0.055500395596027374, -0.03144461289048195, -0.0017711411928758025, -0.021764634177088737, -0.009982428513467312, -0.0019348944770172238, 0.05632437393069267, 0.01632474921643734, -0.01129110436886549, 0.015912475064396858, 0.006522991228848696, 0.009665228426456451, 0.042724817991256714, 0.034255094826221466, 0.006323882844299078, -0.04542335122823715, -0.01858973130583763, -0.010515990667045116, -0.01670394092798233, -0.03704683110117912, 0.0018620791379362345, -0.07281225174665451, 0.03493009880185127, -0.015372267924249172, -0.011412173509597778, 0.04232732579112053, -0.06556537747383118, 0.033720772713422775, -0.036731403321027756, 0.0397176519036293, -0.01352137140929699, 0.017690014094114304, 0.030857305973768234, -0.0014228716026991606, -0.01537939254194498, 0.036982107907533646, -0.03545989841222763, 0.023724619299173355, 0.01511459518224001, 0.04847174510359764, -0.030947258695960045, 0.016651399433612823, 0.04137655720114708, -0.028442176058888435, -0.021464025601744652, 0.003372756764292717, 0.012610041536390781, -0.03988620266318321, -0.0014316464075818658, -0.034012213349342346, 0.03476599231362343, -0.031351715326309204, -0.016989748924970627, 0.01884266920387745, 0.05166802927851677, -0.005991823971271515, -0.030360551550984383, -0.046020179986953735, -0.009152636863291264, -0.08136212080717087, 0.0206372682005167, 0.03581587225198746, 0.014090691693127155, -0.04778837412595749, -0.0037769463378936052, -0.020982710644602776, -0.024659447371959686, 0.012434600852429867, -0.05418136715888977, 0.06484901905059814, -0.03823978826403618, -0.01901637203991413, -0.014071748591959476, -0.04560402035713196, 0.018520016223192215, -0.001148468698374927, -0.02458980306982994, 0.007788615766912699, 0.01287071593105793, 0.012622234411537647, 0.00047100576921366155, -0.032985929399728775, -0.022677285596728325, 0.021429916843771935, 0.0322362557053566, 0.042589087039232254, 0.023928312584757805, 0.08289144188165665, -0.0034659933298826218, 0.007940144278109074, -0.0006862303125672042, 0.003423316404223442, 0.01767600327730179, -0.01865966059267521, -0.012069771997630596, -0.021413249894976616, 0.033022355288267136, 0.0010587621945887804, -0.025317635387182236, -0.0006586848176084459, 0.06860142201185226, -0.00519771920517087, -0.015847088769078255, -0.04899671673774719, 0.014637485146522522, -0.06196102499961853, -0.011059939861297607, -0.04639309644699097, 0.007964689284563065, 0.005026532337069511, -0.006794532295316458, -0.013506145216524601, -0.0567498505115509, 0.2209136039018631, 0.0015600294573232532, 0.0072324262000620365, 0.00598642323166132, -0.03791138529777527, 0.022357992827892303, 0.012784833088517189, -0.004153634421527386, 0.024235287681221962, -0.033359698951244354, 0.003484846092760563, -0.027728324756026268, 0.03147926554083824, 0.036986589431762695, 0.0195436030626297, 0.04373743012547493, -0.048387251794338226, -0.0032542585395276546, -0.005672814790159464, -0.026110678911209106, -0.051742397248744965, -0.023095212876796722, 0.000932470487896353, 0.0009706320124678314, -0.0373365618288517, -0.015116464346647263, 0.06740178167819977, -0.02928442507982254, 0.03333081677556038, 0.0051732901483774185, 0.04044094309210777, 0.00528694735839963, 0.0037018791772425175, -0.03193828836083412, -0.017787182703614235, 0.009572948329150677, -0.03647616505622864, -0.037094686180353165, 0.03800000250339508, 0.04339112713932991, -0.006568847689777613, 0.002716455841436982, 0.05498112365603447, 0.015137936919927597, 0.03749866783618927, 0.039411235600709915, -0.020253971219062805, 0.01085424330085516, -0.008671829476952553, -0.01447191834449768, 0.029858961701393127, -0.047425802797079086, 0.007435064297169447, -0.04562372341752052, -0.051878612488508224, -0.020492754876613617, 0.010294234380126, -0.040705300867557526, 0.006891036406159401, -0.02801116369664669, -0.00421505980193615, -0.00320437946356833, 0.0016814544796943665, -0.053144048899412155, -0.028832213953137398, 0.027178464457392693, 0.02837289869785309, 0.0023187368642538786, 0.016900498420000076, 0.0018818493699654937, 0.008348776958882809, -0.014894535765051842, -0.004712078254669905, -0.02046998031437397, 0.0019172009779140353, -0.00295373797416687, -0.017961613833904266, 0.019987277686595917, -0.025676395744085312, 0.004630401264876127, -0.005676968023180962, -0.033626385033130646, -0.04188770800828934, 0.003808450186625123, 0.027526989579200745, -8.999097190098837e-05, -0.023938730359077454, -0.0032230434007942677, -0.0066841645166277885, 0.022692713886499405, 0.051453132182359695, 0.015436309389770031, 0.013968084007501602, -0.017724692821502686, 0.010561011731624603],
      "k": 30,
      "number_of_candidates" : 45
    }
  }
}
9

Could you do a screenshot? I think what you are getting back is not what you think you sent.

query_vector is not part of:

{"size": 30,
  "query": {
    "knn": {
      "field": "output_dense_vector",
      "vector":[ /* */ ],
      "k": 30,
      "number_of_candidates" : 45
    }
  }
}
10

yea so I tried vector for a few time first , but I got the same error, like :

"message": """[request query.method]: Method must be one of, case insensitive ['HEAD', 'GET', 'POST', 'PUT', 'DELETE', 'PATCH']. Received '"vector"'."""
}

My locally generated embedding is attached above, so if there's any different . I think that could be the only factor, is it that my vector is mal-formatted?

11

Could you please do a screenshot?

12

Yea so my vector is of 1024 dims so it'd be a bit hard to read, but heres the screenshot:

_17248293539001
_172482935390011847×956 337 KB

14

And when I run it, I got :

_17248294819485
_172482948194851915×365 38.8 KB

15

It looks like your query is malformed:

  • You're passing in vector but the documentation states this should be query_vector
  • You're passing in number_of_candidates but it should be num_candidates
  • I'm guessing the error you're getting is from otherwise malformed json in the request.

Here's an example of how to write the knn query using dev tools:

PUT vector-test 
{
  "mappings": {
    "properties": {
      "output_dense_vector": {
        "type": "dense_vector"
      }
    }
  }
}

# Index a few documents to see sizing changes
POST vector-test/_doc
{
  "output_dense_vector": [ 1, 2, 3 ]
}

GET vector-test/_search
{
  "size": 50,
  "query": {
    "knn": {
      "field": "output_dense_vector",
      "query_vector": [ 1, 2, 3],
      "k": 50,
      "num_candidates": 50
    }
  }
}

Note: k and size have to be the size that you want to retrieve. If size is 30 but k is 10, you will only get 10 results. Also num_candidates has to be >= k.

16

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.