Sending query via classic asp works one way but not another

[bilpor (william crawley)]

HI All,

If I send a simple query to Elastic using the following code construct, in the response back I obtain my expected results:

webaddress = "http://localhost:9200/" + index + "/_search?q='hello world'"

set xml = server.Createobject("MSXML2.ServerXMLHTTP")

	xml.Open "GET", webaddress ,False

	xml.setRequestHeader "Authorization", "Basic " & Base64Encode("elastic:elastic123")

	xml.Send

Now if I want to send a much more complex query such that It's not practical or possibly even possible to place the query on a single 'q' parameter as above. i.e.

the query:

get myindex/_search
{
    "query":{"match": {"headline": "overseas territory"},
             "match" : {"bodytext": "british territory"}
            }
}

So I would build this up into a variable. lets call it mySearch

Then my webaddress becomes :

webaddress = "http://localhost:9200/" + index + "/_search"

and xml.send becomes :

xml.Send mySearch

mySearch gets ignored and I receive back a default first 10 records from the variable webaddress.

How should I pass my complex search so that it is recognised. I dont have the luxury of curl

[forloop (Russ Cam)]

You will need to use "POST" in order to be able to send a JSON body to the _search endpoint; as far as I am aware, MSXML2.ServerXMLHTTP cannot send a request body with "GET". I'd also recommend setting

xml.setRequestHeader "Content-Type", "application/json"

If you need to interact with Elasticsearch from Classic ASP more than this, you might be able to save yourself some considerable time by creating a COM Callable Wrapper and using COM interop with the .NET client to interact with Elasticsearch.

[bilpor (william crawley)]

HI changing to POST and also changing my index such that all the properties were declared as lowercase has given me better results, however on return, my response object only has the first 10 records being returned. When I'm expecting more. Via Kibana the 10 returned match the first 10 in the Kibana results window.

[forloop (Russ Cam)]

A search request will return only the top 10 scored documents by default. If you want to return more then you can use "size" on the request, and use it in conjunction with "from" for pagination.

Bear in mind that this is not an efficient way to deeply paginate a large dataset; for this, you may consider using the scroll API or search after API.

[bilpor (william crawley)]

I have had some success using the scroll API. However the issue I now have is that there is nothing being returned that I can test when all data has been returned. I thought I could check the txt.hits.hits.length Property, but on the last page in the Kibana output window it shows as [] but back in classic asp it just says 'Empty' but it's not in an 'Empty' state that classic asp would recognise and it doesn't return the literal word 'Empty'. Calling the scoll API continues to send back a valid scroll_id, took still continues to have a number greater than 0, even though nothing is returned, I've basically got nothing to interpret to tell me when I've reached the end.

[forloop (Russ Cam)]

However the issue I now have is that there is nothing being returned that I can test when all data has been returned.

When no more documents are being returned, you've reached the end. You can calculate how many calls this will be ahead of time from the first call, by using the total in the response of the first response in conjunction with the size parameter that's used.

[bilpor (william crawley)]

Ok, thank you....This seems to work directly via Kibana doing the Math. So I'll fo and try my Classic asp now, though it makes my script a bit more messy :).