Elastic 5.0 rejects "non-caching" underscore (_) parameter in query string of Ajax request URL

[GrahamHannington]

I've developed a tiny web UI for the Elasticsearch HTTP REST API that displays a table of indices, and then lets you delete indices. I find it useful for testing. I call it "Lackygun".

I originally wrote Lackygun to work with Elasticsearch 1.x. I've just dusted it off for Elastic 5.0, and... it didn't work.

Elasticsearch now rejects the Ajax request with error 400 (Bad Request) for the following reason:

request [/stats/docs,store/] contains unrecognized parameter: []

Sure enough, here's the request URL:

http://localhost:9200/_stats/docs,store/?_=1479891338183

I didn't explicitly add that underscore (_) parameter. jQuery did.

I'm using that request with the DataTables (1.10.8) jQuery plug-in.

To suppress that parameter, I added the following property to the DataTable ajax property:

"cache": true

That fixed it.

Why am I bothering reporting this?

Two reasons:

• To ask the Elasticsearch developers: did you specifically decide to reject URLs with an underscore parameter, or did you simply make a blanket decision to reject all parameters not defined by the REST API?
• In case other users of the REST API hit the same issue.

I understand that this might be very old news indeed, but I couldn't find any related information via Google (specific to the Elasticsearch API rejecting this _ query string parameter).

[mainec (Isabel Drost-Fromm)]

Citing

https://www.elastic.co/guide/en/elasticsearch/reference/current/breaking_50_rest_api_changes.html#_strict_rest_query_string_parameter_parsing

Strict REST query string parameter parsingedit
Previous versions of Elasticsearch ignored unrecognized URL query string parameters. This means that extraneous parameters or parameters containing typographical errors would be silently accepted by Elasticsearch. This is dangerous from an end-user perspective because it means a submitted request will silently execute not as intended. This leniency has been removed and Elasticsearch will now fail any request that contains unrecognized query string parameters.

[nik9000 (Nik Everett)]

If the _ parameter is popular enough we might want to throw it away without complaining. I don't think it is a good idea but it is worth thinking about....

I guess is that this hasn't come in before is because this strictness is a 5.0 feature and because most folks have an application between the browser and Elasticsearch. Certainly in production you have to because exposing Elasticsearch to the internet isn't a good thing. Your project makes sense though because it is for administration.

If the _ parameter is for cache busting then maybe you don't need it with this application? Or you could use a header if you wanted to have something. We kind of assumed that that is how you'd control caches in the data center. I expect jQuery doesn't do it because the internet is a scary place where poor http headers get eaten all the time.....

[GrahamHannington]

@nik9000

we might want to throw it away without complaining

Entirely your call. I guess I felt I should at least sensitize you to that _ parameter.

I'm happy Lackygun now works with 5.0. Created a test index with the wrong combination of fields? Forget cURL; just click and kapow! There are many Elastic admin web UIs like it, but this one is mine.

Yes, this is on an intranet behind a firewall, and that is the most likely use case I'm looking at: I write the user documentation for IBM Transaction Analysis Workbench for z/OS. (I realize the not_analyzed index template I describe there will need to be updated to use the keyword type for 5.0.)

[GrahamHannington]

Thanks for the link. For what it's worth, while it broke my code *, I agree with this change.

* You'd think that might've prompted me to read those "Breaking changes" docs, right? That did occur to me, but there have been several Elasticsearch releases since I last used Lackygun. I baulked at trawling through the breaking changes for them all, and my attempts in Google Search didn't return any relevant results. (I need to work on my Google-fu.) All poor excuses. Because I've just gone to the Elastic docs home page, clicked Search, entered "rest api query parameter", and the link you cited was the first result. Bottom line? Your docs and search are working well; I'm embarrassed (not for the first time); next time, I'll use your search. Thanks again.

[mainec (Isabel Drost-Fromm)]

No worries - and no need to be embarrassed. Glad the docs I dug out actually were helpful for you (Also: It's always easier to find something when you know that a) it should exist and b) you roughly know what exactly it is you are looking for)