Trying to extract Lucene from Kibana

[Sandra_Schlichting]

Dear all =)

When I write KQL in the search bar, while having Google Chrome debugging open, I see a request to bsearch is made with a cryptic response such as

eJxEjl1rgzAUhv9Lrr2oijdCb8Za2MB0pnGFj ...  ==

I suppose the translated Lucene is in there? Can anyone explain how I can decode that?

Hugs,
Sandra =)

[stephenb (Stephen Brown)]

Hi @Sandra_Schlichting

Which App are you using Discover? If So you can just use the Inspect to see the actual request and response.

Discover -> Inspect -> Request

That GUID you show above is not an encoded request, I am pretty sure that GUID of the search request so it can the search can run asynchronously.

If you still really want to look at the request, it would be under the request payload not the headers. And that will end up showing you pretty much the same thing as inspect but it is just broken up a bit different.... the request under Discover -> Inspect is the equivalent DSL query

[Sandra_Schlichting]

Dear @stephenb

That is excellent! Thanks! As you probably have guessed I am looking for am automated way to convert KQL to Lucene =)

Ideally I would prefer this, and second an official API to convert KQL to Lucene, but I haven't been able to find such. Do you know if there is a feature request to implement an API for this?

I have never tried to crawl a React app (Kibana) before, so that should be interesting if nothing else =)

Hugs,
Sandra =)

[Sandra_Schlichting]

Hi @stephenb

I just noticed something interesting.

If I disable network in Chrome for the Kibana Discover page, and then enter a KQL string in the search bar, press enter and look that (failed) request, the KQL have already been converted to Lucene. That must mean the conversion is done client side.

My current thought is to write a crawler, would this make it easier?

Or can you give me some info what would make it easier? =)

Hugs,
Sandra =)

[stephenb (Stephen Brown)]

@Sandra_Schlichting

and second an official API to convert KQL to Lucene, but I haven't been able to find such.

No I do not see that in the repo perhaps you should open a Feature Request against Elasticsearch.

With respect to where the actual code in Kibana that does the KQL - DSL conversion ... I do not know.. I asked not sure if / when I will get an answer....

Update Looks like it is here : From Engineering
"It depends a bit on the actual place whether we compile it in the browser or in the Kibana server, but it's def Kibana side. KQL is nothing ES understands."

[Sandra_Schlichting]

Ok, so that is very interesting that the same code also is run server side in some cases.

My experience with crawling is that it is often easier to parse in parameters (KQL in this case) to something where the response comes back from the server, and not the client.

Can you ask Engineering, if they can think of a place in Kibana, where the KQL to Lucene conversion is done server side? And therefore perhaps easier to crawl? =)