App Search API Keys Basic Analytics Info

[alongaks]

Hello,

I have a v8.7 deployment using App Search to crawl websites, indexing content. The site search experience is working nicely. However, I am wondering about the 'Recent queries' stats at the bottom of the page from Kibana > Enterprise Search > App Search > Engines - [meta-enginename] > Analytics.

In this area, I understand in order to get the full analytic functionality like 'click analytics' requires adding in necessary API configs to track the relevant info. Analytics APIs | App Search documentation [8.7] | Elastic

When using the Credentials > API Keys - search-key (type: Public Search Key, Modes: search), I believe the one created with App Search by default, queries shows up in the 'Recent queries' list in the 'Analytics' page of the meta-engine. No custom code is required in the site search box on the webpage for this to work. This functionality is what is desired for the below scenario.

If someone uses a created API key in Credentials > API Keys - name-of-created-key (type: Private API Key, Modes: read-only), these queries are only showing up in the API logs, but not in the 'Recent queries' list.

Is it possible to get 'Recent queries' to show up in the 'Analytics' page for a 'private api key'/read-only mode key?

[joemcelroy (Joseph McElroy)]

Hey!

Is there a particular reason why you need to use a private key instead of a search key? Would help understand the reasoning why you need to use a private key there.

Joe

[alongaks]

Hi, Joe! Appreciate the quick response!

The website search is being provided in two environments - one is a public facing site, the other an 'intranet' site. I believe preference is to not have search queries/resultant data 'leaked' over-the-air, if that makes sense?

I'm not one of the site devs that is using/codifying the search functionality, but I will ask for clarification.

It was something I noticed as of recent and figured it would be ideal to continue to capture that info in some way, even if a private, read-only API key is in use.

[joemcelroy (Joseph McElroy)]

Could be that you are sharing one index between intranet and public facing site, with a signed key to only search a specific set of documents for public consumption? Would be good to hear more about your setup.

Ill find out the differences between search vs signed key in analytics

Joe

[alongaks]

Found out a bit more info about the use of the API keys in App Search.

Apparently there are limitations with the default App Search 'search' API that an Elasticsearch API allows.

Below is an example query that is run using the manually created private-key pointing to Elasticsearch

curl -X POST 'http://URL.com:3002/api/as/v1/engines/meta-engine/elasticsearch/_search' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer search-redacted' \
-d '{"from":0,"size":10,"query":{"bool":{"must":{"multi_match":{"query":"rubbish","fields":[]}},"should":[{"wildcard":{"url":"*.php"}},{"wildcard":{"url":"*.php?*"}},{"wildcard":{"url":"*\/"}}],"minimum_should_match":1}}}'

That comes back with an error...

{"errors":["Authentication method not allowed. Allowed methods: Private API key."]}

But if you run the following query, there are differences in what is available to use in the query, presumably

curl -X POST 'http://URL.com:3002/api/as/v1/engines/meta-engine/search' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer search-redacted' \
-d '{"from":0,"size":10,"query":{"bool":{"must":{"multi_match":{"query":"rubbish","fields":[]}},"should":[{"wildcard":{"url":"*.php"}},{"wildcard":{"url":"*.php?*"}},{"wildcard":{"url":"*\/"}}],"minimum_should_match":1}}}'

... as that returns

{"errors":["Options contains invalid keys: from and size","Query must be a string or number"]}

Can the basic analytics that the default App Search 'public search' API pulls in be obtained when querying against meta-engine/elasticsearch/_search?

[joemcelroy (Joseph McElroy)]

Right! This makes sense why you need to use the private-key. _search endpoint requires the admin key to use and is designed not to be used directly (on the browser) but integrated with your API server.

You have probably discovered this already but the app-search search API and elasticsearch/_search endpoints are very different.

CORRECTION: You can emit analytics events via the passthrough _search endpoint using the analytics headers Elasticsearch search API for App Search | App Search documentation [8.7] | Elastic

Joe

[alongaks]

Thanks for the responses, Joe!

I noticed this in documentation, possibly for Elasticsearch API: Elasticsearch search API for App Search | App Search documentation [8.7] | Elastic

It mentions using the 'X-Enterprise-Search-Analytics' header in the API requests. So I tried it a couple ways, neither of them error out, but one has different analytics results and the other appears to have no analytics result logged.

This one has a result logged into analytics

curl -X POST 'http://URL.com:3002/api/as/v1/engines/meta-engine/elasticsearch/_search' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer private-redacted' \
-H 'X-Enterprise-Search-Analytics: true' \
-d '{"from":0,"size":10,"query":{"bool":{"must":{"multi_match":{"query":"rubbish","fields":[]}},"should":[{"wildcard":{"url":"*.php"}},{"wildcard":{"url":"*.php?*"}},{"wildcard":{"url":"*\/"}}],"minimum_should_match":1}}}'

But it logs the word 'true' not the actual search query being run.

If I do it as just -H 'X-Enterprise-Search-Analytics' it doesn't log anything. I'm wondering if there is a different syntax needed to enable the tracking since it seems to work somewhat without errors, using the private key?

[joemcelroy (Joseph McElroy)]

Oh you're right! I completely forgot about that addition!

So the value needs to be the query text. In your example it would need to be

-H 'X-Enterprise-Search-Analytics: rubbish' \

We cant unfortunately take the query value you added as with _search, you can specify one or more match conditions with different query text. This header allows you to specify the query you want to appear in analytics.

You can also add tags to the analytics query via another header with

-H 'X-Enterprise-Search-Analytics-Tags: tag1,tag2' \

Hope this helps and thanks for flagging.

Joe

[alongaks]

Much appreciated!