Visualise/report search queries from elastic

Elastic Stack Kibana
1

Hi all,

Recently I have implemented a simple search functionality which runs on the ES. The set up and search speed exceeds expectations. I have also installed Kibana as I would like to see/report what people are actually searching. After exploring different sections in Kibana I am getting a little bit puzzled. I am now not sure if such data required for this type of reporting is available within ES and kibana can use it. I am not underestimating the whole project but trying to get a bit better understanding on would is needed to be able to report what users are searching.

I appreciate any assistance.

Regards,
Tom.

2

So you have an application using search in Elasticsearch and you'd like to record metrics about how users are searching. You're going to need a way to get usage metrics out of Elasticsearch.

Unfortunately, I don't think this is something Elasticsearch provides out of the box, so you're going to have to put something of your own in place to get that information. That is, add some kind of service layer in front of Elasticsearch that can capture and record requests and send them back into Elasticsearch (or whatever tool you want to use to explore that data). The easiest way to do this, assuming you're using the REST API, is probably to add a proxy (nginx is a fine choice). That'll give you a way to capture requests, and then you can use something like filebeat to simply index the log files from proxy.

3

Hi Joe,

Thanks a lot for coming back to me. That is kind of what I was suspecting that the Elasticsearch may not necessarily have such information within itself. My implementation was done in Magento where one endpoint capable to "talk" to Elastic Search is being queried through AJAX from the frontend. Your suggestion will be probably as you stated the easiest to implement as I can query proxy url and then from proxy pass the request to the already mentioned endpoint. I may need to implement a bit of logic to remove "noises" like search terms "c", "ch", "cha", "chai" and leave "chair" only. For the first 2 letters I will probably won't search for at all and for the remaining one I will have something to think about over the weekend :slight_smile:

Thank you again, appreciated.

4

Have a look at Packetbeat, it can watch the Elasticsearch HTTP port and report what is being asked.

1 Like
5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.