ElasticSearch - search statistic - like google analytics


(Jacob Dalgaard) #1

Hello
I am looking into using ElasticSearch as a search engine for one of the
projects I am working on. There is still one thing which I need to find an
answer for, and I hope someone inhere can help.
The customer want to be able to see some search statistic, like google
analytics. Most searched words, new search words and so on.

Is there a way to easily setup this type of search statistic?
My idea is something like ElasticSearch stores search history, about the
search request made to the REST API. Then my customer can use Kibana or
some other visual tool to monitor the search history of ElasticSearch.

Another approch could be to set up LogStash to pick up all log entries to
the IIS on search requests, and put em in ElasticSearch. Then they could be
viewed with Kibana. Is anyone aware of a logstash pattern for IIS?

Hope someone can help me with an answer for this.

Regards Jacob

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5dbbf0b8-739a-4201-8500-6dd8efdccb42%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Mark Walkom) #2

ES doesn't store this natively, you'd have to put something in-between the
user and ES to capture and collate this information.

Your LS idea seems like a good one to solve it.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 17 June 2014 20:41, Jacob Dalgaard jacob.dalgaard@gmail.com wrote:

Hello
I am looking into using ElasticSearch as a search engine for one of the
projects I am working on. There is still one thing which I need to find an
answer for, and I hope someone inhere can help.
The customer want to be able to see some search statistic, like google
analytics. Most searched words, new search words and so on.

Is there a way to easily setup this type of search statistic?
My idea is something like ElasticSearch stores search history, about the
search request made to the REST API. Then my customer can use Kibana or
some other visual tool to monitor the search history of ElasticSearch.

Another approch could be to set up LogStash to pick up all log entries to
the IIS on search requests, and put em in ElasticSearch. Then they could be
viewed with Kibana. Is anyone aware of a logstash pattern for IIS?

Hope someone can help me with an answer for this.

Regards Jacob

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/5dbbf0b8-739a-4201-8500-6dd8efdccb42%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/5dbbf0b8-739a-4201-8500-6dd8efdccb42%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624ZzoOu9t8gSu%2B9YuTDBN%2BL%3DTgaBJAgH0xJWeYr-9fc%2Bgg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Jacob Dalgaard) #3

Hello Mark
Thank you for your reply. Then I will look into this approch.

Regards.
Jacob

Den tirsdag den 17. juni 2014 14.06.15 UTC+2 skrev Mark Walkom:

ES doesn't store this natively, you'd have to put something in-between the
user and ES to capture and collate this information.

Your LS idea seems like a good one to solve it.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: ma...@campaignmonitor.com <javascript:>
web: www.campaignmonitor.com

On 17 June 2014 20:41, Jacob Dalgaard <jacob.d...@gmail.com <javascript:>>
wrote:

Hello
I am looking into using ElasticSearch as a search engine for one of the
projects I am working on. There is still one thing which I need to find an
answer for, and I hope someone inhere can help.
The customer want to be able to see some search statistic, like google
analytics. Most searched words, new search words and so on.

Is there a way to easily setup this type of search statistic?
My idea is something like ElasticSearch stores search history, about the
search request made to the REST API. Then my customer can use Kibana or
some other visual tool to monitor the search history of ElasticSearch.

Another approch could be to set up LogStash to pick up all log entries to
the IIS on search requests, and put em in ElasticSearch. Then they could be
viewed with Kibana. Is anyone aware of a logstash pattern for IIS?

Hope someone can help me with an answer for this.

Regards Jacob

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/5dbbf0b8-739a-4201-8500-6dd8efdccb42%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/5dbbf0b8-739a-4201-8500-6dd8efdccb42%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8ee2eb2f-ce2c-4eac-9c89-022c08041893%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Jorge Luis Betancourt Gonzalez) #4

Some time ago I did something similar. But in our case we don't use IIS, and it was a bit more complex because we aggregated logs from different platforms. But it's doable. The must recent experience was feeding Solr logs from a legacy application, and application logs into ELK for analytics purposes.

Greetings,

On Jun 17, 2014, at 9:51 AM, Jacob Dalgaard jacob.dalgaard@gmail.com wrote:

Hello Mark
Thank you for your reply. Then I will look into this approch.

Regards.
Jacob

Den tirsdag den 17. juni 2014 14.06.15 UTC+2 skrev Mark Walkom:
ES doesn't store this natively, you'd have to put something in-between the user and ES to capture and collate this information.

Your LS idea seems like a good one to solve it.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: ma...@campaignmonitor.com
web: www.campaignmonitor.com

On 17 June 2014 20:41, Jacob Dalgaard jacob.d...@gmail.com wrote:
Hello
I am looking into using ElasticSearch as a search engine for one of the projects I am working on. There is still one thing which I need to find an answer for, and I hope someone inhere can help.
The customer want to be able to see some search statistic, like google analytics. Most searched words, new search words and so on.

Is there a way to easily setup this type of search statistic?
My idea is something like ElasticSearch stores search history, about the search request made to the REST API. Then my customer can use Kibana or some other visual tool to monitor the search history of ElasticSearch.

Another approch could be to set up LogStash to pick up all log entries to the IIS on search requests, and put em in ElasticSearch. Then they could be viewed with Kibana. Is anyone aware of a logstash pattern for IIS?

Hope someone can help me with an answer for this.

Regards Jacob

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5dbbf0b8-739a-4201-8500-6dd8efdccb42%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8ee2eb2f-ce2c-4eac-9c89-022c08041893%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

VII Escuela Internacional de Verano en la UCI del 30 de junio al 11 de julio de 2014. Ver www.uci.cu

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/DFBC9BA1-B1EA-4740-8806-5E1EFFBC6CDE%40uci.cu.
For more options, visit https://groups.google.com/d/optout.


(system) #5