Stats about response times


(Florian Munz) #1

Hello,

is there any way to get the response time for a query in the logfile
or an aggregate for a specific time frame? I'd like to graph the
response times we're seeing from elasticsearch.

Cheers,
Florian


(Karussell) #2

You could enable slow query into log feature:

But you can also index this into elasticsearch itself (or use
logstash), then you could search, facet them etc ...

Peter.

On 9 Dez., 15:47, Florian Munz m...@podio.com wrote:

Hello,

is there any way to get the response time for a query in the logfile
or an aggregate for a specific time frame? I'd like to graph the
response times we're seeing from elasticsearch.

Cheers,
Florian


(dungtc) #3

Hello
I have use Elastic search (machine 1) to index 50000 documents in BigCouch (cluster version of CouchDB 1.1.0) on the machine 2. At this moment, cluster contain only one node.
I use couchdb river to do it in the same way with couchdb, because the bigcouch API is the same as couchdb API . All is successful after typing curl. An index has been created and I can search on them. But When I restart Elastic Search, it cann't be retrieve changes from BigCouch any more (chang stream donot work). Here is the error message:
[2011-12-09 16:09:08,643][WARN ][river.couchdb ] [NFL Superpro] [couchdb][couchdb_indexation_test_river_1] failed to read from _changes, throttling....
java.net.SocketException: Unexpected end of file from server
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:770)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:633)
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:767)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:633)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1162)
at org.elasticsearch.river.couchdb.CouchdbRiver$Slurper.run(CouchdbRiver.java:429)
at java.lang.Thread.run(Thread.java:636)

In the machine 2 of BigCouch, I see:
[error] [<0.1515.0>] [04d73d10] Uncaught error in HTTP request: {exit,normal}
[info] [<0.1515.0>] [04d73d10] Stacktrace: [{mochiweb_request,send,2},
{chttpd,send_chunk,2},
{chttpd_db,changes_callback,2},
{fabric_view_changes,
keep_sending_changes,6},
{fabric_view_changes,go,5},
{chttpd,handle_request,1},
{mochiweb_http,headers,5},
{proc_lib,init_p_do_apply,3}]
Please explain it to me and how to solve it? For Couchdb (not BigCouch), all works well but this problem happen with BigCouch and Elastic Search 0.18.4
Thanks a lot
TCD


(David Pilato) #4

It's a big couch problem. Not an ES problem.
It could be relative to the way Bigcouch modify sequence numbers in Strings.

Let me check that.

David :wink:
@dadoonet

Le 9 déc. 2011 à 20:50, Chi Dung Tran dungtctin4@yahoo.com a écrit :

Hello
I have use Elastic search (machine 1) to index 50000 documents in BigCouch (cluster version of CouchDB 1.1.0) on the machine 2. At this moment, cluster contain only one node.
I use couchdb river to do it in the same way with couchdb, because the bigcouch API is the same as couchdb API . All is successful after typing curl. An index has been created and I can search on them. But When I restart Elastic Search, it cann't be retrieve changes from BigCouch any more (chang stream donot work). Here is the error message:
[2011-12-09 16:09:08,643][WARN ][river.couchdb ] [NFL Superpro] [couchdb][couchdb_indexation_test_river_1] failed to read from _changes, throttling....
java.net.SocketException: Unexpected end of file from server
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:770)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:633)
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:767)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:633)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1162)
at org.elasticsearch.river.couchdb.CouchdbRiver$Slurper.run(CouchdbRiver.java:429)
at java.lang.Thread.run(Thread.java:636)

In the machine 2 of BigCouch, I see:
[error] [<0.1515.0>] [04d73d10] Uncaught error in HTTP request: {exit,normal}
[info] [<0.1515.0>] [04d73d10] Stacktrace: [{mochiweb_request,send,2},
{chttpd,send_chunk,2},
{chttpd_db,changes_callback,2},
{fabric_view_changes,
keep_sending_changes,6},
{fabric_view_changes,go,5},
{chttpd,handle_request,1},
{mochiweb_http,headers,5},
{proc_lib,init_p_do_apply,3}]
Please explain it to me and how to solve it? For Couchdb (not BigCouch), all works well but this problem happen with BigCouch and Elastic Search 0.18.4
Thanks a lot
TCD


(David Pilato) #5

It has been fixed 3 weeks ago in 0.18 and now in the external plugin
(https://github.com/elasticsearch/elasticsearch-river-couchdb).

See also here : https://github.com/elasticsearch/elasticsearch/issues/1478

Which version of ES are you using and which version of couchDb plugin are you
using ?

David.

Le 9 décembre 2011 à 21:10, David Pilato david@pilato.fr a écrit :

It's a big couch problem. Not an ES problem.
It could be relative to the way Bigcouch modify sequence numbers in Strings.

Let me check that.

David :wink:
@dadoonet

Le 9 déc. 2011 à 20:50, Chi Dung Tran dungtctin4@yahoo.com a écrit :

Hello
I have use Elastic search (machine 1) to index 50000 documents in BigCouch
(cluster version of CouchDB 1.1.0) on the machine 2. At this moment, cluster
contain only one node.
I use couchdb river to do it in the same way with couchdb, because the
bigcouch API is the same as couchdb API . All is successful after typing
curl. An index has been created and I can search on them. But When I restart
Elastic Search, it cann't be retrieve changes from BigCouch any more (chang
stream donot work). Here is the error message:
[2011-12-09 16:09:08,643][WARN ][river.couchdb ] [NFL Superpro]
[couchdb][couchdb_indexation_test_river_1] failed to read from _changes,
throttling....
java.net.SocketException: Unexpected end of file from server
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:770)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:633)
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:767)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:633)
at
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1162)
at
org.elasticsearch.river.couchdb.CouchdbRiver$Slurper.run(CouchdbRiver.java:429)
at java.lang.Thread.run(Thread.java:636)

In the machine 2 of BigCouch, I see:
[error] [<0.1515.0>] [04d73d10] Uncaught error in HTTP request:
{exit,normal}
[info] [<0.1515.0>] [04d73d10] Stacktrace: [{mochiweb_request,send,2},
{chttpd,send_chunk,2},
{chttpd_db,changes_callback,2},
{fabric_view_changes,
keep_sending_changes,6},
{fabric_view_changes,go,5},
{chttpd,handle_request,1},
{mochiweb_http,headers,5},
{proc_lib,init_p_do_apply,3}]
Please explain it to me and how to solve it? For Couchdb (not BigCouch), all
works well but this problem happen with BigCouch and Elastic Search 0.18.4
Thanks a lot
TCD
--
David Pilato
http://dev.david.pilato.fr/
Twitter : @dadoonet


(Shay Banon) #6

Also, check node stats and index stats APIs, which include search/index/get
stats.

On Fri, Dec 9, 2011 at 9:44 PM, Karussell tableyourtime@googlemail.comwrote:

You could enable slow query into log feature:

https://github.com/elasticsearch/elasticsearch/issues/1462

But you can also index this into elasticsearch itself (or use
logstash), then you could search, facet them etc ...

Peter.

On 9 Dez., 15:47, Florian Munz m...@podio.com wrote:

Hello,

is there any way to get the response time for a query in the logfile
or an aggregate for a specific time frame? I'd like to graph the
response times we're seeing from elasticsearch.

Cheers,
Florian


(dungtc) #7

I'm using Elastic Search 0.18.4 and 0.17.6
This problem happen with both. BigCOuch is 0.4.a (cluster of CouchDB 1.1.0)
I'll try new river
Thanks


Ngu ngoc lon nhat cua doi nguoi la yeu duong - dieu ran thu 15 cua duc Phat (tai sao khong)

TINH YEU KHONG TU NHIEN SINH RA, KHONG TU NHIEN MAT DI, NO CHI CHUYEN TU NGUOI NAY SANG NGUOI KHAC

     NGAY MAI EM DI

BIEN NAM CHAN EM GIAT VE


De : "david@pilato.fr" david@pilato.fr
À : elasticsearch@googlegroups.com
Envoyé le : Vendredi 9 Décembre 2011 21h38
Objet : Re: Problem with river BigCouch

It has been fixed 3 weeks ago in 0.18 and now in the external plugin ( https://github.com/elasticsearch/elasticsearch-river-couchdb ).

See also here : https://github.com/elasticsearch/elasticsearch/issues/1478

Which version of ES are you using and which version of couchDb plugin are you using ?

David.

Le 9 décembre 2011 à 21:10, David Pilato david@pilato.fr a écrit : > It's a big couch problem. Not an ES problem. > It could be relative to the way Bigcouch modify sequence numbers in Strings. > > Let me check that. > > David :wink: > @dadoonet > > > Le 9 déc. 2011 à 20:50, Chi Dung Tran dungtctin4@yahoo.com a écrit : > > > Hello > > I have use Elastic search (machine 1) to index 50000 documents in BigCouch (cluster version of CouchDB 1.1.0) on the machine 2. At this moment, cluster contain only one node. > > I use couchdb river to do it in the same way with couchdb, because the bigcouch API is the same as couchdb API . All is successful after typing curl. An index has been created and I can search on them. But When I restart Elastic Search, it cann't be retrieve changes from BigCouch any more (chang stream donot work). Here is the error message: > > [2011-12-09 16:09:08,643][WARN ][river.couchdb ] [NFL Superpro]
[couchdb][couchdb_indexation_test_river_1] failed to read from _changes, throttling.... > > java.net.SocketException: Unexpected end of file from server > > at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:770) > > at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:633) > > at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:767) > > at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:633) > > at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1162) > > at org.elasticsearch.river.couchdb.CouchdbRiver$Slurper.run(CouchdbRiver.java:429) > > at java.lang.Thread.run(Thread.java:636) > > > > In the machine 2 of BigCouch, I see: > > [error] [<0.1515.0>] [04d73d10] Uncaught error in HTTP request: {exit,normal} > > [info] [<0.1515.0>] [04d73d10] Stacktrace: [{mochiweb_request,send,2}, > >
{chttpd,send_chunk,2}, > > {chttpd_db,changes_callback,2}, > > {fabric_view_changes, > > keep_sending_changes,6}, > > {fabric_view_changes,go,5}, > > {chttpd,handle_request,1}, > > {mochiweb_http,headers,5}, > > {proc_lib,init_p_do_apply,3}] > > Please explain it to me and how to solve it? For Couchdb (not BigCouch), all works well but this problem happen with BigCouch and Elastic Search 0.18.4 > > Thanks a lot > > TCD

--
David Pilato
http://dev.david.pilato.fr/
Twitter : @dadoonet


(dungtc) #8

I'm using ES 0.18.4 with plugin couchdb river 0.18.4, not 1.1.0 on your link

But I think my problem is different, but I'll try to use ES 0.18.4 with this new plugin couchdb 1.1.0
Thanks


Ngu ngoc lon nhat cua doi nguoi la yeu duong - dieu ran thu 15 cua duc Phat (tai sao khong)

TINH YEU KHONG TU NHIEN SINH RA, KHONG TU NHIEN MAT DI, NO CHI CHUYEN TU NGUOI NAY SANG NGUOI KHAC

     NGAY MAI EM DI

BIEN NAM CHAN EM GIAT VE


De : "david@pilato.fr" david@pilato.fr
À : elasticsearch@googlegroups.com
Envoyé le : Vendredi 9 Décembre 2011 21h38
Objet : Re: Problem with river BigCouch

It has been fixed 3 weeks ago in 0.18 and now in the external plugin ( https://github.com/elasticsearch/elasticsearch-river-couchdb ).

See also here : https://github.com/elasticsearch/elasticsearch/issues/1478

Which version of ES are you using and which version of couchDb plugin are you using ?

David.

Le 9 décembre 2011 à 21:10, David Pilato david@pilato.fr a écrit : > It's a big couch problem. Not an ES problem. > It could be relative to the way Bigcouch modify sequence numbers in Strings. > > Let me check that. > > David :wink: > @dadoonet > > > Le 9 déc. 2011 à 20:50, Chi Dung Tran dungtctin4@yahoo.com a écrit : > > > Hello > > I have use Elastic search (machine 1) to index 50000 documents in BigCouch (cluster version of CouchDB 1.1.0) on the machine 2. At this moment, cluster contain only one node. > > I use couchdb river to do it in the same way with couchdb, because the bigcouch API is the same as couchdb API . All is successful after typing curl. An index has been created and I can search on them. But When I restart Elastic Search, it cann't be retrieve changes from BigCouch any more (chang stream donot work). Here is the error message: > > [2011-12-09 16:09:08,643][WARN ][river.couchdb ] [NFL Superpro]
[couchdb][couchdb_indexation_test_river_1] failed to read from _changes, throttling.... > > java.net.SocketException: Unexpected end of file from server > > at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:770) > > at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:633) > > at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:767) > > at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:633) > > at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1162) > > at org.elasticsearch.river.couchdb.CouchdbRiver$Slurper.run(CouchdbRiver.java:429) > > at java.lang.Thread.run(Thread.java:636) > > > > In the machine 2 of BigCouch, I see: > > [error] [<0.1515.0>] [04d73d10] Uncaught error in HTTP request: {exit,normal} > > [info] [<0.1515.0>] [04d73d10] Stacktrace: [{mochiweb_request,send,2}, > >
{chttpd,send_chunk,2}, > > {chttpd_db,changes_callback,2}, > > {fabric_view_changes, > > keep_sending_changes,6}, > > {fabric_view_changes,go,5}, > > {chttpd,handle_request,1}, > > {mochiweb_http,headers,5}, > > {proc_lib,init_p_do_apply,3}] > > Please explain it to me and how to solve it? For Couchdb (not BigCouch), all works well but this problem happen with BigCouch and Elastic Search 0.18.4 > > Thanks a lot > > TCD

--
David Pilato
http://dev.david.pilato.fr/
Twitter : @dadoonet


(system) #9