I am testing elasticsearch since a while and all the sudden today, I am
getting a weird error without having changed anything.
My PHP script indexes ~ 180k documents, and currently stops reliably after
about 40k - 50k documents with the error message mentioned in the subject.
There is nothing in the elasticsearch.log.
On the PHP side I am using curl to connect to the service. I do one request
per document, so it works fine for the first 40k or so docs.
This error does not happen on the same document, but very reliably after
40k - 50k docs, after I deleted and re-created the index and even restartet
the ES service.
Previously I have used "localhost:9200" as the address and changed it to
127.0.0.1, which did not fix the problem.
I am using the ES version 0.19.8 on Debian (stock Debian install with ICU
plugin).
I am testing elasticsearch since a while and all the sudden today, I am
getting a weird error without having changed anything.
My PHP script indexes ~ 180k documents, and currently stops reliably after
about 40k - 50k documents with the error message mentioned in the subject.
There is nothing in the elasticsearch.log.
On the PHP side I am using curl to connect to the service. I do one request
per document, so it works fine for the first 40k or so docs.
This error does not happen on the same document, but very reliably after
40k - 50k docs, after I deleted and re-created the index and even restartet
the ES service.
Can you give us an example of the error you're seeing?
On Friday, October 12, 2012 12:21:21 AM UTC+2, Drew Raines wrote:
Martin Rode wrote:
I am testing elasticsearch since a while and all the sudden today, I am
getting a weird error without having changed anything.
My PHP script indexes ~ 180k documents, and currently stops reliably
after
about 40k - 50k documents with the error message mentioned in the
subject.
There is nothing in the elasticsearch.log.
On the PHP side I am using curl to connect to the service. I do one
request
per document, so it works fine for the first 40k or so docs.
This error does not happen on the same document, but very reliably after
40k - 50k docs, after I deleted and re-created the index and even
restartet
the ES service.
Can you give us an example of the error you're seeing?
On Friday, October 12, 2012 12:21:21 AM UTC+2, Drew Raines wrote:
Martin Rode wrote:
I am testing elasticsearch since a while and all the sudden today, I am
getting a weird error without having changed anything.
My PHP script indexes ~ 180k documents, and currently stops reliably
after
about 40k - 50k documents with the error message mentioned in the
subject.
There is nothing in the elasticsearch.log.
On the PHP side I am using curl to connect to the service. I do one
request
per document, so it works fine for the first 40k or so docs.
This error does not happen on the same document, but very reliably
after
40k - 50k docs, after I deleted and re-created the index and even
restartet
the ES service.
Can you give us an example of the error you're seeing?
Is the ES JVM dying somehow? This suggests that it's not listening
anymore. And if so, you really don't see anything in the log? Try
setting rootLogger to DEBUG in config/logging.yml to see if anything
appears in the log then.
By chance does your server have selinux and if so did it get changed to
enabled?
On Thursday, October 11, 2012 11:43:32 AM UTC-4, Martin Rode wrote:
Hi everybody,
I am testing elasticsearch since a while and all the sudden today, I am
getting a weird error without having changed anything.
My PHP script indexes ~ 180k documents, and currently
stops reliably after about 40k - 50k documents with the error message
mentioned in the subject.
There is nothing in the elasticsearch.log.
On the PHP side I am using curl to connect to the service. I do one
request per document, so it works fine for the first 40k or so docs.
This error does not happen on the same document, but very reliably after
40k - 50k docs, after I deleted and re-created the index and even restartet
the ES service.
Previously I have used "localhost:9200" as the address and changed it to
127.0.0.1, which did not fix the problem.
I am using the ES version 0.19.8 on Debian (stock Debian install with ICU
plugin).
You did not set the SO_REUSEADDR flag, so after "40k -50k documents", you
have forced your system to ran out of sockets. See
also PHP: socket_set_option - Manual
Jörg
On Thursday, October 11, 2012 5:43:32 PM UTC+2, Martin Rode wrote:
Hi everybody,
I am testing elasticsearch since a while and all the sudden today, I am
getting a weird error without having changed anything.
My PHP script indexes ~ 180k documents, and currently
stops reliably after about 40k - 50k documents with the error message
mentioned in the subject.
There is nothing in the elasticsearch.log.
On the PHP side I am using curl to connect to the service. I do one
request per document, so it works fine for the first 40k or so docs.
This error does not happen on the same document, but very reliably after
40k - 50k docs, after I deleted and re-created the index and even restartet
the ES service.
Previously I have used "localhost:9200" as the address and changed it to
127.0.0.1, which did not fix the problem.
I am using the ES version 0.19.8 on Debian (stock Debian install with ICU
plugin).
I was encountering the same problem using cURL in PHP. I was opening and closing a cURL connection with each request to Elasticsearch, and was making hundreds of thousands of requests from the same PHP process.
$this->Curl = curl_init();
// make requests
curl_close( $this->Curl );
When I rewrote my class to use the same cURL connection instance for every request, this problem went away. ( eg. I never call curl_close() in my script, and use $this->Curl for every request ).
I do not know what is limiting the number of cURL connections I'm able to open and close in a single PHP thread. Perhaps someone else can shine some light on this.
As said, that is very basic to socket programming. Operation systems
keep socket addresses reserved after being closed for some minutes, in
order to be prepared for reuse at reopening (TCP state CLOSE_WAIT).
Therefore, you should use persistent connections (HTTP keepalive or
SO_REUSEADDR), otherwise you will quickly consume all the socket
addresses after some thousands request open/close cycles. Note that
socket addresses are usually 16 bit wide, and for user space, you can
go up to around a number of 40-50k sockets totally.
Jörg
Am 06.02.13 09:15, schrieb tbrianjones:
I do not know what is limiting the number of cURL connections I'm able to
open and close in a single PHP thread. Perhaps someone else can shine some
light on this.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
