Hi -- I'm using the HTTP API of ElasticSearch with a Ruby library called
Typhoeus, and I'm trying to track down the source of a pretty common error
I'm seeing.
The error is this: NULL pointer given.
The error, from all the stack traces, is definitely somewhere between
Typhoeus and ElasticSearch, so I thought I'd post here to see if it rings
any bells with anyone.
I currently have ES running across 3 nodes with 40 shards, and have it set
to use between 256 and 512 meg of memory.
I guess maybe that's a pretty low setting for the memory? Could increasing
that help things?
Anyway, thanks for any & all assistance,
Doug.
we'll need the stack trace.
Peter.
The stack trace from the app is here: Stack trace of NULL pointer given exception · GitHub
It's possibly not much use, though, as it doesn't get into the
Elasticsearch internals.
Is that perhaps what you mean?
Is there somewhere I could look for that information?
On 13 December 2011 11:49, Karussell tableyourtime@googlemail.com wrote:
we'll need the stack trace.
Peter.
It does look like a typhoeus usage problem. Can you share the actual
exception and the code which generate it?
All I can say is that I use typhoeus v0.3.2 without any problem
whatsoever. Only gotcha is that you have to make sure not to use the
typhoeus "Easy" object when in a multithreaded environment. It usually
leads to segmentation faults, not null pointers.
Colin
On Tue, Dec 13, 2011 at 9:01 AM, doug livesey biot023@gmail.com wrote:
The stack trace from the app is here: Stack trace of NULL pointer given exception · GitHub
It's possibly not much use, though, as it doesn't get into the Elasticsearch
internals.
Is that perhaps what you mean?
Is there somewhere I could look for that information?On 13 December 2011 11:49, Karussell tableyourtime@googlemail.com wrote:
we'll need the stack trace.
Peter.
Hi -- first, sorry for the really late reply -- I've been off on
emergencies!
A full exception report for the error is here:
And here is the code that created this error:
The code I've put there does actually use the Easy object, but the
environment is not multi-threaded -- it's just a Rails app.
There's lots of code that doesn't use the Easy object, but that's all
rather involved & in a Template pattern that's fit to bursting at the seams
& begging to become a Strategy pattern! 
I've noticed that the error is far more prevalent on one of our servers
than the others, which does suggest that this is a Ruby/Typhoeus issue.
However, in the last week, we've been having issues with the index
disappearing entirely in the middle of the day, which is very worrying.
I'm going to take the servers down in the wee hours tomorrow morning &
ensure I have the latest Elasticsearch and Typhoeus on them, and go from
there, I reckon.
Cheers,
Doug.
On 13 December 2011 18:08, Colin Surprenant colin.surprenant@gmail.comwrote:
It does look like a typhoeus usage problem. Can you share the actual
exception and the code which generate it?All I can say is that I use typhoeus v0.3.2 without any problem
whatsoever. Only gotcha is that you have to make sure not to use the
typhoeus "Easy" object when in a multithreaded environment. It usually
leads to segmentation faults, not null pointers.Colin
On Tue, Dec 13, 2011 at 9:01 AM, doug livesey biot023@gmail.com wrote:
The stack trace from the app is here: Stack trace of NULL pointer given exception · GitHub
It's possibly not much use, though, as it doesn't get into the
Elasticsearch
internals.
Is that perhaps what you mean?
Is there somewhere I could look for that information?On 13 December 2011 11:49, Karussell tableyourtime@googlemail.com
wrote:we'll need the stack trace.
Peter.
Oh, I should add that the code that doesn't use the Easy object is just
as prone to failing with the NULL pointer given error as the code I gave
that does.
On 19 December 2011 08:52, doug livesey biot023@gmail.com wrote:
Hi -- first, sorry for the really late reply -- I've been off on
emergencies!
A full exception report for the error is here:
Full NULL pointer given exception · GitHub
And here is the code that created this error:
health_check_controller.rb · GitHub
The code I've put there does actually use the Easy object, but the
environment is not multi-threaded -- it's just a Rails app.
There's lots of code that doesn't use the Easy object, but that's all
rather involved & in a Template pattern that's fit to bursting at the seams
& begging to become a Strategy pattern!
I've noticed that the error is far more prevalent on one of our servers
than the others, which does suggest that this is a Ruby/Typhoeus issue.
However, in the last week, we've been having issues with the index
disappearing entirely in the middle of the day, which is very worrying.
I'm going to take the servers down in the wee hours tomorrow morning &
ensure I have the latest Elasticsearch and Typhoeus on them, and go from
there, I reckon.
Cheers,
Doug.On 13 December 2011 18:08, Colin Surprenant colin.surprenant@gmail.comwrote:
It does look like a typhoeus usage problem. Can you share the actual
exception and the code which generate it?All I can say is that I use typhoeus v0.3.2 without any problem
whatsoever. Only gotcha is that you have to make sure not to use the
typhoeus "Easy" object when in a multithreaded environment. It usually
leads to segmentation faults, not null pointers.Colin
On Tue, Dec 13, 2011 at 9:01 AM, doug livesey biot023@gmail.com wrote:
The stack trace from the app is here: Stack trace of NULL pointer given exception · GitHub
It's possibly not much use, though, as it doesn't get into the
Elasticsearch
internals.
Is that perhaps what you mean?
Is there somewhere I could look for that information?On 13 December 2011 11:49, Karussell tableyourtime@googlemail.com
wrote:we'll need the stack trace.
Peter.
are you using the latest libcurl?
On Mon, Dec 19, 2011 at 3:54 AM, doug livesey biot023@gmail.com wrote:
Oh, I should add that the code that doesn't use the Easy object is just as
prone to failing with the NULL pointer given error as the code I gave that
does.On 19 December 2011 08:52, doug livesey biot023@gmail.com wrote:
Hi -- first, sorry for the really late reply -- I've been off on
emergencies!
A full exception report for the error is
here: Full NULL pointer given exception · GitHub
And here is the code that created this
error: health_check_controller.rb · GitHub
The code I've put there does actually use the Easy object, but the
environment is not multi-threaded -- it's just a Rails app.
There's lots of code that doesn't use the Easy object, but that's all
rather involved & in a Template pattern that's fit to bursting at the seams
& begging to become a Strategy pattern!
I've noticed that the error is far more prevalent on one of our servers
than the others, which does suggest that this is a Ruby/Typhoeus issue.
However, in the last week, we've been having issues with the index
disappearing entirely in the middle of the day, which is very worrying.
I'm going to take the servers down in the wee hours tomorrow morning &
ensure I have the latest Elasticsearch and Typhoeus on them, and go from
there, I reckon.
Cheers,
Doug.On 13 December 2011 18:08, Colin Surprenant colin.surprenant@gmail.com
wrote:It does look like a typhoeus usage problem. Can you share the actual
exception and the code which generate it?All I can say is that I use typhoeus v0.3.2 without any problem
whatsoever. Only gotcha is that you have to make sure not to use the
typhoeus "Easy" object when in a multithreaded environment. It usually
leads to segmentation faults, not null pointers.Colin
On Tue, Dec 13, 2011 at 9:01 AM, doug livesey biot023@gmail.com wrote:
The stack trace from the app is here: Stack trace of NULL pointer given exception · GitHub
It's possibly not much use, though, as it doesn't get into the
Elasticsearch
internals.
Is that perhaps what you mean?
Is there somewhere I could look for that information?On 13 December 2011 11:49, Karussell tableyourtime@googlemail.com
wrote:we'll need the stack trace.
Peter.
Hmm -- almost certainly not -- that's a good thing to check, cheers!
On 20 December 2011 21:02, Colin Surprenant colin.surprenant@gmail.comwrote:
are you using the latest libcurl?
On Mon, Dec 19, 2011 at 3:54 AM, doug livesey biot023@gmail.com wrote:
Oh, I should add that the code that doesn't use the Easy object is
just as
prone to failing with the NULL pointer given error as the code I gave
that
does.On 19 December 2011 08:52, doug livesey biot023@gmail.com wrote:
Hi -- first, sorry for the really late reply -- I've been off on
emergencies!
A full exception report for the error is
here: Full NULL pointer given exception · GitHub
And here is the code that created this
error: health_check_controller.rb · GitHub
The code I've put there does actually use the Easy object, but the
environment is not multi-threaded -- it's just a Rails app.
There's lots of code that doesn't use the Easy object, but that's all
rather involved & in a Template pattern that's fit to bursting at the
seams
& begging to become a Strategy pattern!
I've noticed that the error is far more prevalent on one of our servers
than the others, which does suggest that this is a Ruby/Typhoeus issue.
However, in the last week, we've been having issues with the index
disappearing entirely in the middle of the day, which is very worrying.
I'm going to take the servers down in the wee hours tomorrow morning &
ensure I have the latest Elasticsearch and Typhoeus on them, and go from
there, I reckon.
Cheers,
Doug.On 13 December 2011 18:08, Colin Surprenant <colin.surprenant@gmail.com
wrote:
It does look like a typhoeus usage problem. Can you share the actual
exception and the code which generate it?All I can say is that I use typhoeus v0.3.2 without any problem
whatsoever. Only gotcha is that you have to make sure not to use the
typhoeus "Easy" object when in a multithreaded environment. It usually
leads to segmentation faults, not null pointers.Colin
On Tue, Dec 13, 2011 at 9:01 AM, doug livesey biot023@gmail.com
wrote:The stack trace from the app is here:
Stack trace of NULL pointer given exception · GitHub
It's possibly not much use, though, as it doesn't get into the
Elasticsearch
internals.
Is that perhaps what you mean?
Is there somewhere I could look for that information?On 13 December 2011 11:49, Karussell tableyourtime@googlemail.com
wrote:we'll need the stack trace.
Peter.
Hi -- really late update. We've only just been able to update all the
packages on our servers (had to update Puppet first!), and the latest
libcurl (and all other libraries) totally fixed this issue.
Cheers,
Doug.
On 20 December 2011 21:05, doug livesey biot023@gmail.com wrote:
Hmm -- almost certainly not -- that's a good thing to check, cheers!
On 20 December 2011 21:02, Colin Surprenant colin.surprenant@gmail.comwrote:
are you using the latest libcurl?
On Mon, Dec 19, 2011 at 3:54 AM, doug livesey biot023@gmail.com wrote:
Oh, I should add that the code that doesn't use the Easy object is
just as
prone to failing with the NULL pointer given error as the code I gave
that
does.On 19 December 2011 08:52, doug livesey biot023@gmail.com wrote:
Hi -- first, sorry for the really late reply -- I've been off on
emergencies!
A full exception report for the error is
here: Full NULL pointer given exception · GitHub
And here is the code that created this
error: health_check_controller.rb · GitHub
The code I've put there does actually use the Easy object, but the
environment is not multi-threaded -- it's just a Rails app.
There's lots of code that doesn't use the Easy object, but that's all
rather involved & in a Template pattern that's fit to bursting at the
seams
& begging to become a Strategy pattern!
I've noticed that the error is far more prevalent on one of our servers
than the others, which does suggest that this is a Ruby/Typhoeus issue.
However, in the last week, we've been having issues with the index
disappearing entirely in the middle of the day, which is very worrying.
I'm going to take the servers down in the wee hours tomorrow morning &
ensure I have the latest Elasticsearch and Typhoeus on them, and go
from
there, I reckon.
Cheers,
Doug.On 13 December 2011 18:08, Colin Surprenant <
colin.surprenant@gmail.com>
wrote:It does look like a typhoeus usage problem. Can you share the actual
exception and the code which generate it?All I can say is that I use typhoeus v0.3.2 without any problem
whatsoever. Only gotcha is that you have to make sure not to use the
typhoeus "Easy" object when in a multithreaded environment. It usually
leads to segmentation faults, not null pointers.Colin
On Tue, Dec 13, 2011 at 9:01 AM, doug livesey biot023@gmail.com
wrote:The stack trace from the app is here:
Stack trace of NULL pointer given exception · GitHub
It's possibly not much use, though, as it doesn't get into the
Elasticsearch
internals.
Is that perhaps what you mean?
Is there somewhere I could look for that information?On 13 December 2011 11:49, Karussell tableyourtime@googlemail.com
wrote:we'll need the stack trace.
Peter.