ES 0.18.3 NPE when set logging level to TRACE

Elastic Stack Elasticsearch
1

Hi, I just ran into an exception, here are the steps to reproduce it.

  1. I have 2 ES servers, vanilla setup
  2. Change the logging level to TRACE in logging.yml
    rootLogger: TRACE, console, file
  3. Shutdown the primary node
  4. Run the command
    $ curl -XGET http://secondaryhost:9200/_cluster/state
    and get the following error.
    {"error":"NullPointerException[null]","status":500}

Nothing left in the log file.

Any suggestions?

Thanks.

-- Liyu

2

You will have to be more specific about the step to recreate it. I started
two elasticsearch instances with TRACE logging, ran cluster state and did
not see the failure. Start from fresh instances.

On Wed, Dec 21, 2011 at 6:29 AM, Liyu liyuyi@gmail.com wrote:

Hi, I just ran into an exception, here are the steps to reproduce it.

  1. I have 2 ES servers, vanilla setup
  2. Change the logging level to TRACE in logging.yml
    rootLogger: TRACE, console, file
  3. Shutdown the primary node
  4. Run the command
    $ curl -XGET http://secondaryhost:9200/_cluster/state
    and get the following error.
    {"error":"NullPointerException[null]","status":500}

Nothing left in the log file.

Any suggestions?

Thanks.

-- Liyu

3

Shay,

Did you shutdown the primary node? and run the cluster state from the
secondary node?

Thanks!

-- Liyu

On Tue, Dec 20, 2011 at 9:09 PM, Shay Banon kimchy@gmail.com wrote:

You will have to be more specific about the step to recreate it. I started
two elasticsearch instances with TRACE logging, ran cluster state and did
not see the failure. Start from fresh instances.

On Wed, Dec 21, 2011 at 6:29 AM, Liyu liyuyi@gmail.com wrote:

Hi, I just ran into an exception, here are the steps to reproduce it.

  1. I have 2 ES servers, vanilla setup
  2. Change the logging level to TRACE in logging.yml
    rootLogger: TRACE, console, file
  3. Shutdown the primary node
  4. Run the command
    $ curl -XGET http://secondaryhost:9200/_cluster/state
    and get the following error.
    {"error":"NullPointerException[null]","status":500}

Nothing left in the log file.

Any suggestions?

Thanks.

-- Liyu

4

You mean shutdown the elected master node, and execute the cluster state
against the second node (which got elected to be master)? Yes. (there is no
primary or secondary node in elasticsearch). Explain the concrete steps
that you do to recreate it.

On Wed, Dec 21, 2011 at 7:54 AM, Liyu Yi liyuyi@gmail.com wrote:

Shay,

Did you shutdown the primary node? and run the cluster state from the
secondary node?

Thanks!

-- Liyu

On Tue, Dec 20, 2011 at 9:09 PM, Shay Banon kimchy@gmail.com wrote:

You will have to be more specific about the step to recreate it. I
started two elasticsearch instances with TRACE logging, ran cluster state
and did not see the failure. Start from fresh instances.

On Wed, Dec 21, 2011 at 6:29 AM, Liyu liyuyi@gmail.com wrote:

Hi, I just ran into an exception, here are the steps to reproduce it.

  1. I have 2 ES servers, vanilla setup
  2. Change the logging level to TRACE in logging.yml
    rootLogger: TRACE, console, file
  3. Shutdown the primary node
  4. Run the command
    $ curl -XGET http://secondaryhost:9200/_cluster/state
    and get the following error.
    {"error":"NullPointerException[null]","status":500}

Nothing left in the log file.

Any suggestions?

Thanks.

-- Liyu

5

Hi, Shay, what you said is correct :slight_smile:
It is the master node I shut down. Also I forgot to mention one thing, I
configured the cluster with unicast since multicast is disabled in my
environment.
If I change the logging level to DEBUG or WARN, everything works perfectly.
And I can see all the details from the log that the master is left and the
secondary is promoted to the new master.
But when set to TRACE, I had this NPE.

On Tuesday, December 20, 2011, Shay Banon kimchy@gmail.com wrote:

You mean shutdown the elected master node, and execute the cluster state
against the second node (which got elected to be master)? Yes. (there is no
primary or secondary node in elasticsearch). Explain the concrete steps
that you do to recreate it.

On Wed, Dec 21, 2011 at 7:54 AM, Liyu Yi liyuyi@gmail.com wrote:

Shay,

Did you shutdown the primary node? and run the cluster state from the
secondary node?

Thanks!

-- Liyu

On Tue, Dec 20, 2011 at 9:09 PM, Shay Banon kimchy@gmail.com wrote:

You will have to be more specific about the step to recreate it. I
started two elasticsearch instances with TRACE logging, ran cluster state
and did not see the failure. Start from fresh instances.

On Wed, Dec 21, 2011 at 6:29 AM, Liyu liyuyi@gmail.com wrote:

Hi, I just ran into an exception, here are the steps to reproduce it.

  1. I have 2 ES servers, vanilla setup
  2. Change the logging level to TRACE in logging.yml
    rootLogger: TRACE, console, file
  3. Shutdown the primary node
  4. Run the command
    $ curl -XGET http://secondaryhost:9200/_cluster/state
    and get the following error.
    {"error":"NullPointerException[null]","status":500}

Nothing left in the log file.

Any suggestions?

Thanks.

-- Liyu

6

Used unicast as well, did not recreate it. Please, provide the steps you do
to recreate it. It should look something like this:

  1. Downloaded elasticsearch
  2. Changed the configuration file to have this and that
  3. Started first node
  4. Started second node
  5. ....

Also, which version are you using?

On Wed, Dec 21, 2011 at 8:46 AM, Liyu Yi liyuyi@gmail.com wrote:

Hi, Shay, what you said is correct :slight_smile:
It is the master node I shut down. Also I forgot to mention one thing, I
configured the cluster with unicast since multicast is disabled in my
environment.
If I change the logging level to DEBUG or WARN, everything works
perfectly. And I can see all the details from the log that the master is
left and the secondary is promoted to the new master.
But when set to TRACE, I had this NPE.

On Tuesday, December 20, 2011, Shay Banon kimchy@gmail.com wrote:

You mean shutdown the elected master node, and execute the cluster state
against the second node (which got elected to be master)? Yes. (there is no
primary or secondary node in elasticsearch). Explain the concrete steps
that you do to recreate it.

On Wed, Dec 21, 2011 at 7:54 AM, Liyu Yi liyuyi@gmail.com wrote:

Shay,

Did you shutdown the primary node? and run the cluster state from the
secondary node?

Thanks!

-- Liyu

On Tue, Dec 20, 2011 at 9:09 PM, Shay Banon kimchy@gmail.com wrote:

You will have to be more specific about the step to recreate it. I
started two elasticsearch instances with TRACE logging, ran cluster state
and did not see the failure. Start from fresh instances.

On Wed, Dec 21, 2011 at 6:29 AM, Liyu liyuyi@gmail.com wrote:

Hi, I just ran into an exception, here are the steps to reproduce it.

  1. I have 2 ES servers, vanilla setup
  2. Change the logging level to TRACE in logging.yml
    rootLogger: TRACE, console, file
  3. Shutdown the primary node
  4. Run the command
    $ curl -XGET http://secondaryhost:9200/_cluster/state
    and get the following error.
    {"error":"NullPointerException[null]","status":500}

Nothing left in the log file.

Any suggestions?

Thanks.

-- Liyu

7

Hello, Shay,

I'll try out a new installation from scratch and let you know the exact
steps to recreate this problem soon.

The version number is 0.18.3. Also the following exception is found from
the log, but is NOT triggered by the cluster state command from the curl
command.

[2011-12-20 23:13:05,958][DEBUG][rest.action.admin.cluster.state]
[Ardroman] failed to handle cluster state
java.lang.NullPointerException
at
org.elasticsearch.cluster.routing.RoutingNode.nodeId(RoutingNode.java:56)
at
org.elasticsearch.rest.action.admin.cluster.state.RestClusterStateAction$1.onResponse(RestClusterStateAction.java:248)
at
org.elasticsearch.rest.action.admin.cluster.state.RestClusterStateAction$1.onResponse(RestClusterStateAction.java:80)
at
org.elasticsearch.action.support.BaseAction$ThreadedActionListener$1.run(BaseAction.java:84)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
[2011-12-20 23:13:06,915][DEBUG][rest.action.admin.cluster.state]
[Ardroman] failed to handle cluster state
java.lang.NullPointerException
at
org.elasticsearch.cluster.routing.RoutingNode.nodeId(RoutingNode.java:56)
at
org.elasticsearch.rest.action.admin.cluster.state.RestClusterStateAction$1.onResponse(RestClusterStateAction.java:248)
at
org.elasticsearch.rest.action.admin.cluster.state.RestClusterStateAction$1.onResponse(RestClusterStateAction.java:80)
at
org.elasticsearch.action.support.BaseAction$ThreadedActionListener$1.run(BaseAction.java:84)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
[2011-12-20 23:13:08,558][DEBUG][rest.action.admin.cluster.state]
[Ardroman] failed to handle cluster state
java.lang.NullPointerException
at
org.elasticsearch.cluster.routing.RoutingNode.nodeId(RoutingNode.java:56)
at
org.elasticsearch.rest.action.admin.cluster.state.RestClusterStateAction$1.onResponse(RestClusterStateAction.java:248)
at
org.elasticsearch.rest.action.admin.cluster.state.RestClusterStateAction$1.onResponse(RestClusterStateAction.java:80)
at
org.elasticsearch.action.support.BaseAction$ThreadedActionListener$1.run(BaseAction.java:84)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)

On Tue, Dec 20, 2011 at 11:50 PM, Shay Banon kimchy@gmail.com wrote:

Used unicast as well, did not recreate it. Please, provide the steps you
do to recreate it. It should look something like this:

  1. Downloaded elasticsearch
  2. Changed the configuration file to have this and that
  3. Started first node
  4. Started second node
  5. ....

Also, which version are you using?

On Wed, Dec 21, 2011 at 8:46 AM, Liyu Yi liyuyi@gmail.com wrote:

Hi, Shay, what you said is correct :slight_smile:
It is the master node I shut down. Also I forgot to mention one thing, I
configured the cluster with unicast since multicast is disabled in my
environment.
If I change the logging level to DEBUG or WARN, everything works
perfectly. And I can see all the details from the log that the master is
left and the secondary is promoted to the new master.
But when set to TRACE, I had this NPE.

On Tuesday, December 20, 2011, Shay Banon kimchy@gmail.com wrote:

You mean shutdown the elected master node, and execute the cluster
state against the second node (which got elected to be master)? Yes. (there
is no primary or secondary node in elasticsearch). Explain the concrete
steps that you do to recreate it.

On Wed, Dec 21, 2011 at 7:54 AM, Liyu Yi liyuyi@gmail.com wrote:

Shay,

Did you shutdown the primary node? and run the cluster state from the
secondary node?

Thanks!

-- Liyu

On Tue, Dec 20, 2011 at 9:09 PM, Shay Banon kimchy@gmail.com wrote:

You will have to be more specific about the step to recreate it. I
started two elasticsearch instances with TRACE logging, ran cluster state
and did not see the failure. Start from fresh instances.

On Wed, Dec 21, 2011 at 6:29 AM, Liyu liyuyi@gmail.com wrote:

Hi, I just ran into an exception, here are the steps to reproduce it.

  1. I have 2 ES servers, vanilla setup
  2. Change the logging level to TRACE in logging.yml
    rootLogger: TRACE, console, file
  3. Shutdown the primary node
  4. Run the command
    $ curl -XGET http://secondaryhost:9200/_cluster/state
    and get the following error.
    {"error":"NullPointerException[null]","status":500}

Nothing left in the log file.

Any suggestions?

Thanks.

-- Liyu