Closed Index missing after restarting the elasticsearch cluster


(error804) #1

We are running elasticsearch in extra large instances of amazon ec2. After
restarting the cluster the closed index are not present in that cluster. So
we are seeking for some help, can anybody have some ideas about that.

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Ivan Brusic) #2

Does the data exist in the data directory (path.data)?

Cheers,

Ivan

On Wed, Sep 18, 2013 at 3:09 AM, pawansharma2045
pawansharma2045@gmail.comwrote:

We are running elasticsearch in extra large instances of amazon ec2.
After restarting the cluster the closed index are not present in that
cluster. So we are seeking for some help, can anybody have some ideas about
that.

Thanks

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(error804) #3

Thanks for the reply.

No data doesn't exist. That index as well as data lost. We have to process
that data again.
You have got any ides about that.

Regards
Pawan

On Wed, Sep 18, 2013 at 9:06 PM, Ivan Brusic ivan@brusic.com wrote:

Does the data exist in the data directory (path.data)?

Cheers,

Ivan

On Wed, Sep 18, 2013 at 3:09 AM, pawansharma2045 <
pawansharma2045@gmail.com> wrote:

We are running elasticsearch in extra large instances of amazon ec2.
After restarting the cluster the closed index are not present in that
cluster. So we are seeking for some help, can anybody have some ideas about
that.

Thanks

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Alexander Reelsen) #4

Hey,

can you create a reproducible example and provide some more information?
I'd like to see

  • how you create this index (any special configuration, also in your .yml
    file)
  • how you close the index?
  • is there anything in your logfiles? any exceptions?
  • What elasticsearch version are you using?

Thanks!

--Alex

On Thu, Sep 19, 2013 at 10:24 AM, Pawan Sharma pawansharma2045@gmail.comwrote:

Thanks for the reply.

No data doesn't exist. That index as well as data lost. We have to process
that data again.
You have got any ides about that.

Regards
Pawan

On Wed, Sep 18, 2013 at 9:06 PM, Ivan Brusic ivan@brusic.com wrote:

Does the data exist in the data directory (path.data)?

Cheers,

Ivan

On Wed, Sep 18, 2013 at 3:09 AM, pawansharma2045 <
pawansharma2045@gmail.com> wrote:

We are running elasticsearch in extra large instances of amazon ec2.
After restarting the cluster the closed index are not present in that
cluster. So we are seeking for some help, can anybody have some ideas about
that.

Thanks

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(error804) #5

Hi,

We use python script to copy index from transient cluster.

This is used to creat index while copying from transient. Here we give source
index [FROM]
and destination Index [To Be copied] .
call('curl -XPOST ' + destination_instances[0] + ':9200/' + dest_index+'/
-d @' + mapping_path + 'mapping', shell=True)

and

For Closing indexes we call

call('curl -XPOST ' + source_instances[0] + ':9200/' + source_index
+'/_close', shell=True)
call('curl -XPOST ' + destination_instances[0] + ':9200/' + dest_index
+'/_close', shell=True)

There is nothing abnormal or any logs or exceptions . Currently we are
using 0.90.3

and nothing special configuration in .yml file.

Regards
Pawan

On Thu, Sep 19, 2013 at 2:21 PM, Alexander Reelsen alr@spinscale.de wrote:

Hey,

can you create a reproducible example and provide some more information?
I'd like to see

  • how you create this index (any special configuration, also in your .yml
    file)
  • how you close the index?
  • is there anything in your logfiles? any exceptions?
  • What elasticsearch version are you using?

Thanks!

--Alex

On Thu, Sep 19, 2013 at 10:24 AM, Pawan Sharma pawansharma2045@gmail.comwrote:

Thanks for the reply.

No data doesn't exist. That index as well as data lost. We have to
process that data again.
You have got any ides about that.

Regards
Pawan

On Wed, Sep 18, 2013 at 9:06 PM, Ivan Brusic ivan@brusic.com wrote:

Does the data exist in the data directory (path.data)?

Cheers,

Ivan

On Wed, Sep 18, 2013 at 3:09 AM, pawansharma2045 <
pawansharma2045@gmail.com> wrote:

We are running elasticsearch in extra large instances of amazon ec2.
After restarting the cluster the closed index are not present in that
cluster. So we are seeking for some help, can anybody have some ideas about
that.

Thanks

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Ivan Brusic) #6

Where there any open indices during the restart?

On Thu, Sep 19, 2013 at 2:19 AM, Pawan Sharma pawansharma2045@gmail.comwrote:

Hi,

We use python script to copy index from transient cluster.

This is used to creat index while copying from transient. Here we give source
index [FROM]
and destination Index [To Be copied] .
call('curl -XPOST ' + destination_instances[0] + ':9200/' + dest_index+'/
-d @' + mapping_path + 'mapping', shell=True)

and

For Closing indexes we call

call('curl -XPOST ' + source_instances[0] + ':9200/' + source_index
+'/_close', shell=True)
call('curl -XPOST ' + destination_instances[0] + ':9200/' + dest_index
+'/_close', shell=True)

There is nothing abnormal or any logs or exceptions . Currently we are
using 0.90.3

and nothing special configuration in .yml file.

Regards
Pawan

On Thu, Sep 19, 2013 at 2:21 PM, Alexander Reelsen alr@spinscale.dewrote:

Hey,

can you create a reproducible example and provide some more information?
I'd like to see

  • how you create this index (any special configuration, also in your .yml
    file)
  • how you close the index?
  • is there anything in your logfiles? any exceptions?
  • What elasticsearch version are you using?

Thanks!

--Alex

On Thu, Sep 19, 2013 at 10:24 AM, Pawan Sharma <pawansharma2045@gmail.com

wrote:

Thanks for the reply.

No data doesn't exist. That index as well as data lost. We have to
process that data again.
You have got any ides about that.

Regards
Pawan

On Wed, Sep 18, 2013 at 9:06 PM, Ivan Brusic ivan@brusic.com wrote:

Does the data exist in the data directory (path.data)?

Cheers,

Ivan

On Wed, Sep 18, 2013 at 3:09 AM, pawansharma2045 <
pawansharma2045@gmail.com> wrote:

We are running elasticsearch in extra large instances of amazon ec2.
After restarting the cluster the closed index are not present in that
cluster. So we are seeking for some help, can anybody have some ideas about
that.

Thanks

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(error804) #7

HI ivan Brusic,

There are open about 30 open indices during the restart.But the problem is
only that the closed indices are missed.

Regards

On Fri, Sep 20, 2013 at 12:17 AM, Ivan Brusic ivan@brusic.com wrote:

Where there any open indices during the restart?

On Thu, Sep 19, 2013 at 2:19 AM, Pawan Sharma pawansharma2045@gmail.comwrote:

Hi,

We use python script to copy index from transient cluster.

This is used to creat index while copying from transient. Here we give source
index [FROM]
and destination Index [To Be copied] .
call('curl -XPOST ' + destination_instances[0] + ':9200/' + dest_index+'/
-d @' + mapping_path + 'mapping', shell=True)

and

For Closing indexes we call

call('curl -XPOST ' + source_instances[0] + ':9200/' + source_index
+'/_close', shell=True)
call('curl -XPOST ' + destination_instances[0] + ':9200/' + dest_index
+'/_close', shell=True)

There is nothing abnormal or any logs or exceptions . Currently we are
using 0.90.3

and nothing special configuration in .yml file.

Regards
Pawan

On Thu, Sep 19, 2013 at 2:21 PM, Alexander Reelsen alr@spinscale.dewrote:

Hey,

can you create a reproducible example and provide some more information?
I'd like to see

  • how you create this index (any special configuration, also in your
    .yml file)
  • how you close the index?
  • is there anything in your logfiles? any exceptions?
  • What elasticsearch version are you using?

Thanks!

--Alex

On Thu, Sep 19, 2013 at 10:24 AM, Pawan Sharma <
pawansharma2045@gmail.com> wrote:

Thanks for the reply.

No data doesn't exist. That index as well as data lost. We have to
process that data again.
You have got any ides about that.

Regards
Pawan

On Wed, Sep 18, 2013 at 9:06 PM, Ivan Brusic ivan@brusic.com wrote:

Does the data exist in the data directory (path.data)?

Cheers,

Ivan

On Wed, Sep 18, 2013 at 3:09 AM, pawansharma2045 <
pawansharma2045@gmail.com> wrote:

We are running elasticsearch in extra large instances of amazon ec2.
After restarting the cluster the closed index are not present in that
cluster. So we are seeking for some help, can anybody have some ideas about
that.

Thanks

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(system) #8