Can I update analyzer settings without recreating the index?


(Nick Dunn) #1

I am looking at using a hosted ES service (Bonsai.io) which offers an
index per customer, pre-created. Usually I create a config file
containing my custom analyzers and send this when creating an index,
however because Bonsai provide the index already, this method will not
work.

I've read the docs for the Updating Settings API for an index but it
seems only certain settings can be modified this way, and not
analyzers.

What are my options here? I've not heard of a way to update index
custom analyzers without recreating an index, but perhaps I've missed
something somewhere! Can I send custom analyzer config another way?


(Shay Banon) #2

You can't change analyzer config on a "live" index using update settings,
you can though close the index, send an update settings with the new
analysis settings, and then open it back, though I am not sure of
close/open are allowed in bonsai...

On Wed, Mar 28, 2012 at 9:43 AM, Nick Dunn nick@nick-dunn.co.uk wrote:

I am looking at using a hosted ES service (Bonsai.io) which offers an
index per customer, pre-created. Usually I create a config file
containing my custom analyzers and send this when creating an index,
however because Bonsai provide the index already, this method will not
work.

I've read the docs for the Updating Settings API for an index but it
seems only certain settings can be modified this way, and not
analyzers.

What are my options here? I've not heard of a way to update index
custom analyzers without recreating an index, but perhaps I've missed
something somewhere! Can I send custom analyzer config another way?


(David Pilato) #3

Hi Shay,

Just to be sure to understand this.

If we want to merge settings (replication), we should (must ?) not close the
index before sending an update settings.

But if we want to add a new analyzer, we must close the index before.

Is that it ?

I tried this in Java :

         // Before merging, we have to close the index

         CloseIndexRequestBuilder cirb =

client.admin().indices().prepareClose(index);

         CloseIndexResponse closeIndexResponse =

cirb.execute().actionGet();

         if (!closeIndexResponse.acknowledged()) throw new

Exception("Could not close index ["+index+"].");

         UpdateSettingsRequestBuilder usrb =

client.admin().indices().prepareUpdateSettings(index);

         // If there are settings for this index, we use it. If not,

using Elasticsearch defaults.

         String source = readIndexSettings(index);

         if (source != null) {

                if (logger.isTraceEnabled()) logger.trace("Found

settings for index "+index+" : " + source);

                usrb.setSettings(source);

         }

         

         usrb.execute().actionGet();

But it failed at the last line with a
org.elasticsearch.indices.IndexMissingException: [twitter] missing

If I remove the closeIndex lines, I don’t get the exception but I get the
following warning :

22:56:48,843 WARN [metadata] [junit.node.transport] [twitter] ignoring non
dynamic index level settings for open indices:
[index.analysis.analyzer.type]

How to close an index and update settings in java ?

Thanks

David.

De : elasticsearch@googlegroups.com [mailto:elasticsearch@googlegroups.com]
De la part de Shay Banon
Envoyé : mercredi 28 mars 2012 10:55
À : elasticsearch@googlegroups.com
Objet : Re: Can I update analyzer settings without recreating the index?

You can't change analyzer config on a "live" index using update settings,
you can though close the index, send an update settings with the new
analysis settings, and then open it back, though I am not sure of close/open
are allowed in bonsai...

On Wed, Mar 28, 2012 at 9:43 AM, Nick Dunn nick@nick-dunn.co.uk wrote:

I am looking at using a hosted ES service (Bonsai.io) which offers an
index per customer, pre-created. Usually I create a config file
containing my custom analyzers and send this when creating an index,
however because Bonsai provide the index already, this method will not
work.

I've read the docs for the Updating Settings API for an index but it
seems only certain settings can be modified this way, and not
analyzers.

What are my options here? I've not heard of a way to update index
custom analyzers without recreating an index, but perhaps I've missed
something somewhere! Can I send custom analyzer config another way?


(Shay Banon) #4

Thats strange..., can you gist hte full failure? This works on 0.19:

curl -XPUT localhost:9200/test
curl -XPOST localhost:9200/test/_close

curl -XPUT localhost:9200/test/_settings -d '{
"index" : {
"something" : "value"
}
}'

On Fri, Mar 30, 2012 at 11:58 PM, David Pilato david@pilato.fr wrote:

Hi Shay,****


Just to be sure to understand this.****


If we want to merge settings (replication), we should (must ?) not close
the index before sending an update settings.****

But if we want to add a new analyzer, we must close the index before.****


Is that it ?****


I tried this in Java :****

         // Before merging, we have to close the index****

         CloseIndexRequestBuilder cirb = client

.admin().indices().prepareClose(index);****

         CloseIndexResponse closeIndexResponse =

cirb.execute().actionGet();****

         *if* (!closeIndexResponse.acknowledged()) *throw* *new*Exception("Could

not close index ["+index+"].");****


         UpdateSettingsRequestBuilder usrb = client

.admin().indices().prepareUpdateSettings(index);****


         // If there are settings for this index, we use it. If not,

using Elasticsearch defaults.****

         String source = readIndexSettings(index);****

         *if* (source != *null*) {****

                *if* (logger.isTraceEnabled()) logger.trace("Found

settings for index "+index+" : " + source);****

                usrb.setSettings(source);****

         }****

         ****

         usrb.execute().actionGet();****

But it failed at the last line with a
org.elasticsearch.indices.IndexMissingException: [twitter] missing****


If I remove the closeIndex lines, I don’t get the exception but I get the
following warning :****

22:56:48,843 WARN [metadata] [junit.node.transport] [twitter] ignoring
non dynamic index level settings for open indices:
[index.analysis.analyzer.type]****


How to close an index and update settings in java ?****


Thanks****

David. ****



De : elasticsearch@googlegroups.com [mailto:
elasticsearch@googlegroups.com] De la part de Shay Banon
Envoyé : mercredi 28 mars 2012 10:55
À : elasticsearch@googlegroups.com
Objet : Re: Can I update analyzer settings without recreating the index?



You can't change analyzer config on a "live" index using update settings,
you can though close the index, send an update settings with the new
analysis settings, and then open it back, though I am not sure of
close/open are allowed in bonsai...****

On Wed, Mar 28, 2012 at 9:43 AM, Nick Dunn nick@nick-dunn.co.uk wrote:**
**

I am looking at using a hosted ES service (Bonsai.io) which offers an
index per customer, pre-created. Usually I create a config file
containing my custom analyzers and send this when creating an index,
however because Bonsai provide the index already, this method will not
work.

I've read the docs for the Updating Settings API for an index but it
seems only certain settings can be modified this way, and not
analyzers.

What are my options here? I've not heard of a way to update index
custom analyzers without recreating an index, but perhaps I've missed
something somewhere! Can I send custom analyzer config another way?****



(Nick Zadrozny) #5

On Wed, Mar 28, 2012 at 1:55 AM, Shay Banon kimchy@gmail.com wrote:

You can't change analyzer config on a "live" index using update settings,
you can though close the index, send an update settings with the new
analysis settings, and then open it back, though I am not sure of
close/open are allowed in bonsai...

As of today, all PUT and POST actions to the /index handler on Bonsai are
restricted. But that's probably just for the beta as we continue to work
our way through the various system- and user-facing stories and whitelist
what's appropriate. (E.g., changing number of replicas should be
restricted, changing the mapping should not.)

Shay, I'm going to send you an overdue ping off-list about our little
project :slight_smile:

--
Nick Zadrozny


(David Pilato) #6

Hi Shay,

I think I understood where the error came from:

I cannot update number of replicas on a closed index.

So some properties have to be updated on a closed index and others on a live
index.

My use case is for the Spring factory.

I want to update settings but I don’t know what settings will set the user.

Is there a way to analyze the settings json file and split it in two
settings files : one to be applied live and another to be applied closed ?

Do you have any class or method in ES that I can use for that ?

Or do I have to parse all the properties to find if one of them is in this
list :
http://www.elasticsearch.org/guide/reference/api/admin-indices-update-settin
gs.html so it has to be applied on a live index ?

Is there any reason that you can’t apply all properties on a closed index ?
Should I raise an issue ?

Thanks

David.

De : elasticsearch@googlegroups.com [mailto:elasticsearch@googlegroups.com]
De la part de Shay Banon
Envoyé : samedi 31 mars 2012 23:02
À : elasticsearch@googlegroups.com
Objet : Re: Can I update analyzer settings without recreating the index?

Thats strange..., can you gist hte full failure? This works on 0.19:

curl -XPUT localhost:9200/test

curl -XPOST localhost:9200/test/_close

curl -XPUT localhost:9200/test/_settings -d '{

"index" : {

    "something" : "value"

}

}'

On Fri, Mar 30, 2012 at 11:58 PM, David Pilato david@pilato.fr wrote:

Hi Shay,

Just to be sure to understand this.

If we want to merge settings (replication), we should (must ?) not close the
index before sending an update settings.

But if we want to add a new analyzer, we must close the index before.

Is that it ?

I tried this in Java :

         // Before merging, we have to close the index

         CloseIndexRequestBuilder cirb =

client.admin().indices().prepareClose(index);

         CloseIndexResponse closeIndexResponse =

cirb.execute().actionGet();

         if (!closeIndexResponse.acknowledged()) throw new

Exception("Could not close index ["+index+"].");

         UpdateSettingsRequestBuilder usrb =

client.admin().indices().prepareUpdateSettings(index);

         // If there are settings for this index, we use it. If not,

using Elasticsearch defaults.

         String source = readIndexSettings(index);

         if (source != null) {

                if (logger.isTraceEnabled()) logger.trace("Found

settings for index "+index+" : " + source);

                usrb.setSettings(source);

         }

         

         usrb.execute().actionGet();

But it failed at the last line with a
org.elasticsearch.indices.IndexMissingException: [twitter] missing

If I remove the closeIndex lines, I don’t get the exception but I get the
following warning :

22:56:48,843 WARN [metadata] [junit.node.transport] [twitter] ignoring non
dynamic index level settings for open indices:
[index.analysis.analyzer.type]

How to close an index and update settings in java ?

Thanks

David.

De : elasticsearch@googlegroups.com [mailto:elasticsearch@googlegroups.com]
De la part de Shay Banon
Envoyé : mercredi 28 mars 2012 10:55
À : elasticsearch@googlegroups.com
Objet : Re: Can I update analyzer settings without recreating the index?

You can't change analyzer config on a "live" index using update settings,
you can though close the index, send an update settings with the new
analysis settings, and then open it back, though I am not sure of close/open
are allowed in bonsai...

On Wed, Mar 28, 2012 at 9:43 AM, Nick Dunn nick@nick-dunn.co.uk wrote:

I am looking at using a hosted ES service (Bonsai.io) which offers an
index per customer, pre-created. Usually I create a config file
containing my custom analyzers and send this when creating an index,
however because Bonsai provide the index already, this method will not
work.

I've read the docs for the Updating Settings API for an index but it
seems only certain settings can be modified this way, and not
analyzers.

What are my options here? I've not heard of a way to update index
custom analyzers without recreating an index, but perhaps I've missed
something somewhere! Can I send custom analyzer config another way?


(Shay Banon) #7

Yea, only the list mentioned in the update settings can be applied
dynamically, while other settings can be applied on a closed index. The
number of replicas can be updated on a live index though...

On Sun, Apr 1, 2012 at 10:49 PM, David Pilato david@pilato.fr wrote:

Hi Shay,****


I think I understood where the error came from:
https://gist.github.com/2278029****


I cannot update number of replicas on a closed index.****


So some properties have to be updated on a closed index and others on a
live index.****



My use case is for the Spring factory.****

I want to update settings but I don’t know what settings will set the user.



Is there a way to analyze the settings json file and split it in two
settings files : one to be applied live and another to be applied closed ?


Do you have any class or method in ES that I can use for that ?****


Or do I have to parse all the properties to find if one of them is in this
list :
http://www.elasticsearch.org/guide/reference/api/admin-indices-update-settings.htmlso it has to be applied on a live index ?



Is there any reason that you can’t apply all properties on a closed index
? Should I raise an issue ?****



Thanks****

David.****



De : elasticsearch@googlegroups.com [mailto:
elasticsearch@googlegroups.com] De la part de Shay Banon
Envoyé : samedi 31 mars 2012 23:02

À : elasticsearch@googlegroups.com
Objet : Re: Can I update analyzer settings without recreating the index?



Thats strange..., can you gist hte full failure? This works on 0.19:****


curl -XPUT localhost:9200/test****

curl -XPOST localhost:9200/test/_close****


curl -XPUT localhost:9200/test/_settings -d '{****

"index" : {****

    "something" : "value"****

}****

}'****



On Fri, Mar 30, 2012 at 11:58 PM, David Pilato david@pilato.fr wrote:***
*

Hi Shay,****


Just to be sure to understand this.****


If we want to merge settings (replication), we should (must ?) not close
the index before sending an update settings.****

But if we want to add a new analyzer, we must close the index before.****


Is that it ?****


I tried this in Java :****

         // Before merging, we have to close the index****

         CloseIndexRequestBuilder cirb = client

.admin().indices().prepareClose(index);****

         CloseIndexResponse closeIndexResponse =

cirb.execute().actionGet();****

         *if* (!closeIndexResponse.acknowledged()) *throw* *new*Exception("Could

not close index ["+index+"].");****


         UpdateSettingsRequestBuilder usrb = client

.admin().indices().prepareUpdateSettings(index);****


         // If there are settings for this index, we use it. If not,

using Elasticsearch defaults.****

         String source = readIndexSettings(index);****

         *if* (source != *null*) {****

                *if* (logger.isTraceEnabled()) logger.trace("Found

settings for index "+index+" : " + source);****

                usrb.setSettings(source);****

         }****

         ****

         usrb.execute().actionGet();****

But it failed at the last line with a
org.elasticsearch.indices.IndexMissingException: [twitter] missing****


If I remove the closeIndex lines, I don’t get the exception but I get the
following warning :****

22:56:48,843 WARN [metadata] [junit.node.transport] [twitter] ignoring
non dynamic index level settings for open indices:
[index.analysis.analyzer.type]****


How to close an index and update settings in java ?****


Thanks****

David. ****



De : elasticsearch@googlegroups.com [mailto:
elasticsearch@googlegroups.com] De la part de Shay Banon
Envoyé : mercredi 28 mars 2012 10:55
À : elasticsearch@googlegroups.com
Objet : Re: Can I update analyzer settings without recreating the index?



You can't change analyzer config on a "live" index using update settings,
you can though close the index, send an update settings with the new
analysis settings, and then open it back, though I am not sure of
close/open are allowed in bonsai...****

On Wed, Mar 28, 2012 at 9:43 AM, Nick Dunn nick@nick-dunn.co.uk wrote:**
**

I am looking at using a hosted ES service (Bonsai.io) which offers an
index per customer, pre-created. Usually I create a config file
containing my custom analyzers and send this when creating an index,
however because Bonsai provide the index already, this method will not
work.

I've read the docs for the Updating Settings API for an index but it
seems only certain settings can be modified this way, and not
analyzers.

What are my options here? I've not heard of a way to update index
custom analyzers without recreating an index, but perhaps I've missed
something somewhere! Can I send custom analyzer config another way?****




(David Pilato) #8

Just for understand, is there any technical reason that we can not update
all settings on a closed index ?
I can understand for an opened index but I don't understand why there is
this restriction ?

My other question is : can I find somewhere in ES source code a method that
I can use to parse (precheck) settings on my side before trying to push to
ES settings that couldn't be applied ?

Thanks,
David.

Le 3 avril 2012 à 16:10, Shay Banon kimchy@gmail.com a écrit :

Yea, only the list mentioned in the update settings can be applied
dynamically, while other settings can be applied on a closed index. The
number of replicas can be updated on a live index though...

On Sun, Apr 1, 2012 at 10:49 PM, David Pilato < david@pilato.fr
mailto:david@pilato.fr > wrote:

Hi Shay,

I think I understood where the error came from:
https://gist.github.com/2278029 https://gist.github.com/2278029

I cannot update number of replicas on a closed index.

So some properties have to be updated on a closed index and others on a
live index.

My use case is for the Spring factory.

I want to update settings but I don’t know what settings will set the user.

Is there a way to analyze the settings json file and split it in two
settings files : one to be applied live and another to be applied closed ?

Do you have any class or method in ES that I can use for that ?

Or do I have to parse all the properties to find if one of them is in this
list :
http://www.elasticsearch.org/guide/reference/api/admin-indices-update-settings.html
http://www.elasticsearch.org/guide/reference/api/admin-indices-update-settings.html
so it has to be applied on a live index ?

Is there any reason that you can’t apply all properties on a closed index ?
Should I raise an issue ?

Thanks

David.

De : elasticsearch@googlegroups.com mailto:elasticsearch@googlegroups.com
[mailto: elasticsearch@googlegroups.com
mailto:elasticsearch@googlegroups.com ] De la part de Shay Banon
Envoyé : samedi 31 mars 2012 23:02

À : elasticsearch@googlegroups.com mailto:elasticsearch@googlegroups.com
Objet : Re: Can I update analyzer settings without recreating the index?

Thats strange..., can you gist hte full failure? This works on 0.19:

curl -XPUT localhost:9200/test

curl -XPOST localhost:9200/test/_close

curl -XPUT localhost:9200/test/_settings -d '{

"index" : {

    "something" : "value"

}

}'

On Fri, Mar 30, 2012 at 11:58 PM, David Pilato < david@pilato.fr
mailto:david@pilato.fr > wrote:

Hi Shay,

Just to be sure to understand this.

If we want to merge settings (replication), we should (must ?) not close
the index before sending an update settings.

But if we want to add a new analyzer, we must close the index before.

Is that it ?

I tried this in Java :

         // Before merging, we have to close the index

         CloseIndexRequestBuilder cirb = client

.admin().indices().prepareClose(index);

         CloseIndexResponse closeIndexResponse =

cirb.execute().actionGet();

         if (!closeIndexResponse.acknowledged()) throw new Exception(

"Could not close index [" +index+ "]." );

         UpdateSettingsRequestBuilder usrb = client

.admin().indices().prepareUpdateSettings(index);

         // If there are settings for this index, we use it. If not,

using Elasticsearch defaults.

         String source = readIndexSettings(index);

         if (source != null ) {

                if ( logger .isTraceEnabled()) logger .trace( "Found

settings for index " +index+ " : " + source);

                usrb.setSettings(source);

         }



         usrb.execute().actionGet();

But it failed at the last line with a
org.elasticsearch.indices.IndexMissingException: [twitter] missing

If I remove the closeIndex lines, I don’t get the exception but I get the
following warning :

22:56:48,843 WARN [metadata] [junit.node.transport] [twitter] ignoring non
dynamic index level settings for open indices:
[index.analysis.analyzer.type]

How to close an index and update settings in java ?

Thanks

David.

De : elasticsearch@googlegroups.com mailto:elasticsearch@googlegroups.com
[mailto: elasticsearch@googlegroups.com
mailto:elasticsearch@googlegroups.com ] De la part de Shay Banon
Envoyé : mercredi 28 mars 2012 10:55
À : elasticsearch@googlegroups.com mailto:elasticsearch@googlegroups.com
Objet : Re: Can I update analyzer settings without recreating the index?

You can't change analyzer config on a "live" index using update settings,
you can though close the index, send an update settings with the new
analysis settings, and then open it back, though I am not sure of
close/open are allowed in bonsai...

On Wed, Mar 28, 2012 at 9:43 AM, Nick Dunn < nick@nick-dunn.co.uk
mailto:nick@nick-dunn.co.uk > wrote:

I am looking at using a hosted ES service (Bonsai.io) which offers an
index per customer, pre-created. Usually I create a config file
containing my custom analyzers and send this when creating an index,
however because Bonsai provide the index already, this method will not
work.

I've read the docs for the Updating Settings API for an index but it
seems only certain settings can be modified this way, and not
analyzers.

What are my options here? I've not heard of a way to update index
custom analyzers without recreating an index, but perhaps I've missed
something somewhere! Can I send custom analyzer config another way?

--
David Pilato
http://dev.david.pilato.fr/
Twitter : @dadoonet


(Shay Banon) #9

On Tue, Apr 3, 2012 at 6:25 PM, david@pilato.fr david@pilato.fr wrote:

**

Just for understand, is there any technical reason that we can not update
all settings on a closed index ?

I can understand for an opened index but I don't understand why there is
this restriction ?

You can update any setting on a closed index, there ins't a restriction
there.

My other question is : can I find somewhere in ES source code a method
that I can use to parse (precheck) settings on my side before trying to
push to ES settings that couldn't be applied ?

There isn't really... . Its spread in each component that reads its own
relevant setting.

Thanks,

David.

Le 3 avril 2012 à 16:10, Shay Banon kimchy@gmail.com a écrit :

Yea, only the list mentioned in the update settings can be applied
dynamically, while other settings can be applied on a closed index. The
number of replicas can be updated on a live index though...

On Sun, Apr 1, 2012 at 10:49 PM, David Pilato < david@pilato.fr > wrote:

Hi Shay,

I think I understood where the error came from:
https://gist.github.com/2278029

I cannot update number of replicas on a closed index.

So some properties have to be updated on a closed index and others on a
live index.

My use case is for the Spring factory.

I want to update settings but I don’t know what settings will set the
user.

Is there a way to analyze the settings json file and split it in two
settings files : one to be applied live and another to be applied closed ?

Do you have any class or method in ES that I can use for that ?

Or do I have to parse all the properties to find if one of them is in
this list :
http://www.elasticsearch.org/guide/reference/api/admin-indices-update-settings.htmlso it has to be applied on a live index ?

Is there any reason that you can’t apply all properties on a closed index
? Should I raise an issue ?

Thanks

David.

À : elasticsearch@googlegroups.com
Objet : Re: Can I update analyzer settings without recreating the
index?

Thats strange..., can you gist hte full failure? This works on 0.19:

curl -XPUT localhost:9200/test

curl -XPOST localhost:9200/test/_close

curl -XPUT localhost:9200/test/_settings -d '{

"index" : {

    "something" : "value"

}

}'

On Fri, Mar 30, 2012 at 11:58 PM, David Pilato < david@pilato.fr > wrote:

Hi Shay,

Just to be sure to understand this.

If we want to merge settings (replication), we should (must ?) not close
the index before sending an update settings.

But if we want to add a new analyzer, we must close the index before.

Is that it ?

I tried this in Java :

         // Before merging, we have to close the index

          CloseIndexRequestBuilder cirb = client.admin().indices().prepareClose(index);

         CloseIndexResponse closeIndexResponse =

cirb.execute().actionGet();

          * if * (!closeIndexResponse.acknowledged()) * throw * * new
  • Exception( "Could not close index [" +index+ "]." );

            UpdateSettingsRequestBuilder usrb = client.admin().indices().prepareUpdateSettings(index);
    
    
    
            // If there are settings for this index, we use it. If not,
    

using Elasticsearch defaults.

         String source = readIndexSettings(index);

          * if * (source != * null * ) {

                 * if * ( logger .isTraceEnabled()) logger .trace( "Found

settings for index " +index+ " : " + source);

                usrb.setSettings(source);

         }



         usrb.execute().actionGet();

But it failed at the last line with a
org.elasticsearch.indices.IndexMissingException: [twitter] missing

If I remove the closeIndex lines, I don’t get the exception but I get the
following warning :

22:56:48,843 WARN [metadata] [junit.node.transport] [twitter] ignoring
non dynamic index level settings for open indices:
[index.analysis.analyzer.type]

How to close an index and update settings in java ?

Thanks

David.

You can't change analyzer config on a "live" index using update settings,
you can though close the index, send an update settings with the new
analysis settings, and then open it back, though I am not sure of
close/open are allowed in bonsai...

On Wed, Mar 28, 2012 at 9:43 AM, Nick Dunn < nick@nick-dunn.co.uk >
wrote:

I am looking at using a hosted ES service (Bonsai.io) which offers an
index per customer, pre-created. Usually I create a config file
containing my custom analyzers and send this when creating an index,
however because Bonsai provide the index already, this method will not
work.

I've read the docs for the Updating Settings API for an index but it
seems only certain settings can be modified this way, and not
analyzers.

What are my options here? I've not heard of a way to update index
custom analyzers without recreating an index, but perhaps I've missed
something somewhere! Can I send custom analyzer config another way?

--
David Pilato
http://dev.david.pilato.fr/
Twitter : @dadoonet


(David Pilato) #10

Heya,

So why did my update of "number of replicas" on the closed index fails with
that error ?
Caused by: org.elasticsearch.indices.IndexMissingException: [twitter]
missing
at
org.elasticsearch.cluster.routing.RoutingTable$Builder.updateNumberOfReplicas(RoutingTable.java:314)

It should work on a closed index as you wrote. Is it a bug or am I doing
something weird ?

David.

Le 4 avril 2012 à 17:25, Shay Banon kimchy@gmail.com a écrit :

On Tue, Apr 3, 2012 at 6:25 PM, david@pilato.fr mailto:david@pilato.fr <
david@pilato.fr mailto:david@pilato.fr > wrote:

Just for understand, is there any technical reason that we can not update
all settings on a closed index ?
I can understand for an opened index but I don't understand why there is
this restriction ?

You can update any setting on a closed index, there ins't a restriction
there.

My other question is : can I find somewhere in ES source code a method
that
I can use to parse (precheck) settings on my side before trying to push to
ES settings that couldn't be applied ?

There isn't really... . Its spread in each component that reads its own
relevant setting.

Thanks,
David.

Le 3 avril 2012 à 16:10, Shay Banon < kimchy@gmail.com
mailto:kimchy@gmail.com > a écrit :

Yea, only the list mentioned in the update settings can be applied
dynamically, while other settings can be applied on a closed index. The
number of replicas can be updated on a live index though...
On Sun, Apr 1, 2012 at 10:49 PM, David Pilato < david@pilato.fr
mailto:david@pilato.fr > wrote:

Hi Shay,

I think I understood where the error came from:
https://gist.github.com/2278029 https://gist.github.com/2278029

I cannot update number of replicas on a closed index.

So some properties have to be updated on a closed index and others on a
live index.

My use case is for the Spring factory.

I want to update settings but I don’t know what settings will set the
user.

Is there a way to analyze the settings json file and split it in two
settings files : one to be applied live and another to be applied closed
?

Do you have any class or method in ES that I can use for that ?

Or do I have to parse all the properties to find if one of them is in
this
list :
http://www.elasticsearch.org/guide/reference/api/admin-indices-update-settings.html
http://www.elasticsearch.org/guide/reference/api/admin-indices-update-settings.html
so it has to be applied on a live index ?

Is there any reason that you can’t apply all properties on a closed
index ?
Should I raise an issue ?

Thanks

David.

De : elasticsearch@googlegroups.com
mailto:elasticsearch@googlegroups.com
[mailto: elasticsearch@googlegroups.com
mailto:elasticsearch@googlegroups.com ] De la part de Shay Banon
Envoyé : samedi 31 mars 2012 23:02

À : elasticsearch@googlegroups.com
mailto:elasticsearch@googlegroups.com
Objet : Re: Can I update analyzer settings without recreating the index?

Thats strange..., can you gist hte full failure? This works on 0.19:

curl -XPUT localhost:9200/test

curl -XPOST localhost:9200/test/_close

curl -XPUT localhost:9200/test/_settings -d '{

"index" : {

    "something" : "value"

}

}'

On Fri, Mar 30, 2012 at 11:58 PM, David Pilato < david@pilato.fr
mailto:david@pilato.fr > wrote:

Hi Shay,

Just to be sure to understand this.

If we want to merge settings (replication), we should (must ?) not close
the index before sending an update settings.

But if we want to add a new analyzer, we must close the index before.

Is that it ?

I tried this in Java :

         // Before merging, we have to close the index

         CloseIndexRequestBuilder cirb = client

.admin().indices().prepareClose(index);

         CloseIndexResponse closeIndexResponse =

cirb.execute().actionGet();

         if (!closeIndexResponse.acknowledged()) throw new

Exception(
"Could not close index [" +index+ "]." );

         UpdateSettingsRequestBuilder usrb = client

.admin().indices().prepareUpdateSettings(index);

         // If there are settings for this index, we use it. If not,

using Elasticsearch defaults.

         String source = readIndexSettings(index);

         if (source != null ) {

                if ( logger .isTraceEnabled()) logger .trace( "Found

settings for index " +index+ " : " + source);

                usrb.setSettings(source);

         }



         usrb.execute().actionGet();

But it failed at the last line with a
org.elasticsearch.indices.IndexMissingException: [twitter] missing

If I remove the closeIndex lines, I don’t get the exception but I get
the
following warning :

22:56:48,843 WARN [metadata] [junit.node.transport] [twitter] ignoring
non
dynamic index level settings for open indices:
[index.analysis.analyzer.type]

How to close an index and update settings in java ?

Thanks

David.

De : elasticsearch@googlegroups.com
mailto:elasticsearch@googlegroups.com
[mailto: elasticsearch@googlegroups.com
mailto:elasticsearch@googlegroups.com ] De la part de Shay Banon
Envoyé : mercredi 28 mars 2012 10:55
À : elasticsearch@googlegroups.com
mailto:elasticsearch@googlegroups.com
Objet : Re: Can I update analyzer settings without recreating the index?

You can't change analyzer config on a "live" index using update
settings,
you can though close the index, send an update settings with the new
analysis settings, and then open it back, though I am not sure of
close/open are allowed in bonsai...

On Wed, Mar 28, 2012 at 9:43 AM, Nick Dunn < nick@nick-dunn.co.uk
mailto:nick@nick-dunn.co.uk > wrote:

I am looking at using a hosted ES service (Bonsai.io) which offers an
index per customer, pre-created. Usually I create a config file
containing my custom analyzers and send this when creating an index,
however because Bonsai provide the index already, this method will not
work.

I've read the docs for the Updating Settings API for an index but it
seems only certain settings can be modified this way, and not
analyzers.

What are my options here? I've not heard of a way to update index
custom analyzers without recreating an index, but perhaps I've missed
something somewhere! Can I send custom analyzer config another way?

--
David Pilato
http://dev.david.pilato.fr/ http://dev.david.pilato.fr/
Twitter : @dadoonet

--
David Pilato
http://dev.david.pilato.fr/
Twitter : @dadoonet


(Shay Banon) #11

It might just be the number of replicas setting, where it should handle
things better in case of a closed index. All other settings should work.
Open an issue for the number of replica settings update problem on a closed
index.

On Wed, Apr 4, 2012 at 6:43 PM, david@pilato.fr david@pilato.fr wrote:

**

Heya,

So why did my update of "number of replicas" on the closed index fails
with that error ?
Caused by: org.elasticsearch.indices.IndexMissingException: [twitter]
missing
at
org.elasticsearch.cluster.routing.RoutingTable$Builder.updateNumberOfReplicas(RoutingTable.java:314)

It should work on a closed index as you wrote. Is it a bug or am I doing
something weird ?

David.

Le 4 avril 2012 à 17:25, Shay Banon kimchy@gmail.com a écrit :

On Tue, Apr 3, 2012 at 6:25 PM, david@pilato.fr < david@pilato.fr > wrote:

Just for understand, is there any technical reason that we can not
update all settings on a closed index ?

I can understand for an opened index but I don't understand why there is
this restriction ?

You can update any setting on a closed index, there ins't a restriction
there.

My other question is : can I find somewhere in ES source code a method
that I can use to parse (precheck) settings on my side before trying to
push to ES settings that couldn't be applied ?

There isn't really... . Its spread in each component that reads its own
relevant setting.

Thanks,

David.

Le 3 avril 2012 à 16:10, Shay Banon < kimchy@gmail.com > a écrit :

Yea, only the list mentioned in the update settings can be applied
dynamically, while other settings can be applied on a closed index. The
number of replicas can be updated on a live index though...
On Sun, Apr 1, 2012 at 10:49 PM, David Pilato < david@pilato.fr > wrote:

Hi Shay,

I think I understood where the error came from:
https://gist.github.com/2278029

I cannot update number of replicas on a closed index.

So some properties have to be updated on a closed index and others on a
live index.

My use case is for the Spring factory.

I want to update settings but I don’t know what settings will set the user.

Is there a way to analyze the settings json file and split it in two
settings files : one to be applied live and another to be applied closed ?

Do you have any class or method in ES that I can use for that ?

Or do I have to parse all the properties to find if one of them is in
this list :
http://www.elasticsearch.org/guide/reference/api/admin-indices-update-settings.htmlso it has to be applied on a live index ?

Is there any reason that you can’t apply all properties on a closed index
? Should I raise an issue ?

Thanks

David.

À : elasticsearch@googlegroups.com
Objet : Re: Can I update analyzer settings without recreating the
index?

Thats strange..., can you gist hte full failure? This works on 0.19:

curl -XPUT localhost:9200/test

curl -XPOST localhost:9200/test/_close

curl -XPUT localhost:9200/test/_settings -d '{

"index" : {

    "something" : "value"

}

}'

On Fri, Mar 30, 2012 at 11:58 PM, David Pilato < david@pilato.fr > wrote:

Hi Shay,

Just to be sure to understand this.

If we want to merge settings (replication), we should (must ?) not close
the index before sending an update settings.

But if we want to add a new analyzer, we must close the index before.

Is that it ?

I tried this in Java :

         // Before merging, we have to close the index

          CloseIndexRequestBuilder cirb = client.admin().indices().prepareClose(index);

         CloseIndexResponse closeIndexResponse =

cirb.execute().actionGet();

          * if * (!closeIndexResponse.acknowledged()) * throw * * new
  • Exception( "Could not close index [" +index+ "]." );

            UpdateSettingsRequestBuilder usrb = client.admin().indices().prepareUpdateSettings(index);
    
    
    
            // If there are settings for this index, we use it. If not,
    

using Elasticsearch defaults.

         String source = readIndexSettings(index);

          * if * (source != * null * ) {

                 * if * ( logger .isTraceEnabled()) logger .trace( "Found

settings for index " +index+ " : " + source);

                usrb.setSettings(source);

         }



         usrb.execute().actionGet();

But it failed at the last line with a
org.elasticsearch.indices.IndexMissingException: [twitter] missing

If I remove the closeIndex lines, I don’t get the exception but I get the
following warning :

22:56:48,843 WARN [metadata] [junit.node.transport] [twitter] ignoring
non dynamic index level settings for open indices:
[index.analysis.analyzer.type]

How to close an index and update settings in java ?

Thanks

David.

You can't change analyzer config on a "live" index using update settings,
you can though close the index, send an update settings with the new
analysis settings, and then open it back, though I am not sure of
close/open are allowed in bonsai...

On Wed, Mar 28, 2012 at 9:43 AM, Nick Dunn < nick@nick-dunn.co.uk >
wrote:

I am looking at using a hosted ES service (Bonsai.io) which offers an
index per customer, pre-created. Usually I create a config file
containing my custom analyzers and send this when creating an index,
however because Bonsai provide the index already, this method will not
work.

I've read the docs for the Updating Settings API for an index but it
seems only certain settings can be modified this way, and not
analyzers.

What are my options here? I've not heard of a way to update index
custom analyzers without recreating an index, but perhaps I've missed
something somewhere! Can I send custom analyzer config another way?

--
David Pilato
http://dev.david.pilato.fr/
Twitter : @dadoonet

--
David Pilato
http://dev.david.pilato.fr/
Twitter : @dadoonet


(David Pilato) #12

Thanks. Opened here :

David

De : elasticsearch@googlegroups.com [mailto:elasticsearch@googlegroups.com]
De la part de Shay Banon
Envoyé : mercredi 4 avril 2012 17:46
À : elasticsearch@googlegroups.com
Objet : Re: Can I update analyzer settings without recreating the index?

It might just be the number of replicas setting, where it should handle
things better in case of a closed index. All other settings should work.
Open an issue for the number of replica settings update problem on a closed
index.

On Wed, Apr 4, 2012 at 6:43 PM, david@pilato.fr david@pilato.fr wrote:

Heya,

So why did my update of "number of replicas" on the closed index fails with
that error ?

Caused by: org.elasticsearch.indices.IndexMissingException: [twitter]
missing

    at

org.elasticsearch.cluster.routing.RoutingTable$Builder.updateNumberOfReplica
s(RoutingTable.java:314)

It should work on a closed index as you wrote. Is it a bug or am I doing
something weird ?

David.

Le 4 avril 2012 à 17:25, Shay Banon kimchy@gmail.com a écrit :

On Tue, Apr 3, 2012 at 6:25 PM, david@pilato.fr < david@pilato.fr > wrote:

Just for understand, is there any technical reason that we can not update
all settings on a closed index ?

I can understand for an opened index but I don't understand why there is
this restriction ?

You can update any setting on a closed index, there ins't a restriction
there.

My other question is : can I find somewhere in ES source code a method that
I can use to parse (precheck) settings on my side before trying to push to
ES settings that couldn't be applied ?

There isn't really... . Its spread in each component that reads its own
relevant setting.

Thanks,

David.

Le 3 avril 2012 à 16:10, Shay Banon < kimchy@gmail.com > a écrit :

Yea, only the list mentioned in the update settings can be applied
dynamically, while other settings can be applied on a closed index. The
number of replicas can be updated on a live index though...

On Sun, Apr 1, 2012 at 10:49 PM, David Pilato < david@pilato.fr > wrote:

Hi Shay,

I think I understood where the error came from:

I cannot update number of replicas on a closed index.

So some properties have to be updated on a closed index and others on a live
index.

My use case is for the Spring factory.

I want to update settings but I don’t know what settings will set the user.

Is there a way to analyze the settings json file and split it in two
settings files : one to be applied live and another to be applied closed ?

Do you have any class or method in ES that I can use for that ?

Or do I have to parse all the properties to find if one of them is in this
list :
http://www.elasticsearch.org/guide/reference/api/admin-indices-update-settin
gs.html so it has to be applied on a live index ?

Is there any reason that you can’t apply all properties on a closed index ?
Should I raise an issue ?

Thanks

David.

De : elasticsearch@googlegroups.com [mailto: elasticsearch@googlegroups.com
] De la part de Shay Banon

Envoyé : samedi 31 mars 2012 23:02

À : elasticsearch@googlegroups.com
Objet : Re: Can I update analyzer settings without recreating the index?

Thats strange..., can you gist hte full failure? This works on 0.19:

curl -XPUT localhost:9200/test

curl -XPOST localhost:9200/test/_close

curl -XPUT localhost:9200/test/_settings -d '{

"index" : {

    "something" : "value"

}

}'

On Fri, Mar 30, 2012 at 11:58 PM, David Pilato < david@pilato.fr > wrote:

Hi Shay,

Just to be sure to understand this.

If we want to merge settings (replication), we should (must ?) not close the
index before sending an update settings.

But if we want to add a new analyzer, we must close the index before.

Is that it ?

I tried this in Java :

         // Before merging, we have to close the index 

         CloseIndexRequestBuilder cirb = client

.admin().indices().prepareClose(index);

         CloseIndexResponse closeIndexResponse =

cirb.execute().actionGet();

         if (!closeIndexResponse.acknowledged()) throw new Exception(

"Could not close index [" +index+ "]." );

         UpdateSettingsRequestBuilder usrb = client

.admin().indices().prepareUpdateSettings(index);

         // If there are settings for this index, we use it. If not,

using Elasticsearch defaults.

         String source = readIndexSettings(index); 

         if (source != null ) { 

                if ( logger .isTraceEnabled()) logger .trace( "Found

settings for index " +index+ " : " + source);

                usrb.setSettings(source); 

         } 

         

         usrb.execute().actionGet(); 

But it failed at the last line with a
org.elasticsearch.indices.IndexMissingException: [twitter] missing

If I remove the closeIndex lines, I don’t get the exception but I get the
following warning :

22:56:48,843 WARN [metadata] [junit.node.transport] [twitter] ignoring non
dynamic index level settings for open indices:
[index.analysis.analyzer.type]

How to close an index and update settings in java ?

Thanks

David.

De : elasticsearch@googlegroups.com [mailto: elasticsearch@googlegroups.com
] De la part de Shay Banon
Envoyé : mercredi 28 mars 2012 10:55
À : elasticsearch@googlegroups.com
Objet : Re: Can I update analyzer settings without recreating the index?

You can't change analyzer config on a "live" index using update settings,
you can though close the index, send an update settings with the new
analysis settings, and then open it back, though I am not sure of close/open
are allowed in bonsai...

On Wed, Mar 28, 2012 at 9:43 AM, Nick Dunn < nick@nick-dunn.co.uk > wrote:

I am looking at using a hosted ES service (Bonsai.io) which offers an
index per customer, pre-created. Usually I create a config file
containing my custom analyzers and send this when creating an index,
however because Bonsai provide the index already, this method will not
work.

I've read the docs for the Updating Settings API for an index but it
seems only certain settings can be modified this way, and not
analyzers.

What are my options here? I've not heard of a way to update index
custom analyzers without recreating an index, but perhaps I've missed
something somewhere! Can I send custom analyzer config another way?

--
David Pilato
http://dev.david.pilato.fr/
Twitter : @dadoonet

--
David Pilato
http://dev.david.pilato.fr/
Twitter : @dadoonet


(system) #13