Removing index settings

Elastic Stack Elasticsearch
1

I have a configuration yml file that I am using to specify the analyzers
that I am using to index our documents. This is working fine, however if I
remove settings from the yml file, after updating elasticsearch, they are
still there.

Is there a mechanism for removing individual properties from an indexes
settings?

As an example, if I have set a number of filters, I see properties like so:
"index.analysis.analyzer.my_analyzer.filter.0" : "my_stop_words",
"index.analysis.analyzer.my_analyzer.filter.1" : "my_stemmer",

If I remove my_stemmer from the set of filters and update my settings the
entry "index.analysis.analyzer.my_analyzer.filter.1" remains.

Is there a way to remove these?

Thanks
Stephen

2

Hi Stephen

On Wed, 2012-07-18 at 17:38 -0700, Stephen Beeson wrote:

I have a configuration yml file that I am using to specify the
analyzers that I am using to index our documents. This is working
fine, however if I remove settings from the yml file, after updating
elasticsearch, they are still there.

Any index settings in a config file are just used as a template for a
new index. They aren't read and applied (eg) every time you restart.

Also, you can't change analyzers on an existing index* so you have to
create a new index and reindex your data from old to new.

  • You actually can change analyzers on an existing index by closing the
    index, making your changes then opening the index. But: this could be
    used for adding analyzers, but should not be used for changing them, as
    your existing data would be indexed incorrectly.

clint

Is there a mechanism for removing individual properties from an
indexes settings?

As an example, if I have set a number of filters, I see properties
like so:
"index.analysis.analyzer.my_analyzer.filter.0" :
"my_stop_words",
"index.analysis.analyzer.my_analyzer.filter.1" : "my_stemmer",

If I remove my_stemmer from the set of filters and update my settings
the entry "index.analysis.analyzer.my_analyzer.filter.1" remains.

Is there a way to remove these?

Thanks
Stephen

3

Thanks for the response Clint.
I've got a few scripts that update various settings, such as the mappings
and the analyzers when we deploy. The one that updates the analyzers does
close and open the index so that part of the updates is working.

For our needs, it's not critical that the documents are indexed exactly.
We're ok if we change the indexing settings and then reindex over time. We
also don't expect to be updating them too much after release.

I'm currently updating analyzer properties successfully, but would be great
if I could remove them when needed.

Stephen

On Thursday, July 19, 2012 5:56:15 PM UTC+10, Clinton Gormley wrote:

Hi Stephen

On Wed, 2012-07-18 at 17:38 -0700, Stephen Beeson wrote:

I have a configuration yml file that I am using to specify the
analyzers that I am using to index our documents. This is working
fine, however if I remove settings from the yml file, after updating
elasticsearch, they are still there.

Any index settings in a config file are just used as a template for a
new index. They aren't read and applied (eg) every time you restart.

Also, you can't change analyzers on an existing index* so you have to
create a new index and reindex your data from old to new.

  • You actually can change analyzers on an existing index by closing the
    index, making your changes then opening the index. But: this could be
    used for adding analyzers, but should not be used for changing them, as
    your existing data would be indexed incorrectly.

clint

Is there a mechanism for removing individual properties from an
indexes settings?

As an example, if I have set a number of filters, I see properties
like so:
"index.analysis.analyzer.my_analyzer.filter.0" :
"my_stop_words",
"index.analysis.analyzer.my_analyzer.filter.1" : "my_stemmer",

If I remove my_stemmer from the set of filters and update my settings
the entry "index.analysis.analyzer.my_analyzer.filter.1" remains.

Is there a way to remove these?

Thanks
Stephen

4

Hi Stephen

On Thu, 2012-07-19 at 01:03 -0700, Stephen Beeson wrote:

Thanks for the response Clint.
I've got a few scripts that update various settings, such as the
mappings and the analyzers when we deploy. The one that updates the
analyzers does close and open the index so that part of the updates is
working.

For our needs, it's not critical that the documents are indexed
exactly. We're ok if we change the indexing settings and then reindex
over time. We also don't expect to be updating them too much after
release.

That's not a great approach - it'll bite you. Changing the analyzers may
make your existing data unsearchable (depending on the changes).

You might consider creating a new index with the new settings, and
creating an alias that points to the old and new index. Then you can
search across both indices using the alias, but the correct analyzers
will be applied to each index

clint

I'm currently updating analyzer properties successfully, but would be
great if I could remove them when needed.

Stephen

On Thursday, July 19, 2012 5:56:15 PM UTC+10, Clinton Gormley wrote:
Hi Stephen

    On Wed, 2012-07-18 at 17:38 -0700, Stephen Beeson wrote: 
    > I have a configuration yml file that I am using to specify
    the 
    > analyzers that I am using to index our documents. This is
    working 
    > fine, however if I remove settings from the yml file, after
    updating 
    > elasticsearch, they are still there. 
    
    Any index settings in a config file are just used as a
    template for a 
    new index.  They aren't read and applied (eg) every time you
    restart. 
    
    Also, you can't change analyzers on an existing index* so you
    have to 
    create a new index and reindex your data from old to new. 
    
    * You actually can change analyzers on an existing index by
    closing the 
    index, making your changes then opening the index.  But: this
    could be 
    used for adding analyzers, but should not be used for changing
    them, as 
    your existing data would be indexed incorrectly. 
    
    clint 
    
    > 
    > 
    > Is there a mechanism for removing individual properties from
    an 
    > indexes settings? 
    > 
    > 
    > As an example, if I have set a number of filters, I see
    properties 
    > like so: 
    >       "index.analysis.analyzer.my_analyzer.filter.0" : 
    > "my_stop_words", 
    >       "index.analysis.analyzer.my_analyzer.filter.1" :
    "my_stemmer", 
    >   
    > If I remove my_stemmer from the set of filters and update my
    settings 
    > the entry "index.analysis.analyzer.my_analyzer.filter.1"
    remains. 
    > 
    > 
    > Is there a way to remove these? 
    > 
    > 
    > Thanks 
    > Stephen
5

I know this is an old thread, but it is most relevant to my similar
question. I have a situation where I have discovered some problems in my
analyzer configurations, to where I have a snowball tokenizer on the search
query, but no snowball being applied in a specific field with a custom
analyzer.
My analyzers were set up like this:

"default": {
    "type": "custom",
    "tokenizer": "standard",
    "filter": [
        "word_delimiter_filter",
        "standard",
        "lowercase",
        "stop",
        "snowball"
    ]
},


"ngram_index_analyzer": {
    "type": "custom",
    "tokenizer": "standard",
    "filter": [
        "standard",
        "lowercase",
        "ngram_filter"
    ]
}

The specific field using the ngram looks like this:

"keywords": {
    "index_name": "keyword",
    "type": "multi_field",
    "fields": {
        "keywords": {
            "index_analyzer": "ngram_index_analyzer",
            "type": "string"
        },
        "untouched": {
            "type": "string",
            "index": "not_analyzed"
        }
    }
},

I realize its going to look obviously wrong, which is why I am trying to
find the best way to fix it on a live production index

So searching for something like "body" was tokenizing to "bodi" and not
matching on the field using the ngram_index_analyzer. My thought was to
remove the "snowball" filter from the default analyzer. I discovered in
testing that you can't remove the field, but rather can only overwrite it
with a new value (which made me thing maybe I could just replace it with
another stop filter).But then the information in this thread did made me
realize that it would potentially screw up existing data tokenized via
snowball.

@Clinton, is that suggestion about creating another index with an adjusted
mapping, and using an alias relevant to my problem? Or is there a better
way to adjust for bad search analyzer settings on a live index?

On Thursday, July 19, 2012 12:38:09 PM UTC+12, Stephen Beeson wrote:

I have a configuration yml file that I am using to specify the analyzers
that I am using to index our documents. This is working fine, however if I
remove settings from the yml file, after updating elasticsearch, they are
still there.

Is there a mechanism for removing individual properties from an indexes
settings?

As an example, if I have set a number of filters, I see properties like so:
"index.analysis.analyzer.my_analyzer.filter.0" : "my_stop_words",
"index.analysis.analyzer.my_analyzer.filter.1" : "my_stemmer",

If I remove my_stemmer from the set of filters and update my settings the
entry "index.analysis.analyzer.my_analyzer.filter.1" remains.

Is there a way to remove these?

Thanks
Stephen

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.