Replace analyzer on _all field

Elastic Stack Elasticsearch
1

Hi,

I have a lot of data in my ES cluster. I'm using the _all field for general
search in my documents, but I recently found out that the standard analyzer
doesn't tokenize words with dots inside (ex: www.google.com - not found
when searching for "google").

So I want to replace the standard analyzer with the simple analyzer for the
_all field.

How do I do that? Please let me know all the steps - replacing,
re-indexing, etc.

Thanks,
Lasse

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CADERWXrJFxvv3wRC_COVmEjkKnOJ4zc7F0bveSs0AK_jMRoMrg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

2

First you should not use _all but prefer copy_to feature which gives more flexibility. See: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-core-types.html#copy-to

That said, changing an analyzer one by another requires to reindex your documents.
So create a new index (or drop the old one and create it again) with new settings/mappings.
Reindex.

HTH

--
David Pilato | Technical Advocate | elasticsearch.com
david.pilato@elasticsearch.com
@dadoonet | @elasticsearchfr | @scrutmydocs

Le 9 octobre 2014 à 09:46:23, Lasse Schou (lasseschou@gmail.com) a écrit:

Hi,

I have a lot of data in my ES cluster. I'm using the _all field for general search in my documents, but I recently found out that the standard analyzer doesn't tokenize words with dots inside (ex: www.google.com - not found when searching for "google").

So I want to replace the standard analyzer with the simple analyzer for the _all field.

How do I do that? Please let me know all the steps - replacing, re-indexing, etc.

Thanks,
Lasse

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CADERWXrJFxvv3wRC_COVmEjkKnOJ4zc7F0bveSs0AK_jMRoMrg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.543656d3.5577f8e1.bc27%40MacBook-Air-de-David.local.
For more options, visit https://groups.google.com/d/optout.

3

Hi,

The copy_to feature looks nice, but for now I'm happy using the _all
feature.

However I don't think my question was fully answered. When creating the new
index, how do I change the _all analyzer? Are you saying that I need to
change the analyzer on each of the fields I've enabled _all on?

Thanks again,
Lasse

On Thursday, October 9, 2014 11:35:30 AM UTC+2, David Pilato wrote:

First you should not use _all but prefer copy_to feature which gives more
flexibility. See:
Elasticsearch Platform — Find real-time answers at scale | Elastic

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-core-types.html#copy-to
T
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-core-types.html#copy-tohat
said, changing an analyzer one by another requires to reindex your
documents.
So create a new index (or drop the old one and create it again) with new
settings/mappings.
Reindex.

HTH

--
David Pilato | Technical Advocate | elasticsearch.com
http://elasticsearch.com
david....@elasticsearch.com <javascript:>
@dadoonet https://twitter.com/dadoonet | @elasticsearchfr
https://twitter.com/elasticsearchfr | @scrutmydocs
http://twitter.com/scrutmydocs
https://twitter.com/scrutmydocs

Le 9 octobre 2014 à 09:46:23, Lasse Schou (lasse...@gmail.com
<javascript:>) a écrit:

Hi,

I have a lot of data in my ES cluster. I'm using the _all field for
general search in my documents, but I recently found out that the standard
analyzer doesn't tokenize words with dots inside (ex: www.google.com -
not found when searching for "google").

So I want to replace the standard analyzer with the simple analyzer for
the _all field.

How do I do that? Please let me know all the steps - replacing,
re-indexing, etc.

Thanks,
Lasse

You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CADERWXrJFxvv3wRC_COVmEjkKnOJ4zc7F0bveSs0AK_jMRoMrg%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CADERWXrJFxvv3wRC_COVmEjkKnOJ4zc7F0bveSs0AK_jMRoMrg%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/468c37f1-456e-4c7a-88e1-f387734f26fe%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

4

Lasse,

The copy_to feature looks nice, but for now I'm happy using the _all
feature.

Glad you're happy with the _all feature.

However I don't think my question was fully answered. When creating the
new index, how do I change the _all analyzer? Are you saying that I need to
change the analyzer on each of the fields I've enabled _all on?

Ok, so not so happy after all! :slight_smile:

This link
http://elasticsearch-users.115913.n3.nabble.com/Specifying-analyzer-for-all-field-td3851732.html
contains a good description of the _all field. I admit that it's also
confusing to me. But I always now disable the _all field. Then:

  1. For a general directory query application, I lock down Elasticsearch to
    disable the _all field, prevent unmapped fields from being added, prevent
    unmapped types from being added, and prevent indexes from being
    automatically created with the addition of the first document.

  2. For processing logs using the ELK stack, I disable the _all field and
    specify the use of the message field as the default. Then all of my
    logstash configurations use the message field but do not modify it. Then I
    create the mapping I wish for the message field and all is well (no pun
    intended!).

Brian

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/b738433d-e81f-4355-8fca-404b254a81ec%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

5

Hi,

It's still not 100% clear to me how to replace the _all analyzer from the
standard analyzer to the simple analyzer. In the link you wrote I see this:

curl -X PUT "localhost:9200/indexName" -d '{ "settings" : { "index" : {
"number_of_shards" : 2, "number_of_replicas" : 1 },
"analysis" : {"analyzer":{"my_analyzer" : {
"tokenizer" : "keyword" }}}
}}'

But can anybody let me know exactly how to change the _all analyzer from
standard to simple?

Is it like this?

curl -X PUT "localhost:9200/indexName" -d '{ "settings" : { "index" : {
"number_of_shards" : 2, "number_of_replicas" : 1 },
"analysis" : {"analyzer":{"simple" }}
}}'

Thanks!

2014-10-29 22:14 GMT+01:00 Brian brian.from.fl@gmail.com:

Lasse,

The copy_to feature looks nice, but for now I'm happy using the _all
feature.

Glad you're happy with the _all feature.

However I don't think my question was fully answered. When creating the
new index, how do I change the _all analyzer? Are you saying that I need to
change the analyzer on each of the fields I've enabled _all on?

Ok, so not so happy after all! :slight_smile:

This link
http://elasticsearch-users.115913.n3.nabble.com/Specifying-analyzer-for-all-field-td3851732.html
contains a good description of the _all field. I admit that it's also
confusing to me. But I always now disable the _all field. Then:

  1. For a general directory query application, I lock down Elasticsearch to
    disable the _all field, prevent unmapped fields from being added, prevent
    unmapped types from being added, and prevent indexes from being
    automatically created with the addition of the first document.

  2. For processing logs using the ELK stack, I disable the _all field and
    specify the use of the message field as the default. Then all of my
    logstash configurations use the message field but do not modify it. Then I
    create the mapping I wish for the message field and all is well (no pun
    intended!).

Brian

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/wq4iD5PQV2M/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/b738433d-e81f-4355-8fca-404b254a81ec%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/b738433d-e81f-4355-8fca-404b254a81ec%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CADERWXovugQgi8mFry-C8WTOYfEXOyTh6gAW_mm%2B9dh-wJQN3Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.