Asciifolding analyzer

Elastic Stack Elasticsearch
1

Hi, I've done a lot of research on groups and ES guides on how to use this
analyzer but somehow it is not working.

Here's my elasticsearch.yml:

index:
analysis:
analyzer:
standard:
tokenizer: standard
type: standard
filter: [standard, lowercase, stop, asciifolding]

I based this link
http://www.elasticsearch.org/guide/reference/index-modules/analysis/ -
aliasing section

I already stopped ES, removed the data/ dir and indexed everything after
changing the file. Still not working.

My ES version is 1.17.0

What may be wrong?

Thanks,

--
Carlos Eduardo da Fonseca

2

You change the standard analyzer, this means that in the mapping, if you set
for a field explicitly to use the standard analyzer (set
analyzer="standard") then it will use it.

If you want to change the default analyzer, then replace the name standard
with default.

On Thu, Jul 28, 2011 at 1:22 AM, Carlos Eduardo da Fonseca <
eduardongu@gmail.com> wrote:

Hi, I've done a lot of research on groups and ES guides on how to use this
analyzer but somehow it is not working.

Here's my elasticsearch.yml:

index:
analysis:
analyzer:
standard:
tokenizer: standard
type: standard
filter: [standard, lowercase, stop, asciifolding]

I based this link
Elasticsearch Platform — Find real-time answers at scale | Elastic -
aliasing section

I already stopped ES, removed the data/ dir and indexed everything after
changing the file. Still not working.

My ES version is 1.17.0

What may be wrong?

Thanks,

--
Carlos Eduardo da Fonseca

3

Hi Shay, thanks for the answer.

I changed it to "default", restarted ES (cleaning data/) and reindexed, but
it doesn't worked...

elasticsearch.yml:

index:
analysis:
analyzer:
default:
tokenizer: standard
type: standard
filter: [standard, lowercase, stop, asciifolding]

On Thu, Jul 28, 2011 at 9:53 AM, Shay Banon kimchy@gmail.com wrote:

You change the standard analyzer, this means that in the mapping, if you
set for a field explicitly to use the standard analyzer (set
analyzer="standard") then it will use it.

If you want to change the default analyzer, then replace the name
standard with default.

On Thu, Jul 28, 2011 at 1:22 AM, Carlos Eduardo da Fonseca <
eduardongu@gmail.com> wrote:

Hi, I've done a lot of research on groups and ES guides on how to use this
analyzer but somehow it is not working.

Here's my elasticsearch.yml:

index:
analysis:
analyzer:
standard:
tokenizer: standard
type: standard
filter: [standard, lowercase, stop, asciifolding]

I based this link
Elasticsearch Platform — Find real-time answers at scale | Elastic -
aliasing section

I already stopped ES, removed the data/ dir and indexed everything after
changing the file. Still not working.

My ES version is 1.17.0

What may be wrong?

Thanks,

--
Carlos Eduardo da Fonseca

--
Carlos Eduardo da Fonseca

4

Gist an example then of why it does not work. One way to make sure it has
taked affect on indices created is to issue get settings and see if its
there.

On Thu, Jul 28, 2011 at 4:09 PM, Carlos Eduardo da Fonseca <
eduardongu@gmail.com> wrote:

Hi Shay, thanks for the answer.

I changed it to "default", restarted ES (cleaning data/) and reindexed, but
it doesn't worked...

elasticsearch.yml:

index:
analysis:
analyzer:
default:
tokenizer: standard
type: standard
filter: [standard, lowercase, stop, asciifolding]

On Thu, Jul 28, 2011 at 9:53 AM, Shay Banon kimchy@gmail.com wrote:

You change the standard analyzer, this means that in the mapping, if you
set for a field explicitly to use the standard analyzer (set
analyzer="standard") then it will use it.

If you want to change the default analyzer, then replace the name
standard with default.

On Thu, Jul 28, 2011 at 1:22 AM, Carlos Eduardo da Fonseca <
eduardongu@gmail.com> wrote:

Hi, I've done a lot of research on groups and ES guides on how to use
this analyzer but somehow it is not working.

Here's my elasticsearch.yml:

index:
analysis:
analyzer:
standard:
tokenizer: standard
type: standard
filter: [standard, lowercase, stop, asciifolding]

I based this link
Elasticsearch Platform — Find real-time answers at scale | Elastic -
aliasing section

I already stopped ES, removed the data/ dir and indexed everything after
changing the file. Still not working.

My ES version is 1.17.0

What may be wrong?

Thanks,

--
Carlos Eduardo da Fonseca

--
Carlos Eduardo da Fonseca

5

Ok, here's a gist with index settings and everything I did after creating
the index with settings via PUT:

6

I'm using this settings and asciifolding works:

/_search?q=title:botao* - OK
/_search?q=botao* - OK

You can remove the "my_delimiter", "porter_stem" and use these
settings, just write in your mappings for field-type "string" for
"index_analyzer" and "search_analyzer" your custom analyzer.

7

That worked! thanks a lot!!!

On Thu, Jul 28, 2011 at 4:41 PM, Vladimir Shkurin vshkurin@gmail.comwrote:

I'm using this settings and asciifolding works:

ElasticSearch Test Queries · GitHub

/_search?q=title:botao* - OK
/_search?q=botao* - OK

You can remove the "my_delimiter", "porter_stem" and use these
settings, just write in your mappings for field-type "string" for
"index_analyzer" and "search_analyzer" your custom analyzer.

--
Carlos Eduardo da Fonseca

8

Strange that it worked :). By default, wildcard and prefix using the
query_string query (which is what q= is translated to) don't get analyzed.
Elasticsearch Platform — Find real-time answers at scale | Elastic.
You can pass analyze_wildcard set to true, which will cause wildcards to be
analyzed as well (though its tricky).

Having said that, the analyze_wildcard parameter is not exposed when using
?q=..., though it can be added easily, something like:
?q=analyze_wildcard=true&q=... .

Most times, a better option is to use the text family of queries:
Elasticsearch Platform — Find real-time answers at scale | Elastic.

On Thu, Jul 28, 2011 at 11:57 PM, Carlos Eduardo da Fonseca <
eduardongu@gmail.com> wrote:

That worked! thanks a lot!!!

On Thu, Jul 28, 2011 at 4:41 PM, Vladimir Shkurin vshkurin@gmail.comwrote:

I'm using this settings and asciifolding works:

ElasticSearch Test Queries · GitHub

/_search?q=title:botao* - OK
/_search?q=botao* - OK

You can remove the "my_delimiter", "porter_stem" and use these
settings, just write in your mappings for field-type "string" for
"index_analyzer" and "search_analyzer" your custom analyzer.

--
Carlos Eduardo da Fonseca

9

Shay, I searched using text and field queries, forgot to mention. I think it
wasn't working before because of missing mappings...

On Fri, Jul 29, 2011 at 3:37 AM, Shay Banon kimchy@gmail.com wrote:

Strange that it worked :). By default, wildcard and prefix using the
query_string query (which is what q= is translated to) don't get analyzed.
Elasticsearch Platform — Find real-time answers at scale | Elastic.
You can pass analyze_wildcard set to true, which will cause wildcards to be
analyzed as well (though its tricky).

Having said that, the analyze_wildcard parameter is not exposed when using
?q=..., though it can be added easily, something like:
?q=analyze_wildcard=true&q=... .

Most times, a better option is to use the text family of queries:
Elasticsearch Platform — Find real-time answers at scale | Elastic.

On Thu, Jul 28, 2011 at 11:57 PM, Carlos Eduardo da Fonseca <
eduardongu@gmail.com> wrote:

That worked! thanks a lot!!!

On Thu, Jul 28, 2011 at 4:41 PM, Vladimir Shkurin vshkurin@gmail.comwrote:

I'm using this settings and asciifolding works:

ElasticSearch Test Queries · GitHub

/_search?q=title:botao* - OK
/_search?q=botao* - OK

You can remove the "my_delimiter", "porter_stem" and use these
settings, just write in your mappings for field-type "string" for
"index_analyzer" and "search_analyzer" your custom analyzer.

--
Carlos Eduardo da Fonseca

--
Carlos Eduardo da Fonseca