I'm trying to set a default index configuration in config/elastisearch.yml but for some reason it doesn't seem to get picked up. Can somebody spot the error?
The ES doesn't seem to show an error when loading. Here is the yml:
On Saturday, May 28, 2011 at 1:21 AM, Yannick Smits wrote:
I’m trying to set a default index configuration in config/elastisearch.yml but for some reason it doesn’t seem to get picked up. Can somebody spot the error?
The ES doesn’t seem to show an error when loading. Here is the yml:
If you have already indexed one document (with default mapping or not),
config files won't be read by ES.
So, to use mapping within config file, you have to start a clean ES cluster
with the config files put in mappings/indexName/typeName.json
I'm trying to set a default index configuration in
config/elastisearch.yml but for some reason it doesn't seem to get picked
up. Can somebody spot the error?The ES doesn't seem to show an error when
loading. Here is the yml: index: analysis: analyzer: default_index:
tokenizer: standard filter: [standard, lowercase, my_synonym, my_snow]
default_search: tokenizer: standard filter: [standard, lowercase, stop]
filter: my_synonym: type: synonym synonyms_path: 'analysers/synonyms.dat'
ignore_case: true my_snow: type: snowball language: English
If you have already indexed one document (with default mapping or not),
config files won't be read by ES.
So, to use mapping within config file, you have to start a clean ES cluster
with the config files put in mappings/indexName/typeName.json
I’m trying to set a default index configuration in config/elastisearch.yml
but for some reason it doesn’t seem to get picked up. Can somebody spot the
error?
The ES doesn’t seem to show an error when loading. Here is the yml:
index:
analysis:
analyzer:
default_index:
tokenizer: standard
filter: [standard, lowercase, my_synonym, my_snow]
default_search:
tokenizer: standard
filter: [standard, lowercase, stop]
filter:
my_synonym:
type: synonym
synonyms_path: 'analysers/synonyms.dat'
ignore_case: true
my_snow:
type: snowball
language: English
Oh and to reply to David: yes I’m aware that settings have to be applied before the index is created. So I removed the index completely using the delete command before setting the new configuration and adding documents.
On Saturday, May 28, 2011 at 1:21 AM, Yannick Smits wrote:
I’m trying to set a default index configuration in config/elastisearch.yml but for some reason it doesn’t seem to get picked up. Can somebody spot the error?
The ES doesn’t seem to show an error when loading. Here is the yml:
index:
analysis:
analyzer:
default_index:
tokenizer: standard
filter: [standard, lowercase, my_synonym, my_snow]
default_search:
tokenizer: standard
filter: [standard, lowercase, stop]
filter:
my_synonym:
type: synonym
synonyms_path: 'analysers/synonyms.dat'
ignore_case: true
my_snow:
type: snowball
language: English
Oh and to reply to David: yes I’m aware that settings have to be applied before the index is created. So I removed the index completely using the delete command before setting the new configuration and adding documents.
I’m trying to set a default index configuration in config/elastisearch.yml but for some reason it doesn’t seem to get picked up. Can somebody spot the error?
The ES doesn’t seem to show an error when loading. Here is the yml:
I tried to search for some of the synonyms in the file which should have matched up with words in my documents, but no results. Also I tested simple search for stemming which didn’t give results either.
Is there a way to get the analysers back from ES to check if they are correctly applied using a command?
Oh and to reply to David: yes I’m aware that settings have to be applied before the index is created. So I removed the index completely using the delete command before setting the new configuration and adding documents.
On Saturday, May 28, 2011 at 1:21 AM, Yannick Smits wrote:
I’m trying to set a default index configuration in config/elastisearch.yml but for some reason it doesn’t seem to get picked up. Can somebody spot the error?
The ES doesn’t seem to show an error when loading. Here is the yml:
index:
analysis:
analyzer:
default_index:
tokenizer: standard
filter: [standard, lowercase, my_synonym, my_snow]
default_search:
tokenizer: standard
filter: [standard, lowercase, stop]
filter:
my_synonym:
type: synonym
synonyms_path: 'analysers/synonyms.dat'
ignore_case: true
my_snow:
type: snowball
language: English
On Saturday, May 28, 2011 at 5:12 PM, Yannick Smits wrote:
I tried to search for some of the synonyms in the file which should have matched up with words in my documents, but no results. Also I tested simple search for stemming which didn’t give results either.
Is there a way to get the analysers back from ES to check if they are correctly applied using a command?
ok. how do you know that its not getting picked up? Can you gist a repro of what you are seeing?
On Saturday, May 28, 2011 at 3:34 PM, Yannick Smits wrote:
Oh and to reply to David: yes I’m aware that settings have to be applied before the index is created. So I removed the index completely using the delete command before setting the new configuration and adding documents.
I’m trying to set a default index configuration in config/elastisearch.yml but for some reason it doesn’t seem to get picked up. Can somebody spot the error?
The ES doesn’t seem to show an error when loading. Here is the yml:
Ok, I got the analyzer settings to shop up in the index _settings request by using a JSON put. Cannot seem to set those using the YAML approach as explained below.
But even while the JSON method of setting it does seem to get through my tests show they are not working / not configured correctly.
On Saturday, May 28, 2011 at 5:12 PM, Yannick Smits wrote:
I tried to search for some of the synonyms in the file which should have matched up with words in my documents, but no results. Also I tested simple search for stemming which didn’t give results either.
Is there a way to get the analysers back from ES to check if they are correctly applied using a command?
Oh and to reply to David: yes I’m aware that settings have to be applied before the index is created. So I removed the index completely using the delete command before setting the new configuration and adding documents.
On Saturday, May 28, 2011 at 1:21 AM, Yannick Smits wrote:
I’m trying to set a default index configuration in config/elastisearch.yml but for some reason it doesn’t seem to get picked up. Can somebody spot the error?
The ES doesn’t seem to show an error when loading. Here is the yml:
index:
analysis:
analyzer:
default_index:
tokenizer: standard
filter: [standard, lowercase, my_synonym, my_snow]
default_search:
tokenizer: standard
filter: [standard, lowercase, stop]
filter:
my_synonym:
type: synonym
synonyms_path: 'analysers/synonyms.dat'
ignore_case: true
my_snow:
type: snowball
language: English
Ok, I got the analyzer settings to shop up in the index _settings request by using a JSON put. Cannot seem to set those using the YAML approach as explained below.
But even while the JSON method of setting it does seem to get through my tests show they are not working / not configured correctly.
On Saturday, May 28, 2011 at 5:12 PM, Yannick Smits wrote:
I tried to search for some of the synonyms in the file which should have matched up with words in my documents, but no results. Also I tested simple search for stemming which didn’t give results either.
Is there a way to get the analysers back from ES to check if they are correctly applied using a command?
Oh and to reply to David: yes I’m aware that settings have to be applied before the index is created. So I removed the index completely using the delete command before setting the new configuration and adding documents.
On Saturday, May 28, 2011 at 1:21 AM, Yannick Smits wrote:
I’m trying to set a default index configuration in config/elastisearch.yml but for some reason it doesn’t seem to get picked up. Can somebody spot the error?
The ES doesn’t seem to show an error when loading. Here is the yml:
index:
analysis:
analyzer:
default_index:
tokenizer: standard
filter: [standard, lowercase, my_synonym, my_snow]
default_search:
tokenizer: standard
filter: [standard, lowercase, stop]
filter:
my_synonym:
type: synonym
synonyms_path: 'analysers/synonyms.dat'
ignore_case: true
my_snow:
type: snowball
language: English
You're using the snowball filter on text that you are indexing but then:
you're using a term query, which doesn't get analysed, instead of
a text|field|query_string query
even if you were using a text|field|query_string query, it probably
would fail as well, because your default search analyzer doesn't
include the snowball filter.
You almost always want to use the same analyzer for indexing and for
searching
Thanks for pointing this out to me. Now using a text query and making the index and search analyzers equal everything works as expected. The only thing I can't seem to get to work is setting the default analyzers inside the elasticsearch.yml. But using the json put method it works well enough for me now.
You're using the snowball filter on text that you are indexing but then:
you're using a term query, which doesn't get analysed, instead of
a text|field|query_string query
even if you were using a text|field|query_string query, it probably
would fail as well, because your default search analyzer doesn't
include the snowball filter.
You almost always want to use the same analyzer for indexing and for searching
You almost always want to use the same analyzer for indexing and for searching
Could you elaborate on why that is? I would imagine if the synonyms are already applied to the index that it doesn't make sense to apply the synonyms on the search query, because they are already matched in the index.
Thanks for pointing this out to me. Now using a text query and making the index and search analyzers equal everything works as expected. The only thing I can't seem to get to work is setting the default analyzers inside the elasticsearch.yml. But using the json put method it works well enough for me now.
You're using the snowball filter on text that you are indexing but then:
you're using a term query, which doesn't get analysed, instead of
a text|field|query_string query
even if you were using a text|field|query_string query, it probably
would fail as well, because your default search analyzer doesn't
include the snowball filter.
You almost always want to use the same analyzer for indexing and for searching
You almost always want to use the same analyzer for indexing and for
searching
Could you elaborate on why that is? I would imagine if the synonyms are
already applied to the index that it doesn't make sense to apply the
synonyms on the search query, because they are already matched in the index.
Thanks for pointing this out to me. Now using a text query and making the
index and search analyzers equal everything works as expected. The only
thing I can't seem to get to work is setting the default analyzers inside
the elasticsearch.yml. But using the json put method it works well enough
for me now.
You're using the snowball filter on text that you are indexing but then:
you're using a term query, which doesn't get analysed, instead of
a text|field|query_string query
even if you were using a text|field|query_string query, it probably
would fail as well, because your default search analyzer doesn't
include the snowball filter.
You almost always want to use the same analyzer for indexing and for
searching
You almost always want to use the same analyzer for indexing and for
searching
Could you elaborate on why that is? I would imagine if the synonyms are
already applied to the index that it doesn't make sense to apply the
synonyms on the search query, because they are already matched in the index.
Sent: zaterdag 4 juni 2011 16:26
To: users@elasticsearch.com
Subject: RE: index configuration
Hi Clint,
Thanks for pointing this out to me. Now using a text query and making the
index and search analyzers equal everything works as expected. The only
thing I can't seem to get to work is setting the default analyzers inside
the elasticsearch.yml. But using the json put method it works well enough
for me now.
You're using the snowball filter on text that you are indexing but then:
you're using a term query, which doesn't get analysed, instead of
a text|field|query_string query
even if you were using a text|field|query_string query, it probably
would fail as well, because your default search analyzer doesn't
include the snowball filter.
You almost always want to use the same analyzer for indexing and for
searching
Could you elaborate on why that is? I would imagine if the synonyms are already applied to the index that it doesn't make sense to apply the synonyms on the search query, because they are already matched in the index.
You're right that for synonyms this probably doesn't matter; however,
for the snowball stemmer you are using you definitely should apply it
to both the query and the index since the the stemmer will cause only
the stemmed version of words to be indexed, while a user will often
search with words that are not stemmed. So for instance, a search for
the word "indexing" will not match anything if all documents in the
index have had the word "indexing" stemmed to "index".
Generally good practice to just use the same analyzer for both the
query and indexing to avoid these types of problems.
You almost always want to use the same analyzer for indexing and for searching
Could you elaborate on why that is? I would imagine if the synonyms are already applied to the index that it doesn't make sense to apply the synonyms on the search query, because they are already matched in the index.
Thanks for pointing this out to me. Now using a text query and making the index and search analyzers equal everything works as expected. The only thing I can't seem to get to work is setting the default analyzers inside the elasticsearch.yml. But using the json put method it works well enough for me now.
You're using the snowball filter on text that you are indexing but then:
you're using a term query, which doesn't get analysed, instead of
a text|field|query_string query
even if you were using a text|field|query_string query, it probably
would fail as well, because your default search analyzer doesn't
include the snowball filter.
You almost always want to use the same analyzer for indexing and for searching
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.