0.90.9 in elasticsearch - indexing with ngrams, search returns 0 results


(thale jacobs) #1

Hello - I am attempting to run a ngram test using elastics 0.90.9. I was
able to replicate the problem I am having in our production system by
following the example from here:

http://blog.rnf.me/2013/exact-substring-search-in-elasticsearch.html

The Mappings looks like this:

{
"mappings": {
"homes": {
"properties": {
"desc": {
"type": "string",
"index_analyzer": "index_ngram",
"search_analyzer": "search_ngram"
}
}
}
},
"settings": {
"analysis": {
"filter": {
"desc_ngram": {
"type": "ngram",
"min_gram": 3,
"max_gram": 8
}
},
"analyzer": {
"index_ngram": {
"type": "custom",
"tokenizer": "keyword",
"filter": [ "desc_ngram", "lowercase" ]
},
"search_ngram": {
"type": "custom",
"tokenizer": "keyword",
"filter": "lowercase"
}
}
}
}
}

The mapping is set up as follows:

curl -s -XPUT 'localhost:9200/listings' -d @settings.json

Three records are inserted into the index as follows:

curl -s -XPUT 'localhost:9200/listings/homes/1' -d '{ "desc": "This is a
lovely home with a large yard." }'
curl -s -XPUT 'localhost:9200/listings/homes/2' -d '{ "desc": "This large
fixer-upper has a gravelly yard." }'
curl -s -XPUT 'localhost:9200/listings/homes/3' -d '{ "desc": "A colonial
mansion with a large yard and a pool." }'

The Search looks like this:
curl -s -XGET 'localhost:9200/listings/homes/_search?pretty=true' -d '{
"query" : { "bool" : { "must" : [ { "match" : { "desc" : { "query" : "large
ya", "type" : "phrase" } } }, { "match" : { "desc" : { "query" : "arge
yar", "type" : "phrase" } } }, { "match" : { "desc" : { "query" : "rge
yard", "type" : "phrase" } } } ] } } }';

The problem is no results are returned. Any input that can be given for a
solution would be appreciated.

Thank you!

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/4bee1be0-7ee4-4667-b4f7-b51371963928%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(David Pilato) #2

Hey!

Just tested it with es 1.0.0.RC1 and it's working fine.
See https://gist.github.com/dadoonet/8456535

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr

Le 16 janvier 2014 at 14:25:02, thale jacobs (thalejacobs@gmail.com) a écrit:

Hello - I am attempting to run a ngram test using elastics 0.90.9. I was able to replicate the problem I am having in our production system by following the example from here:

http://blog.rnf.me/2013/exact-substring-search-in-elasticsearch.html

The Mappings looks like this:

{
"mappings": {
"homes": {
"properties": {
"desc": {
"type": "string",
"index_analyzer": "index_ngram",
"search_analyzer": "search_ngram"
}
}
}
},
"settings": {
"analysis": {
"filter": {
"desc_ngram": {
"type": "ngram",
"min_gram": 3,
"max_gram": 8
}
},
"analyzer": {
"index_ngram": {
"type": "custom",
"tokenizer": "keyword",
"filter": [ "desc_ngram", "lowercase" ]
},
"search_ngram": {
"type": "custom",
"tokenizer": "keyword",
"filter": "lowercase"
}
}
}
}
}

The mapping is set up as follows:

curl -s -XPUT 'localhost:9200/listings' -d @settings.json

Three records are inserted into the index as follows:

curl -s -XPUT 'localhost:9200/listings/homes/1' -d '{ "desc": "This is a lovely home with a large yard." }'
curl -s -XPUT 'localhost:9200/listings/homes/2' -d '{ "desc": "This large fixer-upper has a gravelly yard." }'
curl -s -XPUT 'localhost:9200/listings/homes/3' -d '{ "desc": "A colonial mansion with a large yard and a pool." }'

The Search looks like this:
curl -s -XGET 'localhost:9200/listings/homes/_search?pretty=true' -d '{ "query" : { "bool" : { "must" : [ { "match" : { "desc" : { "query" : "large ya", "type" : "phrase" } } }, { "match" : { "desc" : { "query" : "arge yar", "type" : "phrase" } } }, { "match" : { "desc" : { "query" : "rge yard", "type" : "phrase" } } } ] } } }';

The problem is no results are returned. Any input that can be given for a solution would be appreciated.

Thank you!

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/4bee1be0-7ee4-4667-b4f7-b51371963928%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.52d7f74c.2ae8944a.dc5%40MacBook-Air-de-David.local.
For more options, visit https://groups.google.com/groups/opt_out.


(thale jacobs) #3

Hi David - Thanks for the reply. I just tried it on 90.5, 90.9, 90.10, and
1.0.0.RC1. No results were returned in my searches. I forgot to include
that I have ES running on ubuntu if that makes a difference.

Thale

On Thursday, January 16, 2014 10:14:20 AM UTC-5, David Pilato wrote:

Hey!

Just tested it with es 1.0.0.RC1 and it's working fine.
See https://gist.github.com/dadoonet/8456535

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr

Le 16 janvier 2014 at 14:25:02, thale jacobs (thale...@gmail.com<javascript:>)
a écrit:

Hello - I am attempting to run a ngram test using elastics 0.90.9. I was
able to replicate the problem I am having in our production system by
following the example from here:

http://blog.rnf.me/2013/exact-substring-search-in-elasticsearch.html

The Mappings looks like this:

{
"mappings": {
"homes": {
"properties": {
"desc": {
"type": "string",
"index_analyzer": "index_ngram",
"search_analyzer": "search_ngram"
}
}
}
},
"settings": {
"analysis": {
"filter": {
"desc_ngram": {
"type": "ngram",
"min_gram": 3,
"max_gram": 8
}
},
"analyzer": {
"index_ngram": {
"type": "custom",
"tokenizer": "keyword",
"filter": [ "desc_ngram", "lowercase" ]
},
"search_ngram": {
"type": "custom",
"tokenizer": "keyword",
"filter": "lowercase"
}
}
}
}
}

The mapping is set up as follows:

curl -s -XPUT 'localhost:9200/listings' -d @settings.json

Three records are inserted into the index as follows:

curl -s -XPUT 'localhost:9200/listings/homes/1' -d '{ "desc": "This is a
lovely home with a large yard." }'
curl -s -XPUT 'localhost:9200/listings/homes/2' -d '{ "desc": "This large
fixer-upper has a gravelly yard." }'
curl -s -XPUT 'localhost:9200/listings/homes/3' -d '{ "desc": "A colonial
mansion with a large yard and a pool." }'

The Search looks like this:
curl -s -XGET 'localhost:9200/listings/homes/_search?pretty=true' -d '{
"query" : { "bool" : { "must" : [ { "match" : { "desc" : { "query" : "large
ya", "type" : "phrase" } } }, { "match" : { "desc" : { "query" : "arge
yar", "type" : "phrase" } } }, { "match" : { "desc" : { "query" : "rge
yard", "type" : "phrase" } } } ] } } }';

The problem is no results are returned. Any input that can be given for a
solution would be appreciated.

Thank you!

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/4bee1be0-7ee4-4667-b4f7-b51371963928%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/7bccf38a-eaa7-4e7b-9f82-a364a1d410df%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(David Pilato) #4

So you have specific settings in elasticsearch.yml file?
What result do you actually get?

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr

Le 16 janvier 2014 at 18:06:58, thale jacobs (thalejacobs@gmail.com) a écrit:

Hi David - Thanks for the reply. I just tried it on 90.5, 90.9, 90.10, and 1.0.0.RC1. No results were returned in my searches. I forgot to include that I have ES running on ubuntu if that makes a difference.

Thale

On Thursday, January 16, 2014 10:14:20 AM UTC-5, David Pilato wrote:
Hey!

Just tested it with es 1.0.0.RC1 and it's working fine.
See https://gist.github.com/dadoonet/8456535

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr

Le 16 janvier 2014 at 14:25:02, thale jacobs (thale...@gmail.com) a écrit:

Hello - I am attempting to run a ngram test using elastics 0.90.9. I was able to replicate the problem I am having in our production system by following the example from here:

http://blog.rnf.me/2013/exact-substring-search-in-elasticsearch.html

The Mappings looks like this:

{
"mappings": {
"homes": {
"properties": {
"desc": {
"type": "string",
"index_analyzer": "index_ngram",
"search_analyzer": "search_ngram"
}
}
}
},
"settings": {
"analysis": {
"filter": {
"desc_ngram": {
"type": "ngram",
"min_gram": 3,
"max_gram": 8
}
},
"analyzer": {
"index_ngram": {
"type": "custom",
"tokenizer": "keyword",
"filter": [ "desc_ngram", "lowercase" ]
},
"search_ngram": {
"type": "custom",
"tokenizer": "keyword",
"filter": "lowercase"
}
}
}
}
}

The mapping is set up as follows:

curl -s -XPUT 'localhost:9200/listings' -d @settings.json

Three records are inserted into the index as follows:

curl -s -XPUT 'localhost:9200/listings/homes/1' -d '{ "desc": "This is a lovely home with a large yard." }'
curl -s -XPUT 'localhost:9200/listings/homes/2' -d '{ "desc": "This large fixer-upper has a gravelly yard." }'
curl -s -XPUT 'localhost:9200/listings/homes/3' -d '{ "desc": "A colonial mansion with a large yard and a pool." }'

The Search looks like this:
curl -s -XGET 'localhost:9200/listings/homes/_search?pretty=true' -d '{ "query" : { "bool" : { "must" : [ { "match" : { "desc" : { "query" : "large ya", "type" : "phrase" } } }, { "match" : { "desc" : { "query" : "arge yar", "type" : "phrase" } } }, { "match" : { "desc" : { "query" : "rge yard", "type" : "phrase" } } } ] } } }';

The problem is no results are returned. Any input that can be given for a solution would be appreciated.

Thank you!

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/4bee1be0-7ee4-4667-b4f7-b51371963928%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/7bccf38a-eaa7-4e7b-9f82-a364a1d410df%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.52d811f7.216231b.dc5%40MacBook-Air-de-David.local.
For more options, visit https://groups.google.com/groups/opt_out.


(thale jacobs) #5

Actually, I take back what I said about it not working. The example you
posted up on gist is working. Thanks!...I just need to figure out the
difference between the original example I posted up which is not working
and your working post. Thanks again.

On Thursday, January 16, 2014 12:06:54 PM UTC-5, thale jacobs wrote:

Hi David - Thanks for the reply. I just tried it on 90.5, 90.9, 90.10,
and 1.0.0.RC1. No results were returned in my searches. I forgot to
include that I have ES running on ubuntu if that makes a difference.

Thale

On Thursday, January 16, 2014 10:14:20 AM UTC-5, David Pilato wrote:

Hey!

Just tested it with es 1.0.0.RC1 and it's working fine.
See https://gist.github.com/dadoonet/8456535

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr

Le 16 janvier 2014 at 14:25:02, thale jacobs (thale...@gmail.com) a
écrit:

Hello - I am attempting to run a ngram test using elastics 0.90.9. I
was able to replicate the problem I am having in our production system by
following the example from here:

http://blog.rnf.me/2013/exact-substring-search-in-elasticsearch.html

The Mappings looks like this:

{
"mappings": {
"homes": {
"properties": {
"desc": {
"type": "string",
"index_analyzer": "index_ngram",
"search_analyzer": "search_ngram"
}
}
}
},
"settings": {
"analysis": {
"filter": {
"desc_ngram": {
"type": "ngram",
"min_gram": 3,
"max_gram": 8
}
},
"analyzer": {
"index_ngram": {
"type": "custom",
"tokenizer": "keyword",
"filter": [ "desc_ngram", "lowercase" ]
},
"search_ngram": {
"type": "custom",
"tokenizer": "keyword",
"filter": "lowercase"
}
}
}
}
}

The mapping is set up as follows:

curl -s -XPUT 'localhost:9200/listings' -d @settings.json

Three records are inserted into the index as follows:

curl -s -XPUT 'localhost:9200/listings/homes/1' -d '{ "desc": "This is
a lovely home with a large yard." }'
curl -s -XPUT 'localhost:9200/listings/homes/2' -d '{ "desc": "This
large fixer-upper has a gravelly yard." }'
curl -s -XPUT 'localhost:9200/listings/homes/3' -d '{ "desc": "A
colonial mansion with a large yard and a pool." }'

The Search looks like this:
curl -s -XGET 'localhost:9200/listings/homes/_search?pretty=true' -d '{
"query" : { "bool" : { "must" : [ { "match" : { "desc" : { "query" : "large
ya", "type" : "phrase" } } }, { "match" : { "desc" : { "query" : "arge
yar", "type" : "phrase" } } }, { "match" : { "desc" : { "query" : "rge
yard", "type" : "phrase" } } } ] } } }';

The problem is no results are returned. Any input that can be given for
a solution would be appreciated.

Thank you!

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/4bee1be0-7ee4-4667-b4f7-b51371963928%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d67e3572-f074-4879-9659-59bc2a561bc5%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(system) #6