Filter on not_analyzed field with whitespace/hyphen not working

bashir · August 7, 2012, 11:14pm

I've been scratching my head for some time on this one. Major brownie
points for any guidance:
I have mapping:
mapping :show => { :properties => {...
:tags => {:type => 'string', :index_name => 'tag', :boost => 80, :index
=> 'not_analyzed'},
...}}

tags is an array of tags. For example:
"tags":["rails","ruby on rails"]

I run a query as such(notice no results):

curl -X GET "
http://localhost:9200/magic/_search?from=0&page=1&per_page=24&size=24pretty=true"
-d '{"sort":[[{"score":"desc"}]],"filter":{"terms":{"tags":["ruby on
rails"]}},"size":24,"from":0}'
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 0,
"max_score" : null,
"hits" : [ ]
}

If I run it with "tags":["ruby"], it will match all documents that are
tagged "ruby" and all documents tagged "ruby on rails". I don't want a
substring search, I need an exact search and I thought if the field is
"not_analyzed" that it would be an exact match. How to do this the right
way. Notice that this is a filter and not query.

dadoonet · August 8, 2012, 12:55am

Are you sure that your mapping is applied?
What do you see when doing:
curl -XGET 'http://localhost:9200/magic/yourtype/_mapping'
also, did you try with Term filter instead of terms filter? Same result I suppose.

David

Le 8 août 2012 à 01:14, Bashir Eghbali bashir@gmail.com a écrit :

I've been scratching my head for some time on this one. Major brownie points for any guidance:
I have mapping:
mapping :show => { :properties => {...
:tags => {:type => 'string', :index_name => 'tag', :boost => 80, :index => 'not_analyzed'},
...}}

tags is an array of tags. For example:
"tags":["rails","ruby on rails"]

I run a query as such(notice no results):

curl -X GET "http://localhost:9200/magic/_search?from=0&page=1&per_page=24&size=24pretty=true" -d '{"sort":[[{"score":"desc"}]],"filter":{"terms":{"tags":["ruby on rails"]}},"size":24,"from":0}'
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 0,
"max_score" : null,
"hits" :
}

If I run it with "tags":["ruby"], it will match all documents that are tagged "ruby" and all documents tagged "ruby on rails". I don't want a substring search, I need an exact search and I thought if the field is "not_analyzed" that it would be an exact match. How to do this the right way. Notice that this is a filter and not query.

bashir · August 8, 2012, 2:07am

I get:
"tags":{"type":"string"}

It says type string but when I assign to tags at indexing time, I pass in
an array of strings. I assume it handles the json array fine.

I tried term filter as well to no avail.

Any other ideas? does the terms query get tokenized at query time and then
matched against a verbatim indexed field, resulting in prefix match?

On Tuesday, August 7, 2012 5:55:00 PM UTC-7, David Pilato wrote:

Are you sure that your mapping is applied?
What do you see when doing:

curl -XGET 'http://localhost:9200/magic/yourtype/_mapping'

also, did you try with Term filter instead of terms filter? Same result I
suppose.

David

Le 8 août 2012 à 01:14, Bashir Eghbali <bas...@gmail.com <javascript:>> a
écrit :

I've been scratching my head for some time on this one. Major brownie
points for any guidance:
I have mapping:
mapping :show => { :properties => {...
:tags => {:type => 'string', :index_name => 'tag', :boost => 80, :index
=> 'not_analyzed'},
...}}

tags is an array of tags. For example:
"tags":["rails","ruby on rails"]

I run a query as such(notice no results):

curl -X GET "
http://localhost:9200/magic/_search?from=0&page=1&per_page=24&size=24pretty=true"
-d '{"sort":[[{"score":"desc"}]],"filter":{"terms":{"tags":["ruby on
rails"]}},"size":24,"from":0}'
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 0,
"max_score" : null,
"hits" :
}

If I run it with "tags":["ruby"], it will match all documents that are
tagged "ruby" and all documents tagged "ruby on rails". I don't want a
substring search, I need an exact search and I thought if the field is
"not_analyzed" that it would be an exact match. How to do this the right
way. Notice that this is a filter and not query.

dadoonet · August 8, 2012, 7:03am

So, you can see that your mapping is not applied.
Remove your index magic and create it again.
Send the mapping
Verify that your mapping is correct
And only if correct, send your document and run your query again. It should work.

HTH
David

--

Le 8 août 2012 à 04:07, Bashir Eghbali bashir@gmail.com a écrit :

I get:
"tags":{"type":"string"}

It says type string but when I assign to tags at indexing time, I pass in an array of strings. I assume it handles the json array fine.

I tried term filter as well to no avail.

Any other ideas? does the terms query get tokenized at query time and then matched against a verbatim indexed field, resulting in prefix match?

On Tuesday, August 7, 2012 5:55:00 PM UTC-7, David Pilato wrote:
Are you sure that your mapping is applied?
What do you see when doing:
curl -XGET 'http://localhost:9200/magic/yourtype/_mapping'
also, did you try with Term filter instead of terms filter? Same result I suppose.

David

Le 8 août 2012 à 01:14, Bashir Eghbali bas...@gmail.com a écrit :

I've been scratching my head for some time on this one. Major brownie points for any guidance:
I have mapping:
mapping :show => { :properties => {...
:tags => {:type => 'string', :index_name => 'tag', :boost => 80, :index => 'not_analyzed'},
...}}

tags is an array of tags. For example:
"tags":["rails","ruby on rails"]

I run a query as such(notice no results):

curl -X GET "http://localhost:9200/magic/_search?from=0&page=1&per_page=24&size=24pretty=true" -d '{"sort":[[{"score":"desc"}]],"filter":{"terms":{"tags":["ruby on rails"]}},"size":24,"from":0}'
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 0,
"max_score" : null,
"hits" :
}

If I run it with "tags":["ruby"], it will match all documents that are tagged "ruby" and all documents tagged "ruby on rails". I don't want a substring search, I need an exact search and I thought if the field is "not_analyzed" that it would be an exact match. How to do this the right way. Notice that this is a filter and not query.

bashir · August 8, 2012, 8:01am

If the mapping was not applied, I would venture that no search would work.
Search works in general just returns superset of data or no data on tags
with spaces.
I am not sure what the mapping should look like for the type as there is no
explicit array type in elasticsearch. should I be configuring the index
with a different type/structure? it seems to show the type that was
configured right now which is string.

thanks,
Bashir

On Wed, Aug 8, 2012 at 12:03 AM, David Pilato david@pilato.fr wrote:

So, you can see that your mapping is not applied.
Remove your index magic and create it again.
Send the mapping
Verify that your mapping is correct
And only if correct, send your document and run your query again. It
should work.

HTH
David

--

Le 8 août 2012 à 04:07, Bashir Eghbali bashir@gmail.com a écrit :

I get:
"tags":{"type":"string"}

It says type string but when I assign to tags at indexing time, I pass in
an array of strings. I assume it handles the json array fine.

I tried term filter as well to no avail.

Any other ideas? does the terms query get tokenized at query time and then
matched against a verbatim indexed field, resulting in prefix match?

On Tuesday, August 7, 2012 5:55:00 PM UTC-7, David Pilato wrote:

Are you sure that your mapping is applied?
What do you see when doing:

curl -XGET 'http://localhost:9200/magic/**yourtype/_mapping' http://localhost:9200/magic/yourtype/_mapping'

also, did you try with Term filter instead of terms filter? Same result I
suppose.

David

Le 8 août 2012 à 01:14, Bashir Eghbali bas...@gmail.com a écrit :

I've been scratching my head for some time on this one. Major brownie
points for any guidance:
I have mapping:
mapping :show => { :properties => {...
:tags => {:type => 'string', :index_name => 'tag', :boost => 80, :index
=> 'not_analyzed'},
...}}

tags is an array of tags. For example:
"tags":["rails","ruby on rails"]

I run a query as such(notice no results):

curl -X GET "http://localhost:9200/magic/_**
search?from=0&page=1&per_page=24&size=24pretty=truehttp://localhost:9200/magic/_search?from=0&page=1&per_page=24&size=24pretty=true"
-d '{"sort":[[{"score":"desc"}]],"filter":{"terms":{"tags":["ruby on
rails"]}},"size":24,"from":0}'
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 0,
"max_score" : null,
"hits" :
}

If I run it with "tags":["ruby"], it will match all documents that are
tagged "ruby" and all documents tagged "ruby on rails". I don't want a
substring search, I need an exact search and I thought if the field is
"not_analyzed" that it would be an exact match. How to do this the right
way. Notice that this is a filter and not query.

--
Bashir Eghbali

bashireghbali.com
linkedin http://www.linkedin.com/in/bashireghbali

dadoonet · August 8, 2012, 8:07am

Array of string can be considered as a String type.

So create your mapping like this:

$ curl -XPUT 'http://localhost:9200/magic/yourtype/_mapping' -d '
{
"yourtype" : {
"properties" : {
"message" : {"type" : "string", "index" : "not_analyzed"}
}
}
}
'

--

Le 8 août 2012 à 10:01, Bashir Eghbali bashir@gmail.com a écrit :

If the mapping was not applied, I would venture that no search would work. Search works in general just returns superset of data or no data on tags with spaces.
I am not sure what the mapping should look like for the type as there is no explicit array type in elasticsearch. should I be configuring the index with a different type/structure? it seems to show the type that was configured right now which is string.

thanks,
Bashir

On Wed, Aug 8, 2012 at 12:03 AM, David Pilato david@pilato.fr wrote:
So, you can see that your mapping is not applied.
Remove your index magic and create it again.
Send the mapping
Verify that your mapping is correct
And only if correct, send your document and run your query again. It should work.

HTH
David

--

Le 8 août 2012 à 04:07, Bashir Eghbali bashir@gmail.com a écrit :

I get:
"tags":{"type":"string"}

It says type string but when I assign to tags at indexing time, I pass in an array of strings. I assume it handles the json array fine.

I tried term filter as well to no avail.

Any other ideas? does the terms query get tokenized at query time and then matched against a verbatim indexed field, resulting in prefix match?

On Tuesday, August 7, 2012 5:55:00 PM UTC-7, David Pilato wrote:
Are you sure that your mapping is applied?
What do you see when doing:
curl -XGET 'http://localhost:9200/magic/yourtype/_mapping'
also, did you try with Term filter instead of terms filter? Same result I suppose.

David

Le 8 août 2012 à 01:14, Bashir Eghbali bas...@gmail.com a écrit :

I've been scratching my head for some time on this one. Major brownie points for any guidance:
I have mapping:
mapping :show => { :properties => {...
:tags => {:type => 'string', :index_name => 'tag', :boost => 80, :index => 'not_analyzed'},
...}}

tags is an array of tags. For example:
"tags":["rails","ruby on rails"]

I run a query as such(notice no results):

curl -X GET "http://localhost:9200/magic/_search?from=0&page=1&per_page=24&size=24pretty=true" -d '{"sort":[[{"score":"desc"}]],"filter":{"terms":{"tags":["ruby on rails"]}},"size":24,"from":0}'
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 0,
"max_score" : null,
"hits" :
}

If I run it with "tags":["ruby"], it will match all documents that are tagged "ruby" and all documents tagged "ruby on rails". I don't want a substring search, I need an exact search and I thought if the field is "not_analyzed" that it would be an exact match. How to do this the right way. Notice that this is a filter and not query.

--
Bashir Eghbali

bashireghbali.com
linkedin

dadoonet · August 8, 2012, 8:08am

Sorry. Replace message by tags.

--

Le 8 août 2012 à 10:07, David Pilato david@pilato.fr a écrit :

Array of string can be considered as a String type.

So create your mapping like this:

$ curl -XPUT 'http://localhost:9200/magic/yourtype/_mapping' -d '
{
"yourtype" : {
"properties" : {
"message" : {"type" : "string", "index" : "not_analyzed"}
}
}
}
'

--

Le 8 août 2012 à 10:01, Bashir Eghbali bashir@gmail.com a écrit :

If the mapping was not applied, I would venture that no search would work. Search works in general just returns superset of data or no data on tags with spaces.
I am not sure what the mapping should look like for the type as there is no explicit array type in elasticsearch. should I be configuring the index with a different type/structure? it seems to show the type that was configured right now which is string.

thanks,
Bashir

On Wed, Aug 8, 2012 at 12:03 AM, David Pilato david@pilato.fr wrote:
So, you can see that your mapping is not applied.
Remove your index magic and create it again.
Send the mapping
Verify that your mapping is correct
And only if correct, send your document and run your query again. It should work.

HTH
David

--

Le 8 août 2012 à 04:07, Bashir Eghbali bashir@gmail.com a écrit :

I get:
"tags":{"type":"string"}

It says type string but when I assign to tags at indexing time, I pass in an array of strings. I assume it handles the json array fine.

I tried term filter as well to no avail.

Any other ideas? does the terms query get tokenized at query time and then matched against a verbatim indexed field, resulting in prefix match?

On Tuesday, August 7, 2012 5:55:00 PM UTC-7, David Pilato wrote:
Are you sure that your mapping is applied?
What do you see when doing:
curl -XGET 'http://localhost:9200/magic/yourtype/_mapping'
also, did you try with Term filter instead of terms filter? Same result I suppose.

David

Le 8 août 2012 à 01:14, Bashir Eghbali bas...@gmail.com a écrit :

I've been scratching my head for some time on this one. Major brownie points for any guidance:
I have mapping:
mapping :show => { :properties => {...
:tags => {:type => 'string', :index_name => 'tag', :boost => 80, :index => 'not_analyzed'},
...}}

tags is an array of tags. For example:
"tags":["rails","ruby on rails"]

I run a query as such(notice no results):

curl -X GET "http://localhost:9200/magic/_search?from=0&page=1&per_page=24&size=24pretty=true" -d '{"sort":[[{"score":"desc"}]],"filter":{"terms":{"tags":["ruby on rails"]}},"size":24,"from":0}'
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 0,
"max_score" : null,
"hits" :
}

If I run it with "tags":["ruby"], it will match all documents that are tagged "ruby" and all documents tagged "ruby on rails". I don't want a substring search, I need an exact search and I thought if the field is "not_analyzed" that it would be an exact match. How to do this the right way. Notice that this is a filter and not query.

--
Bashir Eghbali

bashireghbali.com
linkedin

bashir · August 8, 2012, 9:28pm

that is how I've had my mapping, here is my mapping:
:magic => { :properties => {
:tags => {:type => 'string', :index_name => 'tag', :boost => 80, :index
=> 'not_analyzed'},
}}

is this not correct or you are suggesting to get rid of :index_name and
:boost?

On Wednesday, August 8, 2012 1:08:25 AM UTC-7, David Pilato wrote:

Sorry. Replace message by tags.

--

Le 8 août 2012 à 10:07, David Pilato <da...@pilato.fr <javascript:>> a
écrit :

Array of string can be considered as a String type.

So create your mapping like this:

$ curl -XPUT 'http://localhost:9200/magic/yourtype/_mapping' -d '
{
"yourtype" : {
"properties" : {
"message" : {"type" : "string", "index" : "not_analyzed"}
}
}
}
'

--

Le 8 août 2012 à 10:01, Bashir Eghbali <bas...@gmail.com <javascript:>> a
écrit :

If the mapping was not applied, I would venture that no search would work.
Search works in general just returns superset of data or no data on tags
with spaces.
I am not sure what the mapping should look like for the type as there is
no explicit array type in elasticsearch. should I be configuring the index
with a different type/structure? it seems to show the type that was
configured right now which is string.

thanks,
Bashir

On Wed, Aug 8, 2012 at 12:03 AM, David Pilato <da...@pilato.fr<javascript:>

wrote:

So, you can see that your mapping is not applied.
Remove your index magic and create it again.
Send the mapping
Verify that your mapping is correct
And only if correct, send your document and run your query again. It
should work.

HTH
David

--

Le 8 août 2012 à 04:07, Bashir Eghbali <bas...@gmail.com <javascript:>>
a écrit :

I get:
"tags":{"type":"string"}

It says type string but when I assign to tags at indexing time, I pass in
an array of strings. I assume it handles the json array fine.

I tried term filter as well to no avail.

Any other ideas? does the terms query get tokenized at query time and
then matched against a verbatim indexed field, resulting in prefix match?

On Tuesday, August 7, 2012 5:55:00 PM UTC-7, David Pilato wrote:

Are you sure that your mapping is applied?
What do you see when doing:

curl -XGET 'http://localhost:9200/magic/**yourtype/_mapping' http://localhost:9200/magic/yourtype/_mapping'

also, did you try with Term filter instead of terms filter? Same result
I suppose.

David

Le 8 août 2012 à 01:14, Bashir Eghbali bas...@gmail.com a écrit :

I've been scratching my head for some time on this one. Major brownie
points for any guidance:
I have mapping:
mapping :show => { :properties => {...
:tags => {:type => 'string', :index_name => 'tag', :boost => 80,
:index => 'not_analyzed'},
...}}

tags is an array of tags. For example:
"tags":["rails","ruby on rails"]

I run a query as such(notice no results):

curl -X GET "http://localhost:9200/magic/_**
search?from=0&page=1&per_page=24&size=24pretty=truehttp://localhost:9200/magic/_search?from=0&page=1&per_page=24&size=24pretty=true"
-d '{"sort":[[{"score":"desc"}]],"filter":{"terms":{"tags":["ruby
on rails"]}},"size":24,"from":0}'
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 0,
"max_score" : null,
"hits" :
}

If I run it with "tags":["ruby"], it will match all documents that are
tagged "ruby" and all documents tagged "ruby on rails". I don't want a
substring search, I need an exact search and I thought if the field is
"not_analyzed" that it would be an exact match. How to do this the
right way. Notice that this is a filter and not query.

--
Bashir Eghbali

bashireghbali.com
linkedin http://www.linkedin.com/in/bashireghbali

dadoonet · August 8, 2012, 9:34pm

I think it's correct. What are the results now?

--

Le 8 août 2012 à 23:28, Bashir Eghbali bashir@gmail.com a écrit :

that is how I've had my mapping, here is my mapping:
:magic => { :properties => {
:tags => {:type => 'string', :index_name => 'tag', :boost => 80, :index => 'not_analyzed'},
}}

is this not correct or you are suggesting to get rid of :index_name and :boost?

On Wednesday, August 8, 2012 1:08:25 AM UTC-7, David Pilato wrote:
Sorry. Replace message by tags.

--

Le 8 août 2012 à 10:07, David Pilato da...@pilato.fr a écrit :

Array of string can be considered as a String type.

So create your mapping like this:

$ curl -XPUT 'http://localhost:9200/magic/yourtype/_mapping' -d '
{
"yourtype" : {
"properties" : {
"message" : {"type" : "string", "index" : "not_analyzed"}
}
}
}
'

--

Le 8 août 2012 à 10:01, Bashir Eghbali bas...@gmail.com a écrit :

If the mapping was not applied, I would venture that no search would work. Search works in general just returns superset of data or no data on tags with spaces.
I am not sure what the mapping should look like for the type as there is no explicit array type in elasticsearch. should I be configuring the index with a different type/structure? it seems to show the type that was configured right now which is string.

thanks,
Bashir

On Wed, Aug 8, 2012 at 12:03 AM, David Pilato da...@pilato.fr wrote:
So, you can see that your mapping is not applied.
Remove your index magic and create it again.
Send the mapping
Verify that your mapping is correct
And only if correct, send your document and run your query again. It should work.

HTH
David

--

Le 8 août 2012 à 04:07, Bashir Eghbali bas...@gmail.com a écrit :

I get:
"tags":{"type":"string"}

It says type string but when I assign to tags at indexing time, I pass in an array of strings. I assume it handles the json array fine.

I tried term filter as well to no avail.

Any other ideas? does the terms query get tokenized at query time and then matched against a verbatim indexed field, resulting in prefix match?

On Tuesday, August 7, 2012 5:55:00 PM UTC-7, David Pilato wrote:
Are you sure that your mapping is applied?
What do you see when doing:
curl -XGET 'http://localhost:9200/magic/yourtype/_mapping'
also, did you try with Term filter instead of terms filter? Same result I suppose.

David

Le 8 août 2012 à 01:14, Bashir Eghbali bas...@gmail.com a écrit :

I've been scratching my head for some time on this one. Major brownie points for any guidance:
I have mapping:
mapping :show => { :properties => {...
:tags => {:type => 'string', :index_name => 'tag', :boost => 80, :index => 'not_analyzed'},
...}}

tags is an array of tags. For example:
"tags":["rails","ruby on rails"]

I run a query as such(notice no results):

curl -X GET "http://localhost:9200/magic/_search?from=0&page=1&per_page=24&size=24pretty=true" -d '{"sort":[[{"score":"desc"}]],"filter":{"terms":{"tags":["ruby on rails"]}},"size":24,"from":0}'
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 0,
"max_score" : null,
"hits" :
}

If I run it with "tags":["ruby"], it will match all documents that are tagged "ruby" and all documents tagged "ruby on rails". I don't want a substring search, I need an exact search and I thought if the field is "not_analyzed" that it would be an exact match. How to do this the right way. Notice that this is a filter and not query.

--
Bashir Eghbali

bashireghbali.com
linkedin

bashir · August 8, 2012, 10:46pm

same results, still "ruby on rails" returns no results whereas "ruby"
returns items tagged "ruby" or "ruby on rails". It seems like the tag
search is a prefix search namely if I have a tag "rails is great" and I
search for everything tagged "rails" it will return that article whereas I
want it to only return the article when I search for the tag "rails is
great". It factors in whitespace if I search for "ruby " (notice the space
after) it will return no documents. But if I search for "ruby" it will
match both "ruby" and "ruby on rails" which leads me to believe that the
index is tonkenized. Does the boost cause tokenization? not sure how boost
is implemented in lucene. Also specifying an index_name, does that cause
the index to be analyzed thus overwriting what's specified?

thanks,
bashir

On Wed, Aug 8, 2012 at 2:34 PM, David Pilato david@pilato.fr wrote:

I think it's correct. What are the results now?

--

Le 8 août 2012 à 23:28, Bashir Eghbali bashir@gmail.com a écrit :

that is how I've had my mapping, here is my mapping:
:magic => { :properties => {
:tags => {:type => 'string', :index_name => 'tag', :boost => 80,
:index => 'not_analyzed'},
}}

is this not correct or you are suggesting to get rid of :index_name and
:boost?

On Wednesday, August 8, 2012 1:08:25 AM UTC-7, David Pilato wrote:

Sorry. Replace message by tags.

--

Le 8 août 2012 à 10:07, David Pilato da...@pilato.fr a écrit :

Array of string can be considered as a String type.

So create your mapping like this:

$ curl -XPUT 'http://localhost:9200/magic/**yourtype/_mapping' http://localhost:9200/magic/yourtype/_mapping' -d '
{
"yourtype" : {
"properties" : {
"message" : {"type" : "string", "index" : "not_analyzed"}
}
}
}
'

--

Le 8 août 2012 à 10:01, Bashir Eghbali bas...@gmail.com a écrit :

If the mapping was not applied, I would venture that no search would
work. Search works in general just returns superset of data or no data on
tags with spaces.
I am not sure what the mapping should look like for the type as there is
no explicit array type in elasticsearch. should I be configuring the index
with a different type/structure? it seems to show the type that was
configured right now which is string.

thanks,
Bashir

On Wed, Aug 8, 2012 at 12:03 AM, David Pilato da...@pilato.fr wrote:

So, you can see that your mapping is not applied.
Remove your index magic and create it again.
Send the mapping
Verify that your mapping is correct
And only if correct, send your document and run your query again. It
should work.

HTH
David

--

Le 8 août 2012 à 04:07, Bashir Eghbali bas...@gmail.com a écrit :

I get:
"tags":{"type":"string"}

It says type string but when I assign to tags at indexing time, I pass
in an array of strings. I assume it handles the json array fine.

I tried term filter as well to no avail.

Any other ideas? does the terms query get tokenized at query time and
then matched against a verbatim indexed field, resulting in prefix match?

On Tuesday, August 7, 2012 5:55:00 PM UTC-7, David Pilato wrote:

Are you sure that your mapping is applied?
What do you see when doing:

curl -XGET 'http://localhost:9200/magic/**y**ourtype/_mapping' http://localhost:9200/magic/yourtype/_mapping'

also, did you try with Term filter instead of terms filter? Same result
I suppose.

David

Le 8 août 2012 à 01:14, Bashir Eghbali bas...@gmail.com a écrit :

I've been scratching my head for some time on this one. Major brownie
points for any guidance:
I have mapping:
mapping :show => { :properties => {...
:tags => {:type => 'string', :index_name => 'tag', :boost => 80,
:index => 'not_analyzed'},
...}}

tags is an array of tags. For example:
"tags":["rails","ruby on rails"]

I run a query as such(notice no results):

curl -X GET "http://localhost:9200/magic/_****
search?from=0&page=1&per_page=24&size=24pretty=truehttp://localhost:9200/magic/_search?from=0&page=1&per_page=24&size=24pretty=true"
-d '{"sort":[[{"score":"desc"}]],"filter":{"terms":{"tags":["ruby
on rails"]}},"size":24,"from":0}'****
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 0,
"max_score" : null,
"hits" :
}

If I run it with "tags":["ruby"], it will match all documents that are
tagged "ruby" and all documents tagged "ruby on rails". I don't want a
substring search, I need an exact search and I thought if the field is
"not_analyzed" that it would be an exact match. How to do this the
right way. Notice that this is a filter and not query.

--
Bashir Eghbali

bashireghbali.com
linkedin http://www.linkedin.com/in/bashireghbali

--
Bashir Eghbali

bashireghbali.com
linkedin http://www.linkedin.com/in/bashireghbali

dadoonet · August 9, 2012, 5:05am

Could you create a full curl recreation and gist it?
See Elasticsearch Platform — Find real-time answers at scale | Elastic

--

Le 9 août 2012 à 00:46, Bashir Eghbali bashir@gmail.com a écrit :

same results, still "ruby on rails" returns no results whereas "ruby" returns items tagged "ruby" or "ruby on rails". It seems like the tag search is a prefix search namely if I have a tag "rails is great" and I search for everything tagged "rails" it will return that article whereas I want it to only return the article when I search for the tag "rails is great". It factors in whitespace if I search for "ruby " (notice the space after) it will return no documents. But if I search for "ruby" it will match both "ruby" and "ruby on rails" which leads me to believe that the index is tonkenized. Does the boost cause tokenization? not sure how boost is implemented in lucene. Also specifying an index_name, does that cause the index to be analyzed thus overwriting what's specified?

thanks,
bashir

On Wed, Aug 8, 2012 at 2:34 PM, David Pilato david@pilato.fr wrote:
I think it's correct. What are the results now?

--

Le 8 août 2012 à 23:28, Bashir Eghbali bashir@gmail.com a écrit :

that is how I've had my mapping, here is my mapping:
:magic => { :properties => {
:tags => {:type => 'string', :index_name => 'tag', :boost => 80, :index => 'not_analyzed'},
}}

is this not correct or you are suggesting to get rid of :index_name and :boost?

On Wednesday, August 8, 2012 1:08:25 AM UTC-7, David Pilato wrote:
Sorry. Replace message by tags.

--

Le 8 août 2012 à 10:07, David Pilato da...@pilato.fr a écrit :

Array of string can be considered as a String type.

So create your mapping like this:

$ curl -XPUT 'http://localhost:9200/magic/yourtype/_mapping' -d '
{
"yourtype" : {
"properties" : {
"message" : {"type" : "string", "index" : "not_analyzed"}
}
}
}
'

--

Le 8 août 2012 à 10:01, Bashir Eghbali bas...@gmail.com a écrit :

If the mapping was not applied, I would venture that no search would work. Search works in general just returns superset of data or no data on tags with spaces.
I am not sure what the mapping should look like for the type as there is no explicit array type in elasticsearch. should I be configuring the index with a different type/structure? it seems to show the type that was configured right now which is string.

thanks,
Bashir

On Wed, Aug 8, 2012 at 12:03 AM, David Pilato da...@pilato.fr wrote:
So, you can see that your mapping is not applied.
Remove your index magic and create it again.
Send the mapping
Verify that your mapping is correct
And only if correct, send your document and run your query again. It should work.

HTH
David

--

Le 8 août 2012 à 04:07, Bashir Eghbali bas...@gmail.com a écrit :

I get:
"tags":{"type":"string"}

It says type string but when I assign to tags at indexing time, I pass in an array of strings. I assume it handles the json array fine.

I tried term filter as well to no avail.

Any other ideas? does the terms query get tokenized at query time and then matched against a verbatim indexed field, resulting in prefix match?

On Tuesday, August 7, 2012 5:55:00 PM UTC-7, David Pilato wrote:
Are you sure that your mapping is applied?
What do you see when doing:
curl -XGET 'http://localhost:9200/magic/yourtype/_mapping'
also, did you try with Term filter instead of terms filter? Same result I suppose.

David

Le 8 août 2012 à 01:14, Bashir Eghbali bas...@gmail.com a écrit :

I've been scratching my head for some time on this one. Major brownie points for any guidance:
I have mapping:
mapping :show => { :properties => {...
:tags => {:type => 'string', :index_name => 'tag', :boost => 80, :index => 'not_analyzed'},
...}}

tags is an array of tags. For example:
"tags":["rails","ruby on rails"]

I run a query as such(notice no results):

curl -X GET "http://localhost:9200/magic/_search?from=0&page=1&per_page=24&size=24pretty=true" -d '{"sort":[[{"score":"desc"}]],"filter":{"terms":{"tags":["ruby on rails"]}},"size":24,"from":0}'
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 0,
"max_score" : null,
"hits" :
}

If I run it with "tags":["ruby"], it will match all documents that are tagged "ruby" and all documents tagged "ruby on rails". I don't want a substring search, I need an exact search and I thought if the field is "not_analyzed" that it would be an exact match. How to do this the right way. Notice that this is a filter and not query.

--
Bashir Eghbali

bashireghbali.com
linkedin

--
Bashir Eghbali

bashireghbali.com
linkedin