Help with Elastic search multi clause query

Elastic Stack Elasticsearch
1

Hi all
I need some help with Java DSL for Elastic search.

I have the below query:

{

"bool" : {

"must" : {

  "term" : {

    "pl" : "728x90"

  }

},

"must_not" : {

  "term" : {

    "bi" : "qa"

  }

}

}

}

The DSL corresponding is

QueryBuilder biBuilder = QueryBuilders.termQuery(

"bi", val);

QueryBuilder plBuilder = QueryBuilders.termQuery(

"pl", plval);

QueryBuilders.boolQuery()

.must(plBuilder)

.mustNot(biBuilder);

But in the results , I am getting records that has "bi" = "qa". I am
supposed to get results which match "pl=728x90" AND that doesnt have
"bi=qa". Please let me know if I am doing something wrong.

Thanks

G

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

Can you provide a few examples documents that are being matched but should
not be? The mapping would also be helpful.

Term queries are not analyzed, so they will only match exactly
(case sensitive) the token in the index. Also, if you do not need to have
your items ranked/score, it would be more efficient to have that query as a
filter instead.

--
Ivan

On Tue, Mar 26, 2013 at 11:27 AM, george sheeba.george@gmail.com wrote:

Hi all
I need some help with Java DSL for Elastic search.

I have the below query:

{

"bool" : {

"must" : {

  "term" : {

    "pl" : "728x90"

  }

},

"must_not" : {

  "term" : {

    "bi" : "qa"

  }

}

}

}

The DSL corresponding is

QueryBuilder biBuilder = QueryBuilders.termQuery(

"bi", val);

QueryBuilder plBuilder = QueryBuilders.termQuery(

"pl", plval);

QueryBuilders.boolQuery()

.must(plBuilder)

.mustNot(biBuilder);

But in the results , I am getting records that has "bi" = "qa". I am
supposed to get results which match "pl=728x90" AND that doesnt have
"bi=qa". Please let me know if I am doing something wrong.

Thanks

G

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

3

Hi Ivan
Sorry for the delay. The issue turned out that we were using "." (eg :
xyz.1) in the searchable fields . Looks like ES treats "." as special
character and during tokenization it is replaced with space.
I am thinking of using "not_analyzed" for strings that needn't be
tokenized. Do you recommend using cache with filters? In that case I assume
we'll loose all the updates unless the expiry is too short .

Thanks
George

On Wed, Mar 27, 2013 at 11:31 AM, Ivan Brusic ivan@brusic.com wrote:

Can you provide a few examples documents that are being matched but should
not be? The mapping would also be helpful.

Term queries are not analyzed, so they will only match exactly
(case sensitive) the token in the index. Also, if you do not need to have
your items ranked/score, it would be more efficient to have that query as a
filter instead.

--
Ivan

On Tue, Mar 26, 2013 at 11:27 AM, george sheeba.george@gmail.com wrote:

Hi all
I need some help with Java DSL for Elastic search.

I have the below query:

{

"bool" : {

"must" : {

  "term" : {

    "pl" : "728x90"

  }

},

"must_not" : {

  "term" : {

    "bi" : "qa"

  }

}

}

}

The DSL corresponding is

QueryBuilder biBuilder = QueryBuilders.termQuery(

"bi", val);

QueryBuilder plBuilder = QueryBuilders.termQuery(

"pl", plval);

QueryBuilders.boolQuery()

.must(plBuilder)

.mustNot(biBuilder);

But in the results , I am getting records that has "bi" = "qa". I am
supposed to get results which match "pl=728x90" AND that doesnt have
"bi=qa". Please let me know if I am doing something wrong.

Thanks

G

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/34DmI31cMlw/unsubscribe?hl=en-US
.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Sheeba Ann George

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

4

If a string does not need to be tokenized, then not analyzing almost always
makes sense.

The caches should be invalidated when a document is updated. You also have
the option of turning off caching on a filter-by-filter basis.

--
Ivan

On Wed, Apr 3, 2013 at 11:50 AM, Sheeba George sheeba.george@gmail.comwrote:

Hi Ivan
Sorry for the delay. The issue turned out that we were using "." (eg :
xyz.1) in the searchable fields . Looks like ES treats "." as special
character and during tokenization it is replaced with space.
I am thinking of using "not_analyzed" for strings that needn't be
tokenized. Do you recommend using cache with filters? In that case I assume
we'll loose all the updates unless the expiry is too short .

Thanks
George

On Wed, Mar 27, 2013 at 11:31 AM, Ivan Brusic ivan@brusic.com wrote:

Can you provide a few examples documents that are being matched but
should not be? The mapping would also be helpful.

Term queries are not analyzed, so they will only match exactly
(case sensitive) the token in the index. Also, if you do not need to have
your items ranked/score, it would be more efficient to have that query as a
filter instead.

--
Ivan

On Tue, Mar 26, 2013 at 11:27 AM, george sheeba.george@gmail.com wrote:

Hi all
I need some help with Java DSL for Elastic search.

I have the below query:

{

"bool" : {

"must" : {

  "term" : {

    "pl" : "728x90"

  }

},

"must_not" : {

  "term" : {

    "bi" : "qa"

  }

}

}

}

The DSL corresponding is

QueryBuilder biBuilder = QueryBuilders.termQuery(

"bi", val);

QueryBuilder plBuilder = QueryBuilders.termQuery(

"pl", plval);

QueryBuilders.boolQuery()

.must(plBuilder)

.mustNot(biBuilder);

But in the results , I am getting records that has "bi" = "qa". I am
supposed to get results which match "pl=728x90" AND that doesnt have
"bi=qa". Please let me know if I am doing something wrong.

Thanks

G

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/34DmI31cMlw/unsubscribe?hl=en-US
.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
Sheeba Ann George

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

5

Thanks Ivan. When we set cachable as true, are the results of the filter
also cached or just the filter query is cached. Can you please confirm.
I am using Java DSL for filter query and had to wrap with a constant score
(we dont need scoring) query to get the results. Does the below query
look fine?

{

"constant_score" : {

"filter" : {

  "bool" : {

    "must" : {

      "term" : {

        "p" : "120"

      }

    },

    "must" : {

      "terms" : {

        "c" : [ "gc" ],

        "execution" : "bool"

      }

    },

    "must" : {

      "term" : {

        "b" : "bbb"

      }

    }

  }

}

}

}

On Thu, Apr 4, 2013 at 7:39 AM, Ivan Brusic ivan@brusic.com wrote:

If a string does not need to be tokenized, then not analyzing almost
always makes sense.

The caches should be invalidated when a document is updated. You also have
the option of turning off caching on a filter-by-filter basis.

--
Ivan

On Wed, Apr 3, 2013 at 11:50 AM, Sheeba George sheeba.george@gmail.comwrote:

Hi Ivan
Sorry for the delay. The issue turned out that we were using "." (eg :
xyz.1) in the searchable fields . Looks like ES treats "." as special
character and during tokenization it is replaced with space.
I am thinking of using "not_analyzed" for strings that needn't be
tokenized. Do you recommend using cache with filters? In that case I assume
we'll loose all the updates unless the expiry is too short .

Thanks
George

On Wed, Mar 27, 2013 at 11:31 AM, Ivan Brusic ivan@brusic.com wrote:

Can you provide a few examples documents that are being matched but
should not be? The mapping would also be helpful.

Term queries are not analyzed, so they will only match exactly
(case sensitive) the token in the index. Also, if you do not need to have
your items ranked/score, it would be more efficient to have that query as a
filter instead.

--
Ivan

On Tue, Mar 26, 2013 at 11:27 AM, george sheeba.george@gmail.comwrote:

Hi all
I need some help with Java DSL for Elastic search.

I have the below query:

{

"bool" : {

"must" : {

  "term" : {

    "pl" : "728x90"

  }

},

"must_not" : {

  "term" : {

    "bi" : "qa"

  }

}

}

}

The DSL corresponding is

QueryBuilder biBuilder = QueryBuilders.termQuery(

"bi", val);

QueryBuilder plBuilder = QueryBuilders.termQuery(

"pl", plval);

QueryBuilders.boolQuery()

.must(plBuilder)

.mustNot(biBuilder);

But in the results , I am getting records that has "bi" = "qa". I am
supposed to get results which match "pl=728x90" AND that doesnt have
"bi=qa". Please let me know if I am doing something wrong.

Thanks

G

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/34DmI31cMlw/unsubscribe?hl=en-US
.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
Sheeba Ann George

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/34DmI31cMlw/unsubscribe?hl=en-US
.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Sheeba Ann George

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

6

Only filters are cached in Lucene since they are just bitsets. I have not
used a filtered query, so I don't know if it cached, but if the API has a
cache method, then it supports it.

--
Ivan

On Thu, Apr 4, 2013 at 9:17 AM, Sheeba George sheeba.george@gmail.comwrote:

Thanks Ivan. When we set cachable as true, are the results of the filter
also cached or just the filter query is cached. Can you please confirm.
I am using Java DSL for filter query and had to wrap with a constant score
(we dont need scoring) query to get the results. Does the below query
look fine?

{

"constant_score" : {

"filter" : {

  "bool" : {

    "must" : {

      "term" : {

        "p" : "120"

      }

    },

    "must" : {

      "terms" : {

        "c" : [ "gc" ],

        "execution" : "bool"

      }

    },

    "must" : {

      "term" : {

        "b" : "bbb"

      }

    }

  }

}

}

}

On Thu, Apr 4, 2013 at 7:39 AM, Ivan Brusic ivan@brusic.com wrote:

If a string does not need to be tokenized, then not analyzing almost
always makes sense.

The caches should be invalidated when a document is updated. You also
have the option of turning off caching on a filter-by-filter basis.

--
Ivan

On Wed, Apr 3, 2013 at 11:50 AM, Sheeba George sheeba.george@gmail.comwrote:

Hi Ivan
Sorry for the delay. The issue turned out that we were using "." (eg :
xyz.1) in the searchable fields . Looks like ES treats "." as special
character and during tokenization it is replaced with space.
I am thinking of using "not_analyzed" for strings that needn't be
tokenized. Do you recommend using cache with filters? In that case I assume
we'll loose all the updates unless the expiry is too short .

Thanks
George

On Wed, Mar 27, 2013 at 11:31 AM, Ivan Brusic ivan@brusic.com wrote:

Can you provide a few examples documents that are being matched but
should not be? The mapping would also be helpful.

Term queries are not analyzed, so they will only match exactly
(case sensitive) the token in the index. Also, if you do not need to have
your items ranked/score, it would be more efficient to have that query as a
filter instead.

--
Ivan

On Tue, Mar 26, 2013 at 11:27 AM, george sheeba.george@gmail.comwrote:

Hi all
I need some help with Java DSL for Elastic search.

I have the below query:

{

"bool" : {

"must" : {

  "term" : {

    "pl" : "728x90"

  }

},

"must_not" : {

  "term" : {

    "bi" : "qa"

  }

}

}

}

The DSL corresponding is

QueryBuilder biBuilder = QueryBuilders.termQuery(

"bi", val);

QueryBuilder plBuilder = QueryBuilders.termQuery(

"pl", plval);

QueryBuilders.boolQuery()

.must(plBuilder)

.mustNot(biBuilder);

But in the results , I am getting records that has "bi" = "qa". I am
supposed to get results which match "pl=728x90" AND that doesnt have
"bi=qa". Please let me know if I am doing something wrong.

Thanks

G

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/34DmI31cMlw/unsubscribe?hl=en-US
.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
Sheeba Ann George

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/34DmI31cMlw/unsubscribe?hl=en-US
.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Sheeba Ann George

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

7

Thanks a lot, Ivan

Sent from my iPhone

On Apr 5, 2013, at 12:20 PM, Ivan Brusic ivan@brusic.com wrote:

Only filters are cached in Lucene since they are just bitsets. I have not used a filtered query, so I don't know if it cached, but if the API has a cache method, then it supports it.

--
Ivan

On Thu, Apr 4, 2013 at 9:17 AM, Sheeba George sheeba.george@gmail.com wrote:

Thanks Ivan. When we set cachable as true, are the results of the filter also cached or just the filter query is cached. Can you please confirm.
I am using Java DSL for filter query and had to wrap with a constant score (we dont need scoring) query to get the results. Does the below query look fine?
{

"constant_score" : {

"filter" : {

  "bool" : {

    "must" : {

      "term" : {

        "p" : "120"

      }

    },

    "must" : {

      "terms" : {

        "c" : [ "gc" ],

        "execution" : "bool"

      }

    },

    "must" : {

      "term" : {

        "b" : "bbb"

      }

    }

  }

}

}

}

On Thu, Apr 4, 2013 at 7:39 AM, Ivan Brusic ivan@brusic.com wrote:

If a string does not need to be tokenized, then not analyzing almost always makes sense.

The caches should be invalidated when a document is updated. You also have the option of turning off caching on a filter-by-filter basis.

--
Ivan

On Wed, Apr 3, 2013 at 11:50 AM, Sheeba George sheeba.george@gmail.com wrote:

Hi Ivan
Sorry for the delay. The issue turned out that we were using "." (eg : xyz.1) in the searchable fields . Looks like ES treats "." as special character and during tokenization it is replaced with space.
I am thinking of using "not_analyzed" for strings that needn't be tokenized. Do you recommend using cache with filters? In that case I assume we'll loose all the updates unless the expiry is too short .

Thanks
George

On Wed, Mar 27, 2013 at 11:31 AM, Ivan Brusic ivan@brusic.com wrote:

Can you provide a few examples documents that are being matched but should not be? The mapping would also be helpful.

Term queries are not analyzed, so they will only match exactly (case sensitive) the token in the index. Also, if you do not need to have your items ranked/score, it would be more efficient to have that query as a filter instead.

--
Ivan

On Tue, Mar 26, 2013 at 11:27 AM, george sheeba.george@gmail.com wrote:

Hi all
I need some help with Java DSL for Elastic search.

I have the below query:
{

"bool" : {

"must" : {

  "term" : {

    "pl" : "728x90"

  }

},

"must_not" : {

  "term" : {

    "bi" : "qa"

  }

}

}

}

The DSL corresponding is

QueryBuilder biBuilder = QueryBuilders.termQuery(

"bi", val);

QueryBuilder plBuilder = QueryBuilders.termQuery(

"pl", plval);

QueryBuilders.boolQuery()

.must(plBuilder)

.mustNot(biBuilder);

But in the results , I am getting records that has "bi" = "qa". I am supposed to get results which match "pl=728x90" AND that doesnt have "bi=qa". Please let me know if I am doing something wrong.

Thanks

G

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/34DmI31cMlw/unsubscribe?hl=en-US.
To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
Sheeba Ann George

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/34DmI31cMlw/unsubscribe?hl=en-US.
To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Sheeba Ann George

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/34DmI31cMlw/unsubscribe?hl=en-US.
To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.