# Stop Words in Quoted vs Unquoted Search

**URL:** <https://discuss.elastic.co/t/stop-words-in-quoted-vs-unquoted-search/7276>\
**Category:** Elasticsearch\
**Created:** [April 9, 2012, 6:24pm UTC](https://discuss.elastic.co/t/stop-words-in-quoted-vs-unquoted-search/7276 "2012-04-09T18:24:51Z")\
**Posts on this page:** 19\
**Page:** 1

<div class="post-metadata">

**Author:** ![Kenneth\_Loafman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kenneth_loafman/32/2224_2.png) [@Kenneth\_Loafman](https://discuss.elastic.co/u/Kenneth_Loafman)\
**Post date:** [April 9, 2012, 6:24pm UTC](https://discuss.elastic.co/t/stop-words-in-quoted-vs-unquoted-search/7276/1 "2012-04-09T18:24:51Z")

</div>

Hi,

We're seeing an odd problem in search that revolves around stop words.

If I use the query (no quotes):  
worst way to travel  
ES returns nothing.

If I use the query (with quotes):  
"worst way to travel"  
ES returns matches.

Why the difference? Do I need to remove stop words manually from a  
non-quoted query?

...Ken

---

<div class="post-metadata">

**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\
**Post date:** [April 11, 2012, 11:30am UTC](https://discuss.elastic.co/t/stop-words-in-quoted-vs-unquoted-search/7276/2 "2012-04-11T11:30:19Z")

</div>

Thats strange, the first option simply translates to a boolean query and  
the other one translates to a phrase query. Can you gist a recreation?

On Mon, Apr 9, 2012 at 9:24 PM, Kenneth Loafman  
[kenneth.loafman@gmail.com](mailto:kenneth.loafman@gmail.com)wrote:

> Hi,
> 
> We're seeing an odd problem in search that revolves around stop words.
> 
> If I use the query (no quotes):  
> worst way to travel  
> ES returns nothing.
> 
> If I use the query (with quotes):  
> "worst way to travel"  
> ES returns matches.
> 
> Why the difference? Do I need to remove stop words manually from a  
> non-quoted query?
> 
> ...Ken

---

<div class="post-metadata">

**Author:** ![Kenneth\_Loafman\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kenneth_loafman_2/32/2224_2.png) [@Kenneth\_Loafman\_2](https://discuss.elastic.co/u/Kenneth_Loafman_2)\
**Post date:** [April 11, 2012, 2:30pm UTC](https://discuss.elastic.co/t/stop-words-in-quoted-vs-unquoted-search/7276/3 "2012-04-11T14:30:30Z")

</div>

Here's the gist: [Stop Words in Quoted vs Unquoted Search · GitHub](https://gist.github.com/2359644)

...Thanks,  
...Ken

On Wed, Apr 11, 2012 at 6:30 AM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote:

> Thats strange, the first option simply translates to a boolean query and  
> the other one translates to a phrase query. Can you gist a recreation?
> 
> On Mon, Apr 9, 2012 at 9:24 PM, Kenneth Loafman \<[kenneth.loafman@gmail.com](mailto:kenneth.loafman@gmail.com)
> 
> > wrote:
> 
> > Hi,
> > 
> > We're seeing an odd problem in search that revolves around stop words.
> > 
> > If I use the query (no quotes):  
> > worst way to travel  
> > ES returns nothing.
> > 
> > If I use the query (with quotes):  
> > "worst way to travel"  
> > ES returns matches.
> > 
> > Why the difference? Do I need to remove stop words manually from a  
> > non-quoted query?
> > 
> > ...Ken

---

<div class="post-metadata">

**Author:** ![Igor\_Motov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_motov/32/45193_2.png) [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)\
**Post date:** [April 11, 2012, 4:13pm UTC](https://discuss.elastic.co/t/stop-words-in-quoted-vs-unquoted-search/7276/4 "2012-04-11T16:13:30Z")

</div>

This recreation doesn't seem to recreate the problem. The issue is most  
likely in the mapping, and unfortunately, it's not visible in your example.  
This is what I am getting when I am trying to run  
it: [Stop Words in Quoted vs Unquoted Search · GitHub](https://gist.github.com/2360279) (I had to remove publish\_datetime  
sort, since it's not populated in your gist).

On Wednesday, April 11, 2012 10:30:30 AM UTC-4, Kenneth Loafman wrote:

> Here's the gist: [Stop Words in Quoted vs Unquoted Search · GitHub](https://gist.github.com/2359644)
> 
> ...Thanks,  
> ...Ken
> 
> On Wed, Apr 11, 2012 at 6:30 AM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote:
> 
> > Thats strange, the first option simply translates to a boolean query and  
> > the other one translates to a phrase query. Can you gist a recreation?
> > 
> > On Mon, Apr 9, 2012 at 9:24 PM, Kenneth Loafman \<  
> > [kenneth.loafman@gmail.com](mailto:kenneth.loafman@gmail.com)\> wrote:
> > 
> > > Hi,
> > > 
> > > We're seeing an odd problem in search that revolves around stop words.
> > > 
> > > If I use the query (no quotes):  
> > > worst way to travel  
> > > ES returns nothing.
> > > 
> > > If I use the query (with quotes):  
> > > "worst way to travel"  
> > > ES returns matches.
> > > 
> > > Why the difference? Do I need to remove stop words manually from a  
> > > non-quoted query?
> > > 
> > > ...Ken

---

<div class="post-metadata">

**Author:** ![Kenneth\_Loafman\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kenneth_loafman_2/32/2224_2.png) [@Kenneth\_Loafman\_2](https://discuss.elastic.co/u/Kenneth_Loafman_2)\
**Post date:** [April 11, 2012, 8:28pm UTC](https://discuss.elastic.co/t/stop-words-in-quoted-vs-unquoted-search/7276/5 "2012-04-11T20:28:26Z")

</div>

It will run without the publish\_datetime sort, that's part of the gist I  
posted. Does NOT matter whether its populated or not.

...Ken

On Wed, Apr 11, 2012 at 11:13 AM, Igor Motov [imotov@gmail.com](mailto:imotov@gmail.com) wrote:

> This recreation doesn't seem to recreate the problem. The issue is most  
> likely in the mapping, and unfortunately, it's not visible in your example.  
> This is what I am getting when I am trying to run it:  
> [Stop Words in Quoted vs Unquoted Search · GitHub](https://gist.github.com/2360279) (I had to remove publish\_datetime sort,  
> since it's not populated in your gist).
> 
> On Wednesday, April 11, 2012 10:30:30 AM UTC-4, Kenneth Loafman wrote:
> 
> > Here's the gist: [https://gist.github.com/\*\*2359644](https://gist.github.com/**2359644)[https://gist.github.com/2359644](https://gist.github.com/2359644)
> > 
> > ...Thanks,  
> > ...Ken
> > 
> > On Wed, Apr 11, 2012 at 6:30 AM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote:
> > 
> > > Thats strange, the first option simply translates to a boolean query and  
> > > the other one translates to a phrase query. Can you gist a recreation?
> > > 
> > > On Mon, Apr 9, 2012 at 9:24 PM, Kenneth Loafman \<  
> > > [kenneth.loafman@gmail.com](mailto:kenneth.loafman@gmail.com)\> wrote:
> > > 
> > > > Hi,
> > > > 
> > > > We're seeing an odd problem in search that revolves around stop words.
> > > > 
> > > > If I use the query (no quotes):  
> > > > worst way to travel  
> > > > ES returns nothing.
> > > > 
> > > > If I use the query (with quotes):  
> > > > "worst way to travel"  
> > > > ES returns matches.
> > > > 
> > > > Why the difference? Do I need to remove stop words manually from a  
> > > > non-quoted query?
> > > > 
> > > > ...Ken

---

<div class="post-metadata">

**Author:** ![Kenneth\_Loafman\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kenneth_loafman_2/32/2224_2.png) [@Kenneth\_Loafman\_2](https://discuss.elastic.co/u/Kenneth_Loafman_2)\
**Post date:** [April 11, 2012, 8:55pm UTC](https://discuss.elastic.co/t/stop-words-in-quoted-vs-unquoted-search/7276/6 "2012-04-11T20:55:58Z")

</div>

Just to make sure, I updated the original gist with a test that has the  
full mapping. It does not work. The output is from 0.19.1.

...Ken

On Wed, Apr 11, 2012 at 3:28 PM, Kenneth Loafman [kenneth@loafman.com](mailto:kenneth@loafman.com)wrote:

> It will run without the publish\_datetime sort, that's part of the gist I  
> posted. Does NOT matter whether its populated or not.
> 
> ...Ken
> 
> On Wed, Apr 11, 2012 at 11:13 AM, Igor Motov [imotov@gmail.com](mailto:imotov@gmail.com) wrote:
> 
> > This recreation doesn't seem to recreate the problem. The issue is most  
> > likely in the mapping, and unfortunately, it's not visible in your example.  
> > This is what I am getting when I am trying to run it:  
> > [Stop Words in Quoted vs Unquoted Search · GitHub](https://gist.github.com/2360279) (I had to remove publish\_datetime sort,  
> > since it's not populated in your gist).
> > 
> > On Wednesday, April 11, 2012 10:30:30 AM UTC-4, Kenneth Loafman wrote:
> > 
> > > Here's the gist: [https://gist.github.com/\*\*2359644](https://gist.github.com/**2359644)[https://gist.github.com/2359644](https://gist.github.com/2359644)
> > > 
> > > ...Thanks,  
> > > ...Ken
> > > 
> > > On Wed, Apr 11, 2012 at 6:30 AM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote:
> > > 
> > > > Thats strange, the first option simply translates to a boolean query  
> > > > and the other one translates to a phrase query. Can you gist a recreation?
> > > > 
> > > > On Mon, Apr 9, 2012 at 9:24 PM, Kenneth Loafman \<  
> > > > [kenneth.loafman@gmail.com](mailto:kenneth.loafman@gmail.com)\> wrote:
> > > > 
> > > > > Hi,
> > > > > 
> > > > > We're seeing an odd problem in search that revolves around stop words.
> > > > > 
> > > > > If I use the query (no quotes):  
> > > > > worst way to travel  
> > > > > ES returns nothing.
> > > > > 
> > > > > If I use the query (with quotes):  
> > > > > "worst way to travel"  
> > > > > ES returns matches.
> > > > > 
> > > > > Why the difference? Do I need to remove stop words manually from a  
> > > > > non-quoted query?
> > > > > 
> > > > > ...Ken

---

<div class="post-metadata">

**Author:** ![Kenneth\_Loafman\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kenneth_loafman_2/32/2224_2.png) [@Kenneth\_Loafman\_2](https://discuss.elastic.co/u/Kenneth_Loafman_2)\
**Post date:** [April 11, 2012, 9:13pm UTC](https://discuss.elastic.co/t/stop-words-in-quoted-vs-unquoted-search/7276/7 "2012-04-11T21:13:13Z")

</div>

0.18.7 gives the same results.

BTW, original is here: [Stop Words in Quoted vs Unquoted Search · GitHub](https://gist.github.com/2359644)

...Ken

On Wed, Apr 11, 2012 at 3:55 PM, Kenneth Loafman [kenneth@loafman.com](mailto:kenneth@loafman.com)wrote:

> Just to make sure, I updated the original gist with a test that has the  
> full mapping. It does not work. The output is from 0.19.1.
> 
> ...Ken
> 
> On Wed, Apr 11, 2012 at 3:28 PM, Kenneth Loafman [kenneth@loafman.com](mailto:kenneth@loafman.com)wrote:
> 
> > It will run without the publish\_datetime sort, that's part of the gist I  
> > posted. Does NOT matter whether its populated or not.
> > 
> > ...Ken
> > 
> > On Wed, Apr 11, 2012 at 11:13 AM, Igor Motov [imotov@gmail.com](mailto:imotov@gmail.com) wrote:
> > 
> > > This recreation doesn't seem to recreate the problem. The issue is most  
> > > likely in the mapping, and unfortunately, it's not visible in your example.  
> > > This is what I am getting when I am trying to run it:  
> > > [Stop Words in Quoted vs Unquoted Search · GitHub](https://gist.github.com/2360279) (I had to remove publish\_datetime  
> > > sort, since it's not populated in your gist).
> > > 
> > > On Wednesday, April 11, 2012 10:30:30 AM UTC-4, Kenneth Loafman wrote:
> > > 
> > > > Here's the gist: [https://gist.github.com/\*\*2359644](https://gist.github.com/**2359644)[https://gist.github.com/2359644](https://gist.github.com/2359644)
> > > > 
> > > > ...Thanks,  
> > > > ...Ken
> > > > 
> > > > On Wed, Apr 11, 2012 at 6:30 AM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote:
> > > > 
> > > > > Thats strange, the first option simply translates to a boolean query  
> > > > > and the other one translates to a phrase query. Can you gist a recreation?
> > > > > 
> > > > > On Mon, Apr 9, 2012 at 9:24 PM, Kenneth Loafman \<  
> > > > > [kenneth.loafman@gmail.com](mailto:kenneth.loafman@gmail.com)\> wrote:
> > > > > 
> > > > > > Hi,
> > > > > > 
> > > > > > We're seeing an odd problem in search that revolves around stop words.
> > > > > > 
> > > > > > If I use the query (no quotes):  
> > > > > > worst way to travel  
> > > > > > ES returns nothing.
> > > > > > 
> > > > > > If I use the query (with quotes):  
> > > > > > "worst way to travel"  
> > > > > > ES returns matches.
> > > > > > 
> > > > > > Why the difference? Do I need to remove stop words manually from a  
> > > > > > non-quoted query?
> > > > > > 
> > > > > > ...Ken

---

<div class="post-metadata">

**Author:** ![Igor\_Motov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_motov/32/45193_2.png) [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)\
**Post date:** [April 11, 2012, 10:06pm UTC](https://discuss.elastic.co/t/stop-words-in-quoted-vs-unquoted-search/7276/8 "2012-04-11T22:06:14Z")

</div>

I still cannot reproduce it. Something is missing. Most likely it's index  
settings. Could you share your config file or add output of the following  
command to the repro?

curl -XGET "localhost:9200/test/\_settings"

On Wednesday, April 11, 2012 5:13:13 PM UTC-4, Kenneth Loafman wrote:

> 0.18.7 gives the same results.
> 
> BTW, original is here: [Stop Words in Quoted vs Unquoted Search · GitHub](https://gist.github.com/2359644)
> 
> ...Ken
> 
> On Wed, Apr 11, 2012 at 3:55 PM, Kenneth Loafman [kenneth@loafman.com](mailto:kenneth@loafman.com)wrote:
> 
> > Just to make sure, I updated the original gist with a test that has the  
> > full mapping. It does not work. The output is from 0.19.1.
> > 
> > ...Ken
> > 
> > On Wed, Apr 11, 2012 at 3:28 PM, Kenneth Loafman [kenneth@loafman.com](mailto:kenneth@loafman.com)wrote:
> > 
> > > It will run without the publish\_datetime sort, that's part of the gist I  
> > > posted. Does NOT matter whether its populated or not.
> > > 
> > > ...Ken
> > > 
> > > On Wed, Apr 11, 2012 at 11:13 AM, Igor Motov [imotov@gmail.com](mailto:imotov@gmail.com) wrote:
> > > 
> > > > This recreation doesn't seem to recreate the problem. The issue is most  
> > > > likely in the mapping, and unfortunately, it's not visible in your example.  
> > > > This is what I am getting when I am trying to run it:  
> > > > [Stop Words in Quoted vs Unquoted Search · GitHub](https://gist.github.com/2360279) (I had to remove publish\_datetime  
> > > > sort, since it's not populated in your gist).
> > > > 
> > > > On Wednesday, April 11, 2012 10:30:30 AM UTC-4, Kenneth Loafman wrote:
> > > > 
> > > > > Here's the gist: [https://gist.github.com/\*\*2359644](https://gist.github.com/**2359644)[https://gist.github.com/2359644](https://gist.github.com/2359644)
> > > > > 
> > > > > ...Thanks,  
> > > > > ...Ken
> > > > > 
> > > > > On Wed, Apr 11, 2012 at 6:30 AM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote:
> > > > > 
> > > > > > Thats strange, the first option simply translates to a boolean query  
> > > > > > and the other one translates to a phrase query. Can you gist a recreation?
> > > > > > 
> > > > > > On Mon, Apr 9, 2012 at 9:24 PM, Kenneth Loafman \<  
> > > > > > [kenneth.loafman@gmail.com](mailto:kenneth.loafman@gmail.com)\> wrote:
> > > > > > 
> > > > > > > Hi,
> > > > > > > 
> > > > > > > We're seeing an odd problem in search that revolves around stop  
> > > > > > > words.
> > > > > > > 
> > > > > > > If I use the query (no quotes):  
> > > > > > > worst way to travel  
> > > > > > > ES returns nothing.
> > > > > > > 
> > > > > > > If I use the query (with quotes):  
> > > > > > > "worst way to travel"  
> > > > > > > ES returns matches.
> > > > > > > 
> > > > > > > Why the difference? Do I need to remove stop words manually from a  
> > > > > > > non-quoted query?
> > > > > > > 
> > > > > > > ...Ken

---

<div class="post-metadata">

**Author:** ![Kenneth\_Loafman\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kenneth_loafman_2/32/2224_2.png) [@Kenneth\_Loafman\_2](https://discuss.elastic.co/u/Kenneth_Loafman_2)\
**Post date:** [April 11, 2012, 10:12pm UTC](https://discuss.elastic.co/t/stop-words-in-quoted-vs-unquoted-search/7276/9 "2012-04-11T22:12:48Z")

</div>

Here it is:

~$ curl -XGET "localhost:9200/test/\_settings?pretty=true"  
{  
"test" : {  
"settings" : {  
"index.number\_of\_shards" : "2",  
"index.number\_of\_replicas" : "1",  
"index.version.created" : "190199"  
}  
}  
}

On Wed, Apr 11, 2012 at 5:06 PM, Igor Motov [imotov@gmail.com](mailto:imotov@gmail.com) wrote:

> I still cannot reproduce it. Something is missing. Most likely it's index  
> settings. Could you share your config file or add output of the following  
> command to the repro?
> 
> curl -XGET "localhost:9200/test/\_settings"
> 
> On Wednesday, April 11, 2012 5:13:13 PM UTC-4, Kenneth Loafman wrote:
> 
> > 0.18.7 gives the same results.
> > 
> > BTW, original is here: [https://gist.github.com/\*\*2359644](https://gist.github.com/**2359644)[https://gist.github.com/2359644](https://gist.github.com/2359644)
> > 
> > ...Ken
> > 
> > On Wed, Apr 11, 2012 at 3:55 PM, Kenneth Loafman [kenneth@loafman.com](mailto:kenneth@loafman.com)wrote:
> > 
> > > Just to make sure, I updated the original gist with a test that has the  
> > > full mapping. It does not work. The output is from 0.19.1.
> > > 
> > > ...Ken
> > > 
> > > On Wed, Apr 11, 2012 at 3:28 PM, Kenneth Loafman [kenneth@loafman.com](mailto:kenneth@loafman.com)wrote:
> > > 
> > > > It will run without the publish\_datetime sort, that's part of the gist  
> > > > I posted. Does NOT matter whether its populated or not.
> > > > 
> > > > ...Ken
> > > > 
> > > > On Wed, Apr 11, 2012 at 11:13 AM, Igor Motov [imotov@gmail.com](mailto:imotov@gmail.com) wrote:
> > > > 
> > > > > This recreation doesn't seem to recreate the problem. The issue is  
> > > > > most likely in the mapping, and unfortunately, it's not visible in your  
> > > > > example. This is what I am getting when I am trying to run it:  
> > > > > [https://gist.github.com/\*\*2360279](https://gist.github.com/**2360279) [https://gist.github.com/2360279](https://gist.github.com/2360279) (I had to remove publish\_datetime sort, since it's not populated in your  
> > > > > gist).
> > > > > 
> > > > > On Wednesday, April 11, 2012 10:30:30 AM UTC-4, Kenneth Loafman wrote:
> > > > > 
> > > > > > Here's the gist: [https://gist.github.com/\*\*\*\*2359644](https://gist.github.com/ **** 2359644)[https://gist.github.com/2359644](https://gist.github.com/2359644)
> > > > > > 
> > > > > > ...Thanks,  
> > > > > > ...Ken
> > > > > > 
> > > > > > On Wed, Apr 11, 2012 at 6:30 AM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote:
> > > > > > 
> > > > > > > Thats strange, the first option simply translates to a boolean query  
> > > > > > > and the other one translates to a phrase query. Can you gist a recreation?
> > > > > > > 
> > > > > > > On Mon, Apr 9, 2012 at 9:24 PM, Kenneth Loafman \<  
> > > > > > > [kenneth.loafman@gmail.com](mailto:kenneth.loafman@gmail.com)\> wrote:
> > > > > > > 
> > > > > > > > Hi,
> > > > > > > > 
> > > > > > > > We're seeing an odd problem in search that revolves around stop  
> > > > > > > > words.
> > > > > > > > 
> > > > > > > > If I use the query (no quotes):  
> > > > > > > > worst way to travel  
> > > > > > > > ES returns nothing.
> > > > > > > > 
> > > > > > > > If I use the query (with quotes):  
> > > > > > > > "worst way to travel"  
> > > > > > > > ES returns matches.
> > > > > > > > 
> > > > > > > > Why the difference? Do I need to remove stop words manually from a  
> > > > > > > > non-quoted query?
> > > > > > > > 
> > > > > > > > ...Ken

---

<div class="post-metadata">

**Author:** ![Kenneth\_Loafman\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kenneth_loafman_2/32/2224_2.png) [@Kenneth\_Loafman\_2](https://discuss.elastic.co/u/Kenneth_Loafman_2)\
**Post date:** [April 11, 2012, 10:17pm UTC](https://discuss.elastic.co/t/stop-words-in-quoted-vs-unquoted-search/7276/10 "2012-04-11T22:17:11Z")

</div>

Here's elasticsearch.yml just in case...

gateway:  
type: local

path :  
data : /mnt/search-data-dev/node0/data  
work : /mnt/search-data-dev/node0/work  
logs : /mnt/search-data-dev/node0/logs

index :  
number\_of\_shards : 2  
number\_of\_replicas : 1

bootstrap:  
mlockall: true

action:  
disable\_delete\_all\_indexes: true

network :  
host : localhost

discovery.zen.ping.multicast:  
enabled: false

discovery.zen.ping.unicast:  
hosts: ["localhost:9300","localhost:9301"]

# Shard level query and fetch threshold logging.

#index.search.slowlog.level: TRACE  
index.search.slowlog.threshold.query.warn: 10s  
index.search.slowlog.threshold.query.info: 5s  
#index.search.slowlog.threshold.query.debug: 2s  
#index.search.slowlog.threshold.query.trace: 500ms

index.search.slowlog.threshold.fetch.warn: 1s  
index.search.slowlog.threshold.fetch.info: 800ms  
#index.search.slowlog.threshold.fetch.debug: 500ms  
#index.search.slowlog.threshold.fetch.trace: 200ms

On Wed, Apr 11, 2012 at 5:12 PM, Kenneth Loafman [kenneth@loafman.com](mailto:kenneth@loafman.com)wrote:

> Here it is:
> 
> ~$ curl -XGET "localhost:9200/test/\_settings?pretty=true"  
> {  
> "test" : {  
> "settings" : {  
> "index.number\_of\_shards" : "2",  
> "index.number\_of\_replicas" : "1",  
> "index.version.created" : "190199"  
> }  
> }  
> }
> 
> On Wed, Apr 11, 2012 at 5:06 PM, Igor Motov [imotov@gmail.com](mailto:imotov@gmail.com) wrote:
> 
> > I still cannot reproduce it. Something is missing. Most likely it's index  
> > settings. Could you share your config file or add output of the following  
> > command to the repro?
> > 
> > curl -XGET "localhost:9200/test/\_settings"
> > 
> > On Wednesday, April 11, 2012 5:13:13 PM UTC-4, Kenneth Loafman wrote:
> > 
> > > 0.18.7 gives the same results.
> > > 
> > > BTW, original is here: [https://gist.github.com/\*\*2359644](https://gist.github.com/**2359644)[https://gist.github.com/2359644](https://gist.github.com/2359644)
> > > 
> > > ...Ken
> > > 
> > > On Wed, Apr 11, 2012 at 3:55 PM, Kenneth Loafman [kenneth@loafman.com](mailto:kenneth@loafman.com)wrote:
> > > 
> > > > Just to make sure, I updated the original gist with a test that has the  
> > > > full mapping. It does not work. The output is from 0.19.1.
> > > > 
> > > > ...Ken
> > > > 
> > > > On Wed, Apr 11, 2012 at 3:28 PM, Kenneth Loafman [kenneth@loafman.com](mailto:kenneth@loafman.com)wrote:
> > > > 
> > > > > It will run without the publish\_datetime sort, that's part of the gist  
> > > > > I posted. Does NOT matter whether its populated or not.
> > > > > 
> > > > > ...Ken
> > > > > 
> > > > > On Wed, Apr 11, 2012 at 11:13 AM, Igor Motov [imotov@gmail.com](mailto:imotov@gmail.com) wrote:
> > > > > 
> > > > > > This recreation doesn't seem to recreate the problem. The issue is  
> > > > > > most likely in the mapping, and unfortunately, it's not visible in your  
> > > > > > example. This is what I am getting when I am trying to run it:  
> > > > > > [https://gist.github.com/\*\*2360279](https://gist.github.com/**2360279) [https://gist.github.com/2360279](https://gist.github.com/2360279) (I had to remove publish\_datetime sort, since it's not populated in your  
> > > > > > gist).
> > > > > > 
> > > > > > On Wednesday, April 11, 2012 10:30:30 AM UTC-4, Kenneth Loafman wrote:
> > > > > > 
> > > > > > > Here's the gist: [https://gist.github.com/\*\*\*\*2359644](https://gist.github.com/ **** 2359644)[https://gist.github.com/2359644](https://gist.github.com/2359644)
> > > > > > > 
> > > > > > > ...Thanks,  
> > > > > > > ...Ken
> > > > > > > 
> > > > > > > On Wed, Apr 11, 2012 at 6:30 AM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com)wrote:
> > > > > > > 
> > > > > > > > Thats strange, the first option simply translates to a boolean  
> > > > > > > > query and the other one translates to a phrase query. Can you gist a  
> > > > > > > > recreation?
> > > > > > > > 
> > > > > > > > On Mon, Apr 9, 2012 at 9:24 PM, Kenneth Loafman \<  
> > > > > > > > [kenneth.loafman@gmail.com](mailto:kenneth.loafman@gmail.com)\> wrote:
> > > > > > > > 
> > > > > > > > > Hi,
> > > > > > > > > 
> > > > > > > > > We're seeing an odd problem in search that revolves around stop  
> > > > > > > > > words.
> > > > > > > > > 
> > > > > > > > > If I use the query (no quotes):  
> > > > > > > > > worst way to travel  
> > > > > > > > > ES returns nothing.
> > > > > > > > > 
> > > > > > > > > If I use the query (with quotes):  
> > > > > > > > > "worst way to travel"  
> > > > > > > > > ES returns matches.
> > > > > > > > > 
> > > > > > > > > Why the difference? Do I need to remove stop words manually from  
> > > > > > > > > a non-quoted query?
> > > > > > > > > 
> > > > > > > > > ...Ken

---

<div class="post-metadata">

**Author:** ![Igor\_Motov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_motov/32/45193_2.png) [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)\
**Post date:** [April 11, 2012, 10:30pm UTC](https://discuss.elastic.co/t/stop-words-in-quoted-vs-unquoted-search/7276/11 "2012-04-11T22:30:04Z")

</div>

The tricky part is this:

{"ok":true,"\_shards":{"total":4,_"successful":3_,"failed":0}}

I don't quite understand how this is possible. If you had one node running,  
you should have gotten "total":4, "successful":2. If you had two nodes  
running, you should have gotten "total":4, "successful":4. Having  
"total":4, "successful":3 most likely indicates that there is an issue with  
allocating one of the shards on one of the nodes. Do you see any errors in  
the log file on one of the nodes?

On Wednesday, April 11, 2012 6:17:11 PM UTC-4, Kenneth Loafman wrote:

> Here's elasticsearch.yml just in case...
> 
> gateway:  
> type: local
> 
> path :  
> data : /mnt/search-data-dev/node0/data  
> work : /mnt/search-data-dev/node0/work  
> logs : /mnt/search-data-dev/node0/logs
> 
> index :  
> number\_of\_shards : 2  
> number\_of\_replicas : 1
> 
> bootstrap:  
> mlockall: true
> 
> action:  
> disable\_delete\_all\_indexes: true
> 
> network :  
> host : localhost
> 
> discovery.zen.ping.multicast:  
> enabled: false
> 
> discovery.zen.ping.unicast:  
> hosts: ["localhost:9300","localhost:9301"]
> 
> # Shard level query and fetch threshold logging.
> 
> #index.search.slowlog.level: TRACE  
> index.search.slowlog.threshold.query.warn: 10s  
> index.search.slowlog.threshold.query.info: 5s  
> #index.search.slowlog.threshold.query.debug: 2s  
> #index.search.slowlog.threshold.query.trace: 500ms
> 
> index.search.slowlog.threshold.fetch.warn: 1s  
> index.search.slowlog.threshold.fetch.info: 800ms  
> #index.search.slowlog.threshold.fetch.debug: 500ms  
> #index.search.slowlog.threshold.fetch.trace: 200ms
> 
> On Wed, Apr 11, 2012 at 5:12 PM, Kenneth Loafman [kenneth@loafman.com](mailto:kenneth@loafman.com)wrote:
> 
> > Here it is:
> > 
> > ~$ curl -XGET "localhost:9200/test/\_settings?pretty=true"  
> > {  
> > "test" : {  
> > "settings" : {  
> > "index.number\_of\_shards" : "2",  
> > "index.number\_of\_replicas" : "1",  
> > "index.version.created" : "190199"  
> > }  
> > }  
> > }
> > 
> > On Wed, Apr 11, 2012 at 5:06 PM, Igor Motov [imotov@gmail.com](mailto:imotov@gmail.com) wrote:
> > 
> > > I still cannot reproduce it. Something is missing. Most likely it's  
> > > index settings. Could you share your config file or add output of the  
> > > following command to the repro?
> > > 
> > > curl -XGET "localhost:9200/test/\_settings"
> > > 
> > > On Wednesday, April 11, 2012 5:13:13 PM UTC-4, Kenneth Loafman wrote:
> > > 
> > > > 0.18.7 gives the same results.
> > > > 
> > > > BTW, original is here: [https://gist.github.com/\*\*2359644](https://gist.github.com/**2359644)[https://gist.github.com/2359644](https://gist.github.com/2359644)
> > > > 
> > > > ...Ken
> > > > 
> > > > On Wed, Apr 11, 2012 at 3:55 PM, Kenneth Loafman [kenneth@loafman.com](mailto:kenneth@loafman.com)wrote:
> > > > 
> > > > > Just to make sure, I updated the original gist with a test that has  
> > > > > the full mapping. It does not work. The output is from 0.19.1.
> > > > > 
> > > > > ...Ken
> > > > > 
> > > > > On Wed, Apr 11, 2012 at 3:28 PM, Kenneth Loafman [kenneth@loafman.com](mailto:kenneth@loafman.com)wrote:
> > > > > 
> > > > > > It will run without the publish\_datetime sort, that's part of the  
> > > > > > gist I posted. Does NOT matter whether its populated or not.
> > > > > > 
> > > > > > ...Ken
> > > > > > 
> > > > > > On Wed, Apr 11, 2012 at 11:13 AM, Igor Motov [imotov@gmail.com](mailto:imotov@gmail.com)wrote:
> > > > > > 
> > > > > > > This recreation doesn't seem to recreate the problem. The issue is  
> > > > > > > most likely in the mapping, and unfortunately, it's not visible in your  
> > > > > > > example. This is what I am getting when I am trying to run it:  
> > > > > > > [https://gist.github.com/\*\*2360279](https://gist.github.com/**2360279) [https://gist.github.com/2360279](https://gist.github.com/2360279) (I had to remove publish\_datetime sort, since it's not populated in your  
> > > > > > > gist).
> > > > > > > 
> > > > > > > On Wednesday, April 11, 2012 10:30:30 AM UTC-4, Kenneth Loafman  
> > > > > > > wrote:
> > > > > > > 
> > > > > > > > Here's the gist: [https://gist.github.com/\*\*\*\*2359644](https://gist.github.com/ **** 2359644)[https://gist.github.com/2359644](https://gist.github.com/2359644)
> > > > > > > > 
> > > > > > > > ...Thanks,  
> > > > > > > > ...Ken
> > > > > > > > 
> > > > > > > > On Wed, Apr 11, 2012 at 6:30 AM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com)wrote:
> > > > > > > > 
> > > > > > > > > Thats strange, the first option simply translates to a boolean  
> > > > > > > > > query and the other one translates to a phrase query. Can you gist a  
> > > > > > > > > recreation?
> > > > > > > > > 
> > > > > > > > > On Mon, Apr 9, 2012 at 9:24 PM, Kenneth Loafman \<  
> > > > > > > > > [kenneth.loafman@gmail.com](mailto:kenneth.loafman@gmail.com)\> wrote:
> > > > > > > > > 
> > > > > > > > > > Hi,
> > > > > > > > > > 
> > > > > > > > > > We're seeing an odd problem in search that revolves around stop  
> > > > > > > > > > words.
> > > > > > > > > > 
> > > > > > > > > > If I use the query (no quotes):  
> > > > > > > > > > worst way to travel  
> > > > > > > > > > ES returns nothing.
> > > > > > > > > > 
> > > > > > > > > > If I use the query (with quotes):  
> > > > > > > > > > "worst way to travel"  
> > > > > > > > > > ES returns matches.
> > > > > > > > > > 
> > > > > > > > > > Why the difference? Do I need to remove stop words manually from  
> > > > > > > > > > a non-quoted query?
> > > > > > > > > > 
> > > > > > > > > > ...Ken

---

<div class="post-metadata">

**Author:** ![Kenneth\_Loafman\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kenneth_loafman_2/32/2224_2.png) [@Kenneth\_Loafman\_2](https://discuss.elastic.co/u/Kenneth_Loafman_2)\
**Post date:** [April 12, 2012, 1:11am UTC](https://discuss.elastic.co/t/stop-words-in-quoted-vs-unquoted-search/7276/12 "2012-04-12T01:11:42Z")

</div>

The test script does not do any delays after creating the index, but if you  
put in a sleep after creating the index, all 4 shards are created. It does  
not happen (the 3/4 shards thing) on other systems, but the lack of search  
results happens on all 3 systems.

1. my desktop, 2 nodes on the same machine, 0.19.1,
2. remote test machine, same config as above,
3. production cluster, 4 machines, 1 ES node per machine, 0.18.7

All machines are running Ubuntu Lucid, 64-bit, java 1.6.

I think it's the filtered query that's causing the problem. Stripping out  
the filter part makes it work correctly. The query I sent in the gist is  
actually a minimized version of the query we use in production, as is the  
mapping. The full query fails as well, which is what got this started.

...Ken

On Wed, Apr 11, 2012 at 5:30 PM, Igor Motov [imotov@gmail.com](mailto:imotov@gmail.com) wrote:

> The tricky part is this:
> 
> {"ok":true,"\_shards":{"total":4,_"successful":3_,"failed":0}}
> 
> I don't quite understand how this is possible. If you had one node  
> running, you should have gotten "total":4, "successful":2. If you had two  
> nodes running, you should have gotten "total":4, "successful":4. Having  
> "total":4, "successful":3 most likely indicates that there is an issue with  
> allocating one of the shards on one of the nodes. Do you see any errors in  
> the log file on one of the nodes?
> 
> On Wednesday, April 11, 2012 6:17:11 PM UTC-4, Kenneth Loafman wrote:
> 
> > Here's elasticsearch.yml just in case...
> > 
> > gateway:  
> > type: local
> > 
> > path :  
> > data : /mnt/search-data-dev/node0/\*\*data  
> > work : /mnt/search-data-dev/node0/\*\*work  
> > logs : /mnt/search-data-dev/node0/\*\*logs
> > 
> > index :  
> > number\_of\_shards : 2  
> > number\_of\_replicas : 1
> > 
> > bootstrap:  
> > mlockall: true
> > 
> > action:  
> > disable\_delete\_all\_indexes: true
> > 
> > network :  
> > host : localhost
> > 
> > discovery.zen.ping.multicast:  
> > enabled: false
> > 
> > discovery.zen.ping.unicast:  
> > hosts: ["localhost:9300","localhost:\*\*9301"]
> > 
> > # Shard level query and fetch threshold logging.
> > 
> > #index.search.slowlog.level: TRACE  
> > index.search.slowlog.\*\*threshold.query.warn: 10s  
> > index.search.slowlog.\*\*threshold.query.info[http://index.search.slowlog.threshold.query.info](http://index.search.slowlog.threshold.query.info):  
> > 5s  
> > #index.search.slowlog.\*\*threshold.query.debug: 2s  
> > #index.search.slowlog.\*\*threshold.query.trace: 500ms
> > 
> > index.search.slowlog.\*\*threshold.fetch.warn: 1s  
> > index.search.slowlog.\*\*threshold.fetch.info[http://index.search.slowlog.threshold.fetch.info](http://index.search.slowlog.threshold.fetch.info):  
> > 800ms  
> > #index.search.slowlog.\*\*threshold.fetch.debug: 500ms  
> > #index.search.slowlog.\*\*threshold.fetch.trace: 200ms
> > 
> > On Wed, Apr 11, 2012 at 5:12 PM, Kenneth Loafman [kenneth@loafman.com](mailto:kenneth@loafman.com)wrote:
> > 
> > > Here it is:
> > > 
> > > ~$ curl -XGET "localhost:9200/test/\_\*\*settings?pretty=true"  
> > > {  
> > > "test" : {  
> > > "settings" : {  
> > > "index.number\_of\_shards" : "2",  
> > > "index.number\_of\_replicas" : "1",  
> > > "index.version.created" : "190199"  
> > > }  
> > > }  
> > > }
> > > 
> > > On Wed, Apr 11, 2012 at 5:06 PM, Igor Motov [imotov@gmail.com](mailto:imotov@gmail.com) wrote:
> > > 
> > > > I still cannot reproduce it. Something is missing. Most likely it's  
> > > > index settings. Could you share your config file or add output of the  
> > > > following command to the repro?
> > > > 
> > > > curl -XGET "localhost:9200/test/\_\*\*settings"
> > > > 
> > > > On Wednesday, April 11, 2012 5:13:13 PM UTC-4, Kenneth Loafman wrote:
> > > > 
> > > > > 0.18.7 gives the same results.
> > > > > 
> > > > > BTW, original is here: [https://gist.github.com/\*\*\*\*2359644](https://gist.github.com/ **** 2359644)[https://gist.github.com/2359644](https://gist.github.com/2359644)
> > > > > 
> > > > > ...Ken
> > > > > 
> > > > > On Wed, Apr 11, 2012 at 3:55 PM, Kenneth Loafman \<[kenneth@loafman.com](mailto:kenneth@loafman.com)
> > > > > 
> > > > > > wrote:
> > > > > 
> > > > > > Just to make sure, I updated the original gist with a test that has  
> > > > > > the full mapping. It does not work. The output is from 0.19.1.
> > > > > > 
> > > > > > ...Ken
> > > > > > 
> > > > > > On Wed, Apr 11, 2012 at 3:28 PM, Kenneth Loafman \<[kenneth@loafman.com](mailto:kenneth@loafman.com)
> > > > > > 
> > > > > > > wrote:
> > > > > > 
> > > > > > > It will run without the publish\_datetime sort, that's part of the  
> > > > > > > gist I posted. Does NOT matter whether its populated or not.
> > > > > > > 
> > > > > > > ...Ken
> > > > > > > 
> > > > > > > On Wed, Apr 11, 2012 at 11:13 AM, Igor Motov [imotov@gmail.com](mailto:imotov@gmail.com)wrote:
> > > > > > > 
> > > > > > > > This recreation doesn't seem to recreate the problem. The issue is  
> > > > > > > > most likely in the mapping, and unfortunately, it's not visible in your  
> > > > > > > > example. This is what I am getting when I am trying to run it:  
> > > > > > > > [https://gist.github.com/\*\*23\*\*60279](https://gist.github.com/ **23** 60279)[https://gist.github.com/2360279](https://gist.github.com/2360279) (I had to remove publish\_datetime sort, since it's not populated in your  
> > > > > > > > gist).
> > > > > > > > 
> > > > > > > > On Wednesday, April 11, 2012 10:30:30 AM UTC-4, Kenneth Loafman  
> > > > > > > > wrote:
> > > > > > > > 
> > > > > > > > > Here's the gist: [https://gist.github.com/\*\*\*\*\*\*2359644](https://gist.github.com/ ****** 2359644)[https://gist.github.com/2359644](https://gist.github.com/2359644)
> > > > > > > > > 
> > > > > > > > > ...Thanks,  
> > > > > > > > > ...Ken
> > > > > > > > > 
> > > > > > > > > On Wed, Apr 11, 2012 at 6:30 AM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com)wrote:
> > > > > > > > > 
> > > > > > > > > > Thats strange, the first option simply translates to a boolean  
> > > > > > > > > > query and the other one translates to a phrase query. Can you gist a  
> > > > > > > > > > recreation?
> > > > > > > > > > 
> > > > > > > > > > On Mon, Apr 9, 2012 at 9:24 PM, Kenneth Loafman \<  
> > > > > > > > > > [kenneth.loafman@gmail.com](mailto:kenneth.loafman@gmail.com)\> wrote:
> > > > > > > > > > 
> > > > > > > > > > > Hi,
> > > > > > > > > > > 
> > > > > > > > > > > We're seeing an odd problem in search that revolves around stop  
> > > > > > > > > > > words.
> > > > > > > > > > > 
> > > > > > > > > > > If I use the query (no quotes):  
> > > > > > > > > > > worst way to travel  
> > > > > > > > > > > ES returns nothing.
> > > > > > > > > > > 
> > > > > > > > > > > If I use the query (with quotes):  
> > > > > > > > > > > "worst way to travel"  
> > > > > > > > > > > ES returns matches.
> > > > > > > > > > > 
> > > > > > > > > > > Why the difference? Do I need to remove stop words manually  
> > > > > > > > > > > from a non-quoted query?
> > > > > > > > > > > 
> > > > > > > > > > > ...Ken

---

<div class="post-metadata">

**Author:** ![Kenneth\_Loafman\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kenneth_loafman_2/32/2224_2.png) [@Kenneth\_Loafman\_2](https://discuss.elastic.co/u/Kenneth_Loafman_2)\
**Post date:** [April 13, 2012, 1:04am UTC](https://discuss.elastic.co/t/stop-words-in-quoted-vs-unquoted-search/7276/13 "2012-04-13T01:04:01Z")

</div>

Ping. Any thoughts?

...Ken

On Wed, Apr 11, 2012 at 8:11 PM, Kenneth Loafman [kenneth@loafman.com](mailto:kenneth@loafman.com)wrote:

> The test script does not do any delays after creating the index, but if  
> you put in a sleep after creating the index, all 4 shards are created. It  
> does not happen (the 3/4 shards thing) on other systems, but the lack of  
> search results happens on all 3 systems.
> 
> 1. my desktop, 2 nodes on the same machine, 0.19.1,
> 2. remote test machine, same config as above,
> 3. production cluster, 4 machines, 1 ES node per machine, 0.18.7
> 
> All machines are running Ubuntu Lucid, 64-bit, java 1.6.
> 
> I think it's the filtered query that's causing the problem. Stripping out  
> the filter part makes it work correctly. The query I sent in the gist is  
> actually a minimized version of the query we use in production, as is the  
> mapping. The full query fails as well, which is what got this started.
> 
> ...Ken
> 
> On Wed, Apr 11, 2012 at 5:30 PM, Igor Motov [imotov@gmail.com](mailto:imotov@gmail.com) wrote:
> 
> > The tricky part is this:
> > 
> > {"ok":true,"\_shards":{"total":4,_"successful":3_,"failed":0}}
> > 
> > I don't quite understand how this is possible. If you had one node  
> > running, you should have gotten "total":4, "successful":2. If you had two  
> > nodes running, you should have gotten "total":4, "successful":4. Having  
> > "total":4, "successful":3 most likely indicates that there is an issue with  
> > allocating one of the shards on one of the nodes. Do you see any errors in  
> > the log file on one of the nodes?
> > 
> > On Wednesday, April 11, 2012 6:17:11 PM UTC-4, Kenneth Loafman wrote:
> > 
> > > Here's elasticsearch.yml just in case...
> > > 
> > > gateway:  
> > > type: local
> > > 
> > > path :  
> > > data : /mnt/search-data-dev/node0/\*\*data  
> > > work : /mnt/search-data-dev/node0/\*\*work  
> > > logs : /mnt/search-data-dev/node0/\*\*logs
> > > 
> > > index :  
> > > number\_of\_shards : 2  
> > > number\_of\_replicas : 1
> > > 
> > > bootstrap:  
> > > mlockall: true
> > > 
> > > action:  
> > > disable\_delete\_all\_indexes: true
> > > 
> > > network :  
> > > host : localhost
> > > 
> > > discovery.zen.ping.multicast:  
> > > enabled: false
> > > 
> > > discovery.zen.ping.unicast:  
> > > hosts: ["localhost:9300","localhost:\*\*9301"]
> > > 
> > > # Shard level query and fetch threshold logging.
> > > 
> > > #index.search.slowlog.level: TRACE  
> > > index.search.slowlog.\*\*threshold.query.warn: 10s  
> > > index.search.slowlog.\*\*threshold.query.info[http://index.search.slowlog.threshold.query.info](http://index.search.slowlog.threshold.query.info):  
> > > 5s  
> > > #index.search.slowlog.\*\*threshold.query.debug: 2s  
> > > #index.search.slowlog.\*\*threshold.query.trace: 500ms
> > > 
> > > index.search.slowlog.\*\*threshold.fetch.warn: 1s  
> > > index.search.slowlog.\*\*threshold.fetch.info[http://index.search.slowlog.threshold.fetch.info](http://index.search.slowlog.threshold.fetch.info):  
> > > 800ms  
> > > #index.search.slowlog.\*\*threshold.fetch.debug: 500ms  
> > > #index.search.slowlog.\*\*threshold.fetch.trace: 200ms
> > > 
> > > On Wed, Apr 11, 2012 at 5:12 PM, Kenneth Loafman [kenneth@loafman.com](mailto:kenneth@loafman.com)wrote:
> > > 
> > > > Here it is:
> > > > 
> > > > ~$ curl -XGET "localhost:9200/test/\_\*\*settings?pretty=true"  
> > > > {  
> > > > "test" : {  
> > > > "settings" : {  
> > > > "index.number\_of\_shards" : "2",  
> > > > "index.number\_of\_replicas" : "1",  
> > > > "index.version.created" : "190199"  
> > > > }  
> > > > }  
> > > > }
> > > > 
> > > > On Wed, Apr 11, 2012 at 5:06 PM, Igor Motov [imotov@gmail.com](mailto:imotov@gmail.com) wrote:
> > > > 
> > > > > I still cannot reproduce it. Something is missing. Most likely it's  
> > > > > index settings. Could you share your config file or add output of the  
> > > > > following command to the repro?
> > > > > 
> > > > > curl -XGET "localhost:9200/test/\_\*\*settings"
> > > > > 
> > > > > On Wednesday, April 11, 2012 5:13:13 PM UTC-4, Kenneth Loafman wrote:
> > > > > 
> > > > > > 0.18.7 gives the same results.
> > > > > > 
> > > > > > BTW, original is here: [https://gist.github.com/\*\*\*\*2359644](https://gist.github.com/ **** 2359644)[https://gist.github.com/2359644](https://gist.github.com/2359644)
> > > > > > 
> > > > > > ...Ken
> > > > > > 
> > > > > > On Wed, Apr 11, 2012 at 3:55 PM, Kenneth Loafman \<  
> > > > > > [kenneth@loafman.com](mailto:kenneth@loafman.com)\> wrote:
> > > > > > 
> > > > > > > Just to make sure, I updated the original gist with a test that has  
> > > > > > > the full mapping. It does not work. The output is from 0.19.1.
> > > > > > > 
> > > > > > > ...Ken
> > > > > > > 
> > > > > > > On Wed, Apr 11, 2012 at 3:28 PM, Kenneth Loafman \<  
> > > > > > > [kenneth@loafman.com](mailto:kenneth@loafman.com)\> wrote:
> > > > > > > 
> > > > > > > > It will run without the publish\_datetime sort, that's part of the  
> > > > > > > > gist I posted. Does NOT matter whether its populated or not.
> > > > > > > > 
> > > > > > > > ...Ken
> > > > > > > > 
> > > > > > > > On Wed, Apr 11, 2012 at 11:13 AM, Igor Motov [imotov@gmail.com](mailto:imotov@gmail.com)wrote:
> > > > > > > > 
> > > > > > > > > This recreation doesn't seem to recreate the problem. The issue is  
> > > > > > > > > most likely in the mapping, and unfortunately, it's not visible in your  
> > > > > > > > > example. This is what I am getting when I am trying to run it:  
> > > > > > > > > [https://gist.github.com/\*\*23\*\*60279](https://gist.github.com/ **23** 60279)[https://gist.github.com/2360279](https://gist.github.com/2360279) (I had to remove publish\_datetime sort, since it's not populated in your  
> > > > > > > > > gist).
> > > > > > > > > 
> > > > > > > > > On Wednesday, April 11, 2012 10:30:30 AM UTC-4, Kenneth Loafman  
> > > > > > > > > wrote:
> > > > > > > > > 
> > > > > > > > > > Here's the gist: [https://gist.github.com/\*\*\*\*\*\*2359644](https://gist.github.com/ ****** 2359644)[https://gist.github.com/2359644](https://gist.github.com/2359644)
> > > > > > > > > > 
> > > > > > > > > > ...Thanks,  
> > > > > > > > > > ...Ken
> > > > > > > > > > 
> > > > > > > > > > On Wed, Apr 11, 2012 at 6:30 AM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com)wrote:
> > > > > > > > > > 
> > > > > > > > > > > Thats strange, the first option simply translates to a boolean  
> > > > > > > > > > > query and the other one translates to a phrase query. Can you gist a  
> > > > > > > > > > > recreation?
> > > > > > > > > > > 
> > > > > > > > > > > On Mon, Apr 9, 2012 at 9:24 PM, Kenneth Loafman \<  
> > > > > > > > > > > [kenneth.loafman@gmail.com](mailto:kenneth.loafman@gmail.com)\> wrote:
> > > > > > > > > > > 
> > > > > > > > > > > > Hi,
> > > > > > > > > > > > 
> > > > > > > > > > > > We're seeing an odd problem in search that revolves around stop  
> > > > > > > > > > > > words.
> > > > > > > > > > > > 
> > > > > > > > > > > > If I use the query (no quotes):  
> > > > > > > > > > > > worst way to travel  
> > > > > > > > > > > > ES returns nothing.
> > > > > > > > > > > > 
> > > > > > > > > > > > If I use the query (with quotes):  
> > > > > > > > > > > > "worst way to travel"  
> > > > > > > > > > > > ES returns matches.
> > > > > > > > > > > > 
> > > > > > > > > > > > Why the difference? Do I need to remove stop words manually  
> > > > > > > > > > > > from a non-quoted query?
> > > > > > > > > > > > 
> > > > > > > > > > > > ...Ken

---

<div class="post-metadata">

**Author:** ![Igor\_Motov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_motov/32/45193_2.png) [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)\
**Post date:** [April 13, 2012, 1:38am UTC](https://discuss.elastic.co/t/stop-words-in-quoted-vs-unquoted-search/7276/14 "2012-04-13T01:38:52Z")

</div>

I ran out of ideas. I tried reproducing it on Mac and Ubuntu using your  
config and 3 different versions of java without success. I get results for  
both searches every time.

Can you reproduce it with a freshly downloaded version of elasticsearch?  
Basically, download elasticsearch 0.19.1 from [elasticsearch.org](http://elasticsearch.org), unzip  
it, don't change configuration, start two nodes and run stop-words-test.sh.

On Thursday, April 12, 2012 9:04:01 PM UTC-4, Kenneth Loafman wrote:

> Ping. Any thoughts?
> 
> ...Ken
> 
> On Wed, Apr 11, 2012 at 8:11 PM, Kenneth Loafman [kenneth@loafman.com](mailto:kenneth@loafman.com)wrote:
> 
> > The test script does not do any delays after creating the index, but if  
> > you put in a sleep after creating the index, all 4 shards are created. It  
> > does not happen (the 3/4 shards thing) on other systems, but the lack of  
> > search results happens on all 3 systems.
> > 
> > 1. my desktop, 2 nodes on the same machine, 0.19.1,
> > 2. remote test machine, same config as above,
> > 3. production cluster, 4 machines, 1 ES node per machine, 0.18.7
> > 
> > All machines are running Ubuntu Lucid, 64-bit, java 1.6.
> > 
> > I think it's the filtered query that's causing the problem. Stripping  
> > out the filter part makes it work correctly. The query I sent in the gist  
> > is actually a minimized version of the query we use in production, as is  
> > the mapping. The full query fails as well, which is what got this started.
> > 
> > ...Ken
> > 
> > On Wed, Apr 11, 2012 at 5:30 PM, Igor Motov [imotov@gmail.com](mailto:imotov@gmail.com) wrote:
> > 
> > > The tricky part is this:
> > > 
> > > {"ok":true,"\_shards":{"total":4,_"successful":3_,"failed":0}}
> > > 
> > > I don't quite understand how this is possible. If you had one node  
> > > running, you should have gotten "total":4, "successful":2. If you had two  
> > > nodes running, you should have gotten "total":4, "successful":4. Having  
> > > "total":4, "successful":3 most likely indicates that there is an issue with  
> > > allocating one of the shards on one of the nodes. Do you see any errors in  
> > > the log file on one of the nodes?
> > > 
> > > On Wednesday, April 11, 2012 6:17:11 PM UTC-4, Kenneth Loafman wrote:
> > > 
> > > > Here's elasticsearch.yml just in case...
> > > > 
> > > > gateway:  
> > > > type: local
> > > > 
> > > > path :  
> > > > data : /mnt/search-data-dev/node0/\*\*data  
> > > > work : /mnt/search-data-dev/node0/\*\*work  
> > > > logs : /mnt/search-data-dev/node0/\*\*logs
> > > > 
> > > > index :  
> > > > number\_of\_shards : 2  
> > > > number\_of\_replicas : 1
> > > > 
> > > > bootstrap:  
> > > > mlockall: true
> > > > 
> > > > action:  
> > > > disable\_delete\_all\_indexes: true
> > > > 
> > > > network :  
> > > > host : localhost
> > > > 
> > > > discovery.zen.ping.multicast:  
> > > > enabled: false
> > > > 
> > > > discovery.zen.ping.unicast:  
> > > > hosts: ["localhost:9300","localhost:\*\*9301"]
> > > > 
> > > > # Shard level query and fetch threshold logging.
> > > > 
> > > > #index.search.slowlog.level: TRACE  
> > > > index.search.slowlog.\*\*threshold.query.warn: 10s  
> > > > index.search.slowlog.\*\*threshold.query.info[http://index.search.slowlog.threshold.query.info](http://index.search.slowlog.threshold.query.info):  
> > > > 5s  
> > > > #index.search.slowlog.\*\*threshold.query.debug: 2s  
> > > > #index.search.slowlog.\*\*threshold.query.trace: 500ms
> > > > 
> > > > index.search.slowlog.\*\*threshold.fetch.warn: 1s  
> > > > index.search.slowlog.\*\*threshold.fetch.info[http://index.search.slowlog.threshold.fetch.info](http://index.search.slowlog.threshold.fetch.info):  
> > > > 800ms  
> > > > #index.search.slowlog.\*\*threshold.fetch.debug: 500ms  
> > > > #index.search.slowlog.\*\*threshold.fetch.trace: 200ms
> > > > 
> > > > On Wed, Apr 11, 2012 at 5:12 PM, Kenneth Loafman [kenneth@loafman.com](mailto:kenneth@loafman.com)wrote:
> > > > 
> > > > > Here it is:
> > > > > 
> > > > > ~$ curl -XGET "localhost:9200/test/\_\*\*settings?pretty=true"  
> > > > > {  
> > > > > "test" : {  
> > > > > "settings" : {  
> > > > > "index.number\_of\_shards" : "2",  
> > > > > "index.number\_of\_replicas" : "1",  
> > > > > "index.version.created" : "190199"  
> > > > > }  
> > > > > }  
> > > > > }
> > > > > 
> > > > > On Wed, Apr 11, 2012 at 5:06 PM, Igor Motov [imotov@gmail.com](mailto:imotov@gmail.com) wrote:
> > > > > 
> > > > > > I still cannot reproduce it. Something is missing. Most likely it's  
> > > > > > index settings. Could you share your config file or add output of the  
> > > > > > following command to the repro?
> > > > > > 
> > > > > > curl -XGET "localhost:9200/test/\_\*\*settings"
> > > > > > 
> > > > > > On Wednesday, April 11, 2012 5:13:13 PM UTC-4, Kenneth Loafman wrote:
> > > > > > 
> > > > > > > 0.18.7 gives the same results.
> > > > > > > 
> > > > > > > BTW, original is here: [https://gist.github.com/\*\*\*\*2359644](https://gist.github.com/ **** 2359644)[https://gist.github.com/2359644](https://gist.github.com/2359644)
> > > > > > > 
> > > > > > > ...Ken
> > > > > > > 
> > > > > > > On Wed, Apr 11, 2012 at 3:55 PM, Kenneth Loafman \<  
> > > > > > > [kenneth@loafman.com](mailto:kenneth@loafman.com)\> wrote:
> > > > > > > 
> > > > > > > > Just to make sure, I updated the original gist with a test that has  
> > > > > > > > the full mapping. It does not work. The output is from 0.19.1.
> > > > > > > > 
> > > > > > > > ...Ken
> > > > > > > > 
> > > > > > > > On Wed, Apr 11, 2012 at 3:28 PM, Kenneth Loafman \<  
> > > > > > > > [kenneth@loafman.com](mailto:kenneth@loafman.com)\> wrote:
> > > > > > > > 
> > > > > > > > > It will run without the publish\_datetime sort, that's part of the  
> > > > > > > > > gist I posted. Does NOT matter whether its populated or not.
> > > > > > > > > 
> > > > > > > > > ...Ken
> > > > > > > > > 
> > > > > > > > > On Wed, Apr 11, 2012 at 11:13 AM, Igor Motov [imotov@gmail.com](mailto:imotov@gmail.com)wrote:
> > > > > > > > > 
> > > > > > > > > > This recreation doesn't seem to recreate the problem. The issue  
> > > > > > > > > > is most likely in the mapping, and unfortunately, it's not visible in your  
> > > > > > > > > > example. This is what I am getting when I am trying to run it:  
> > > > > > > > > > [https://gist.github.com/\*\*23\*\*60279](https://gist.github.com/ **23** 60279)[https://gist.github.com/2360279](https://gist.github.com/2360279) (I had to remove publish\_datetime sort, since it's not populated in your  
> > > > > > > > > > gist).
> > > > > > > > > > 
> > > > > > > > > > On Wednesday, April 11, 2012 10:30:30 AM UTC-4, Kenneth Loafman  
> > > > > > > > > > wrote:
> > > > > > > > > > 
> > > > > > > > > > > Here's the gist: [https://gist.github.com/\*\*\*\*\*\*2359644](https://gist.github.com/ ****** 2359644)[https://gist.github.com/2359644](https://gist.github.com/2359644)
> > > > > > > > > > > 
> > > > > > > > > > > ...Thanks,  
> > > > > > > > > > > ...Ken
> > > > > > > > > > > 
> > > > > > > > > > > On Wed, Apr 11, 2012 at 6:30 AM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com)wrote:
> > > > > > > > > > > 
> > > > > > > > > > > > Thats strange, the first option simply translates to a boolean  
> > > > > > > > > > > > query and the other one translates to a phrase query. Can you gist a  
> > > > > > > > > > > > recreation?
> > > > > > > > > > > > 
> > > > > > > > > > > > On Mon, Apr 9, 2012 at 9:24 PM, Kenneth Loafman \<  
> > > > > > > > > > > > [kenneth.loafman@gmail.com](mailto:kenneth.loafman@gmail.com)\> wrote:
> > > > > > > > > > > > 
> > > > > > > > > > > > > Hi,
> > > > > > > > > > > > > 
> > > > > > > > > > > > > We're seeing an odd problem in search that revolves around  
> > > > > > > > > > > > > stop words.
> > > > > > > > > > > > > 
> > > > > > > > > > > > > If I use the query (no quotes):  
> > > > > > > > > > > > > worst way to travel  
> > > > > > > > > > > > > ES returns nothing.
> > > > > > > > > > > > > 
> > > > > > > > > > > > > If I use the query (with quotes):  
> > > > > > > > > > > > > "worst way to travel"  
> > > > > > > > > > > > > ES returns matches.
> > > > > > > > > > > > > 
> > > > > > > > > > > > > Why the difference? Do I need to remove stop words manually  
> > > > > > > > > > > > > from a non-quoted query?
> > > > > > > > > > > > > 
> > > > > > > > > > > > > ...Ken

---

<div class="post-metadata">

**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\
**Post date:** [April 13, 2012, 12:39pm UTC](https://discuss.elastic.co/t/stop-words-in-quoted-vs-unquoted-search/7276/15 "2012-04-13T12:39:42Z")

</div>

The refresh result of total 3 failed 0 can happen, effectively, the index  
gets created, and shards are starting to be allocated. The index requests  
will work once a primary shard is available. Then the refresh goes and not  
all replica shards have been allocated yet. I don't think thats the  
problem, but you can add a call to the health API with wait\_for\_status set  
to green after the index gets created. But, does that solve the problem?  
Thats what I missed here...

On Fri, Apr 13, 2012 at 4:38 AM, Igor Motov [imotov@gmail.com](mailto:imotov@gmail.com) wrote:

> I ran out of ideas. I tried reproducing it on Mac and Ubuntu using your  
> config and 3 different versions of java without success. I get results for  
> both searches every time.
> 
> Can you reproduce it with a freshly downloaded version of elasticsearch?  
> Basically, download elasticsearch 0.19.1 from [elasticsearch.org](http://elasticsearch.org), unzip  
> it, don't change configuration, start two nodes and run stop-words-test.sh.
> 
> On Thursday, April 12, 2012 9:04:01 PM UTC-4, Kenneth Loafman wrote:
> 
> > Ping. Any thoughts?
> > 
> > ...Ken
> > 
> > On Wed, Apr 11, 2012 at 8:11 PM, Kenneth Loafman [kenneth@loafman.com](mailto:kenneth@loafman.com)wrote:
> > 
> > > The test script does not do any delays after creating the index, but if  
> > > you put in a sleep after creating the index, all 4 shards are created. It  
> > > does not happen (the 3/4 shards thing) on other systems, but the lack of  
> > > search results happens on all 3 systems.
> > > 
> > > 1. my desktop, 2 nodes on the same machine, 0.19.1,
> > > 2. remote test machine, same config as above,
> > > 3. production cluster, 4 machines, 1 ES node per machine, 0.18.7
> > > 
> > > All machines are running Ubuntu Lucid, 64-bit, java 1.6.
> > > 
> > > I think it's the filtered query that's causing the problem. Stripping  
> > > out the filter part makes it work correctly. The query I sent in the gist  
> > > is actually a minimized version of the query we use in production, as is  
> > > the mapping. The full query fails as well, which is what got this started.
> > > 
> > > ...Ken
> > > 
> > > On Wed, Apr 11, 2012 at 5:30 PM, Igor Motov [imotov@gmail.com](mailto:imotov@gmail.com) wrote:
> > > 
> > > > The tricky part is this:
> > > > 
> > > > {"ok":true,"\_shards":{"total":\*\*4,_"successful":3_,"failed":0}}
> > > > 
> > > > I don't quite understand how this is possible. If you had one node  
> > > > running, you should have gotten "total":4, "successful":2. If you had two  
> > > > nodes running, you should have gotten "total":4, "successful":4. Having  
> > > > "total":4, "successful":3 most likely indicates that there is an issue with  
> > > > allocating one of the shards on one of the nodes. Do you see any errors in  
> > > > the log file on one of the nodes?
> > > > 
> > > > On Wednesday, April 11, 2012 6:17:11 PM UTC-4, Kenneth Loafman wrote:
> > > > 
> > > > > Here's elasticsearch.yml just in case...
> > > > > 
> > > > > gateway:  
> > > > > type: local
> > > > > 
> > > > > path :  
> > > > > data : /mnt/search-data-dev/node0/ **dat** a  
> > > > > work : /mnt/search-data-dev/node0/ **wor** k  
> > > > > logs : /mnt/search-data-dev/node0/ **log** s
> > > > > 
> > > > > index :  
> > > > > number\_of\_shards : 2  
> > > > > number\_of\_replicas : 1
> > > > > 
> > > > > bootstrap:  
> > > > > mlockall: true
> > > > > 
> > > > > action:  
> > > > > disable\_delete\_all\_indexes: true
> > > > > 
> > > > > network :  
> > > > > host : localhost
> > > > > 
> > > > > discovery.zen.ping.multicast:  
> > > > > enabled: false
> > > > > 
> > > > > discovery.zen.ping.unicast:  
> > > > > hosts: ["localhost:9300","localhost: **9** 301"]
> > > > > 
> > > > > # Shard level query and fetch threshold logging.
> > > > > 
> > > > > #index.search.slowlog.level: TRACE  
> > > > > index.search.slowlog. **threshold**.query.warn: 10s  
> > > > > index.search.slowlog. **threshold**.query.info[http://index.search.slowlog.threshold.query.info](http://index.search.slowlog.threshold.query.info):  
> > > > > 5s  
> > > > > #index.search.slowlog. **threshol** d.query.debug: 2s  
> > > > > #index.search.slowlog. **threshol** d.query.trace: 500ms
> > > > > 
> > > > > index.search.slowlog. **threshold**.fetch.warn: 1s  
> > > > > index.search.slowlog. **threshold**.fetch.info[http://index.search.slowlog.threshold.fetch.info](http://index.search.slowlog.threshold.fetch.info):  
> > > > > 800ms  
> > > > > #index.search.slowlog. **threshol** d.fetch.debug: 500ms  
> > > > > #index.search.slowlog. **threshol** d.fetch.trace: 200ms
> > > > > 
> > > > > On Wed, Apr 11, 2012 at 5:12 PM, Kenneth Loafman [kenneth@loafman.com](mailto:kenneth@loafman.com)wrote:
> > > > > 
> > > > > > Here it is:
> > > > > > 
> > > > > > ~$ curl -XGET "localhost:9200/test/\_ **settings**?pretty=true"  
> > > > > > {  
> > > > > > "test" : {  
> > > > > > "settings" : {  
> > > > > > "index.number\_of\_shards" : "2",  
> > > > > > "index.number\_of\_replicas" : "1",  
> > > > > > "index.version.created" : "190199"  
> > > > > > }  
> > > > > > }  
> > > > > > }
> > > > > > 
> > > > > > On Wed, Apr 11, 2012 at 5:06 PM, Igor Motov [imotov@gmail.com](mailto:imotov@gmail.com) wrote:
> > > > > > 
> > > > > > > I still cannot reproduce it. Something is missing. Most likely it's  
> > > > > > > index settings. Could you share your config file or add output of the  
> > > > > > > following command to the repro?
> > > > > > > 
> > > > > > > curl -XGET "localhost:9200/test/\_ **settings**"
> > > > > > > 
> > > > > > > On Wednesday, April 11, 2012 5:13:13 PM UTC-4, Kenneth Loafman wrote:
> > > > > > > 
> > > > > > > > 0.18.7 gives the same results.
> > > > > > > > 
> > > > > > > > BTW, original is here: [https://gist.github.com/\*\*\*\*\*\*2359644](https://gist.github.com/ ****** 2359644)[https://gist.github.com/2359644](https://gist.github.com/2359644)
> > > > > > > > 
> > > > > > > > ...Ken
> > > > > > > > 
> > > > > > > > On Wed, Apr 11, 2012 at 3:55 PM, Kenneth Loafman \<  
> > > > > > > > [kenneth@loafman.com](mailto:kenneth@loafman.com)\> wrote:
> > > > > > > > 
> > > > > > > > > Just to make sure, I updated the original gist with a test that  
> > > > > > > > > has the full mapping. It does not work. The output is from 0.19.1.
> > > > > > > > > 
> > > > > > > > > ...Ken
> > > > > > > > > 
> > > > > > > > > On Wed, Apr 11, 2012 at 3:28 PM, Kenneth Loafman \<  
> > > > > > > > > [kenneth@loafman.com](mailto:kenneth@loafman.com)\> wrote:
> > > > > > > > > 
> > > > > > > > > > It will run without the publish\_datetime sort, that's part of the  
> > > > > > > > > > gist I posted. Does NOT matter whether its populated or not.
> > > > > > > > > > 
> > > > > > > > > > ...Ken
> > > > > > > > > > 
> > > > > > > > > > On Wed, Apr 11, 2012 at 11:13 AM, Igor Motov [imotov@gmail.com](mailto:imotov@gmail.com)wrote:
> > > > > > > > > > 
> > > > > > > > > > > This recreation doesn't seem to recreate the problem. The issue  
> > > > > > > > > > > is most likely in the mapping, and unfortunately, it's not visible in your  
> > > > > > > > > > > example. This is what I am getting when I am trying to run it:  
> > > > > > > > > > > [https://gist.github.com/\*\*23\*\*\*\*60279](https://gist.github.com/ **23**** 60279)[https://gist.github.com/2360279](https://gist.github.com/2360279) (I had to remove publish\_datetime sort, since it's not populated in your  
> > > > > > > > > > > gist).
> > > > > > > > > > > 
> > > > > > > > > > > On Wednesday, April 11, 2012 10:30:30 AM UTC-4, Kenneth Loafman  
> > > > > > > > > > > wrote:
> > > > > > > > > > > 
> > > > > > > > > > > > Here's the gist: [https://gist.github.com/\*\*\*\*\*\*\*\*2359644](https://gist.github.com/ ******** 2359644)[https://gist.github.com/2359644](https://gist.github.com/2359644)
> > > > > > > > > > > > 
> > > > > > > > > > > > ...Thanks,  
> > > > > > > > > > > > ...Ken
> > > > > > > > > > > > 
> > > > > > > > > > > > On Wed, Apr 11, 2012 at 6:30 AM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com)wrote:
> > > > > > > > > > > > 
> > > > > > > > > > > > > Thats strange, the first option simply translates to a boolean  
> > > > > > > > > > > > > query and the other one translates to a phrase query. Can you gist a  
> > > > > > > > > > > > > recreation?
> > > > > > > > > > > > > 
> > > > > > > > > > > > > On Mon, Apr 9, 2012 at 9:24 PM, Kenneth Loafman \<  
> > > > > > > > > > > > > [kenneth.loafman@gmail.com](mailto:kenneth.loafman@gmail.com)\> wrote:
> > > > > > > > > > > > > 
> > > > > > > > > > > > > > Hi,
> > > > > > > > > > > > > > 
> > > > > > > > > > > > > > We're seeing an odd problem in search that revolves around  
> > > > > > > > > > > > > > stop words.
> > > > > > > > > > > > > > 
> > > > > > > > > > > > > > If I use the query (no quotes):  
> > > > > > > > > > > > > > worst way to travel  
> > > > > > > > > > > > > > ES returns nothing.
> > > > > > > > > > > > > > 
> > > > > > > > > > > > > > If I use the query (with quotes):  
> > > > > > > > > > > > > > "worst way to travel"  
> > > > > > > > > > > > > > ES returns matches.
> > > > > > > > > > > > > > 
> > > > > > > > > > > > > > Why the difference? Do I need to remove stop words manually  
> > > > > > > > > > > > > > from a non-quoted query?
> > > > > > > > > > > > > > 
> > > > > > > > > > > > > > ...Ken

---

<div class="post-metadata">

**Author:** ![Kenneth\_Loafman\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kenneth_loafman_2/32/2224_2.png) [@Kenneth\_Loafman\_2](https://discuss.elastic.co/u/Kenneth_Loafman_2)\
**Post date:** [April 13, 2012, 6:35pm UTC](https://discuss.elastic.co/t/stop-words-in-quoted-vs-unquoted-search/7276/16 "2012-04-13T18:35:59Z")

</div>

Thanks for the help Shay and Igor.

I tried Igor's request on my desktop and it worked. I upgraded my desktop  
to 0.19.2 and it worked. I upgraded our test machine to 0.19.2, and it  
failed. My desktop and the test machine have identical setups, so now I'm  
even more confused. This may resolve down to something as trivial as a  
minor version number on some support package. AArgh!

On Fri, Apr 13, 2012 at 7:39 AM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote:

> The refresh result of total 3 failed 0 can happen, effectively, the index  
> gets created, and shards are starting to be allocated. The index requests  
> will work once a primary shard is available. Then the refresh goes and not  
> all replica shards have been allocated yet. I don't think thats the  
> problem, but you can add a call to the health API with wait\_for\_status set  
> to green after the index gets created. But, does that solve the problem?  
> Thats what I missed here...
> 
> On Fri, Apr 13, 2012 at 4:38 AM, Igor Motov [imotov@gmail.com](mailto:imotov@gmail.com) wrote:
> 
> > I ran out of ideas. I tried reproducing it on Mac and Ubuntu using your  
> > config and 3 different versions of java without success. I get results for  
> > both searches every time.
> > 
> > Can you reproduce it with a freshly downloaded version of elasticsearch?  
> > Basically, download elasticsearch 0.19.1 from [elasticsearch.org](http://elasticsearch.org), unzip  
> > it, don't change configuration, start two nodes and run stop-words-test.sh.
> > 
> > On Thursday, April 12, 2012 9:04:01 PM UTC-4, Kenneth Loafman wrote:
> > 
> > > Ping. Any thoughts?
> > > 
> > > ...Ken
> > > 
> > > On Wed, Apr 11, 2012 at 8:11 PM, Kenneth Loafman [kenneth@loafman.com](mailto:kenneth@loafman.com)wrote:
> > > 
> > > > The test script does not do any delays after creating the index, but if  
> > > > you put in a sleep after creating the index, all 4 shards are created. It  
> > > > does not happen (the 3/4 shards thing) on other systems, but the lack of  
> > > > search results happens on all 3 systems.
> > > > 
> > > > 1. my desktop, 2 nodes on the same machine, 0.19.1,
> > > > 2. remote test machine, same config as above,
> > > > 3. production cluster, 4 machines, 1 ES node per machine, 0.18.7
> > > > 
> > > > All machines are running Ubuntu Lucid, 64-bit, java 1.6.
> > > > 
> > > > I think it's the filtered query that's causing the problem. Stripping  
> > > > out the filter part makes it work correctly. The query I sent in the gist  
> > > > is actually a minimized version of the query we use in production, as is  
> > > > the mapping. The full query fails as well, which is what got this started.
> > > > 
> > > > ...Ken
> > > > 
> > > > On Wed, Apr 11, 2012 at 5:30 PM, Igor Motov [imotov@gmail.com](mailto:imotov@gmail.com) wrote:
> > > > 
> > > > > The tricky part is this:
> > > > > 
> > > > > {"ok":true,"\_shards":{"total":\*\*4,_"successful":3_,"failed":0}}
> > > > > 
> > > > > I don't quite understand how this is possible. If you had one node  
> > > > > running, you should have gotten "total":4, "successful":2. If you had two  
> > > > > nodes running, you should have gotten "total":4, "successful":4. Having  
> > > > > "total":4, "successful":3 most likely indicates that there is an issue with  
> > > > > allocating one of the shards on one of the nodes. Do you see any errors in  
> > > > > the log file on one of the nodes?
> > > > > 
> > > > > On Wednesday, April 11, 2012 6:17:11 PM UTC-4, Kenneth Loafman wrote:
> > > > > 
> > > > > > Here's elasticsearch.yml just in case...
> > > > > > 
> > > > > > gateway:  
> > > > > > type: local
> > > > > > 
> > > > > > path :  
> > > > > > data : /mnt/search-data-dev/node0/ **dat** a  
> > > > > > work : /mnt/search-data-dev/node0/ **wor** k  
> > > > > > logs : /mnt/search-data-dev/node0/ **log** s
> > > > > > 
> > > > > > index :  
> > > > > > number\_of\_shards : 2  
> > > > > > number\_of\_replicas : 1
> > > > > > 
> > > > > > bootstrap:  
> > > > > > mlockall: true
> > > > > > 
> > > > > > action:  
> > > > > > disable\_delete\_all\_indexes: true
> > > > > > 
> > > > > > network :  
> > > > > > host : localhost
> > > > > > 
> > > > > > discovery.zen.ping.multicast:  
> > > > > > enabled: false
> > > > > > 
> > > > > > discovery.zen.ping.unicast:  
> > > > > > hosts: ["localhost:9300","localhost: **9** 301"]
> > > > > > 
> > > > > > # Shard level query and fetch threshold logging.
> > > > > > 
> > > > > > #index.search.slowlog.level: TRACE  
> > > > > > index.search.slowlog. **threshold**.query.warn: 10s  
> > > > > > index.search.slowlog. **threshold**.query.info[http://index.search.slowlog.threshold.query.info](http://index.search.slowlog.threshold.query.info):  
> > > > > > 5s  
> > > > > > #index.search.slowlog. **threshol** d.query.debug: 2s  
> > > > > > #index.search.slowlog. **threshol** d.query.trace: 500ms
> > > > > > 
> > > > > > index.search.slowlog. **threshold**.fetch.warn: 1s  
> > > > > > index.search.slowlog. **threshold**.fetch.info[http://index.search.slowlog.threshold.fetch.info](http://index.search.slowlog.threshold.fetch.info):  
> > > > > > 800ms  
> > > > > > #index.search.slowlog. **threshol** d.fetch.debug: 500ms  
> > > > > > #index.search.slowlog. **threshol** d.fetch.trace: 200ms
> > > > > > 
> > > > > > On Wed, Apr 11, 2012 at 5:12 PM, Kenneth Loafman \<[kenneth@loafman.com](mailto:kenneth@loafman.com)
> > > > > > 
> > > > > > > wrote:
> > > > > > 
> > > > > > > Here it is:
> > > > > > > 
> > > > > > > ~$ curl -XGET "localhost:9200/test/\_ **settings**?pretty=true"  
> > > > > > > {  
> > > > > > > "test" : {  
> > > > > > > "settings" : {  
> > > > > > > "index.number\_of\_shards" : "2",  
> > > > > > > "index.number\_of\_replicas" : "1",  
> > > > > > > "index.version.created" : "190199"  
> > > > > > > }  
> > > > > > > }  
> > > > > > > }
> > > > > > > 
> > > > > > > On Wed, Apr 11, 2012 at 5:06 PM, Igor Motov [imotov@gmail.com](mailto:imotov@gmail.com)wrote:
> > > > > > > 
> > > > > > > > I still cannot reproduce it. Something is missing. Most likely it's  
> > > > > > > > index settings. Could you share your config file or add output of the  
> > > > > > > > following command to the repro?
> > > > > > > > 
> > > > > > > > curl -XGET "localhost:9200/test/\_ **settings**"
> > > > > > > > 
> > > > > > > > On Wednesday, April 11, 2012 5:13:13 PM UTC-4, Kenneth Loafman  
> > > > > > > > wrote:
> > > > > > > > 
> > > > > > > > > 0.18.7 gives the same results.
> > > > > > > > > 
> > > > > > > > > BTW, original is here: [https://gist.github.com/\*\*\*\*\*\*2359644](https://gist.github.com/ ****** 2359644)[https://gist.github.com/2359644](https://gist.github.com/2359644)
> > > > > > > > > 
> > > > > > > > > ...Ken
> > > > > > > > > 
> > > > > > > > > On Wed, Apr 11, 2012 at 3:55 PM, Kenneth Loafman \<  
> > > > > > > > > [kenneth@loafman.com](mailto:kenneth@loafman.com)\> wrote:
> > > > > > > > > 
> > > > > > > > > > Just to make sure, I updated the original gist with a test that  
> > > > > > > > > > has the full mapping. It does not work. The output is from 0.19.1.
> > > > > > > > > > 
> > > > > > > > > > ...Ken
> > > > > > > > > > 
> > > > > > > > > > On Wed, Apr 11, 2012 at 3:28 PM, Kenneth Loafman \<  
> > > > > > > > > > [kenneth@loafman.com](mailto:kenneth@loafman.com)\> wrote:
> > > > > > > > > > 
> > > > > > > > > > > It will run without the publish\_datetime sort, that's part of  
> > > > > > > > > > > the gist I posted. Does NOT matter whether its populated or not.
> > > > > > > > > > > 
> > > > > > > > > > > ...Ken
> > > > > > > > > > > 
> > > > > > > > > > > On Wed, Apr 11, 2012 at 11:13 AM, Igor Motov [imotov@gmail.com](mailto:imotov@gmail.com)wrote:
> > > > > > > > > > > 
> > > > > > > > > > > > This recreation doesn't seem to recreate the problem. The issue  
> > > > > > > > > > > > is most likely in the mapping, and unfortunately, it's not visible in your  
> > > > > > > > > > > > example. This is what I am getting when I am trying to run it:  
> > > > > > > > > > > > [https://gist.github.com/\*\*23\*\*\*\*60279](https://gist.github.com/ **23**** 60279)[https://gist.github.com/2360279](https://gist.github.com/2360279) (I had to remove publish\_datetime sort, since it's not populated in your  
> > > > > > > > > > > > gist).
> > > > > > > > > > > > 
> > > > > > > > > > > > On Wednesday, April 11, 2012 10:30:30 AM UTC-4, Kenneth Loafman  
> > > > > > > > > > > > wrote:
> > > > > > > > > > > > 
> > > > > > > > > > > > > Here's the gist: [https://gist.github.com/\*\*\*\*\*\*\*\*2359644](https://gist.github.com/ ******** 2359644)[https://gist.github.com/2359644](https://gist.github.com/2359644)
> > > > > > > > > > > > > 
> > > > > > > > > > > > > ...Thanks,  
> > > > > > > > > > > > > ...Ken
> > > > > > > > > > > > > 
> > > > > > > > > > > > > On Wed, Apr 11, 2012 at 6:30 AM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com)wrote:
> > > > > > > > > > > > > 
> > > > > > > > > > > > > > Thats strange, the first option simply translates to a  
> > > > > > > > > > > > > > boolean query and the other one translates to a phrase query. Can you gist  
> > > > > > > > > > > > > > a recreation?
> > > > > > > > > > > > > > 
> > > > > > > > > > > > > > On Mon, Apr 9, 2012 at 9:24 PM, Kenneth Loafman \<  
> > > > > > > > > > > > > > [kenneth.loafman@gmail.com](mailto:kenneth.loafman@gmail.com)\> wrote:
> > > > > > > > > > > > > > 
> > > > > > > > > > > > > > > Hi,
> > > > > > > > > > > > > > > 
> > > > > > > > > > > > > > > We're seeing an odd problem in search that revolves around  
> > > > > > > > > > > > > > > stop words.
> > > > > > > > > > > > > > > 
> > > > > > > > > > > > > > > If I use the query (no quotes):  
> > > > > > > > > > > > > > > worst way to travel  
> > > > > > > > > > > > > > > ES returns nothing.
> > > > > > > > > > > > > > > 
> > > > > > > > > > > > > > > If I use the query (with quotes):  
> > > > > > > > > > > > > > > "worst way to travel"  
> > > > > > > > > > > > > > > ES returns matches.
> > > > > > > > > > > > > > > 
> > > > > > > > > > > > > > > Why the difference? Do I need to remove stop words manually  
> > > > > > > > > > > > > > > from a non-quoted query?
> > > > > > > > > > > > > > > 
> > > > > > > > > > > > > > > ...Ken

---

<div class="post-metadata">

**Author:** ![Igor\_Motov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_motov/32/45193_2.png) [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)\
**Post date:** [April 13, 2012, 6:56pm UTC](https://discuss.elastic.co/t/stop-words-in-quoted-vs-unquoted-search/7276/17 "2012-04-13T18:56:58Z")

</div>

So, it might be timing or environment issue then. Here is a few things you  
can try that might helps us to figure out what it is:

1. repeat the second query several times on the machine where it fails to  
see if it will start working
2. add curl -XGET  
'[http://127.0.0.2:9200/\_cluster/health?waitForStatus=green](http://127.0.0.2:9200/_cluster/health?waitForStatus=green)' before \_refresh
3. compare version of java and filesystem where data is located.

On Friday, April 13, 2012 2:35:59 PM UTC-4, Kenneth Loafman wrote:

> Thanks for the help Shay and Igor.
> 
> I tried Igor's request on my desktop and it worked. I upgraded my desktop  
> to 0.19.2 and it worked. I upgraded our test machine to 0.19.2, and it  
> failed. My desktop and the test machine have identical setups, so now I'm  
> even more confused. This may resolve down to something as trivial as a  
> minor version number on some support package. AArgh!
> 
> On Fri, Apr 13, 2012 at 7:39 AM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote:
> 
> > The refresh result of total 3 failed 0 can happen, effectively, the index  
> > gets created, and shards are starting to be allocated. The index requests  
> > will work once a primary shard is available. Then the refresh goes and not  
> > all replica shards have been allocated yet. I don't think thats the  
> > problem, but you can add a call to the health API with wait\_for\_status set  
> > to green after the index gets created. But, does that solve the problem?  
> > Thats what I missed here...
> > 
> > On Fri, Apr 13, 2012 at 4:38 AM, Igor Motov [imotov@gmail.com](mailto:imotov@gmail.com) wrote:
> > 
> > > I ran out of ideas. I tried reproducing it on Mac and Ubuntu using your  
> > > config and 3 different versions of java without success. I get results for  
> > > both searches every time.
> > > 
> > > Can you reproduce it with a freshly downloaded version of elasticsearch?  
> > > Basically, download elasticsearch 0.19.1 from [elasticsearch.org](http://elasticsearch.org), unzip  
> > > it, don't change configuration, start two nodes and run stop-words-test.sh.
> > > 
> > > On Thursday, April 12, 2012 9:04:01 PM UTC-4, Kenneth Loafman wrote:
> > > 
> > > > Ping. Any thoughts?
> > > > 
> > > > ...Ken
> > > > 
> > > > On Wed, Apr 11, 2012 at 8:11 PM, Kenneth Loafman [kenneth@loafman.com](mailto:kenneth@loafman.com)wrote:
> > > > 
> > > > > The test script does not do any delays after creating the index, but  
> > > > > if you put in a sleep after creating the index, all 4 shards are created.  
> > > > > It does not happen (the 3/4 shards thing) on other systems, but the lack  
> > > > > of search results happens on all 3 systems.
> > > > > 
> > > > > 1. my desktop, 2 nodes on the same machine, 0.19.1,
> > > > > 2. remote test machine, same config as above,
> > > > > 3. production cluster, 4 machines, 1 ES node per machine, 0.18.7
> > > > > 
> > > > > All machines are running Ubuntu Lucid, 64-bit, java 1.6.
> > > > > 
> > > > > I think it's the filtered query that's causing the problem. Stripping  
> > > > > out the filter part makes it work correctly. The query I sent in the gist  
> > > > > is actually a minimized version of the query we use in production, as is  
> > > > > the mapping. The full query fails as well, which is what got this started.
> > > > > 
> > > > > ...Ken
> > > > > 
> > > > > On Wed, Apr 11, 2012 at 5:30 PM, Igor Motov [imotov@gmail.com](mailto:imotov@gmail.com) wrote:
> > > > > 
> > > > > > The tricky part is this:
> > > > > > 
> > > > > > {"ok":true,"\_shards":{"total":\*\*4,_"successful":3_,"failed":0}}
> > > > > > 
> > > > > > I don't quite understand how this is possible. If you had one node  
> > > > > > running, you should have gotten "total":4, "successful":2. If you had two  
> > > > > > nodes running, you should have gotten "total":4, "successful":4. Having  
> > > > > > "total":4, "successful":3 most likely indicates that there is an issue with  
> > > > > > allocating one of the shards on one of the nodes. Do you see any errors in  
> > > > > > the log file on one of the nodes?
> > > > > > 
> > > > > > On Wednesday, April 11, 2012 6:17:11 PM UTC-4, Kenneth Loafman wrote:
> > > > > > 
> > > > > > > Here's elasticsearch.yml just in case...
> > > > > > > 
> > > > > > > gateway:  
> > > > > > > type: local
> > > > > > > 
> > > > > > > path :  
> > > > > > > data : /mnt/search-data-dev/node0/ **dat** a  
> > > > > > > work : /mnt/search-data-dev/node0/ **wor** k  
> > > > > > > logs : /mnt/search-data-dev/node0/ **log** s
> > > > > > > 
> > > > > > > index :  
> > > > > > > number\_of\_shards : 2  
> > > > > > > number\_of\_replicas : 1
> > > > > > > 
> > > > > > > bootstrap:  
> > > > > > > mlockall: true
> > > > > > > 
> > > > > > > action:  
> > > > > > > disable\_delete\_all\_indexes: true
> > > > > > > 
> > > > > > > network :  
> > > > > > > host : localhost
> > > > > > > 
> > > > > > > discovery.zen.ping.multicast:  
> > > > > > > enabled: false
> > > > > > > 
> > > > > > > discovery.zen.ping.unicast:  
> > > > > > > hosts: ["localhost:9300","localhost: **9** 301"]
> > > > > > > 
> > > > > > > # Shard level query and fetch threshold logging.
> > > > > > > 
> > > > > > > #index.search.slowlog.level: TRACE  
> > > > > > > index.search.slowlog. **threshold**.query.warn: 10s  
> > > > > > > index.search.slowlog. **threshold**.query.info[http://index.search.slowlog.threshold.query.info](http://index.search.slowlog.threshold.query.info):  
> > > > > > > 5s  
> > > > > > > #index.search.slowlog. **threshol** d.query.debug: 2s  
> > > > > > > #index.search.slowlog. **threshol** d.query.trace: 500ms
> > > > > > > 
> > > > > > > index.search.slowlog. **threshold**.fetch.warn: 1s  
> > > > > > > index.search.slowlog. **threshold**.fetch.info[http://index.search.slowlog.threshold.fetch.info](http://index.search.slowlog.threshold.fetch.info):  
> > > > > > > 800ms  
> > > > > > > #index.search.slowlog. **threshol** d.fetch.debug: 500ms  
> > > > > > > #index.search.slowlog. **threshol** d.fetch.trace: 200ms
> > > > > > > 
> > > > > > > On Wed, Apr 11, 2012 at 5:12 PM, Kenneth Loafman \<  
> > > > > > > [kenneth@loafman.com](mailto:kenneth@loafman.com)\> wrote:
> > > > > > > 
> > > > > > > > Here it is:
> > > > > > > > 
> > > > > > > > ~$ curl -XGET "localhost:9200/test/\_ **settings**?pretty=true"  
> > > > > > > > {  
> > > > > > > > "test" : {  
> > > > > > > > "settings" : {  
> > > > > > > > "index.number\_of\_shards" : "2",  
> > > > > > > > "index.number\_of\_replicas" : "1",  
> > > > > > > > "index.version.created" : "190199"  
> > > > > > > > }  
> > > > > > > > }  
> > > > > > > > }
> > > > > > > > 
> > > > > > > > On Wed, Apr 11, 2012 at 5:06 PM, Igor Motov [imotov@gmail.com](mailto:imotov@gmail.com)wrote:
> > > > > > > > 
> > > > > > > > > I still cannot reproduce it. Something is missing. Most likely  
> > > > > > > > > it's index settings. Could you share your config file or add output of the  
> > > > > > > > > following command to the repro?
> > > > > > > > > 
> > > > > > > > > curl -XGET "localhost:9200/test/\_ **settings**"
> > > > > > > > > 
> > > > > > > > > On Wednesday, April 11, 2012 5:13:13 PM UTC-4, Kenneth Loafman  
> > > > > > > > > wrote:
> > > > > > > > > 
> > > > > > > > > > 0.18.7 gives the same results.
> > > > > > > > > > 
> > > > > > > > > > BTW, original is here: [https://gist.github.com/\*\*\*\*\*\*2359644](https://gist.github.com/ ****** 2359644)[https://gist.github.com/2359644](https://gist.github.com/2359644)
> > > > > > > > > > 
> > > > > > > > > > ...Ken
> > > > > > > > > > 
> > > > > > > > > > On Wed, Apr 11, 2012 at 3:55 PM, Kenneth Loafman \<  
> > > > > > > > > > [kenneth@loafman.com](mailto:kenneth@loafman.com)\> wrote:
> > > > > > > > > > 
> > > > > > > > > > > Just to make sure, I updated the original gist with a test that  
> > > > > > > > > > > has the full mapping. It does not work. The output is from 0.19.1.
> > > > > > > > > > > 
> > > > > > > > > > > ...Ken
> > > > > > > > > > > 
> > > > > > > > > > > On Wed, Apr 11, 2012 at 3:28 PM, Kenneth Loafman \<  
> > > > > > > > > > > [kenneth@loafman.com](mailto:kenneth@loafman.com)\> wrote:
> > > > > > > > > > > 
> > > > > > > > > > > > It will run without the publish\_datetime sort, that's part of  
> > > > > > > > > > > > the gist I posted. Does NOT matter whether its populated or not.
> > > > > > > > > > > > 
> > > > > > > > > > > > ...Ken
> > > > > > > > > > > > 
> > > > > > > > > > > > On Wed, Apr 11, 2012 at 11:13 AM, Igor Motov [imotov@gmail.com](mailto:imotov@gmail.com)wrote:
> > > > > > > > > > > > 
> > > > > > > > > > > > > This recreation doesn't seem to recreate the problem. The  
> > > > > > > > > > > > > issue is most likely in the mapping, and unfortunately, it's not visible in  
> > > > > > > > > > > > > your example. This is what I am getting when I am trying to run it:  
> > > > > > > > > > > > > [https://gist.github.com/\*\*23\*\*\*\*60279](https://gist.github.com/ **23**** 60279)[https://gist.github.com/2360279](https://gist.github.com/2360279) (I had to remove publish\_datetime sort, since it's not populated in your  
> > > > > > > > > > > > > gist).
> > > > > > > > > > > > > 
> > > > > > > > > > > > > On Wednesday, April 11, 2012 10:30:30 AM UTC-4, Kenneth  
> > > > > > > > > > > > > Loafman wrote:
> > > > > > > > > > > > > 
> > > > > > > > > > > > > > Here's the gist: [https://gist.github.com/\*\*\*\*\*\*\*\*2359644](https://gist.github.com/ ******** 2359644)[https://gist.github.com/2359644](https://gist.github.com/2359644)
> > > > > > > > > > > > > > 
> > > > > > > > > > > > > > ...Thanks,  
> > > > > > > > > > > > > > ...Ken
> > > > > > > > > > > > > > 
> > > > > > > > > > > > > > On Wed, Apr 11, 2012 at 6:30 AM, Shay Banon \<[kimchy@gmail.com](mailto:kimchy@gmail.com)
> > > > > > > > > > > > > > 
> > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > 
> > > > > > > > > > > > > > > Thats strange, the first option simply translates to a  
> > > > > > > > > > > > > > > boolean query and the other one translates to a phrase query. Can you gist  
> > > > > > > > > > > > > > > a recreation?
> > > > > > > > > > > > > > > 
> > > > > > > > > > > > > > > On Mon, Apr 9, 2012 at 9:24 PM, Kenneth Loafman \<  
> > > > > > > > > > > > > > > [kenneth.loafman@gmail.com](mailto:kenneth.loafman@gmail.com)\> wrote:
> > > > > > > > > > > > > > > 
> > > > > > > > > > > > > > > > Hi,
> > > > > > > > > > > > > > > > 
> > > > > > > > > > > > > > > > We're seeing an odd problem in search that revolves around  
> > > > > > > > > > > > > > > > stop words.
> > > > > > > > > > > > > > > > 
> > > > > > > > > > > > > > > > If I use the query (no quotes):  
> > > > > > > > > > > > > > > > worst way to travel  
> > > > > > > > > > > > > > > > ES returns nothing.
> > > > > > > > > > > > > > > > 
> > > > > > > > > > > > > > > > If I use the query (with quotes):  
> > > > > > > > > > > > > > > > "worst way to travel"  
> > > > > > > > > > > > > > > > ES returns matches.
> > > > > > > > > > > > > > > > 
> > > > > > > > > > > > > > > > Why the difference? Do I need to remove stop words  
> > > > > > > > > > > > > > > > manually from a non-quoted query?
> > > > > > > > > > > > > > > > 
> > > > > > > > > > > > > > > > ...Ken

---

<div class="post-metadata">

**Author:** ![Kenneth\_Loafman\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kenneth_loafman_2/32/2224_2.png) [@Kenneth\_Loafman\_2](https://discuss.elastic.co/u/Kenneth_Loafman_2)\
**Post date:** [April 13, 2012, 9:45pm UTC](https://discuss.elastic.co/t/stop-words-in-quoted-vs-unquoted-search/7276/18 "2012-04-13T21:45:41Z")

</div>

It's turned out to be a Java version issue. My development machine  
has sun-java6-jre 6.26-2lucid1 while the rest have 6.24-1build0. I managed  
to update while it was still available. The 6.26 version has been deleted  
due to a license conflict with Oracle, so I've got a working version on my  
desktop where the rest are borked.

Igor, I tried 1 & 2 from your last message, but 3 was the real kicker.  
Thanks.

...Ken

On Fri, Apr 13, 2012 at 1:56 PM, Igor Motov [imotov@gmail.com](mailto:imotov@gmail.com) wrote:

> So, it might be timing or environment issue then. Here is a few things you  
> can try that might helps us to figure out what it is:
> 
> 1. repeat the second query several times on the machine where it fails to  
> see if it will start working
> 2. add curl -XGET '  
> [http://127.0.0.2:9200/\_cluster/health?waitForStatus=green](http://127.0.0.2:9200/_cluster/health?waitForStatus=green)' before \_refresh
> 3. compare version of java and filesystem where data is located.
> 
> On Friday, April 13, 2012 2:35:59 PM UTC-4, Kenneth Loafman wrote:
> 
> > Thanks for the help Shay and Igor.
> > 
> > I tried Igor's request on my desktop and it worked. I upgraded my  
> > desktop to 0.19.2 and it worked. I upgraded our test machine to 0.19.2,  
> > and it failed. My desktop and the test machine have identical setups, so  
> > now I'm even more confused. This may resolve down to something as trivial  
> > as a minor version number on some support package. AArgh!
> > 
> > On Fri, Apr 13, 2012 at 7:39 AM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote:
> > 
> > > The refresh result of total 3 failed 0 can happen, effectively, the  
> > > index gets created, and shards are starting to be allocated. The index  
> > > requests will work once a primary shard is available. Then the refresh goes  
> > > and not all replica shards have been allocated yet. I don't think thats the  
> > > problem, but you can add a call to the health API with wait\_for\_status set  
> > > to green after the index gets created. But, does that solve the problem?  
> > > Thats what I missed here...
> > > 
> > > On Fri, Apr 13, 2012 at 4:38 AM, Igor Motov [imotov@gmail.com](mailto:imotov@gmail.com) wrote:
> > > 
> > > > I ran out of ideas. I tried reproducing it on Mac and Ubuntu using your  
> > > > config and 3 different versions of java without success. I get results for  
> > > > both searches every time.
> > > > 
> > > > Can you reproduce it with a freshly downloaded version of  
> > > > elasticsearch? Basically, download elasticsearch 0.19.1 from  
> > > > [elasticsearch.org](http://elasticsearch.org), unzip it, don't change configuration, start two  
> > > > nodes and run stop-words-test.sh.
> > > > 
> > > > On Thursday, April 12, 2012 9:04:01 PM UTC-4, Kenneth Loafman wrote:
> > > > 
> > > > > Ping. Any thoughts?
> > > > > 
> > > > > ...Ken
> > > > > 
> > > > > On Wed, Apr 11, 2012 at 8:11 PM, Kenneth Loafman [kenneth@loafman.com](mailto:kenneth@loafman.com)wrote:
> > > > > 
> > > > > > The test script does not do any delays after creating the index, but  
> > > > > > if you put in a sleep after creating the index, all 4 shards are created.  
> > > > > > It does not happen (the 3/4 shards thing) on other systems, but the lack  
> > > > > > of search results happens on all 3 systems.
> > > > > > 
> > > > > > 1. my desktop, 2 nodes on the same machine, 0.19.1,
> > > > > > 2. remote test machine, same config as above,
> > > > > > 3. production cluster, 4 machines, 1 ES node per machine, 0.18.7
> > > > > > 
> > > > > > All machines are running Ubuntu Lucid, 64-bit, java 1.6.
> > > > > > 
> > > > > > I think it's the filtered query that's causing the problem.  
> > > > > > Stripping out the filter part makes it work correctly. The query I sent  
> > > > > > in the gist is actually a minimized version of the query we use in  
> > > > > > production, as is the mapping. The full query fails as well, which is what  
> > > > > > got this started.
> > > > > > 
> > > > > > ...Ken
> > > > > > 
> > > > > > On Wed, Apr 11, 2012 at 5:30 PM, Igor Motov [imotov@gmail.com](mailto:imotov@gmail.com) wrote:
> > > > > > 
> > > > > > > The tricky part is this:
> > > > > > > 
> > > > > > > {"ok":true,"\_shards":{"total":\*\*_4,"successful":3_,"failed":0}}
> > > > > > > 
> > > > > > > I don't quite understand how this is possible. If you had one node  
> > > > > > > running, you should have gotten "total":4, "successful":2. If you had two  
> > > > > > > nodes running, you should have gotten "total":4, "successful":4. Having  
> > > > > > > "total":4, "successful":3 most likely indicates that there is an issue with  
> > > > > > > allocating one of the shards on one of the nodes. Do you see any errors in  
> > > > > > > the log file on one of the nodes?
> > > > > > > 
> > > > > > > On Wednesday, April 11, 2012 6:17:11 PM UTC-4, Kenneth Loafman wrote:
> > > > > > > 
> > > > > > > > Here's elasticsearch.yml just in case...
> > > > > > > > 
> > > > > > > > gateway:  
> > > > > > > > type: local
> > > > > > > > 
> > > > > > > > path :  
> > > > > > > > data : /mnt/search-data-dev/node0/\*\*dat**a  
> > > > > > > > work : /mnt/search-data-dev/node0/\*\*wor**k  
> > > > > > > > logs : /mnt/search-data-dev/node0/\*\*log\*\*\*\*s
> > > > > > > > 
> > > > > > > > index :  
> > > > > > > > number\_of\_shards : 2  
> > > > > > > > number\_of\_replicas : 1
> > > > > > > > 
> > > > > > > > bootstrap:  
> > > > > > > > mlockall: true
> > > > > > > > 
> > > > > > > > action:  
> > > > > > > > disable\_delete\_all\_indexes: true
> > > > > > > > 
> > > > > > > > network :  
> > > > > > > > host : localhost
> > > > > > > > 
> > > > > > > > discovery.zen.ping.multicast:  
> > > > > > > > enabled: false
> > > > > > > > 
> > > > > > > > discovery.zen.ping.unicast:  
> > > > > > > > hosts: ["localhost:9300","localhost:\*\*9\*\*\*\*301"]
> > > > > > > > 
> > > > > > > > # Shard level query and fetch threshold logging.
> > > > > > > > 
> > > > > > > > #index.search.slowlog.level: TRACE  
> > > > > > > > index.search.slowlog. **threshold** \*\*.query.warn: 10s  
> > > > > > > > index.search.slowlog. **threshold** \*\*.query.info[http://index.search.slowlog.threshold.query.info](http://index.search.slowlog.threshold.query.info):  
> > > > > > > > 5s  
> > > > > > > > #index.search.slowlog.\*\*threshol**d.query.debug: 2s  
> > > > > > > > #index.search.slowlog.\*\*threshol**d.query.trace: 500ms
> > > > > > > > 
> > > > > > > > index.search.slowlog. **threshold** \*\*.fetch.warn: 1s  
> > > > > > > > index.search.slowlog. **threshold** \*\*.fetch.info[http://index.search.slowlog.threshold.fetch.info](http://index.search.slowlog.threshold.fetch.info):  
> > > > > > > > 800ms  
> > > > > > > > #index.search.slowlog.\*\*threshol**d.fetch.debug: 500ms  
> > > > > > > > #index.search.slowlog.\*\*threshol**d.fetch.trace: 200ms
> > > > > > > > 
> > > > > > > > On Wed, Apr 11, 2012 at 5:12 PM, Kenneth Loafman \<  
> > > > > > > > [kenneth@loafman.com](mailto:kenneth@loafman.com)\> wrote:
> > > > > > > > 
> > > > > > > > > Here it is:
> > > > > > > > > 
> > > > > > > > > ~$ curl -XGET "localhost:9200/test/\_ **settings** \*\*?pretty=true"  
> > > > > > > > > {  
> > > > > > > > > "test" : {  
> > > > > > > > > "settings" : {  
> > > > > > > > > "index.number\_of\_shards" : "2",  
> > > > > > > > > "index.number\_of\_replicas" : "1",  
> > > > > > > > > "index.version.created" : "190199"  
> > > > > > > > > }  
> > > > > > > > > }  
> > > > > > > > > }
> > > > > > > > > 
> > > > > > > > > On Wed, Apr 11, 2012 at 5:06 PM, Igor Motov [imotov@gmail.com](mailto:imotov@gmail.com)wrote:
> > > > > > > > > 
> > > > > > > > > > I still cannot reproduce it. Something is missing. Most likely  
> > > > > > > > > > it's index settings. Could you share your config file or add output of the  
> > > > > > > > > > following command to the repro?
> > > > > > > > > > 
> > > > > > > > > > curl -XGET "localhost:9200/test/\_ **settings** \*\*"
> > > > > > > > > > 
> > > > > > > > > > On Wednesday, April 11, 2012 5:13:13 PM UTC-4, Kenneth Loafman  
> > > > > > > > > > wrote:
> > > > > > > > > > 
> > > > > > > > > > > 0.18.7 gives the same results.
> > > > > > > > > > > 
> > > > > > > > > > > BTW, original is here: [https://gist.github.com/\*\*\*\*\*\*\*\*2359644](https://gist.github.com/ ******** 2359644)[https://gist.github.com/2359644](https://gist.github.com/2359644)
> > > > > > > > > > > 
> > > > > > > > > > > ...Ken
> > > > > > > > > > > 
> > > > > > > > > > > On Wed, Apr 11, 2012 at 3:55 PM, Kenneth Loafman \<  
> > > > > > > > > > > [kenneth@loafman.com](mailto:kenneth@loafman.com)\> wrote:
> > > > > > > > > > > 
> > > > > > > > > > > > Just to make sure, I updated the original gist with a test that  
> > > > > > > > > > > > has the full mapping. It does not work. The output is from 0.19.1.
> > > > > > > > > > > > 
> > > > > > > > > > > > ...Ken
> > > > > > > > > > > > 
> > > > > > > > > > > > On Wed, Apr 11, 2012 at 3:28 PM, Kenneth Loafman \<  
> > > > > > > > > > > > [kenneth@loafman.com](mailto:kenneth@loafman.com)\> wrote:
> > > > > > > > > > > > 
> > > > > > > > > > > > > It will run without the publish\_datetime sort, that's part of  
> > > > > > > > > > > > > the gist I posted. Does NOT matter whether its populated or not.
> > > > > > > > > > > > > 
> > > > > > > > > > > > > ...Ken
> > > > > > > > > > > > > 
> > > > > > > > > > > > > On Wed, Apr 11, 2012 at 11:13 AM, Igor Motov \<[imotov@gmail.com](mailto:imotov@gmail.com)
> > > > > > > > > > > > > 
> > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > 
> > > > > > > > > > > > > > This recreation doesn't seem to recreate the problem. The  
> > > > > > > > > > > > > > issue is most likely in the mapping, and unfortunately, it's not visible in  
> > > > > > > > > > > > > > your example. This is what I am getting when I am trying to run it:  
> > > > > > > > > > > > > > [https://gist.github.com/\*\*23\*\*\*\*\*\*60279](https://gist.github.com/ **23****** 60279)[https://gist.github.com/2360279](https://gist.github.com/2360279) (I had to remove publish\_datetime sort, since it's not populated in your  
> > > > > > > > > > > > > > gist).
> > > > > > > > > > > > > > 
> > > > > > > > > > > > > > On Wednesday, April 11, 2012 10:30:30 AM UTC-4, Kenneth  
> > > > > > > > > > > > > > Loafman wrote:
> > > > > > > > > > > > > > 
> > > > > > > > > > > > > > > Here's the gist: [https://gist.github.com/\*\*\*\*\*\*\*\*\*\*2359644](https://gist.github.com/ ********** 2359644)[https://gist.github.com/2359644](https://gist.github.com/2359644)
> > > > > > > > > > > > > > > 
> > > > > > > > > > > > > > > ...Thanks,  
> > > > > > > > > > > > > > > ...Ken
> > > > > > > > > > > > > > > 
> > > > > > > > > > > > > > > On Wed, Apr 11, 2012 at 6:30 AM, Shay Banon \<  
> > > > > > > > > > > > > > > [kimchy@gmail.com](mailto:kimchy@gmail.com)\> wrote:
> > > > > > > > > > > > > > > 
> > > > > > > > > > > > > > > > Thats strange, the first option simply translates to a  
> > > > > > > > > > > > > > > > boolean query and the other one translates to a phrase query. Can you gist  
> > > > > > > > > > > > > > > > a recreation?
> > > > > > > > > > > > > > > > 
> > > > > > > > > > > > > > > > On Mon, Apr 9, 2012 at 9:24 PM, Kenneth Loafman \<  
> > > > > > > > > > > > > > > > [kenneth.loafman@gmail.com](mailto:kenneth.loafman@gmail.com)\> wrote:
> > > > > > > > > > > > > > > > 
> > > > > > > > > > > > > > > > > Hi,
> > > > > > > > > > > > > > > > > 
> > > > > > > > > > > > > > > > > We're seeing an odd problem in search that revolves around  
> > > > > > > > > > > > > > > > > stop words.
> > > > > > > > > > > > > > > > > 
> > > > > > > > > > > > > > > > > If I use the query (no quotes):  
> > > > > > > > > > > > > > > > > worst way to travel  
> > > > > > > > > > > > > > > > > ES returns nothing.
> > > > > > > > > > > > > > > > > 
> > > > > > > > > > > > > > > > > If I use the query (with quotes):  
> > > > > > > > > > > > > > > > > "worst way to travel"  
> > > > > > > > > > > > > > > > > ES returns matches.
> > > > > > > > > > > > > > > > > 
> > > > > > > > > > > > > > > > > Why the difference? Do I need to remove stop words  
> > > > > > > > > > > > > > > > > manually from a non-quoted query?
> > > > > > > > > > > > > > > > > 
> > > > > > > > > > > > > > > > > ...Ken

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 3:32am UTC](https://discuss.elastic.co/t/stop-words-in-quoted-vs-unquoted-search/7276/19 "2017-07-06T03:32:38Z")

</div>