_all field matching fewer results

[luka (Luka Zitnik)]

Hi!

I have a request of this form:

POST http://localhost:9200/_search
{
"query": {
"match": {
"_all": {
"query": "foo bar",
"operator": "and"
}
}
},
"size": 20
}

and it returns fewer hits than this one:

POST http://localhost:9200/_search
{
"query": {
"match": {
"baz": {
"query": "foo bar",
"operator": "and"
}
}
},
"size": 20
}

Does this make any sense?

[Jaspreet_Singh]

Yes it is pretty obvious.
In the first search, you are searching in all fields via _all.
The second query, searches only within field named baz. Hence, the second query results are bound to be lesser and a subset of the first query results.

[luka (Luka Zitnik)]

Thank you Jaspreet_Singh. That would be obvious to me too. The problem that I'm facing is the other way round - I get fewer results when using _all.

Other things I know I use is ngram filter and boost.

I would really like to know what's going on. If you know a good debugging technique, please share.

[Jaspreet_Singh]

My bad and I apologize for overlooking that detail.
Can you share your mappings as well?

[luka (Luka Zitnik)]

I can't. At least I think, in this direction, I would have to make a complete example to prove the case, including data. But I can say that I've noticed that, apart from the mapping being "normal", only a few prebuilt analyzers make consistent results. It feels like _all will always use one of those analyzers. The standard analyzer maybe.

[Jaspreet_Singh]

So here is something I put together, using all defaults (including analyzer standard for text types). It gives exactly expected results, search in _all gives more results than a search in specific field.

PUT test_all
{
  "mappings": {
    "docs":{
      "properties": {
        "title":{
          "type": "text"
        },
        "body":{
          "type": "text"
        }
      }
    }
  }
}

PUT test_all/docs/1
{
  "title":"foo bar",
  "body":"nothing"
}
PUT test_all/docs/2
{
  "title":"nothing",
  "body":"foo bar"
}

GET test_all/_search
{
  "query": {
    "match": {
      "_all": {
        "query": "foo bar",
        "operator": "and"
      }
    }
  }
}

{
  "took": 0,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 2,
    "max_score": 0.5063205,
    "hits": [
      {
        "_index": "test_all",
        "_type": "docs",
        "_id": "2",
        "_score": 0.5063205,
        "_source": {
          "title": "nothing",
          "body": "foo bar"
        }
      },
      {
        "_index": "test_all",
        "_type": "docs",
        "_id": "1",
        "_score": 0.5063205,
        "_source": {
          "title": "foo bar",
          "body": "nothing"
        }
      }
    ]
  }
}

GET test_all/_search
{
"query": {
    "match": {
      "title": {
        "query": "foo bar",
        "operator": "and"
      }
    }
  }
}

{
  "took": 0,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 1,
    "max_score": 0.51623213,
    "hits": [
      {
        "_index": "test_all",
        "_type": "docs",
        "_id": "1",
        "_score": 0.51623213,
        "_source": {
          "title": "foo bar",
          "body": "nothing"
        }
      }
    ]
  }
}