Constructing complex NEST query

[gswartz (Geoff Swartz)]

Our index consists of a text field for notes. The notes can be tied to multiple entity types in our system such as a job, worker, company, etc. So, I need to somehow construct the search query based on what type of entity the user wants to restrict the notes type to. So, for example, if a user is looking for all notes with a specific keyword that are also company notes, I want to do something like below (I think).

var searchResponse = client.Search<NotesAll>(s => s
    .From(viewModel.Page * retval.PageSize)
    .Size(retval.PageSize)
    .Query(q => q
         .Match(m => m
            .Field(f => f.Notes)
            .Query(viewModel.Keyword.ToLower())
         )
    )
    .Query(q => q.Exists(y => y.Field(x => x.Company)))
    .Sort(x => x.Descending(y => y.LastModifiedDate))
);

Trying to match the keyword in the notes where also, the company field is not null.

The problem I have is that I have to create the second Query for Exists based on what type of search they're running and I'm not sure how to do that without having a bunch of case statements. I was hoping with NEST I would have a search object that I could add additional criteria to. Maybe it does and I'm just not seeing how. Any thoughts? Thanks!

[gswartz (Geoff Swartz)]

I eventually patched something together that kind of works, but not what I'm looking for. Rather than limiting it to records that have a match to my search terms AND also have a value in the company_id field, it appears to be disregarding the keyword and returning 10000 results.

var queryContainer = new List<QueryContainer>();
var mustContainer = new List<QueryContainer>();

foreach (var term in terms)
{
    queryContainer.Add(new TermQuery { Field = "notes", Value = term.ToLower()});
}

if (viewModel.Company)
    mustContainer.Add(new ExistsQuery
    {
        Field = "company_id",
    });

var boolQuery = new BoolQuery()
{
    Should = queryContainer,
    Must = mustContainer
};

var searchRequest = new SearchRequest<NotesAll>
{
    From = viewModel.Page * retval.PageSize,
    Size = retval.PageSize,
    Query = boolQuery
};

The resulting json from this query is this.

{
    "from": 0,
    "query": {
        "bool": {
            "must": [{
                "exists": {
                    "field": "company_id"
                }
            }],
            "should": [{
                "term": {
                    "notes": {
                        "value": "geoff"
                    }
                }
            }, {
                "term": {
                    "notes": {
                        "value": "smith"
                    }
                }
            }]
        }
    },
    "size": 20
}

So, how can I change this in order to get any keywrod matches for (geoff or smith) in notes AND requiring of a value in the company_id field? Thanks.

[gswartz (Geoff Swartz)]

I think I figured it out. I changed should to filter and that seems to give me what I'm looking for.

[forloop (Russ Cam)]

Glad to hear that you have some working for you, @gswartz.

One thing about should clauses with the presence of a must clause is that a document does not have to be a match for any of them i.e. none of them have to match. In this scenario, should clauses act as a boosting signal to calculating a relevancy score in a more is better approach. If this is not the behaviour desired and at least one of the should clauses must match, you can use the minimum_should_match parameter to specify a number or percentage of them that must match.