Elasticsearch .Net client ignoring Term in Query?

Elastic Search
1

Hi, I am trying to update_by_query using the following

var updateResult = await ElasticSearch.ClientPool.Client.UpdateByQueryAsync<ElasticSearch.ESMPacketDocument>("mpackets", u => u
    .Query(q => q
        .Bool(b => b                                
            .Must(f => f
                .Term(t => t.Field("fields.processed").Value(0))
                .Prefix(p => p.Field("mqtt.topic").Value(TopicFilter))
            )
        )
    )
    .MaxDocs(MessagesPerQuery)
    .Script(s => s
        .Source($"ctx._source.fields.processed = {CycleKey}")
    )
    .Refresh(true), stoppingToken);

However the generated query (turning debug info on) is

# Audit trail of this API call:
 - [2] MaxRetriesReached: Took: 00:00:00.0000006
# OriginalException: Elastic.Transport.TransportException: Maximum number of retries reached. Call: Status code 409 from: POST /mpackets/_update_by_query?pretty=true&error_trace=true&refresh=true
# Request:
{
  "max_docs": 1000,
  "query": {
    "bool": {
      "must": {
        "prefix": {
          "mqtt.topic": {
            "value": "topicname1"
          }
        }
      }
    }
  },
  "script": {
    "source": "ctx._source.fields.processed = 1287344637"
  }
}

As you can see the term part of the query is ignored... Any thoughts?

However it seems fine when used one a Search as below

  var searchResponse = await ElasticSearch.ClientPool.Client.SearchAsync<ElasticSearch.ESMPacketDocument>(u => u
                        .Index("mpackets")
                        .Query(q => q
                            .Bool(b => b
                                .Filter(f => f
                                    .Term(t => t.Field(F => F.fields.processed).Value(CycleKey))
                                )
                            )
                        )
                        .Size(MessagesPerQuery)
                        .Sort(s => s
                            .Field(f => f.timestamp, s => s.Order(SortOrder.Asc))
                        ), stoppingToken);

Update---
It seems to have to do with having two items in there... The .Term and .Prefix. If I swap the order the other one comes through... how do I get both in the query?

2

Hi @kk123,

I'm not super familiar with .net client for Elasticsearch, but I've looked into docs and found examples like this:

var firstSearchResponse = client.Search<Project>(s => s
    .Query(q => q
        .Term(p => p.Name, "x") && q
        .Term(p => p.Name, "y")
    )
);

Might it work if you change your query to be like this?

var updateResult = await ElasticSearch.ClientPool.Client.UpdateByQueryAsync<ElasticSearch.ESMPacketDocument>("mpackets", u => u
    .Query(q => q
        .Bool(b => b                                
            .Must(f => f
                .Term(t => t.Field("fields.processed").Value(0)) && f
                .Prefix(p => p.Field("mqtt.topic").Value(TopicFilter))
            )
        )
    )
    .MaxDocs(MessagesPerQuery)
    .Script(s => s
        .Source($"ctx._source.fields.processed = {CycleKey}")
    )
    .Refresh(true), stoppingToken);
3

Hi @Artem_Shelkovnikov, thanks for your reply. I managed to solve it in the end through trial and error. The problem was that I should not chain f.Term(...).Prefix(...)... I should rather have done

f=> f.Term(...), f=> f.Prefix(...)

so the Query portion should read

.Query(q => q
        .Bool(b => b                                
            .Must(f => f.Term(t => t.Field("fields.processed").Value(0)),
                       f => f.Prefix(p => p.Field("mqtt.topic").Value(TopicFilter))
            )
        )
    )

Hopefully that is useful to others...