Querying Elasticsearch with NEST error: "Input string was not in a correct format."

Elastic Stack Elasticsearch
1

Hi, I'm using NEST (latest version 7.0.0-alpha2) and trying to query Elasticsearch (7.0) with following:

    var filters = new List<Func<QueryContainerDescriptor<MyDocument>, QueryContainer>>();

    filters.Add(fq => fq.Term(t => t.Field(f => f.Status).Value("0")));//status=0
    filters.Add(fq => fq.Term(t => t.Field(f => f.FreeRound).Value(false)));//freeRound = false
    filters.Add(fq => fq.Term(t => t.Field(f => f.PlayerId).Value(playerId)));//playerId
    filters.Add(fq => fq.DateRange(dr => dr.Field(f => f.eventTime).GreaterThanOrEquals(startDate)));//from Date
    if (endDate != null)
        filters.Add(fq => fq.DateRange(dr => dr.Field(f => f.eventTime)LessThanOrEquals(endDate)));//to Date
    if (gameList.Count > 0)
        filters.Add(fq => fq.Terms(t => t.Field(f => f.ExtGameId).Terms(gameList)));//games

    try
    {
        var responsesResult = _client.Search<MyDocument>(s => s
            .Query(q=> q
                .Bool(bq => bq
                    .Filter(filters)
                )
            )
            .Aggregations(a => a.Sum("sum_amount", sum => sum.Field(f => f.Amount )))
        );

        var sumAmount = responsesResult.Aggregations.Sum("sum_amount");

    }
    catch (Exception e)
    {
        Console.WriteLine(e);
        return null;
    }

But, for some reason, this fails with a message:

Elasticsearch.Net.UnexpectedElasticsearchClientException: Input string was not in a correct format. ---> System.FormatException: Input string was not in a correct format.
   at System.Number.StringToNumber(String str, NumberStyles options, NumberBuffer& number, NumberFormatInfo info, Boolean parseDecimal)
   at System.Number.ParseDecimal(String value, NumberStyles options, NumberFormatInfo numfmt)
   at Elasticsearch.Net.DecimalFormatter.Deserialize(JsonReader& reader, IJsonFormatterResolver formatterResolver)
   at Deserialize(Object[] , JsonReader& , IJsonFormatterResolver )
   at Elasticsearch.Net.DynamicMethodAnonymousFormatter`1.Deserialize(JsonReader& reader, IJsonFormatterResolver formatterResolver)
   at Nest.SourceFormatter`1.Deserialize(JsonReader& reader, IJsonFormatterResolver formatterResolver)
   at Deserialize(Object[] , JsonReader& , IJsonFormatterResolver )
   at Elasticsearch.Net.DynamicMethodAnonymousFormatter`1.Deserialize(JsonReader& reader, IJsonFormatterResolver formatterResolver)
   at Nest.ReadAsFormatter`2.Deserialize(JsonReader& reader, IJsonFormatterResolver formatterResolver)
   at Elasticsearch.Net.CollectionFormatterBase`4.Deserialize(JsonReader& reader, IJsonFormatterResolver formatterResolver)
   at Deserialize(Object[] , JsonReader& , IJsonFormatterResolver )
   at Elasticsearch.Net.DynamicMethodAnonymousFormatter`1.Deserialize(JsonReader& reader, IJsonFormatterResolver formatterResolver)
   at Nest.ReadAsFormatter`2.Deserialize(JsonReader& reader, IJsonFormatterResolver formatterResolver)
   at Deserialize(Object[] , JsonReader& , IJsonFormatterResolver )
   at Elasticsearch.Net.DynamicMethodAnonymousFormatter`1.Deserialize(JsonReader& reader, IJsonFormatterResolver formatterResolver)
   at Elasticsearch.Net.JsonSerializer.Deserialize[T](Byte[] bytes, Int32 offset, IJsonFormatterResolver resolver)
   at Elasticsearch.Net.JsonSerializer.Deserialize[T](Byte[] bytes, IJsonFormatterResolver resolver)
   at Elasticsearch.Net.JsonSerializer.Deserialize[T](Stream stream, IJsonFormatterResolver resolver)
   at Nest.InternalSerializer.Deserialize[T](Stream stream)
   at Elasticsearch.Net.ResponseBuilder.SetBody[TResponse](ApiCallDetails details, RequestData requestData, Stream responseStream, String mimeType)
   at Elasticsearch.Net.ResponseBuilder.ToResponse[TResponse](RequestData requestData, Exception ex, Nullable`1 statusCode, IEnumerable`1 warnings, Stream responseStream, String mimeType)
   at Elasticsearch.Net.HttpConnection.Request[TResponse](RequestData requestData)
   at Elasticsearch.Net.RequestPipeline.CallElasticsearch[TResponse](RequestData requestData)
   at Elasticsearch.Net.Transport`1.Request[TResponse](HttpMethod method, String path, PostData data, IRequestParameters requestParameters)
   --- End of inner exception stack trace ---
   at Elasticsearch.Net.Transport`1.Request[TResponse](HttpMethod method, String path, PostData data, IRequestParameters requestParameters)
   at Elasticsearch.Net.ElasticLowLevelClient.DoRequest[TResponse](HttpMethod method, String path, PostData data, IRequestParameters requestParameters)
   at Nest.ElasticClient.DoRequest[TRequest,TResponse](TRequest p, IRequestParameters parameters, Action`1 forceConfiguration)
   at Nest.ElasticClient.Search[T,TResult](ISearchRequest request)
   at Nest.ElasticClient.Search[T](ISearchRequest request)
   at Nest.ElasticClient.Search[T,TResult](Func`2 selector)
   at Nest.ElasticClient.Search[T](Func`2 selector)

And here are the mappings for relevant fields:

  "amount": {
    "type": "double"
  },
  "eventTime": {
    "type": "date"
  },
  "extGameId": {
    "type": "keyword"
  },
  "freeRound": {
    "type": "boolean"
  },
  "playerId": {
    "type": "long"
  },
  "status": {
    "type": "keyword"
  }
}

And the POCO:

[Number(NumberType.Double)]
public decimal Amount { get; set; }
[Date]
public DateTime eventTime { get; set; }
[Keyword]
public string ExtGameId { get; set; }
[Boolean]
public decimal FreeRound { get; set; }
[Number(NumberType.Long)]
public long PlayerId { get; set; }
[Keyword]
public string Status { get; set; }
2

EDIT:
This is strange, when I try to do a simple search with the match all query and without any aggregations (just to see if I can get a basic operation done) :slight_smile:

        var responsesResult = _client.Search<MyDocument>(s => s
            .Size(10)
            .Query(q => q.MatchAll())
        );

I get jet this error:

Elasticsearch.Net.UnexpectedElasticsearchClientException: expected:'Number Token', actual:'"3"', at offset:459 ---> Elasticsearch.Net.JsonParsingException: expected:'Number Token', actual:'"3"', at offset:459
   at Elasticsearch.Net.JsonReader.ReadInt64()
   at Elasticsearch.Net.JsonReader.ReadInt32()
   at Deserialize(Object[] , JsonReader& , IJsonFormatterResolver )
   at Elasticsearch.Net.DynamicMethodAnonymousFormatter`1.Deserialize(JsonReader& reader, IJsonFormatterResolver formatterResolver)
   at Nest.SourceFormatter`1.Deserialize(JsonReader& reader, IJsonFormatterResolver formatterResolver)
   at Deserialize(Object[] , JsonReader& , IJsonFormatterResolver )
   at Elasticsearch.Net.DynamicMethodAnonymousFormatter`1.Deserialize(JsonReader& reader, IJsonFormatterResolver formatterResolver)
   at Nest.ReadAsFormatter`2.Deserialize(JsonReader& reader, IJsonFormatterResolver formatterResolver)
   at Elasticsearch.Net.CollectionFormatterBase`4.Deserialize(JsonReader& reader, IJsonFormatterResolver formatterResolver)
   at Deserialize(Object[] , JsonReader& , IJsonFormatterResolver )
   at Elasticsearch.Net.DynamicMethodAnonymousFormatter`1.Deserialize(JsonReader& reader, IJsonFormatterResolver formatterResolver)
   at Nest.ReadAsFormatter`2.Deserialize(JsonReader& reader, IJsonFormatterResolver formatterResolver)
   at Deserialize(Object[] , JsonReader& , IJsonFormatterResolver )
   at Elasticsearch.Net.DynamicMethodAnonymousFormatter`1.Deserialize(JsonReader& reader, IJsonFormatterResolver formatterResolver)
   at Elasticsearch.Net.JsonSerializer.Deserialize[T](Byte[] bytes, Int32 offset, IJsonFormatterResolver resolver)
   at Elasticsearch.Net.JsonSerializer.Deserialize[T](Byte[] bytes, IJsonFormatterResolver resolver)
   at Elasticsearch.Net.JsonSerializer.Deserialize[T](Stream stream, IJsonFormatterResolver resolver)
   at Nest.InternalSerializer.Deserialize[T](Stream stream)
   at Elasticsearch.Net.ResponseBuilder.SetBody[TResponse](ApiCallDetails details, RequestData requestData, Stream responseStream, String mimeType)
   at Elasticsearch.Net.ResponseBuilder.ToResponse[TResponse](RequestData requestData, Exception ex, Nullable`1 statusCode, IEnumerable`1 warnings, Stream responseStream, String mimeType)
   at Elasticsearch.Net.HttpConnection.Request[TResponse](RequestData requestData)
   at Elasticsearch.Net.RequestPipeline.CallElasticsearch[TResponse](RequestData requestData)
   at Elasticsearch.Net.Transport`1.Request[TResponse](HttpMethod method, String path, PostData data, IRequestParameters requestParameters)
   --- End of inner exception stack trace ---
   at Elasticsearch.Net.Transport`1.Request[TResponse](HttpMethod method, String path, PostData data, IRequestParameters requestParameters)
   at Elasticsearch.Net.ElasticLowLevelClient.DoRequest[TResponse](HttpMethod method, String path, PostData data, IRequestParameters requestParameters)
   at Nest.ElasticClient.DoRequest[TRequest,TResponse](TRequest p, IRequestParameters parameters, Action`1 forceConfiguration)
   at Nest.ElasticClient.Search[T,TResult](ISearchRequest request)
   at Nest.ElasticClient.Search[T](ISearchRequest request)
   at Nest.ElasticClient.Search[T,TResult](Func`2 selector)
   at Nest.ElasticClient.Search[T](Func`2 selector)

Could it be something with mapping, even I can't see where?!

3

The serializer used in NEST 7.x is strict about how types are deserialized. In this example, the response contains a JSON field the the string value "3", but I suspect MyDocument has a property of a numeric type like int. The serializer does not coerce the string value to a numeric value.

I would recommend fixing the way in which documents are being indexed if you can, so that "3" is indexed as 3.

4

Hi, and thanks for your answer. The data are being collected from log files (which contain requests and responses to and from our API endpoint) using FileBeat and Logstash and are stored (indexed) to Elasticsearch. To be honest, I'm not responsible for that part but will see what can be done

5

Hi, I'm wondering why are data stored as a string and thus bypassing mappings? I've checked the documentation and found that coerce parameter is true by default. So shouldn't conversion be done? Why "3" isn't converted to 3 during indexing?

6

"3" may be being coerced to 3 for indexing, but it will be unchanged in the original JSON document sent to Elasticsearch, which is persisted and returned as _source (the individual value indexed, and the field on the JSON document are two different items of data).

If you can, I would recommend sending the field in the JSON document as 3. Alternatively, you would need to model that field as a string property on the POCO.

7

Thanks once again for your effort on this. Will do so.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.