Hi, i am querying an REST api on my applications and indexing response json as elastic documents. Some of the field are integers so first time the index is create with numeric type for those fields but later the api some days can return "N/A" on those field instead of an integer and then ELK fails to index the full document with error.
Is there anyway to tell elasticsearch to ignore those invalid field values or set them to other value like -1 but index the rest of the document?
Invalid NEST response built from a unsuccessful low level call on PUT: /ueb-metrics-2017.04/object/7953676b-1723-419a-b078-cac7112be454_backup_copy_target_a8_2017.04.04_23.55
# Audit trail of this API call:
- [1] BadResponse: Node: https://xxxxx:8443/ Took: 00:00:00.1406970
# ServerError: ServerError: 400Type: mapper_parsing_exception Reason: "failed to parse [target.max_size]" CausedBy: "Type: number_format_exception Reason: "For input string: "N/A"""
# OriginalException: System.Net.WebException: The remote server returned an error: (400) Bad Request.
at System.Net.HttpWebRequest.EndGetResponse (IAsyncResult asyncResult) <0x400f8b90 + 0x00197> in <filename unknown>:0
at System.Net.HttpWebRequest.GetResponse () <0x400ef080 + 0x00053> in <filename unknown>:0
at Elasticsearch.Net.HttpConnection.Request[TReturn] (Elasticsearch.Net.RequestData requestData) <0x400dd000 + 0x0029f> in <filename unknown>:0
# Request:
<Request stream not captured or already read to completion by serializer. Set DisableDirectStreaming() on ConnectionSettings to force it to be set on the response.>
# Response:
<Response stream not captured or already read to completion by serializer. Set DisableDirectStreaming() on ConnectionSettings to force it to be set on the response.>