Hi folks, I am trying to index a fairly high-volume log source, and am using the ElasticNEST client in a C# app to do so. On most of my batches, I am getting an Invalid NEST response
error.
As far as I can see, the error is always CausedBy: ""
, which is not particularly enlightening. There are null
values (strings) in my objects, but I would have thought that was fine?
Invalid NEST response built from a successful low level call on POST: /powermta-general-2017.03.13/_bulk
# Invalid Bulk items:
operation[0]: index returned 429 _index: powermta-general-2017.03.13 _type: object _id: AVrIPNQBs9B18ytIfFDC _version: 0 error: Type: es_rejected_execution_exception Reason: "rejected execution of org.elasticsearch.transport.TransportService$7@4582ee19 on EsThreadPoolExecutor[bulk, queue capacity = 50, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor@46b4acda[Running, pool size = 2, active threads = 2, queued tasks = 50, completed tasks = 762653]]" CausedBy: ""
# Audit trail of this API call:
- [1] PingSuccess: Node: https://cloudclusternode.eu-west-1.aws.found.io:9243/ Took: 00:00:00.2298987
- [2] HealthyResponse: Node: https://cloudclusternode.eu-west-1.aws.found.io:9243/ Took: 00:00:00.2854329
# Request:
{"index":{"_type":"object","_id":null}}
{"beat":{"hostname":"MARKDESKTOP","filename":"\\\\ca-mta01\\e$\\logs\\process\\acct-2017-03-13-0199.csv"},"dlv":{"destinationIp":"1.2.3.4","esmtpAvailable":"ENHANCEDSTATUSCODES,PIPELINING,8BITMIME,SIZE,DELIVERBY,STARTTLS","size":0,"sourceIp":"37.221.216.177","type":"smtp"},"dsn":{"action":"relayed","diag":"smtp;250 2.0.0 v2DEgCup011544 Message accepted for delivery","diagNormalized":"250 2.0.0 v2DEgCup011544 Message accepted for delivery","diagCode":250,"mta":{"name":"al-ip4-mx-vip2.mailserver.net","address":"144.160.235.144","provider":"other","domain":"mailserver.net"},"status":"2.0.0 (success)","statusCode":200},"header":{},"job":{"name":"emaildomainlz_46","id":0,"instance":"emaildomain","campaignId":46},"src":{"type":"api"},"recordType":{"code":"d","name":"delivered"},"vmta":{"name":"emaildomainlz"},"message":"d,2017-03-13 14:42:22+0000,2017-03-13 14:41:18+0000,emaildomainlz.F.6495.46.0@emaildomain.sgml2.com,jmraft@destinationdomain.net,,relayed,2.0.0 (success),smtp;250 2.0.0 v2DEgCup011544 Message accepted for delivery,al-ip4-mx-vip2.mailserver.net (144.160.235.144),,api,,smtp,37.221.216.177,144.160.235.144,\"ENHANCEDSTATUSCODES,PIPELINING,8BITMIME,SIZE,DELIVERBY,STARTTLS\",47489,emaildomainlz,emaildomainlz_46,,destinationdomain.net/emaildomainlz,,,,,,,,,,,,,,","orig":{"address":"emaildomainlz.f.6495.46.0@emaildomain.com","domain":"emaildomain.com"},"queue":"destinationdomain.net/emaildomainlz","rcpt":{"address":"user@destinationdomain.net","domain":"destinationdomain.net"},"timeLogged":"2017-03-13T14:42:22+00:00","timeQueued":"2017-03-13T14:41:18+00:00"}
# Response:
{"took":0,"errors":true,"items":[{"index":{"_index":"powermta-general-2017.03.13","_type":"object","_id":"AVrIPNQBs9B18ytIfFDC","status":429,"error":{"type":"es_rejected_execution_exception","reason":"rejected execution of org.elasticsearch.transport.TransportService$7@4582ee19 on EsThreadPoolExecutor[bulk, queue capacity = 50, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor@46b4acda[Running, pool size = 2, active threads = 2, queued tasks = 50, completed tasks = 762653]]"}}}]}
(Currently using NEST v5.2.0 and Elasticsearch.net v5.2.0)