Error while copying content to a stream

Elastic Stack Elasticsearch
1

I am trying to push 80mb document to elasticsearch cluster and getting "Error while copying content to a stream".

I have also increase heap size as -Xms6g, -Xmx6g still getting an error.

Error details as below

OriginalException Message: Error while copying content to a stream. 
OriginalException StackTrace:    at System.Net.Http.HttpContent.CopyToAsyncCore(ValueTask copyTask)
   at System.Net.Http.HttpConnection.SendRequestContentAsync(HttpRequestMessage request, HttpContentWriteStream stream, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
   at Elasticsearch.Net.HttpConnection.RequestAsync[TResponse](RequestData requestData, CancellationToken cancellationToken) 
   
InnerException Message: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. 

InnerException StackTrace:    at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
   at System.Net.Http.HttpConnection.WriteAsync(ReadOnlyMemory`1 source)
   at System.IO.Stream.CopyToAsyncInternal(Stream destination, Int32 bufferSize, CancellationToken cancellationToken)
   at System.Net.Http.HttpContent.CopyToAsyncCore(ValueTask copyTask)
   
   
Invalid NEST response built from a unsuccessful low level call on PUT: /index-apttus.micsrosoft.tenant.sanitycheck/document/9ff83b6f-fb1c-4d80-8e11-62cc22faa4a9?pipeline=attachments
# Audit trail of this API call:
 - [1] BadRequest: Node: http://localhost:9200/ Took: 00:00:01.7456023
# OriginalException: System.Net.Http.HttpRequestException: Error while copying content to a stream. ---> System.IO.IOException: Unable to read data from the transport connection: An established connection was aborted by the software in your host machine. ---> System.Net.Sockets.SocketException: An established connection was aborted by the software in your host machine
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnection.WriteAsync(ReadOnlyMemory`1 source)
   at System.IO.Stream.CopyToAsyncInternal(Stream destination, Int32 bufferSize, CancellationToken cancellationToken)
   at System.Net.Http.HttpContent.CopyToAsyncCore(ValueTask copyTask)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpContent.CopyToAsyncCore(ValueTask copyTask)
   at System.Net.Http.HttpConnection.SendRequestContentAsync(HttpRequestMessage request, HttpContentWriteStream stream, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
   at Elasticsearch.Net.HttpConnection.RequestAsync[TResponse](RequestData requestData, CancellationToken cancellationToken)
# Request:

# Response:


Unsuccessful low level call on PUT: /index-apttus.micsrosoft.tenant.sanitycheck/document/9ff83b6f-fb1c-4d80-8e11-62cc22faa4a9?pipeline=attachments
# Audit trail of this API call:
 - [1] BadRequest: Node: http://localhost:9200/ Took: 00:00:01.7456023
# OriginalException: System.Net.Http.HttpRequestException: Error while copying content to a stream. ---> System.IO.IOException: Unable to read data from the transport connection: An established connection was aborted by the software in your host machine. ---> System.Net.Sockets.SocketException: An established connection was aborted by the software in your host machine
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnection.WriteAsync(ReadOnlyMemory`1 source)
   at System.IO.Stream.CopyToAsyncInternal(Stream destination, Int32 bufferSize, CancellationToken cancellationToken)
   at System.Net.Http.HttpContent.CopyToAsyncCore(ValueTask copyTask)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpContent.CopyToAsyncCore(ValueTask copyTask)
   at System.Net.Http.HttpConnection.SendRequestContentAsync(HttpRequestMessage request, HttpContentWriteStream stream, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
   at Elasticsearch.Net.HttpConnection.RequestAsync[TResponse](RequestData requestData, CancellationToken cancellationToken)
# Request:

# Response:

Could you please let me know a fix or any clue for same.

2

Please update on this.

3

It's quite hard to read your question because it is poorly-formatted, and poorly-formatted posts often don't get a useful response. Please format logs etc. using the </> button to make it easier to read, and check the preview of your post before sending it to make sure it looks ok.

As far as I can see you've only provided client-side logs, which basically say that the server shut the connection down. The server logs ought to contain more information about why this happened.

4

I am trying to push the 80mb document to ElasticSearch cluster and getting "Error while copying content to a stream".

FYI, The Used HEAP sized readched at 99%.

Additional Information

We have developed the .NetCore WebApi Application using NEST library for CRUD (Create, read, update, delete) Operation in ElasticSearch.
We have setup ElasticSearch with Ingest plug-in on kubernetes cluster with HeapSize 2gb (On cloud).

Goal: Add 100,000 documents (Per Document Size: 13MB to 15MB) in ElasticSearch in 10 - 15 Minutes

Could you please suggest the ideal ElasticSearch configuration or ElasticSearch configuration for high performance for the above requirement.

Thanks in Advanced.

5

Thank you for reformatting your post more clearly.

You need to look at the server logs.

6

Thanks for reply, Your work really appreciated.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.