# Is it possible to recover logs that failed to transfer to ElasticSearch?

**URL:** https://discuss.elastic.co/t/is-it-possible-to-recover-logs-that-failed-to-transfer-to-elasticsearch/329401
**Category:** Logstash
**Created:** [April 5, 2023, 8:35am UTC](https://discuss.elastic.co/t/is-it-possible-to-recover-logs-that-failed-to-transfer-to-elasticsearch/329401 "2023-04-05T08:35:20Z")
**Posts on this page:** 2
**Page:** 1

<div class="post-metadata">

### Author: ![r.fujii](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/r.fujii/32/119454_2.png) [@r.fujii](https://discuss.elastic.co/u/r.fujii)
#### Post date: [April 5, 2023, 8:35am UTC](https://discuss.elastic.co/t/is-it-possible-to-recover-logs-that-failed-to-transfer-to-elasticsearch/329401/1 "2023-04-05T08:35:20Z")

</div>

The following configuration is used to obtain the server's audit logs and forward them to Elasticsearch.

AuditBeat -\> Logstash -\> Elasticsearch

However, on Elasticsearch, the number of shards exceeded max\_shards\_per\_node, and the following WARNING was output on Logstash and index creation failed.

> Could not index event to Elasticsearch. {:status=\>400, :action=\>["index", {:\_id=\>nil, :\_index=\>"INDEX\_NAME", :\_type=\>"\_doc", :\_routing=\>nil}, #LogStash::Event:0x14f497d4], :response=\>{"index"=\>{"\_index"=\>"INDEX\_NAME", "\_type"=\>"\_doc", "\_id"=\>nil, "status"=\>400, "error"=\>{"type"=\>"illegal\_argument\_exception", "reason"=\>"Validation Failed: 1: this action would add [10] total shards, but this cluster currently has [6010]/[3000] maximum shards open;"}}}}

Currently, max\_shards\_per\_node is set to a large value, so index creation is possible. However, the indices for logs of the periods that could not be transferred have not been created.

Is there any way to retrieve these lost logs?

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [May 3, 2023, 8:36am UTC](https://discuss.elastic.co/t/is-it-possible-to-recover-logs-that-failed-to-transfer-to-elasticsearch/329401/2 "2023-05-03T08:36:14Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
