# Logs are missing in ES

**URL:** <https://discuss.elastic.co/t/logs-are-missing-in-es/116793>\
**Category:** Elasticsearch\
**Created:** [January 24, 2018, 3:50am UTC](https://discuss.elastic.co/t/logs-are-missing-in-es/116793 "2018-01-24T03:50:36Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![shwesinhan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shwesinhan/32/78663_2.png) [@shwesinhan](https://discuss.elastic.co/u/shwesinhan)\
**Post date:** [January 24, 2018, 3:50am UTC](https://discuss.elastic.co/t/logs-are-missing-in-es/116793/1 "2018-01-24T03:50:36Z")

</div>

Hi there!

Let me say my scenario first, logstash index from AWS CWL and inject to AWS ES domain.

We have application logs stored in Amazon CloudWatch Logs.  
Eg: Under Test-Log-Group, App1-Log-Stream, App2-Log-Stream, App3-Log-Stream, App4-Log-Stream, App5-Log-Stream  
These log streams will get the logs continously.

These are used plugins \>\> logstash-input-cloudwatch-logs input plugin and logstash-output-amazon\_es output plugin.

The problem is that some logs are missing in ES Domain when I do search with text value in kibana ui.

When I check in logstash logs i saw

2018-01-24T08:43:04,967][ERROR][logstash.outputs.amazones] Attempted to send a bulk request to Elasticsearch configured at '["https://search-bbbbbb-.es.amazonaws.com:443"]', but an error occurred and it failed! Are you sure you can reach elasticsearch from this machine using the configuration provided? {:client\_config=\>{:hosts=\>["https://search-bbbbbb-.es.amazonaws.com:443"], :region=\>"ap-southeast-1", :transport\_options=\>{:request=\>{:open\_timeout=\>0, :timeout=\>60}, :proxy=\>nil, :headers=\>{"Content-Type"=\>"application/json"}}, :transport\_class=\>Elasticsearch::Transport::Transport::HTTP::AWS}, :error\_message=\>"search-bbbbbb-.es.amazonaws.com:443 failed to respond", :error\_class=\>"Faraday::ClientError", :backtrace=\>nil}  
[2018-01-24T08:43:05,217][WARN][logstash.outputs.amazones] Failed to flush outgoing items {:outgoing\_count=\>1, :exception=\>"Faraday::ClientError", :backtrace=\>nil}

But even I don't see above error time, still missing logs. ☹  
does anybody have experienced this kind of issue?

Thank you.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 21, 2018, 3:50am UTC](https://discuss.elastic.co/t/logs-are-missing-in-es/116793/2 "2018-02-21T03:50:38Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.