# Trying to use Logstash to index from AWS cloudwatch logs and Inject to ElasticSearch

**URL:** <https://discuss.elastic.co/t/trying-to-use-logstash-to-index-from-aws-cloudwatch-logs-and-inject-to-elasticsearch/116416>\
**Category:** Logstash\
**Created:** [January 22, 2018, 1:49am UTC](https://discuss.elastic.co/t/trying-to-use-logstash-to-index-from-aws-cloudwatch-logs-and-inject-to-elasticsearch/116416 "2018-01-22T01:49:27Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![shwesinhan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shwesinhan/32/78663_2.png) [@shwesinhan](https://discuss.elastic.co/u/shwesinhan)\
**Post date:** [January 22, 2018, 1:49am UTC](https://discuss.elastic.co/t/trying-to-use-logstash-to-index-from-aws-cloudwatch-logs-and-inject-to-elasticsearch/116416/1 "2018-01-22T01:49:28Z")

</div>

Hello elastic team!

Good day to you.

May I ask about logstash index from CWL and inject to ES

I have application logs stored in Amazon CloudWatch Logs.  
Eg: Under Test-Log-Group, App1-Log-Stream, App2-Log-Stream, App3-Log-Stream, App4-Log-Stream, App5-Log-Stream  
These log streams will get the logs continously.

I'm trying to use Logstash to index from AWS CloudWatch Logs and format some logs in logstash pipeline and only ship the formatted logs to AWS ElasticSearch Domain.

These are used plugins \>\> `logstash-input-cloudwatch-logs` input plugin, do `grokking` the logs and `logstash-output-amazon_es` output plugin.

`The problem is that logs are missing in ES Domain when I do monitor in kibana ui.`  
Eg: index 500-logs from cloudwatch log, grokking 50 logs which is necessary to store in ES domain.  
And ship that 50 logs to ES.  
At that time, sometime I see my grokked logs, sometimes I don’t see my some grokked log.

Please advice to me which part makes cause this problem `input plugin`? `output plugin`? (I’ve checked my grokked pattern in grok debugger. They’re ok.)

`Does cloudwatch_logs/.sincedb* support mulitple log streams?`

And may I know recommended logstash input plugin of AWS Cloudwatch Logs and logstash output plugin of AWS ElasticSearch.  
I can't seem to find another plugins to accomplish this so far.

Thank you.

Can someone please help on this?????

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 19, 2018, 1:49am UTC](https://discuss.elastic.co/t/trying-to-use-logstash-to-index-from-aws-cloudwatch-logs-and-inject-to-elasticsearch/116416/2 "2018-02-19T01:49:35Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.