# Urgent help for elasticsearch and logstash demo

**URL:** <https://discuss.elastic.co/t/urgent-help-for-elasticsearch-and-logstash-demo/154306>\
**Category:** Logstash\
**Created:** [October 27, 2018, 10:20pm UTC](https://discuss.elastic.co/t/urgent-help-for-elasticsearch-and-logstash-demo/154306 "2018-10-27T22:20:50Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![murali12180](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/murali12180/32/47810_2.png) [@murali12180](https://discuss.elastic.co/u/murali12180)\
**Post date:** [October 27, 2018, 10:20pm UTC](https://discuss.elastic.co/t/urgent-help-for-elasticsearch-and-logstash-demo/154306/1 "2018-10-27T22:20:50Z")

</div>

Hi,

My CURL XPUT command works fine using its mapping however, i'm having trouble loading the same file from s3 to AWS Elasticsearch. Using logstash conf file (below), dumps the s3 json data file into "message" field in elasticsearch. I need the data to be distributed to mapping fields when I use conf file to transfer s3 data to elasticsearch.

input {  
s3 {  
bucket =\> “my bucket”  
access\_key\_id =\> ""  
secret\_access\_key =\> ""  
region =\> "us-east-1"  
}  
}  
output {  
amazon\_es {  
hosts =\> ["[https://vpc-xxxxxx.us-east-1.es.amazonaws.com](https://vpc-xxxxxx.us-east-1.es.amazonaws.com)"]  
region =\> "us-east-1"  
aws\_access\_key\_id =\> ''  
aws\_secret\_access\_key =\> ''  
index =\> "dldemo"  
document\_type =\> "doc"  
}  
}

please advise.

---

<div class="post-metadata">

**Author:** ![murali12180](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/murali12180/32/47810_2.png) [@murali12180](https://discuss.elastic.co/u/murali12180)\
**Post date:** [October 27, 2018, 10:56pm UTC](https://discuss.elastic.co/t/urgent-help-for-elasticsearch-and-logstash-demo/154306/2 "2018-10-27T22:56:34Z")

</div>

Never mind .. i found the solution - i have added codec to the s3 as ... codec =\> json after region=? "us-east-1" in input section..

---

<div class="post-metadata">

**Author:** ![bloke](https://avatars.discourse-cdn.com/v4/letter/b/b5e925/32.png) [@bloke](https://discuss.elastic.co/u/bloke)\
**Post date:** [October 27, 2018, 10:57pm UTC](https://discuss.elastic.co/t/urgent-help-for-elasticsearch-and-logstash-demo/154306/3 "2018-10-27T22:57:33Z")

</div>

Are you missing codec's?

[https://www.elastic.co/guide/en/logstash/current/plugins-inputs-s3.html#plugins-inputs-s3-codec](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-s3.html#plugins-inputs-s3-codec)

And maybe?

[https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-codec](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-codec)

---

<div class="post-metadata">

**Author:** ![murali12180](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/murali12180/32/47810_2.png) [@murali12180](https://discuss.elastic.co/u/murali12180)\
**Post date:** [October 27, 2018, 11:18pm UTC](https://discuss.elastic.co/t/urgent-help-for-elasticsearch-and-logstash-demo/154306/4 "2018-10-27T23:18:43Z")

</div>

Thanks for the help. It works.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 24, 2018, 11:18pm UTC](https://discuss.elastic.co/t/urgent-help-for-elasticsearch-and-logstash-demo/154306/5 "2018-11-24T23:18:47Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.