# Data does not get loaded into ElasticSearch

**URL:** <https://discuss.elastic.co/t/data-does-not-get-loaded-into-elasticsearch/153206>\
**Category:** Elasticsearch\
**Created:** [October 19, 2018, 5:17pm UTC](https://discuss.elastic.co/t/data-does-not-get-loaded-into-elasticsearch/153206 "2018-10-19T17:17:20Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![DevGhdh](https://avatars.discourse-cdn.com/v4/letter/d/b5ac83/32.png) [@DevGhdh](https://discuss.elastic.co/u/DevGhdh)\
**Post date:** [October 19, 2018, 5:17pm UTC](https://discuss.elastic.co/t/data-does-not-get-loaded-into-elasticsearch/153206/1 "2018-10-19T17:17:20Z")

</div>

I am trying to load into my AWS ElasticSearch module from my EC2 instance but am not able to load any data into my ES - am very new to this and any help is highly appreciated.

**I created the index below in Kibana:**  
PUT logstash-trades  
{  
"settings":{  
"number\_of\_shards":1,  
"number\_of\_replicas":1  
},  
"mappings":{  
"tracetrddoc1":{  
"properties":{  
"cusip":{  
"type":"text"  
},  
"reportingpartyside":{  
"type":"text"  
},  
"tradedate":{  
"type":"date"  
},  
"marketsegment":{  
"type":"integer"  
},  
"sectorid":{  
"type":"integer"  
}  
}  
}  
}  
}  
**Below is my logstash config file:**  
input {  
file {  
path =\> "/home/ec2-user/es\_data/temptrds.csv"  
start\_position =\> "beginning"  
}  
}  
filter {  
csv {  
separator =\> "|"  
columns =\> ["issuepriceid", "tradedate", "tradeweek", "trademonth", "tradequarter", "tradeyear", "sequencenumber", "reportingpartyside", "effectivedate", "price", "yield", "spread", "estimatedquantity", "yearstomaturity", "marketspread", "issueid", "cusip", "liquidityscore", "maturity", "issuedate", "paymenttype", "interestratetype", "paymentfrequency", "redemption", "countrycode", "marketsegment", "snprating", "moodyrating", "isocurrencycode", "sectorid", "issueamount", "amountoutstanding", "callable", "putable", "sinkable", "makewhole", "tracecode", "industry", "parent\_party\_id", "isinvestmentgrade", "productid"]  
}  
mutate {convert =\> ["issuepriceid", "integer"]}  
mutate {convert =\> ["sequencenumber", "integer"]}  
mutate {convert =\> ["price", "float"]}  
mutate {convert =\> ["yield", "float"]}  
mutate {convert =\> ["spread", "float"]}  
mutate {convert =\> ["estimatedquantity", "float"]}   
mutate {convert =\> ["yearstomaturity", "integer"]}  
mutate {convert =\> ["marketspread", "float"]}   
mutate {convert =\> ["issueid", "integer"]}  
mutate {convert =\> ["liquidityscore", "integer"]}  
mutate {convert =\> ["marketsegment", "integer"]}  
mutate {convert =\> ["snprating", "integer"]}  
mutate {convert =\> ["moodyrating", "integer"]}  
mutate {convert =\> ["isocurrencycode", "integer"]}  
mutate {convert =\> ["sectorid", "integer"]}  
mutate {convert =\> ["issueamount", "float"]}  
mutate {convert =\> ["amountoutstanding", "float"]}  
mutate {convert =\> ["productid", "integer"]}   
}

output {  
elasticsearch {  
hosts =\> ["[https://vpc-xxx.us-east-1.es.amazonaws.com:443](https://vpc-xxx.us-east-1.es.amazonaws.com:443)"]  
index =\> "logstash-trades"   
}   
}

After running the command: sudo bin/logstash -f /home/ec2-user/es\_data/logstash2.config --path.settings /etc/logstash

When i check the number of documents in my index by GET /\_cat/indices?v - i see 0 docs.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [October 19, 2018, 5:32pm UTC](https://discuss.elastic.co/t/data-does-not-get-loaded-into-elasticsearch/153206/2 "2018-10-19T17:32:33Z")

</div>

AWS ES does as far as I know not work with the elasticsearch output plugin, so I believe you need to install and use the amazon\_es output plugin.

---

<div class="post-metadata">

**Author:** ![DevGhdh](https://avatars.discourse-cdn.com/v4/letter/d/b5ac83/32.png) [@DevGhdh](https://discuss.elastic.co/u/DevGhdh)\
**Post date:** [October 19, 2018, 5:56pm UTC](https://discuss.elastic.co/t/data-does-not-get-loaded-into-elasticsearch/153206/3 "2018-10-19T17:56:47Z")

</div>

Thanks so much for your response.

I installed plugin: sudo bin/logstash-plugin install logstash-output-amazon\_es

And changed my output as well to:  
output {  
amazon\_es {  
hosts =\> ["[https://xxx.us-east-1.es.amazonaws.com:443](https://xxx.us-east-1.es.amazonaws.com:443)"]  
index =\> "logstash-trades"   
}   
}  
But am getting error below  
[2018-10-19T17:50:04,202][ERROR][logstash.pipeline] Error registering plugin {:pipeline\_id=\>"main", :plugin=\>"#LogStash::OutputDelegator:0x26117f44:0x1378162c run\>"}

[2018-10-19T17:50:04,207][ERROR][logstash.pipeline] Pipeline aborted due to error {:pipeline\_id=\>"main", :exception=\>#\<NoMethodError: undefined methole/jruby/2.3.0/gems/aws-sdk-core-2.11.136/lib/aws-sdk-core/signers/v4.rb:45:in `initialize'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-oudapter.rb:111:in`perform\_request'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-amazon\_es-6.4.0-java/lib/logstash/outputs/amazon\_es/r/bundle/jruby/2.3.0/gems/logstash-output-amazon\_es-6.4.0-java/lib/logstash/outputs/amazon\_es/http\_client/pool.rb:245:in `block in healthcheck!'", "org/jrubyems/logstash-output-amazon_es-6.4.0-java/lib/logstash/outputs/amazon_es/http_client/pool.rb:241:in`healthcheck!'", "/usr/share/logstash/vendor/bundle/jruby/ttp\_client/pool.rb:341:in `update_urls'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-amazon_es-6.4.0-java/lib/logstash/outputs/amazoy/2.3.0/gems/logstash-output-amazon_es-6.4.0-java/lib/logstash/outputs/amazon_es/http_client.rb:316:in`build\_pool'", "/usr/share/logstash/vendor/bundle/jrub/http\_client.rb:66:in `initialize'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-amazon_es-6.4.0-java/lib/logstash/outputs/amazon_es//bundle/jruby/2.3.0/gems/logstash-output-amazon_es-6.4.0-java/lib/logstash/outputs/amazon_es/http_client_builder.rb:98:in`build'", "/usr/share/logstash/vendputs/amazon\_es.rb:253:in `build_client'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-amazon_es-6.4.0-java/lib/logstash/outputs/amazoyExt.java:102:in`register'", "org/logstash/config/ir/compiler/AbstractOutputDelegatorExt.java:46:in `register'", "/usr/share/logstash/logstash-core/lib/logsib/logstash/pipeline.rb:253:in`block in register\_plugins'", "org/jruby/RubyArray.java:1734:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/pipelsh/pipeline.rb:594:in`maybe\_setup\_out\_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:263:in `start_workers'", "/usr/share/logstash/lsh-core/lib/logstash/pipeline.rb:160:in`block in start'"], :thread=\>"#\<Thread:0x1378162c run\>"}

[2018-10-19T17:50:04,228][ERROR][logstash.agent] Failed to execute action {:id=\>:main, :action\_type=\>LogStash::ConvergeResult::FailedAction, :messse", :backtrace=\>nil}

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [October 19, 2018, 5:58pm UTC](https://discuss.elastic.co/t/data-does-not-get-loaded-into-elasticsearch/153206/4 "2018-10-19T17:58:06Z")

</div>

I suspect you may need to also add credentials, but have never used that plugin myself.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 16, 2018, 6:10pm UTC](https://discuss.elastic.co/t/data-does-not-get-loaded-into-elasticsearch/153206/5 "2018-11-16T18:10:01Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.