# java.lang.OutOfMemoryError: Java heap space when transferring data from jdbc to elasticsearch via logstash

**URL:** <https://discuss.elastic.co/t/java-lang-outofmemoryerror-java-heap-space-when-transferring-data-from-jdbc-to-elasticsearch-via-logstash/120067>\
**Category:** Logstash\
**Created:** [February 15, 2018, 8:32pm UTC](https://discuss.elastic.co/t/java-lang-outofmemoryerror-java-heap-space-when-transferring-data-from-jdbc-to-elasticsearch-via-logstash/120067 "2018-02-15T20:32:58Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![famo](https://avatars.discourse-cdn.com/v4/letter/f/a698b9/32.png) [@famo](https://discuss.elastic.co/u/famo)\
**Post date:** [February 15, 2018, 8:32pm UTC](https://discuss.elastic.co/t/java-lang-outofmemoryerror-java-heap-space-when-transferring-data-from-jdbc-to-elasticsearch-via-logstash/120067/1 "2018-02-15T20:32:58Z")

</div>

I have a huge postgres database with 20 million rows and i want to transfer it to elasticsearch via logstash . I followed the advice mentioned here and I test it for a simple database with 300 rows and all things worked fine but when i tested it for my main database i allways cross with error:

> nargess@nargess-Surface-Book:/usr/share/logstash/bin$ sudo ./logstash -w 1 -f students.conf --path.data /usr/share/logstash/data/students/ --path.settings /etc/logstash  
> Sending Logstash's logs to /var/log/logstash which is now configured via log4j2.properties  
> java.lang.OutOfMemoryError: Java heap space  
> Dumping heap to java\_pid3453.hprof ...  
> Heap dump file created [13385912484 bytes in 53.304 secs]  
> Exception in thread "Ruby-0-Thread-11: /usr/share/logstash/vendor/bundle/jruby/1.9/gems/puma-2.16.0-java/lib/puma/thread\_pool.rb:216" java.lang.ArrayIndexOutOfBoundsException: -1  
> at org.jruby.runtime.ThreadContext.popRubyClass(ThreadContext.java:729)  
> at org.jruby.runtime.ThreadContext.postYield(ThreadContext.java:1292)  
> at org.jruby.runtime.ContextAwareBlockBody.post(ContextAwareBlockBody.java:29)  
> at org.jruby.runtime.Interpreted19Block.yield(Interpreted19Block.java:198)  
> at org.jruby.runtime.Interpreted19Block.call(Interpreted19Block.java:125)  
> at org.jruby.runtime.Block.call(Block.java:101)  
> at org.jruby.RubyProc.call(RubyProc.java:300)  
> at org.jruby.RubyProc.call(RubyProc.java:230)  
> at org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:103)  
> at java.lang.Thread.run(Thread.java:748)  
> The signal INT is in use by the JVM and will not work correctly on this platform  
> Error: Your application used more memory than the safety cap of 12G.  
> Specify -J-Xmx####m to increase it (#### = cap size in MB).  
> Specify -w for full OutOfMemoryError stack trace

Although I go to file /etc/logstash/jvm.options and set -Xms256m  
-Xmx12000m, but I have had these errors yet. I have 13g memory free. how can i send my data to Elasticsearch with this memory ? this is the student-index.json that i use in elasticsearch

> {  
> "aliases": {},  
> "warmers": {},  
> "mappings": {  
> "tab\_students\_dfe": {  
> "properties": {  
> "stcode": {  
> "type": "text"  
> },  
> "voroodi": {  
> "type": "integer"  
> },  
> "name": {  
> "type": "text"  
> },  
> "family": {  
> "type": "text"  
> },  
> "namp": {  
> "type": "text"  
> },  
> "lastupdate": {  
> "type": "date"  
> },  
> "picture": {  
> "type": "text"  
> },  
> "uniquename": {  
> "type": "text"  
> }  
> }  
> }  
> },  
> "settings": {  
> "index": {  
> "number\_of\_shards": "5",  
> "number\_of\_replicas": "1"  
> }  
> }  
> }

then i try to insert this index in Elasticsearch by

> curl -XPUT --header "Content-Type: application/json"  
> [http://localhost:9200/students](http://localhost:9200/students) -d @postgres-index.json

and next, this is my configuration fil in /usr/shar/logstash/bin/students.conf file

> input {  
> jdbc {  
> jdbc\_connection\_string =\> "jdbc:postgresql://localhost:5432/postgres"  
> jdbc\_user =\> "postgres"  
> jdbc\_password =\> "postgres"

# The path to downloaded jdbc driver

jdbc\_driver\_library =\> "./postgresql-42.2.1.jar"  
jdbc\_driver\_class =\> "org.postgresql.Driver"

# The path to the file containing the query

statement =\> "select \* from students"  
}  
}  
filter {  
aggregate {  
task\_id =\> "%{stcode}"  
code =\> "  
map['stcode'] = event.get('stcode')  
map['voroodi'] = event.get('voroodi')  
map['name'] = event.get('name')  
map['family'] = event.get('family')  
map['namp'] = event.get('namp')  
map['uniquename'] = event.get('uniquename')  
event.cancel()  
"  
push\_previous\_map\_as\_event =\> true  
timeout =\> 5  
}  
}  
output {  
elasticsearch {  
document\_id =\> "%{stcode}"  
document\_type =\> "postgres"  
index =\> "students"  
codec =\> "json"  
hosts =\> ["127.0.0.1:9200"]  
}  
}

Thank you for your help

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 15, 2018, 8:33pm UTC](https://discuss.elastic.co/t/java-lang-outofmemoryerror-java-heap-space-when-transferring-data-from-jdbc-to-elasticsearch-via-logstash/120067/2 "2018-03-15T20:33:11Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.