# Logstash JDBC plugin not working without LIMIT set in statement

**URL:** <https://discuss.elastic.co/t/logstash-jdbc-plugin-not-working-without-limit-set-in-statement/109989>\
**Category:** Logstash\
**Created:** [December 1, 2017, 11:50pm UTC](https://discuss.elastic.co/t/logstash-jdbc-plugin-not-working-without-limit-set-in-statement/109989 "2017-12-01T23:50:30Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![arisbanach](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@arisbanach](https://discuss.elastic.co/u/arisbanach)\
**Post date:** [December 1, 2017, 11:50pm UTC](https://discuss.elastic.co/t/logstash-jdbc-plugin-not-working-without-limit-set-in-statement/109989/1 "2017-12-01T23:50:30Z")

</div>

I am not sure if there is another issue causing this, but I have done some testing and found an odd issue with my Logstash instance. I am trying to import a very large Postgresql database from a remote server and if I set a LIMIT in the statement of the Logstash pipeline config file, it works. I can set it to LIMIT 10 or LIMIT 1000000 and they will both work fine and start importing at the same time. If I don't set a LIMIT, it just hangs forever. If I set LIMIT ALL it also hangs forever. If I set LIMIT 100000000 it also hangs forever. I'm not sure what is causing this.

I set the logging level to debug and looked at the logs. When the LIMIT is not set, it gets to

```
[2017-12-01T23:42:03,221][INFO][logstash.pipeline] Pipeline started {"pipeline.id"=>"main"}
[2017-12-01T23:42:03,231][INFO][logstash.agent] Pipelines running {:count=>1, :pipelines=>["main"]}
[2017-12-01T23:42:03,651][INFO][logstash.inputs.jdbc] (0.049546s) SELECT version()

```

and then every other log after that says "pushing flush onto pipeline and just repeats endlessly.

If I set LIMIT 1000000 I get:

```
[2017-12-01T23:42:03,221][INFO][logstash.pipeline] Pipeline started {"pipeline.id"=>"main"}
[2017-12-01T23:42:03,231][INFO][logstash.agent] Pipelines running {:count=>1, :pipelines=>["main"]}
[2017-12-01T23:42:03,651][INFO][logstash.inputs.jdbc] (0.049546s) SELECT version()
[2017-12-01T23:42:04,501][INFO][logstash.inputs.jdbc] (0.842951s) SELECT count(*) AS "count" FROM (SELECT * FROM tracking LIMIT 1000000) AS "t1" LIMIT 1

```

and it proceeds to work correctly.

Here is my config file:

```
input {
  jdbc {
    jdbc_driver_library => "/home/ubuntu/postgresql-42.1.4.jar"
    jdbc_driver_class => "org.postgresql.Driver"
    jdbc_connection_string => "url"
    jdbc_user => "user"
    jdbc_password => "password"
    statement => "SELECT * FROM tracking LIMIT 1000000"
    jdbc_paging_enabled => "true"
    jdbc_page_size => "1000"
  }
}
output {
  elasticsearch {
    hosts => ["localhost"]
    index => "index-name"
  }
}
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 29, 2017, 11:50pm UTC](https://discuss.elastic.co/t/logstash-jdbc-plugin-not-working-without-limit-set-in-statement/109989/2 "2017-12-29T23:50:38Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.