# Logstash JDBC Input infinite loop

**URL:** <https://discuss.elastic.co/t/logstash-jdbc-input-infinite-loop/108739>\
**Category:** Logstash\
**Created:** [November 22, 2017, 3:23pm UTC](https://discuss.elastic.co/t/logstash-jdbc-input-infinite-loop/108739 "2017-11-22T15:23:25Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![mc1392](https://avatars.discourse-cdn.com/v4/letter/m/57b2e6/32.png) [@mc1392](https://discuss.elastic.co/u/mc1392)\
**Post date:** [November 22, 2017, 3:23pm UTC](https://discuss.elastic.co/t/logstash-jdbc-input-infinite-loop/108739/1 "2017-11-22T15:23:25Z")

</div>

I'm using Logstash 5.6.3 and I think I am stuck in an infinite loop.

I'm using the upsert functionality, with my Id field as the document\_id.

Seems the same record is getting updated multiple times.  
The process never stops.

Here us my config:

```
input {
  jdbc {
    jdbc_driver_library => "/usr/share/java/mysql-connector-java-5.1.38.jar"
    jdbc_driver_class => "com.mysql.jdbc.Driver"
    jdbc_connection_string => "jdbc:mysql://database_url"
    jdbc_user => "user"
    jdbc_password => "password"
    use_column_value=> true
    tracking_column=>"Id"

    statement => "select Id,.....;"
  }
}

output {

stdout {codec=>json}

        elasticsearch{
                hosts=>["https://elastic_cloud_instance:9243"]
                index=>"my-data"
                user=>"user"
                password=>"password"

                doc_as_upsert=>true
                document_id=>"%{id}"
                action=>"update"

        }
}
```

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [November 22, 2017, 3:32pm UTC](https://discuss.elastic.co/t/logstash-jdbc-input-infinite-loop/108739/2 "2017-11-22T15:32:30Z")

</div>

Have a look at [this thread](https://discuss.elastic.co/t/jdbc-tracking-a-string-column-for-unique-ingestion/108662/6).

---

<div class="post-metadata">

**Author:** ![mc1392](https://avatars.discourse-cdn.com/v4/letter/m/57b2e6/32.png) [@mc1392](https://discuss.elastic.co/u/mc1392)\
**Post date:** [November 22, 2017, 3:35pm UTC](https://discuss.elastic.co/t/logstash-jdbc-input-infinite-loop/108739/3 "2017-11-22T15:35:00Z")

</div>

Are you saying that I have to use a timestamp instead?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [November 22, 2017, 3:35pm UTC](https://discuss.elastic.co/t/logstash-jdbc-input-infinite-loop/108739/4 "2017-11-22T15:35:59Z")

</div>

No, but make sure you sort it correctly and lowercase the tracking\_column name.

---

<div class="post-metadata">

**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\
**Post date:** [November 22, 2017, 3:39pm UTC](https://discuss.elastic.co/t/logstash-jdbc-input-infinite-loop/108739/5 "2017-11-22T15:39:34Z")

</div>

[https://www.elastic.co/guide/en/logstash/current/plugins-inputs-jdbc.html#plugins-inputs-jdbc-lowercase\_column\_names](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-jdbc.html#plugins-inputs-jdbc-lowercase_column_names)

Column names are lower cased by default. This is because all logstash examples and docs are always shown with `lower_case_and_underscored_field_names`. Its a convention that is useful in large collaborations when different people over time add or change the logstash config expecting lower case.

---

<div class="post-metadata">

**Author:** ![mc1392](https://avatars.discourse-cdn.com/v4/letter/m/57b2e6/32.png) [@mc1392](https://discuss.elastic.co/u/mc1392)\
**Post date:** [November 22, 2017, 3:45pm UTC](https://discuss.elastic.co/t/logstash-jdbc-input-infinite-loop/108739/6 "2017-11-22T15:45:37Z")

</div>

A permissions issue caused the process to restart.  
Couldn't write to a directory.

Once I ran with sudo, no errors.

In any case, here is the config file:

```
input {
  jdbc {
    jdbc_driver_library => "/usr/share/java/mysql-connector-java-5.1.38.jar"
    jdbc_driver_class => "com.mysql.jdbc.Driver"
    jdbc_connection_string => "jdbc:mysql://database_url"
    jdbc_user => "user"
    jdbc_password => "password"
    tracking_column=>"Id"
    statement => "select Id,....from.... order by Id;"
  }
}

output {

stdout {}

        elasticsearch{
                hosts=>["https://elastic_cloud_instance:9243"]
                index=>"my-data"
                user=>"user"
                password=>"password"
                doc_as_upsert=>true
                document_id=>"%{id}"
                action=>"update"
        }
}
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 20, 2017, 3:45pm UTC](https://discuss.elastic.co/t/logstash-jdbc-input-infinite-loop/108739/7 "2017-12-20T15:45:49Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.