# Logstash not updating documents JDBC Postgres

**URL:** <https://discuss.elastic.co/t/logstash-not-updating-documents-jdbc-postgres/87322>\
**Category:** Logstash\
**Created:** [May 27, 2017, 5:08am UTC](https://discuss.elastic.co/t/logstash-not-updating-documents-jdbc-postgres/87322 "2017-05-27T05:08:10Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![ImTheDeveloper](https://avatars.discourse-cdn.com/v4/letter/i/7ba0ec/32.png) [@ImTheDeveloper](https://discuss.elastic.co/u/ImTheDeveloper)\
**Post date:** [May 27, 2017, 5:08am UTC](https://discuss.elastic.co/t/logstash-not-updating-documents-jdbc-postgres/87322/1 "2017-05-27T05:08:10Z")

</div>

Hi All,

Every hour my data is updated in Postgres. The data in my table is updated each hour with a full drop of the table and a new bulk insert of the data. Right now, I am running logstash JDBC on a schedule so that 5 minutes after each hour I insert the data into elasticsearch.

My output config is as follows:

```
input {
    jdbc {
        # Postgres jdbc connection string to our database, mydb
        jdbc_connection_string => "jdbc:postgresql://localhost:5432/xxxxx"
        # The user we wish to execute our statement as
        jdbc_user => "xxx"
        jdbc_password => "xxxx"
       # The path to our downloaded jdbc driver
        jdbc_driver_library => "/usr/share/java/postgresql-jdbc4.jar"
        # The name of the driver class for Postgresql
        jdbc_driver_class => "org.postgresql.Driver"
        # Schedule for input
        schedule => "5 * * * *"
        # our query
        statement => "select planet.id, planet.x || ':' || planet.y || ':' || planet.z coords, planet.x, planet.y, planet.z ,planetname,rulername,race,planet.size,planet.score,planet.value$
    }
}

output {
    elasticsearch {
        hosts => ["localhost:9200"]
        index => "universe"
        document_type => "planet"
        document_id => "%{id}"
        template => "/etc/logstash/universe_template.json"
        template_name => "universe"
        template_overwrite => true
        manage_template => true
    }
}

```

I have noticed however that even though my logstash flow is running I am not seeing the documents updated in elasticsearch. Infact the timestamp is still being shown as yesterday, the first time I ran the script. I assumed by setting my document\_id to the ID field of my table that it would create the new document each time and the version number would update in elasticsearch.

Here is an example of the data in postgres:

 ![](https://us1.discourse-cdn.com/elastic/original/3X/0/7/077059dbb6f15723cb0ca3c58a5bf881a7820e7a.png)

Here is an example of the current output I see in elasticsearch:

 ![](https://us1.discourse-cdn.com/elastic/original/3X/3/e/3e271f309134895166fa4a1ed94b7aef723e97de.png)

I assume I'm doing something wrong with the document\_id but I cant see why.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [May 29, 2017, 6:56pm UTC](https://discuss.elastic.co/t/logstash-not-updating-documents-jdbc-postgres/87322/2 "2017-05-29T18:56:12Z")

</div>

Things look okay. What if you use a simple `stdout { codec => rubydebug }` output to take ES out of the equation. Are you getting a full database dump to your log each time the jdbc input is scheduled to run?

---

<div class="post-metadata">

**Author:** ![ImTheDeveloper](https://avatars.discourse-cdn.com/v4/letter/i/7ba0ec/32.png) [@ImTheDeveloper](https://discuss.elastic.co/u/ImTheDeveloper)\
**Post date:** [May 30, 2017, 5:00am UTC](https://discuss.elastic.co/t/logstash-not-updating-documents-jdbc-postgres/87322/3 "2017-05-30T05:00:37Z")

</div>

In fact it does seem to be working. It is doing a complete replace, I just didn't see the @version number change at all.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [May 30, 2017, 5:11am UTC](https://discuss.elastic.co/t/logstash-not-updating-documents-jdbc-postgres/87322/4 "2017-05-30T05:11:34Z")

</div>

The `@version` field isn't the document's version, it's the Logstash schema version (which isn't very useful).

---

<div class="post-metadata">

**Author:** ![ImTheDeveloper](https://avatars.discourse-cdn.com/v4/letter/i/7ba0ec/32.png) [@ImTheDeveloper](https://discuss.elastic.co/u/ImTheDeveloper)\
**Post date:** [May 30, 2017, 5:33am UTC](https://discuss.elastic.co/t/logstash-not-updating-documents-jdbc-postgres/87322/5 "2017-05-30T05:33:05Z")

</div>

That makes sense now thanks

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 27, 2017, 5:33am UTC](https://discuss.elastic.co/t/logstash-not-updating-documents-jdbc-postgres/87322/6 "2017-06-27T05:33:34Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.