# Unable to get the updated value of certain columns in logstash 8.6.1 version

**URL:** <https://discuss.elastic.co/t/unable-to-get-the-updated-value-of-certain-columns-in-logstash-8-6-1-version/366539>\
**Category:** Logstash\
**Created:** [September 13, 2024, 12:49pm UTC](https://discuss.elastic.co/t/unable-to-get-the-updated-value-of-certain-columns-in-logstash-8-6-1-version/366539 "2024-09-13T12:49:55Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![mj22manish](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mj22manish/32/137578_2.png) [@mj22manish](https://discuss.elastic.co/u/mj22manish)\
**Post date:** [September 13, 2024, 12:49pm UTC](https://discuss.elastic.co/t/unable-to-get-the-updated-value-of-certain-columns-in-logstash-8-6-1-version/366539/1 "2024-09-13T12:49:55Z")

</div>

Why is the below query not getting the updated value of some columns even after there is an updated value for couple of columns ? Here is my logstash pipeline.

```auto
input {
  jdbc {
    jdbc_driver_library => "mysql-connector-j-8.0.33.jar"
    jdbc_driver_class => "com.mysql.jdbc.Driver"
    jdbc_connection_string => "abc?useTimezone=true&useLegacyDatetimeCode=false&serverTimezone=UTC"
    jdbc_user => "cricket"
    jdbc_paging_enabled => true
    tracking_column => "process_update_dttm"
    use_column_value => true
    tracking_column_type => "timestamp"
    schedule => "*/5 * * * *"
    last_run_metadata_path => "/etc/logstash/java-libs/.logstash_jdbc_last_run_7"
    statement_filepath => "/etc/logstash/java-libs/test.sql"
      }
    }
  
output {
   elasticsearch {
    hosts => ["xyx"]
    ssl => true
    action => "create"
    cacert => "CA.pem"
    data_stream => true
    data_stream_type => "logs"
    data_stream_dataset => "something"
    data_stream_namespace => "ds"
    document_id => "%{process_id}_%{process_update_dttm}"   
    scripted_upsert => true # This ensures that if the document exists, it is updated with the script
    script => '%{[@metadata][script]}' # Executes the script defined in the filter section
    script_params => {
      "Received" => "%{Received}",
      "Sent" => "%{Sent}"
    }
     } 
stdout{ codec => rubydebug }
}

```

---

<div class="post-metadata">

**Author:** ![Artem\_Shelkovnikov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/artem_shelkovnikov/32/134322_2.png) [@Artem\_Shelkovnikov](https://discuss.elastic.co/u/Artem_Shelkovnikov)\
**Post date:** [September 16, 2024, 8:17am UTC](https://discuss.elastic.co/t/unable-to-get-the-updated-value-of-certain-columns-in-logstash-8-6-1-version/366539/2 "2024-09-16T08:17:57Z")

</div>

Category updated on September 16, 2024, 8:17am UTC

From #Elastic Search to #Logstash

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [September 16, 2024, 1:41pm UTC](https://discuss.elastic.co/t/unable-to-get-the-updated-value-of-certain-columns-in-logstash-8-6-1-version/366539/3 "2024-09-16T13:41:16Z")

</div>

> [@mj22manish](#):
>
> Why is the below query not getting the updated value of some columns even after there is an updated value for couple of columns ?

You mean update in your Elasticsearch?

You are using a data stream, data streams are append only and you cannot update it through logstash.

If you want to update your data you need to use normal indices, not data streams.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 14, 2024, 1:41pm UTC](https://discuss.elastic.co/t/unable-to-get-the-updated-value-of-certain-columns-in-logstash-8-6-1-version/366539/4 "2024-10-14T13:41:16Z")

</div>

Closed on October 14, 2024, 1:41pm UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.