# Logstash run pipeline but not send data to output

**URL:** <https://discuss.elastic.co/t/logstash-run-pipeline-but-not-send-data-to-output/270960>\
**Category:** Elasticsearch\
**Created:** [April 22, 2021, 12:58pm UTC](https://discuss.elastic.co/t/logstash-run-pipeline-but-not-send-data-to-output/270960 "2021-04-22T12:58:48Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Omar1](https://avatars.discourse-cdn.com/v4/letter/o/73ab20/32.png) [@Omar1](https://discuss.elastic.co/u/Omar1)\
**Post date:** [April 22, 2021, 12:58pm UTC](https://discuss.elastic.co/t/logstash-run-pipeline-but-not-send-data-to-output/270960/1 "2021-04-22T12:58:48Z")

</div>

I'm trying to use logstash Jdbc plugin to synchronize data from a Postgres database to some ouput (elastic, rabbitMq).

The problem is that logstash is running well every hour (I configured the cronJob like that).  
Logstash is successfully reading the database but for some reason doesn't send the messages right away.  
However, after a few hours, messages are successfully sent.

I first thought it was a perfs issue because of a high volume of messages but even with just one message the behavior is the same.

I tested locally on docker with `docker-compose`, I don't have the issue but in a Kubernetes pod it doesn't work.

pipeline.conf

```
input {
      jdbc {
        jdbc_driver_class => "${JDBC_DRIVER_CLASS}"
        jdbc_connection_string => "${JDBC_CONNECTION_STRING}"
        jdbc_user => "${JDBC_USER}"
        jdbc_password => "${JDBC_PASSWORD}"
        jdbc_paging_enabled => false
        codec => "json"
        tracking_column => "sync_unix_ts"
        use_column_value => true
        tracking_column_type => "numeric"
        schedule => "${LOGSTASH_CRONEXPRESSION}"
        statement_filepath => "/usr/share/logstash/query/elasticsearch-query.sql"
      }
    }
    filter {
      json {
        source => "fieldjson"
      }
      mutate {  
        remove_field => ["fieldjson","sync_unix_ts"]
      }
    }
    
    output {
      elasticsearch {
          index => "index_name"
          document_id => "%{id}"
          hosts => ["${ELASTICSEARCH_HOST}"]
      }
    }

```

logstash.yaml

```
http.host: "0.0.0.0"
xpack.monitoring.enabled: false

```

I don't know if it is a buffer issue or not...

Thanks for your answers

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [April 22, 2021, 1:12pm UTC](https://discuss.elastic.co/t/logstash-run-pipeline-but-not-send-data-to-output/270960/2 "2021-04-22T13:12:53Z")

</div>

Where are you seeing the messages? In Kibana? After how many hours do you see them? It could be related to time zone configuration.

Can you run your pipeline using the `stdout` output, and see if the messages appears?

---

<div class="post-metadata">

**Author:** ![Omar1](https://avatars.discourse-cdn.com/v4/letter/o/73ab20/32.png) [@Omar1](https://discuss.elastic.co/u/Omar1)\
**Post date:** [April 22, 2021, 2:57pm UTC](https://discuss.elastic.co/t/logstash-run-pipeline-but-not-send-data-to-output/270960/3 "2021-04-22T14:57:30Z")

</div>

thanks for your answer.

I see the messages directly on 'elastic' with request on index (ex. 'kibana').  
this is my cron expression : 0 \* \* \* \*.

For the tests situations, i modify my cron expression to '\*/15 \* \* \* \*' (At every 15th minute) and using stdout but messages not appears.

In logstash log, he run every 15th minute and execute a query in input and get 10 messages but not send.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 20, 2021, 2:57pm UTC](https://discuss.elastic.co/t/logstash-run-pipeline-but-not-send-data-to-output/270960/4 "2021-05-20T14:57:43Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.