# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=507 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 508 --- ## [Which Postgres JDBC Driver to use for Logstash 7.10.0 - Java 1.8.0\_131 - Ruby 2.3.7p456 (2018-03-28 revision 63024)](https://discuss.elastic.co/t/which-postgres-jdbc-driver-to-use-for-logstash-7-10-0-java-1-8-0-131-ruby-2-3-7p456-2018-03-28-revision-63024/191473)
**Author:** [@jbowker](https://discuss.elastic.co/u/jbowker)\ **Replies:** 0\ **Last updated:** [July 20, 2019, 8:03am UTC](https://discuss.elastic.co/t/which-postgres-jdbc-driver-to-use-for-logstash-7-10-0-java-1-8-0-131-ruby-2-3-7p456-2018-03-28-revision-63024/191473 "2019-07-20T08:03:28Z")
My conf file: input { jdbc { type =\> "ECSC-1-1" jdbc\_connection\_string =\> "jdbc:postgresql://db-postgresql-lon1-68952-do-user-1949559-0.db.ondigitalocean.com/emis" jdbc\_user =\> "doadmin" jdbc\_password =\> "z04t1h3hr5ggy1… --- ## [Determining reason for stuck grok filter](https://discuss.elastic.co/t/determining-reason-for-stuck-grok-filter/191257)
**Author:** [@victor73](https://discuss.elastic.co/u/victor73)\ **Replies:** 8\ **Last updated:** [July 20, 2019, 12:17am UTC](https://discuss.elastic.co/t/determining-reason-for-stuck-grok-filter/191257 "2019-07-20T00:17:22Z")
Hi all, I've tried using logstash versions 6.8.1, 7.0 and 7.2 and, regardless of version, logstash seems to stop it's processing and begin consuming all the available CPU. It seems to be stuck with the grok filter. I su… --- ## [Convert event timestamp](https://discuss.elastic.co/t/convert-event-timestamp/190858)
**Author:** [@eclipsed450](https://discuss.elastic.co/u/eclipsed450)\ **Replies:** 4\ **Last updated:** [July 19, 2019, 10:37pm UTC](https://discuss.elastic.co/t/convert-event-timestamp/190858 "2019-07-19T22:37:22Z")
Hi, I would like to convert the timestamp from some syslog-type messages into a time-formatted field. It's currently in this format: 19/07/11 01:15:26. I can't seem to find the right syntax to covert this into a format … --- ## [S3 input not passing metadata](https://discuss.elastic.co/t/s3-input-not-passing-metadata/191220)
**Author:** [@eclipsed450](https://discuss.elastic.co/u/eclipsed450)\ **Replies:** 16\ **Last updated:** [July 19, 2019, 10:36pm UTC](https://discuss.elastic.co/t/s3-input-not-passing-metadata/191220 "2019-07-19T22:36:43Z")
Hi, I'm having a weird issue. When logstash pulls messages down from S3 and passes them over to ES, not all of the messages get their S3 metadata key value. Some messages get %{\[@metadata\]\[s3\]\[key\]} as their file field,… --- ## [Logstash 7.2.0 not starting windows](https://discuss.elastic.co/t/logstash-7-2-0-not-starting-windows/191453)
**Author:** [@vinay11283](https://discuss.elastic.co/u/vinay11283)\ **Replies:** 0\ **Last updated:** [July 19, 2019, 8:38pm UTC](https://discuss.elastic.co/t/logstash-7-2-0-not-starting-windows/191453 "2019-07-19T20:38:09Z")
Hello, Why does it point to /usr/local/meven on windows ?? Anyway I can point it to my windows profile .m2 repo ? c:\\tools\\logstash-7.2.0\\logstash-7.2.0\\bin\>logstash.bat \[ERROR\] 2019-07-19 16:36:10.698 \[mai… --- ## [Creating an array of values from a combination of range and list values](https://discuss.elastic.co/t/creating-an-array-of-values-from-a-combination-of-range-and-list-values/191274)
**Author:** [@cromagnon35](https://discuss.elastic.co/u/cromagnon35)\ **Replies:** 3\ **Last updated:** [July 19, 2019, 4:07pm UTC](https://discuss.elastic.co/t/creating-an-array-of-values-from-a-combination-of-range-and-list-values/191274 "2019-07-19T16:07:39Z")
Hello, I've got an interesting problem I'm hoping someone might be able to assist with. I'm using logstash to ingest data from Oracle which in by itself is working fine. However, I have a field called ziparea which has… --- ## [Logstash running as a Service](https://discuss.elastic.co/t/logstash-running-as-a-service/191418)
**Author:** [@edster](https://discuss.elastic.co/u/edster)\ **Replies:** 1\ **Last updated:** [July 19, 2019, 3:16pm UTC](https://discuss.elastic.co/t/logstash-running-as-a-service/191418 "2019-07-19T15:16:39Z")
Logstash as a service. After the initial pull of data. Does logstash run through each record and updates based on ID? Is this something that takes a long time because it goes through everything that has been pulled? --- ## [X-Pack Logstash monitoring and management issue](https://discuss.elastic.co/t/x-pack-logstash-monitoring-and-management-issue/191417)
**Author:** [@Sergey\_RU](https://discuss.elastic.co/u/Sergey_RU)\ **Replies:** 1\ **Last updated:** [July 19, 2019, 3:04pm UTC](https://discuss.elastic.co/t/x-pack-logstash-monitoring-and-management-issue/191417 "2019-07-19T15:04:07Z")
Hi, I'm a newbie in ELK Stack. I just installed ELK on CentOS 7 VM as RPM (7.2.0). Revers proxy is Nginx. SELinux was disabled. Then I enabled trial period for Enterprise ELK. Then I enabled X-Pack in all ELK configurat… --- ## [Netflow codec writes to multiple indexes](https://discuss.elastic.co/t/netflow-codec-writes-to-multiple-indexes/191356)
**Author:** [@rysiornia](https://discuss.elastic.co/u/rysiornia)\ **Replies:** 2\ **Last updated:** [July 19, 2019, 2:38pm UTC](https://discuss.elastic.co/t/netflow-codec-writes-to-multiple-indexes/191356 "2019-07-19T14:38:15Z")
Hello, I have logstash in 6.8.1 version my problem is that when I'm using two pipelines one for metricbeat and other for netflow I can't see metricbeat data. In elasticsearch I can see two indexes metricbeat-\* and netfl… --- ## [Logstash, http filter, problem with sending arrays in body](https://discuss.elastic.co/t/logstash-http-filter-problem-with-sending-arrays-in-body/191337)
**Author:** [@agone](https://discuss.elastic.co/u/agone)\ **Replies:** 2\ **Last updated:** [July 19, 2019, 2:17pm UTC](https://discuss.elastic.co/t/logstash-http-filter-problem-with-sending-arrays-in-body/191337 "2019-07-19T14:17:04Z")
I am trying to use http filter plugin to make POST requests to third party API while processing evens in logstash. This API requires requests to be json-encoded in following format: { "type": "alert", "source":… --- ## [Compatible field type](https://discuss.elastic.co/t/compatible-field-type/191346)
**Author:** [@theo1991](https://discuss.elastic.co/u/theo1991)\ **Replies:** 2\ **Last updated:** [July 19, 2019, 1:40pm UTC](https://discuss.elastic.co/t/compatible-field-type/191346 "2019-07-19T13:40:50Z")
Hi everybody, I want to use visualizations in Kibana and it went right until I want to use specific fields. I want to count the number of events in my logs, which are at the field event\_count. In my grok filter, the e… --- ## [HTTP Filter With Array](https://discuss.elastic.co/t/http-filter-with-array/191306)
**Author:** [@wwalker](https://discuss.elastic.co/u/wwalker)\ **Replies:** 1\ **Last updated:** [July 19, 2019, 1:05pm UTC](https://discuss.elastic.co/t/http-filter-with-array/191306 "2019-07-19T13:05:49Z")
I have a field, assigned\_to\_current, that occasionally contains an array of values. I want to pass each of these values to the HTTP filter and then have it replace each value with the http result value. I guess like th… --- ## [Please help with logstash Filter](https://discuss.elastic.co/t/please-help-with-logstash-filter/191381)
**Author:** [@alfred\_vikas\_paul](https://discuss.elastic.co/u/alfred_vikas_paul)\ **Replies:** 1\ **Last updated:** [July 19, 2019, 12:48pm UTC](https://discuss.elastic.co/t/please-help-with-logstash-filter/191381 "2019-07-19T12:48:25Z")
Hi, i am sending some json message using axios.get() . in Logstash i am receiving GET /?{%22level%22:%22INFO%20From%20Web%20Client%22,%22time%22:%22Wed%20Jul%2017%2018:26:43%202019%22,%22kbID%22:%220%22,%22info%22:%22Cu… --- ## [How to send a single email after csv file saved and attached to email output plugin](https://discuss.elastic.co/t/how-to-send-a-single-email-after-csv-file-saved-and-attached-to-email-output-plugin/186136)
**Author:** [@Ennr](https://discuss.elastic.co/u/Ennr)\ **Replies:** 6\ **Last updated:** [July 10, 2019, 10:28am UTC](https://discuss.elastic.co/t/how-to-send-a-single-email-after-csv-file-saved-and-attached-to-email-output-plugin/186136 "2019-07-10T10:28:55Z")
My requirement is send a file ( extracting data from elastic search index is to save in CSV or excel format it contains 1000 records ) in a single mail through logstash output email plugin. is it possible or not? we… --- ## [Missing Data - SQL Server 2017 to Elasticsearch](https://discuss.elastic.co/t/missing-data-sql-server-2017-to-elasticsearch/191342)
**Author:** [@martb](https://discuss.elastic.co/u/martb)\ **Replies:** 0\ **Last updated:** [July 19, 2019, 7:58am UTC](https://discuss.elastic.co/t/missing-data-sql-server-2017-to-elasticsearch/191342 "2019-07-19T07:58:04Z")
Hi, Im ingesting data from a SQL Server 2017 database to Elasticsearch using logstash v6.2.4. Im finding that although I have a last run identifier Im missing data (source compared to output). This source data is stati… --- ## [Hosted Logstash (on elastic.co) with Docker+Logspout?](https://discuss.elastic.co/t/hosted-logstash-on-elastic-co-with-docker-logspout/191317)
**Author:** [@Sandeep\_Srinivasa](https://discuss.elastic.co/u/Sandeep_Srinivasa)\ **Replies:** 1\ **Last updated:** [July 19, 2019, 5:53am UTC](https://discuss.elastic.co/t/hosted-logstash-on-elastic-co-with-docker-logspout/191317 "2019-07-19T05:53:19Z")
hi guys, I'm just getting into starting to try out elasticsearch based logging - Logstash. I'm using the hosted service. I already use a logging service (like papertrail, loggly, logdna) all of which are piped through … --- ## [Combining the logs in logstash](https://discuss.elastic.co/t/combining-the-logs-in-logstash/190682)
**Author:** [@vinu89](https://discuss.elastic.co/u/vinu89)\ **Replies:** 9\ **Last updated:** [July 19, 2019, 4:34am UTC](https://discuss.elastic.co/t/combining-the-logs-in-logstash/190682 "2019-07-19T04:34:01Z")
Hi, I need to combine the two log lines using the message id i tried aggregate filter plugin but i cant achieve it.Please guide Jun 5 19:24:18 gpigw11 postfix/qmgr\[37531\]: 3077F92E45: from=\<\*\*\*\*\*\*\>, size=2801, nrcpt=1… --- ## [Dir.glob got stuck](https://discuss.elastic.co/t/dir-glob-got-stuck/191297)
**Author:** [@Ram\_Sharan\_Mittal](https://discuss.elastic.co/u/Ram_Sharan_Mittal)\ **Replies:** 0\ **Last updated:** [July 19, 2019, 2:10am UTC](https://discuss.elastic.co/t/dir-glob-got-stuck/191297 "2019-07-19T02:10:53Z")
following code is stuck and it is not moving further in a custom plugin temp\_dir = /tmp #{METADATA\_FILE\_EXTENSION} = .metadata metadata\_files = Dir.glob(temp\_dir, "\*\*" "\*#{METADATA\_FILE\_EXTENSION}") --- ## [Logstash crashes with OOM](https://discuss.elastic.co/t/logstash-crashes-with-oom/191277)
**Author:** [@buytamil](https://discuss.elastic.co/u/buytamil)\ **Replies:** 0\ **Last updated:** [July 18, 2019, 9:06pm UTC](https://discuss.elastic.co/t/logstash-crashes-with-oom/191277 "2019-07-18T21:06:04Z")
Logstash version - 6.2.0 logstash.yml has pipeline.workers: 8 pipeline.batch.size: 250 pipleline.batch.delay: 5 log.level: warn path.log : /usr/share/logstash It is running in pod in kubernetes with min 3g and max… --- ## [Multiple Filebeat configurations on different servers sending to 1 logstash](https://discuss.elastic.co/t/multiple-filebeat-configurations-on-different-servers-sending-to-1-logstash/191265)
**Author:** [@userR](https://discuss.elastic.co/u/userR)\ **Replies:** 3\ **Last updated:** [July 18, 2019, 6:56pm UTC](https://discuss.elastic.co/t/multiple-filebeat-configurations-on-different-servers-sending-to-1-logstash/191265 "2019-07-18T18:56:56Z")
It is my understanding that Logstash can handle multiple instances of filebeat. My question is whether there is a pro to configuring multiple Logstash instances to handle 3-4 filebeat instances (all on different servers)… --- ## [Splitting HTTP\_Poller JSON Results](https://discuss.elastic.co/t/splitting-http-poller-json-results/191250)
**Author:** [@wwalker](https://discuss.elastic.co/u/wwalker)\ **Replies:** 4\ **Last updated:** [July 18, 2019, 6:53pm UTC](https://discuss.elastic.co/t/splitting-http-poller-json-results/191250 "2019-07-18T18:53:39Z")
Making a call to a system for changes to tickets in a given time period, it returns the results as a single line of JSON. How can I split these out into separate events. Below is the returned results. { "result": \[{ "… --- ## [How to use Logstash with Kibana hosted in Amazon WS](https://discuss.elastic.co/t/how-to-use-logstash-with-kibana-hosted-in-amazon-ws/191237)
**Author:** [@Arioule](https://discuss.elastic.co/u/Arioule)\ **Replies:** 0\ **Last updated:** [July 18, 2019, 3:55pm UTC](https://discuss.elastic.co/t/how-to-use-logstash-with-kibana-hosted-in-amazon-ws/191237 "2019-07-18T15:55:12Z")
Also, I have not found documentation regarding how to use a local Logstash application with a Kibana application hosted in AWS. Is there any way to do it ? --- ## [How to use Logstash with Kibana hosted in AWS](https://discuss.elastic.co/t/how-to-use-logstash-with-kibana-hosted-in-aws/191235)
**Author:** [@Arioule](https://discuss.elastic.co/u/Arioule)\ **Replies:** 0\ **Last updated:** [July 18, 2019, 3:53pm UTC](https://discuss.elastic.co/t/how-to-use-logstash-with-kibana-hosted-in-aws/191235 "2019-07-18T15:53:02Z")
Hi Also, I have not found documentation regarding how to use a local Logstash application with a Kibana application hosted in AWS. Is there any way to do it ? Regards, --- ## [Grok S3 object "path"](https://discuss.elastic.co/t/grok-s3-object-path/190390)
**Author:** [@eclipsed450](https://discuss.elastic.co/u/eclipsed450)\ **Replies:** 8\ **Last updated:** [July 18, 2019, 2:29pm UTC](https://discuss.elastic.co/t/grok-s3-object-path/190390 "2019-07-18T14:29:17Z")
Hi all, I'm trying to figure out a way to grok out the key of an S3 object. Here is an example key: elasticmapreduce/j-fjtnfnfk56/containers/application\_1111111111111\_0169/container\_1111111111111\_0169\_01\_000029/stderr.… --- ## [Logstash input to monitor java processess](https://discuss.elastic.co/t/logstash-input-to-monitor-java-processess/191080)
**Author:** [@Abdul\_Qaadir](https://discuss.elastic.co/u/Abdul_Qaadir)\ **Replies:** 4\ **Last updated:** [July 18, 2019, 2:21pm UTC](https://discuss.elastic.co/t/logstash-input-to-monitor-java-processess/191080 "2019-07-18T14:21:22Z")
Hi Elastic Team, I have set of java applications running on my unix server. I want to monitor the heath , state (live, or dead) , memory consumption, cpu on a dashboard. How to pass the java processing running on my un… --- ## [CPU usage of Logstash too high when used with S3 Input plugin](https://discuss.elastic.co/t/cpu-usage-of-logstash-too-high-when-used-with-s3-input-plugin/191143)
**Author:** [@Pista](https://discuss.elastic.co/u/Pista)\ **Replies:** 3\ **Last updated:** [July 18, 2019, 1:40pm UTC](https://discuss.elastic.co/t/cpu-usage-of-logstash-too-high-when-used-with-s3-input-plugin/191143 "2019-07-18T13:40:29Z")
Working on infrastructure built over AWS, since there are some special cases that logs are stored only in S3 buckets, if we want to use ELK to analyse these logs, we need to use 'S3 Input plugin' \[1\]. Unfortunately, the… --- ## [Ruby exception occurred: undefined method \`unpack1' for nil:NilClass](https://discuss.elastic.co/t/ruby-exception-occurred-undefined-method-unpack1-for-nil-nilclass/191134)
**Author:** [@Nagesh\_Kumar](https://discuss.elastic.co/u/Nagesh_Kumar)\ **Replies:** 2\ **Last updated:** [July 18, 2019, 1:24pm UTC](https://discuss.elastic.co/t/ruby-exception-occurred-undefined-method-unpack1-for-nil-nilclass/191134 "2019-07-18T13:24:54Z")
Hi, I am trying with below logstash (7.0 version) conf file to decode base64 logs to send elasticsearch but its throwing below error - \[2019-07-17T19:01:36,791\]\[ERROR\]\[logstash.filters.ruby \] Ruby exception occurred: … --- ## [Adding compound word token filter to a template results in “Failed to install template - response code 500 contacting Elasticsearch”](https://discuss.elastic.co/t/adding-compound-word-token-filter-to-a-template-results-in-failed-to-install-template-response-code-500-contacting-elasticsearch/190914)
**Author:** [@Marta1](https://discuss.elastic.co/u/Marta1)\ **Replies:** 9\ **Last updated:** [July 18, 2019, 12:22pm UTC](https://discuss.elastic.co/t/adding-compound-word-token-filter-to-a-template-results-in-failed-to-install-template-response-code-500-contacting-elasticsearch/190914 "2019-07-18T12:22:08Z")
After adding hyphenation decompounder to my custom template, logstash fails to install it: "Got response code '500' contacting Elasticsearch at URL...". I have already had different token filters like stopwords and stem… --- ## [Grok parse failure on working grok debugger](https://discuss.elastic.co/t/grok-parse-failure-on-working-grok-debugger/191039)
**Author:** [@Shawcs](https://discuss.elastic.co/u/Shawcs)\ **Replies:** 6\ **Last updated:** [July 18, 2019, 10:52am UTC](https://discuss.elastic.co/t/grok-parse-failure-on-working-grok-debugger/191039 "2019-07-18T10:52:21Z")
Hello, I have a \_grokparsefailure but I cant find why as my grok test all succed. here is the log: Loaded archive '56b530f7-ddc2-40c3-9a26-3551c850e897' to hosts running group.\\nDeploying to API Gateway 'GW-1'...\\nDep… --- ## [Http poller with --insecure mode](https://discuss.elastic.co/t/http-poller-with-insecure-mode/191121)
**Author:** [@TsuWeiQuan](https://discuss.elastic.co/u/TsuWeiQuan)\ **Replies:** 1\ **Last updated:** [July 18, 2019, 10:01am UTC](https://discuss.elastic.co/t/http-poller-with-insecure-mode/191121 "2019-07-18T10:01:12Z")
Hello, I am trying to get data from a salt server via RestAPI. I can curl the data with --insecure mode. However, it is using a self-signed cert and I am unable to get the webserver cert. I would like to use http\_polle… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=506) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=508)