# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=577 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 578 --- ## [S3 input plugin - External Bucket](https://discuss.elastic.co/t/s3-input-plugin-external-bucket/173709)
**Author:** [@Roberto\_1](https://discuss.elastic.co/u/Roberto_1)\ **Replies:** 0\ **Last updated:** [March 25, 2019, 9:44am UTC](https://discuss.elastic.co/t/s3-input-plugin-external-bucket/173709 "2019-03-25T09:44:18Z")
Hello, I'would like to use the Logstash S3 input plugin in order to retrieve the logs in a Bucket S3. The problem is that I've to define a "Local Path" in my S3 plugin configuration but, most likely because of me, I ca… --- ## [Filter csv Logstash](https://discuss.elastic.co/t/filter-csv-logstash/174124)
**Author:** [@Lebedev](https://discuss.elastic.co/u/Lebedev)\ **Replies:** 10\ **Last updated:** [March 27, 2019, 2:10pm UTC](https://discuss.elastic.co/t/filter-csv-logstash/174124 "2019-03-27T14:10:50Z")
Hi, everyone. I am have same problem with add csv file to elastic. Could you kindly help me understand what is wrong. My logstash config: input { file { path =\> "C:\\Saratov\_report\\ \*.csv" start\_position =\> "beginni… --- ## [How to handle dictionary from python script in logstash](https://discuss.elastic.co/t/how-to-handle-dictionary-from-python-script-in-logstash/173960)
**Author:** [@ioannisnikolakis](https://discuss.elastic.co/u/ioannisnikolakis)\ **Replies:** 3\ **Last updated:** [March 27, 2019, 1:23pm UTC](https://discuss.elastic.co/t/how-to-handle-dictionary-from-python-script-in-logstash/173960 "2019-03-27T13:23:06Z")
Hello elastic and logstash experts, I am trying out using a python script as input (exec) to a logstash config file. The python script spits out a dictionary like the one depicted below once every 1 second: {'\_score': … --- ## [Is it possible to make Logstash file-output write thru tee or not use concurrent writes?](https://discuss.elastic.co/t/is-it-possible-to-make-logstash-file-output-write-thru-tee-or-not-use-concurrent-writes/174129)
**Author:** [@bjosve](https://discuss.elastic.co/u/bjosve)\ **Replies:** 0\ **Last updated:** [March 27, 2019, 1:07pm UTC](https://discuss.elastic.co/t/is-it-possible-to-make-logstash-file-output-write-thru-tee-or-not-use-concurrent-writes/174129 "2019-03-27T13:07:37Z")
Hi, A short question. Is it possible to make Logstash file-output write thru tee or not use concurrent writes? Best Regards, Bjorn --- ## [Logstash with ILM like old behavior](https://discuss.elastic.co/t/logstash-with-ilm-like-old-behavior/174088)
**Author:** [@gruselglatz](https://discuss.elastic.co/u/gruselglatz)\ **Replies:** 2\ **Last updated:** [March 27, 2019, 12:44pm UTC](https://discuss.elastic.co/t/logstash-with-ilm-like-old-behavior/174088 "2019-03-27T12:44:48Z")
Hi, i want to imitate the old logstash behavior with creating daily indices and naming it per day, but only with ILM and deleting it after xx days. I cant get it to work and i am not really fit with the new ilm setting… --- ## [Deploy Logstash pipelines from stage to production](https://discuss.elastic.co/t/deploy-logstash-pipelines-from-stage-to-production/174118)
**Author:** [@Abhilash\_B](https://discuss.elastic.co/u/Abhilash_B)\ **Replies:** 0\ **Last updated:** [March 27, 2019, 12:10pm UTC](https://discuss.elastic.co/t/deploy-logstash-pipelines-from-stage-to-production/174118 "2019-03-27T12:10:17Z")
Hey team, Is there any way to deploy centralized pipelines (which are managed through kibana) from a staging elastic cloud cluster to a production elastic cloud cluster? All I can think of now is to just copy-paste the … --- ## [After mutate split and add\_field,my message is repetition](https://discuss.elastic.co/t/after-mutate-split-and-add-field-my-message-is-repetition/174115)
**Author:** [@kerwin](https://discuss.elastic.co/u/kerwin)\ **Replies:** 0\ **Last updated:** [March 27, 2019, 11:58am UTC](https://discuss.elastic.co/t/after-mutate-split-and-add-field-my-message-is-repetition/174115 "2019-03-27T11:58:53Z")
logstash,kibana and elastic version: 6.5.1 select same index。 some document， kibana show some field value is repetition， some document didn\`t. error kibana screenshot： source message: 2019-03-27 17:06:45.344 {"w… --- ## [Elasticsearch filter plugin does not work as expected](https://discuss.elastic.co/t/elasticsearch-filter-plugin-does-not-work-as-expected/174108)
**Author:** [@takeno](https://discuss.elastic.co/u/takeno)\ **Replies:** 0\ **Last updated:** [March 27, 2019, 11:34am UTC](https://discuss.elastic.co/t/elasticsearch-filter-plugin-does-not-work-as-expected/174108 "2019-03-27T11:34:44Z")
I'm trying to calculate job duration time using elasticsearch filter. The problem is that I can not get job.duration when I input the whole json. logstash conf: input { stdin { codec =\> "json" } } filter { … --- ## [How do I create my own Index name?](https://discuss.elastic.co/t/how-do-i-create-my-own-index-name/172890)
**Author:** [@Blason](https://discuss.elastic.co/u/Blason)\ **Replies:** 7\ **Last updated:** [March 27, 2019, 11:17am UTC](https://discuss.elastic.co/t/how-do-i-create-my-own-index-name/172890 "2019-03-27T11:17:46Z")
Hi Guys, Can someone please help me on creating new Index name isntead of logstash-\* or filebeat-\* Something like hashes-\*. I think I need to create my new mapping and then post it on elasticsearch? The thing happenin… --- ## [Combine Logs from two input files](https://discuss.elastic.co/t/combine-logs-from-two-input-files/174097)
**Author:** [@Senz79](https://discuss.elastic.co/u/Senz79)\ **Replies:** 0\ **Last updated:** [March 27, 2019, 10:53am UTC](https://discuss.elastic.co/t/combine-logs-from-two-input-files/174097 "2019-03-27T10:53:55Z")
Hello I have two logs files Log file A - 1,2,3,4 5,6,7,8 Log file B - xyz Can we combine both the log files as 1,2,3,4,XYZ 5,6,7,8,XYZ Please guide Thanks --- ## [Logstash reads only one line](https://discuss.elastic.co/t/logstash-reads-only-one-line/174095)
**Author:** [@angham](https://discuss.elastic.co/u/angham)\ **Replies:** 0\ **Last updated:** [March 27, 2019, 10:43am UTC](https://discuss.elastic.co/t/logstash-reads-only-one-line/174095 "2019-03-27T10:43:49Z")
we need to filter the select query but logstash reads just this line : 15:49:05,479 DEBUG \[org.hibernate.stat.internal.ConcurrentStatisticsImpl\] (default task-43) HHH000117: HQL: select T from translator\_MenuTranslator … --- ## [Multiple pipeline pattern for REST API](https://discuss.elastic.co/t/multiple-pipeline-pattern-for-rest-api/174094)
**Author:** [@Sackey](https://discuss.elastic.co/u/Sackey)\ **Replies:** 0\ **Last updated:** [March 27, 2019, 10:39am UTC](https://discuss.elastic.co/t/multiple-pipeline-pattern-for-rest-api/174094 "2019-03-27T10:39:49Z")
Good day. I try to achieve this pattern using Logstash and pipelines. # Auth - pipeline.id: api-getToken path.config: "api-getToken.conf" # Count of pages - pipeline.id: api-getPages path.config: "api-getPages.c… --- ## [Logstash Filter to convert comma separated syslog message body to JSON format](https://discuss.elastic.co/t/logstash-filter-to-convert-comma-separated-syslog-message-body-to-json-format/173959)
**Author:** [@akj01](https://discuss.elastic.co/u/akj01)\ **Replies:** 3\ **Last updated:** [March 27, 2019, 10:35am UTC](https://discuss.elastic.co/t/logstash-filter-to-convert-comma-separated-syslog-message-body-to-json-format/173959 "2019-03-27T10:35:45Z")
We have one of the application server forwarding syslog to logstash. The "message" body of the log contains few unwanted fields inserted by the application server and remaining required fields as well. Providing below th… --- ## [Logstash Dead Letter Queue](https://discuss.elastic.co/t/logstash-dead-letter-queue/174093)
**Author:** [@wiliamauc85](https://discuss.elastic.co/u/wiliamauc85)\ **Replies:** 0\ **Last updated:** [March 27, 2019, 10:35am UTC](https://discuss.elastic.co/t/logstash-dead-letter-queue/174093 "2019-03-27T10:35:39Z")
Hi, I've been trying to setup a dead letter queue \> \[WARN \] 2019-03-27 10:19:14.917 \[\[main\]\>worker0\] elasticsearch - Could not index event to Elasticsearch. {:status=\>400, :action=\>\["index", {:\_id=\>nil, :\_index=\>"hpc-w… --- ## [How do I print object contents as part of logging?](https://discuss.elastic.co/t/how-do-i-print-object-contents-as-part-of-logging/174087)
**Author:** [@blueren](https://discuss.elastic.co/u/blueren)\ **Replies:** 0\ **Last updated:** [March 27, 2019, 10:24am UTC](https://discuss.elastic.co/t/how-do-i-print-object-contents-as-part-of-logging/174087 "2019-03-27T10:24:36Z")
I would like to look at the contents of the input that gets sent to the filters. I have the following code inserted in one of the filters. However, I run into ruby errors. How do I print the content of \[netflow\] which ha… --- ## [Faraday::ConnectionFailed](https://discuss.elastic.co/t/faraday-connectionfailed/174082)
**Author:** [@dhaval.kathiriya](https://discuss.elastic.co/u/dhaval.kathiriya)\ **Replies:** 0\ **Last updated:** [March 27, 2019, 10:03am UTC](https://discuss.elastic.co/t/faraday-connectionfailed/174082 "2019-03-27T10:03:15Z")
Hello, I have using LS filter for fire query with ES and get matched record from ES. but i have getting this error \[2019-03-27T15:10:29,993\]\[WARN \]\[logstash.filters.elasticsearch\] Failed to query elasticsearch for pre… --- ## [Get the correct timestamp from the log file and set to @timestamp](https://discuss.elastic.co/t/get-the-correct-timestamp-from-the-log-file-and-set-to-timestamp/173821)
**Author:** [@Loudone](https://discuss.elastic.co/u/Loudone)\ **Replies:** 2\ **Last updated:** [March 27, 2019, 9:54am UTC](https://discuss.elastic.co/t/get-the-correct-timestamp-from-the-log-file-and-set-to-timestamp/173821 "2019-03-27T09:54:21Z")
Dear All, I'm currently trying to define log timestamp to @timestamp. But for some reasons, it seems not working properly my input log look like: \[25/Mar/2019:08:10:56 +0100\] GET "/foo/bar\_" 200 1 0 On logstash sid… --- ## [Error"Could not finde jruby in C:\\User\\samyo\\Desktop\\ELK-6.5.0\\logstash-6.5.0\\vendor\\jruby"](https://discuss.elastic.co/t/error-could-not-finde-jruby-in-c-user-samyo-desktop-elk-6-5-0-logstash-6-5-0-vendor-jruby/174070)
**Author:** [@samyo](https://discuss.elastic.co/u/samyo)\ **Replies:** 0\ **Last updated:** [March 27, 2019, 9:17am UTC](https://discuss.elastic.co/t/error-could-not-finde-jruby-in-c-user-samyo-desktop-elk-6-5-0-logstash-6-5-0-vendor-jruby/174070 "2019-03-27T09:17:34Z")
i am trying to get Logstash to run and read from my "simple.conf" file. i am using Logstash Version 6.5.0 , on Windows 10. i used the nssm to creat and conect Logstash into the "simple.conf" file. After i went into t… --- ## [GROK filter for my log file JBoss](https://discuss.elastic.co/t/grok-filter-for-my-log-file-jboss/173949)
**Author:** [@markov](https://discuss.elastic.co/u/markov)\ **Replies:** 3\ **Last updated:** [March 27, 2019, 9:17am UTC](https://discuss.elastic.co/t/grok-filter-for-my-log-file-jboss/173949 "2019-03-27T09:17:20Z")
I'm new to the ELK stack and I'm trying to setup a GROK filter but I'm not having any luck. Any help is greatly appreciated. Thanks, this is a line from my log file : 2019-03-11 11:46:50,202 \[1552304809856\]|192.168.0… --- ## [Install azure\_event\_hubs input plugin for logstash 5.6](https://discuss.elastic.co/t/install-azure-event-hubs-input-plugin-for-logstash-5-6/174037)
**Author:** [@thangld](https://discuss.elastic.co/u/thangld)\ **Replies:** 1\ **Last updated:** [March 27, 2019, 7:45am UTC](https://discuss.elastic.co/t/install-azure-event-hubs-input-plugin-for-logstash-5-6/174037 "2019-03-27T07:45:29Z")
Hi all, I'm trying to install azure\_event\_hubs input plugin azure for logstash 5.6 but got this error: /usr/share/logstash/bin/logstash-plugin install --no-verify --version 1.1.0 logstash-input-azure\_event\_hubs Installi… --- ## [Parallel watches in logstash](https://discuss.elastic.co/t/parallel-watches-in-logstash/174054)
**Author:** [@GiteshKhanna](https://discuss.elastic.co/u/GiteshKhanna)\ **Replies:** 0\ **Last updated:** [March 27, 2019, 7:01am UTC](https://discuss.elastic.co/t/parallel-watches-in-logstash/174054 "2019-03-27T07:01:18Z")
my path is defined as path=\> "/mnt/logs/x.log-\*" So it should be watching every file present /mnt/logs/x.log-1 /mnt/logs/x.log-2 and so on... But does logstash support watching multiple files in parallel? Or is it … --- ## [Help with filter/mutate, replace netflow snmp field](https://discuss.elastic.co/t/help-with-filter-mutate-replace-netflow-snmp-field/174053)
**Author:** [@otoce](https://discuss.elastic.co/u/otoce)\ **Replies:** 0\ **Last updated:** [March 27, 2019, 6:56am UTC](https://discuss.elastic.co/t/help-with-filter-mutate-replace-netflow-snmp-field/174053 "2019-03-27T06:56:20Z")
Hi, i'm doing something wrong so please help. Input from 2 sources: \`input { snmp { walk =\> \["1.3.6.1.2.1.2.2.1.2"\] hosts =\> \[{host =\> "udp:xxx" community =\> "public" version =\> "3" retries =\> 5 timeout =\> … --- ## [Xpath filtering](https://discuss.elastic.co/t/xpath-filtering/173574)
**Author:** [@sverma](https://discuss.elastic.co/u/sverma)\ **Replies:** 6\ **Last updated:** [March 27, 2019, 1:00am UTC](https://discuss.elastic.co/t/xpath-filtering/173574 "2019-03-27T01:00:45Z")
I am parsing an xml file via logstash and in my xml structure I have few repeated tags as below : \<.....\> \<.......\> \900G\ \ **Author:** [@Ed\_K](https://discuss.elastic.co/u/Ed_K)\ **Replies:** 4\ **Last updated:** [March 26, 2019, 6:19pm UTC](https://discuss.elastic.co/t/help-with-parsing-kafka-key/173542 "2019-03-26T18:19:04Z") Hello, I currently use \[@metadata\]\[kafka\]\[key\] to set the document ID of all my messages from the kafka input. I want to implement a way to also route the message to the appropriate index. There are 2 options I am thinki… --- ## [Logstash JDBC plugin + Steaming filter + delta import when a child table has changed](https://discuss.elastic.co/t/logstash-jdbc-plugin-steaming-filter-delta-import-when-a-child-table-has-changed/173909)
**Author:** [@Fazolo](https://discuss.elastic.co/u/Fazolo)\ **Replies:** 1\ **Last updated:** [March 26, 2019, 11:13pm UTC](https://discuss.elastic.co/t/logstash-jdbc-plugin-steaming-filter-delta-import-when-a-child-table-has-changed/173909 "2019-03-26T23:13:18Z")
Hello, I'm using the JDBC input plugin with the streaming filter to enrich the base data with extra data found in an other tables: input { jdbc { jdbc\_connection\_string =\> "jdbc:mysql://mysql… --- ## [Logstash connecting to both QA and PROD?](https://discuss.elastic.co/t/logstash-connecting-to-both-qa-and-prod/173987)
**Author:** [@titan1978](https://discuss.elastic.co/u/titan1978)\ **Replies:** 9\ **Last updated:** [March 26, 2019, 10:01pm UTC](https://discuss.elastic.co/t/logstash-connecting-to-both-qa-and-prod/173987 "2019-03-26T22:01:11Z")
When Logstash bootstraps in my container, the logstash output is indicating it is registering BOTH my qa and prod endpoints. I have a series of if-else checks to connect to one or the other. But it doesn't look like logs… --- ## [Can't get filebeat clients to ship to logstash](https://discuss.elastic.co/t/cant-get-filebeat-clients-to-ship-to-logstash/173562)
**Author:** [@boxclever38](https://discuss.elastic.co/u/boxclever38)\ **Replies:** 1\ **Last updated:** [March 26, 2019, 9:35pm UTC](https://discuss.elastic.co/t/cant-get-filebeat-clients-to-ship-to-logstash/173562 "2019-03-26T21:35:09Z")
I have 3 VMs setup: Server A: Elastic Stack. Everything works great. filebeat -\> logstash -\> elasticsearch Kibana loads everything up and we're looking good. Let's say it has an IP address of 10.10.10.10 Server B + C… --- ## [TLS with anonymous rsyslog clients](https://discuss.elastic.co/t/tls-with-anonymous-rsyslog-clients/173963)
**Author:** [@George\_Kraft](https://discuss.elastic.co/u/George_Kraft)\ **Replies:** 0\ **Last updated:** [March 26, 2019, 3:46pm UTC](https://discuss.elastic.co/t/tls-with-anonymous-rsyslog-clients/173963 "2019-03-26T15:46:51Z")
I want anonymous rsyslog clients using the Logstash server’s x509/fingerprint to authenticate the Logstash server for a secure TLS connection; however, I don’t want the Logstash server to authenticate all of the client… --- ## [IllegalStateException when starting logstash with pipeline.workers](https://discuss.elastic.co/t/illegalstateexception-when-starting-logstash-with-pipeline-workers/174007)
**Author:** [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Replies:** 0\ **Last updated:** [March 26, 2019, 8:29pm UTC](https://discuss.elastic.co/t/illegalstateexception-when-starting-logstash-with-pipeline-workers/174007 "2019-03-26T20:29:39Z")
I have a weired problem. as soon as I turn on pipeline.workers:1 on my piplelines.yml file logstash fails starting up with following error. \[2019-03-26T20:24:12,469\]\[ERROR\]\[org.logstash.Logstash \] java.lang.Illega… --- ## [Logstash 6.7 release date?](https://discuss.elastic.co/t/logstash-6-7-release-date/173547)
**Author:** [@Rajachak](https://discuss.elastic.co/u/Rajachak)\ **Replies:** 2\ **Last updated:** [March 26, 2019, 6:47pm UTC](https://discuss.elastic.co/t/logstash-6-7-release-date/173547 "2019-03-26T18:47:19Z")
Hello, I have a couple of quick questions: -Will Logstash 6.7 be Java 11 compatible? -When will Logstash 6.7 be released? (a rough is fine; I just need to plan for an upcoming deployment) thanks, RC [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=576) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=578)