# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=435 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 436 --- ## [Elastic filter plugin https error (ssl?)](https://discuss.elastic.co/t/elastic-filter-plugin-https-error-ssl/210711)
**Author:** [@\_Louw](https://discuss.elastic.co/u/_Louw)\ **Replies:** 1\ **Last updated:** [December 6, 2019, 1:51pm UTC](https://discuss.elastic.co/t/elastic-filter-plugin-https-error-ssl/210711 "2019-12-06T13:51:12Z")
Hi all, I am having trouble getting logstash to only do something, when a change in an index is detected. I have an index inputs, where the docs needs to be split out in separate files - but only if the doc is not alrea… --- ## [Measure time elapsed between apache entering request and tomcat exiting request](https://discuss.elastic.co/t/measure-time-elapsed-between-apache-entering-request-and-tomcat-exiting-request/210748)
**Author:** [@Otmane\_Faouzi](https://discuss.elastic.co/u/Otmane_Faouzi)\ **Replies:** 4\ **Last updated:** [December 6, 2019, 1:34pm UTC](https://discuss.elastic.co/t/measure-time-elapsed-between-apache-entering-request-and-tomcat-exiting-request/210748 "2019-12-06T13:34:17Z")
Hi, We have developed some sort of proxy for a client's need. Incoming requests are processed and send it to the final destination. We need to measure the transit time between us, which is concretely the time between t… --- ## [Http filter](https://discuss.elastic.co/t/http-filter/210592)
**Author:** [@raged](https://discuss.elastic.co/u/raged)\ **Replies:** 1\ **Last updated:** [December 6, 2019, 1:33pm UTC](https://discuss.elastic.co/t/http-filter/210592 "2019-12-06T13:33:34Z")
I have a desire to utilize a rest API call that would reach out to a web service that I would use to determine where the client's location is and tag this data to my event. I am currently already enriching data using JT… --- ## [Logstash couldnot read full data in oracle via jdbc code](https://discuss.elastic.co/t/logstash-couldnot-read-full-data-in-oracle-via-jdbc-code/210539)
**Author:** [@saravana\_hariharan](https://discuss.elastic.co/u/saravana_hariharan)\ **Replies:** 2\ **Last updated:** [December 6, 2019, 12:47pm UTC](https://discuss.elastic.co/t/logstash-couldnot-read-full-data-in-oracle-via-jdbc-code/210539 "2019-12-06T12:47:58Z")
logstash couldnot read full data in oracle via jdbc code logstash read only the first 3 rows of record please help me --- ## [Logstash Output File - Add data in filename](https://discuss.elastic.co/t/logstash-output-file-add-data-in-filename/210545)
**Author:** [@giorgiogale](https://discuss.elastic.co/u/giorgiogale)\ **Replies:** 2\ **Last updated:** [December 6, 2019, 9:02am UTC](https://discuss.elastic.co/t/logstash-output-file-add-data-in-filename/210545 "2019-12-06T09:02:08Z")
Hi, i would like to add the current date into the name of the file created by logstash process. The date isn't an information that i received in input, but i have to get it. I had succeeded, adding the following code … --- ## [Syslog filters for Versa SDWAN](https://discuss.elastic.co/t/syslog-filters-for-versa-sdwan/210830)
**Author:** [@DC123](https://discuss.elastic.co/u/DC123)\ **Replies:** 0\ **Last updated:** [December 6, 2019, 6:21am UTC](https://discuss.elastic.co/t/syslog-filters-for-versa-sdwan/210830 "2019-12-06T06:21:33Z")
Hi I saw a earlier query on Versa Networks logs POC with Logstash . Has anyone done this ? Thanks! --- ## [Logstash has fetched the data but not shipped into Elasticsearch no error in logstash cmd](https://discuss.elastic.co/t/logstash-has-fetched-the-data-but-not-shipped-into-elasticsearch-no-error-in-logstash-cmd/210632)
**Author:** [@swati\_dupare](https://discuss.elastic.co/u/swati_dupare)\ **Replies:** 14\ **Last updated:** [December 6, 2019, 5:02am UTC](https://discuss.elastic.co/t/logstash-has-fetched-the-data-but-not-shipped-into-elasticsearch-no-error-in-logstash-cmd/210632 "2019-12-06T05:02:51Z")
Hi, I am fetching data from another database server. data is fetching but not inserted into elastcisearch, No error is showing in logstash cmd --- ## [Use s3 backup\_bucket option and create folder structure dynamically](https://discuss.elastic.co/t/use-s3-backup-bucket-option-and-create-folder-structure-dynamically/210797)
**Author:** [@user2416](https://discuss.elastic.co/u/user2416)\ **Replies:** 0\ **Last updated:** [December 6, 2019, 12:59am UTC](https://discuss.elastic.co/t/use-s3-backup-bucket-option-and-create-folder-structure-dynamically/210797 "2019-12-06T00:59:32Z")
Hi, I am trying to read AWS cloudtrail logs stored in S3 bucket in the below folder structure AWSLogs/accountnumber/CloudTrail/region/year/month/day and send the processed files to the backup S3 bucket using s3 backup\_… --- ## [Update offline logstash-input-s3 plugin](https://discuss.elastic.co/t/update-offline-logstash-input-s3-plugin/210796)
**Author:** [@sili.redwork](https://discuss.elastic.co/u/sili.redwork)\ **Replies:** 0\ **Last updated:** [December 6, 2019, 12:51am UTC](https://discuss.elastic.co/t/update-offline-logstash-input-s3-plugin/210796 "2019-12-06T00:51:07Z")
Hello, I am trying to install logstash-input-s3 plugin from logstash v 6.5.0 into logstash v 6.0.0 on RHEL7 Installed logstash v 6.5.0 on server1 and prepared offline plugin pack with below command and copied generat… --- ## [JSON Parsing Error - Logstash](https://discuss.elastic.co/t/json-parsing-error-logstash/210768)
**Author:** [@Booooooo](https://discuss.elastic.co/u/Booooooo)\ **Replies:** 1\ **Last updated:** [December 5, 2019, 8:52pm UTC](https://discuss.elastic.co/t/json-parsing-error-logstash/210768 "2019-12-05T20:52:31Z")
Hi, I am trying to ingest a JSON file but I'm getting errors and none of the key value pairs are being extrated to any fields. I currently have this configuration: input { file { path =\> "/home/path\_to\_jso… --- ## [How to set an Environment variable in Http\_poller](https://discuss.elastic.co/t/how-to-set-an-environment-variable-in-http-poller/210757)
**Author:** [@kamalesh](https://discuss.elastic.co/u/kamalesh)\ **Replies:** 0\ **Last updated:** [December 5, 2019, 5:59pm UTC](https://discuss.elastic.co/t/how-to-set-an-environment-variable-in-http-poller/210757 "2019-12-05T17:59:08Z")
Hi All, I am trying to use Environment variable in http\_poller in the Url part but not sure on how to do it and where it need to be configured Can any one please let me know where i need to set the environment variable… --- ## [Logstash Fingerprint for Regulations](https://discuss.elastic.co/t/logstash-fingerprint-for-regulations/210675)
**Author:** [@anilozyurt](https://discuss.elastic.co/u/anilozyurt)\ **Replies:** 1\ **Last updated:** [December 5, 2019, 5:40pm UTC](https://discuss.elastic.co/t/logstash-fingerprint-for-regulations/210675 "2019-12-05T17:40:56Z")
Hi, We're using ELK stack to centralize logs of our systems and need to encrypt some of these log types(access logs, audit logs etc.). Logstash fingerprint seems suitable for this but we have to check if the log content… --- ## [Logstash runing with the flowing configuration](https://discuss.elastic.co/t/logstash-runing-with-the-flowing-configuration/210756)
**Author:** [@kumar4](https://discuss.elastic.co/u/kumar4)\ **Replies:** 0\ **Last updated:** [December 5, 2019, 5:40pm UTC](https://discuss.elastic.co/t/logstash-runing-with-the-flowing-configuration/210756 "2019-12-05T17:40:39Z")
i have the follwing config for logstash (shown in image) and i want to know how to specify the index name and howa to start data collect and wher to put this file thnak you --- ## [Is this the best way to compare and enrich two datasets?](https://discuss.elastic.co/t/is-this-the-best-way-to-compare-and-enrich-two-datasets/210738)
**Author:** [@onetrickwolf](https://discuss.elastic.co/u/onetrickwolf)\ **Replies:** 0\ **Last updated:** [December 5, 2019, 3:38pm UTC](https://discuss.elastic.co/t/is-this-the-best-way-to-compare-and-enrich-two-datasets/210738 "2019-12-05T15:38:01Z")
Hi I have a similar use case to this: https://discuss.elastic.co/t/compare-two-datasets/129006/3 However, we are an e-commerce platform and are looking to flag data from bad sellers/buyers instead of IP addresses. So we… --- ## [Logstash WMI plugin logstash-input-wmi](https://discuss.elastic.co/t/logstash-wmi-plugin-logstash-input-wmi/209774)
**Author:** [@jeaolucas](https://discuss.elastic.co/u/jeaolucas)\ **Replies:** 0\ **Last updated:** [November 27, 2019, 11:28pm UTC](https://discuss.elastic.co/t/logstash-wmi-plugin-logstash-input-wmi/209774 "2019-11-27T23:28:10Z")
Hi i have the next problem when i try to excecute my pipeline logstash -f config\\logstash-wmi-simple.conf this is the content of file // WMI -\> Logstash -\> Elasticsearch pipeline. input { # prueba local wmi { host… --- ## [JMX conf multiple hosts](https://discuss.elastic.co/t/jmx-conf-multiple-hosts/210736)
**Author:** [@Gidi\_Kalef](https://discuss.elastic.co/u/Gidi_Kalef)\ **Replies:** 0\ **Last updated:** [December 5, 2019, 3:09pm UTC](https://discuss.elastic.co/t/jmx-conf-multiple-hosts/210736 "2019-12-05T15:09:31Z")
Hi , is it possible to add multiple hosts to jmx.conf: { //Required, JMX listening host/ip "host" : "192.168.1.2", //Required, JMX listening port "port" : 1335, //Optional, the username to connect to JMX "usernam… --- ## [Get message field of last event](https://discuss.elastic.co/t/get-message-field-of-last-event/210735)
**Author:** [@ja123](https://discuss.elastic.co/u/ja123)\ **Replies:** 0\ **Last updated:** [December 5, 2019, 2:55pm UTC](https://discuss.elastic.co/t/get-message-field-of-last-event/210735 "2019-12-05T14:55:34Z")
How can I get the message field of previous event in elasticsearch? The document ID is dynamically { "request" : { "url" : "https://reiseauskunft.bahn.de/bin/stboard.exe/dn?rt=1&input=Stuttgart%20Hauptbahnhof&boardTy… --- ## [Syslog output debugging](https://discuss.elastic.co/t/syslog-output-debugging/210733)
**Author:** [@grimshad](https://discuss.elastic.co/u/grimshad)\ **Replies:** 0\ **Last updated:** [December 5, 2019, 2:51pm UTC](https://discuss.elastic.co/t/syslog-output-debugging/210733 "2019-12-05T14:51:48Z")
How can I output the final syslog output to stdout to debug it prior to setting it up to be sent to the syslog server? --- ## [Output to Kafka](https://discuss.elastic.co/t/output-to-kafka/210732)
**Author:** [@Haalanam](https://discuss.elastic.co/u/Haalanam)\ **Replies:** 0\ **Last updated:** [December 5, 2019, 2:40pm UTC](https://discuss.elastic.co/t/output-to-kafka/210732 "2019-12-05T14:40:48Z")
i have a logstash conf file that looks like this: input { jdbc { jdbc\_driver\_class =\> "Java::oracle.jdbc.driver.OracleDriver" jdbc\_connection\_string =\> "jdbc:oracle:thin:@//the-s2.db.orac… --- ## [Logstash Grok problem](https://discuss.elastic.co/t/logstash-grok-problem/210710)
**Author:** [@LuigiDelavega](https://discuss.elastic.co/u/LuigiDelavega)\ **Replies:** 0\ **Last updated:** [December 5, 2019, 12:41pm UTC](https://discuss.elastic.co/t/logstash-grok-problem/210710 "2019-12-05T12:41:02Z")
Hello, I'm using a grok filter for my logs. They come from severals folders and they have a common pattern i already manage to create : grok { match =\> { "message" =\> "%{DATESTAMP:time}\\|%{LOGLEVEL:Level}\\|%{DA… --- ## [Tcp input plugin errors](https://discuss.elastic.co/t/tcp-input-plugin-errors/210673)
**Author:** [@annk](https://discuss.elastic.co/u/annk)\ **Replies:** 0\ **Last updated:** [December 5, 2019, 10:11am UTC](https://discuss.elastic.co/t/tcp-input-plugin-errors/210673 "2019-12-05T10:11:37Z")
\[logstash.inputs.tcp \] Error in Netty pipeline: java.io.IOException: Connection reset by peer I've 3 logstahs nodes(docker containers) in our cluster and I am also seeing "Number of nodes " going up and down in th… --- ## [How to write Logstash input to get data from Rest API](https://discuss.elastic.co/t/how-to-write-logstash-input-to-get-data-from-rest-api/210687)
**Author:** [@katara](https://discuss.elastic.co/u/katara)\ **Replies:** 0\ **Last updated:** [December 5, 2019, 11:26am UTC](https://discuss.elastic.co/t/how-to-write-logstash-input-to-get-data-from-rest-api/210687 "2019-12-05T11:26:44Z")
Hi team, I'm new to ELK ( using 7.2.0) and I need to connect to ServiceNow through a Rest API connection, I'm not exactly sure of how to proceed with it. Can anyone give me a simple example of how the Logstash conf f… --- ## [Logstash-input-jdbc handle geo shape for mysql geometry field](https://discuss.elastic.co/t/logstash-input-jdbc-handle-geo-shape-for-mysql-geometry-field/210677)
**Author:** [@xiangqian](https://discuss.elastic.co/u/xiangqian)\ **Replies:** 0\ **Last updated:** [December 5, 2019, 10:36am UTC](https://discuss.elastic.co/t/logstash-input-jdbc-handle-geo-shape-for-mysql-geometry-field/210677 "2019-12-05T10:36:43Z")
when i use logstash-input-jdbc to transform data from mysql to es,i find a problem which seems like can not convert ASCII-8bit to UTF-8;this is not plugin'bug ;because mysql use a different format to encode geometry. wh… --- ## [Error parsing csv when files are being added](https://discuss.elastic.co/t/error-parsing-csv-when-files-are-being-added/210071)
**Author:** [@ansamHox](https://discuss.elastic.co/u/ansamHox)\ **Replies:** 2\ **Last updated:** [December 5, 2019, 10:00am UTC](https://discuss.elastic.co/t/error-parsing-csv-when-files-are-being-added/210071 "2019-12-05T10:00:57Z")
I'm copying log files generated on dba server on my linux machine (where logstash instance is) with crontab job every day at 06:30. Files are during generation being written each minute, which means that if generation is… --- ## [How do I give input file path from a text file?](https://discuss.elastic.co/t/how-do-i-give-input-file-path-from-a-text-file/210666)
**Author:** [@Ramesh\_Raghavan](https://discuss.elastic.co/u/Ramesh_Raghavan)\ **Replies:** 0\ **Last updated:** [December 5, 2019, 9:40am UTC](https://discuss.elastic.co/t/how-do-i-give-input-file-path-from-a-text-file/210666 "2019-12-05T09:40:53Z")
if I have a input field in config file like this. input { file { path =\> "/tmp/log.csv" start\_position =\> "beginning" } } How do I give input file path from a text file? --- ## [Logstash unable to gsub a pattern](https://discuss.elastic.co/t/logstash-unable-to-gsub-a-pattern/210273)
**Author:** [@Abhishek\_Kumar4](https://discuss.elastic.co/u/Abhishek_Kumar4)\ **Replies:** 1\ **Last updated:** [December 5, 2019, 9:35am UTC](https://discuss.elastic.co/t/logstash-unable-to-gsub-a-pattern/210273 "2019-12-05T09:35:21Z")
I am sending below log file from filbeat to Logstash. The below is the output from filbeat to Logstash This output I got by setting filebeat output to stdout to see what is being sent to Logstash \\u001b\[m\\u001b\[32m\[201… --- ## [Pb using logstash Helm chart and X-Pack](https://discuss.elastic.co/t/pb-using-logstash-helm-chart-and-x-pack/209979)
**Author:** [@cabrinoob](https://discuss.elastic.co/u/cabrinoob)\ **Replies:** 1\ **Last updated:** [December 5, 2019, 7:18am UTC](https://discuss.elastic.co/t/pb-using-logstash-helm-chart-and-x-pack/209979 "2019-12-05T07:18:08Z")
Hi, I'am trying to deploy Logstash on my Kubernetes cluster using this helm chart : I just modified the values.yaml to have the input and output I need like this : inputs: main: |- input { kinesis { … --- ## [Configure logstast to bind to an IP](https://discuss.elastic.co/t/configure-logstast-to-bind-to-an-ip/210627)
**Author:** [@ShiviKaya](https://discuss.elastic.co/u/ShiviKaya)\ **Replies:** 0\ **Last updated:** [December 5, 2019, 6:12am UTC](https://discuss.elastic.co/t/configure-logstast-to-bind-to-an-ip/210627 "2019-12-05T06:12:47Z")
My server has two interfaces, public and private. I need to send logs to a kafka server which is not reachable from the private interface. Is there a way to configure logstash to use the public interface because it uses … --- ## [Alert when Exception Logstash](https://discuss.elastic.co/t/alert-when-exception-logstash/210621)
**Author:** [@Anuwa\_A](https://discuss.elastic.co/u/Anuwa_A)\ **Replies:** 0\ **Last updated:** [December 5, 2019, 5:18am UTC](https://discuss.elastic.co/t/alert-when-exception-logstash/210621 "2019-12-05T05:18:57Z")
I'm using logstash to sync mysql data to elasticserch. Is there a way to enable alert notification/email when there is any exception happen in that process. --- ## [All logs are entered twice](https://discuss.elastic.co/t/all-logs-are-entered-twice/210603)
**Author:** [@matthew.simpson](https://discuss.elastic.co/u/matthew.simpson)\ **Replies:** 0\ **Last updated:** [December 4, 2019, 10:28pm UTC](https://discuss.elastic.co/t/all-logs-are-entered-twice/210603 "2019-12-04T22:28:03Z")
Hey everyone, we have filebeat set up on two VM's and Elasticsearch, Kibana and Logstash working on a 3rd VM. We also have Meraki Syslogs parsing into Logstash. All of our log entries are being duplicated from all sourc… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=434) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=436)