# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=979 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 980 --- ## [How correctly debug GROK's in logstash](https://discuss.elastic.co/t/how-correctly-debug-groks-in-logstash/86289)
**Author:** [@Igor\_Gerasimow](https://discuss.elastic.co/u/Igor_Gerasimow)\ **Replies:** 1\ **Last updated:** [May 23, 2017, 3:49am UTC](https://discuss.elastic.co/t/how-correctly-debug-groks-in-logstash/86289 "2017-05-23T03:49:28Z")
Hello. I'm sure that my issues is very popular: how i can add fields from grok - for example 2017-05-18 13:52:56 "GET /?method=get\_match&match\_id=1598784326®ion=eune&include\_timeline=true HTTP/1.1" 200 1 … --- ## [No traking sql\_last\_value value in jdbc plugin](https://discuss.elastic.co/t/no-traking-sql-last-value-value-in-jdbc-plugin/48219)
**Author:** [@jalen\_wang](https://discuss.elastic.co/u/jalen_wang)\ **Replies:** 1\ **Last updated:** [May 22, 2017, 9:19pm UTC](https://discuss.elastic.co/t/no-traking-sql-last-value-value-in-jdbc-plugin/48219 "2017-05-22T21:19:39Z")
I'm going to use jdbc plugin to get data from oracle database,below is my sample data: OP\_TYPE OP\_DATE OBJ\_NAME LAG HOSTNAME GROUP\_NAME RECORDLENGTH TIMESTAMP FILESEQNO FILERBA ID --… --- ## [Nothing outputting at all](https://discuss.elastic.co/t/nothing-outputting-at-all/86695)
**Author:** [@Jaxon\_Kochel](https://discuss.elastic.co/u/Jaxon_Kochel)\ **Replies:** 1\ **Last updated:** [May 22, 2017, 6:41pm UTC](https://discuss.elastic.co/t/nothing-outputting-at-all/86695 "2017-05-22T18:41:09Z")
So I'm assuming i broke my config file somewhere as I'm no longer getting terminal output for any files, and they havent yet been read. I'm not getting nay terminal errors, is there any way for me to debug what is going … --- ## [Send batch of multiline events to http output](https://discuss.elastic.co/t/send-batch-of-multiline-events-to-http-output/86721)
**Author:** [@deeps123](https://discuss.elastic.co/u/deeps123)\ **Replies:** 0\ **Last updated:** [May 22, 2017, 5:42pm UTC](https://discuss.elastic.co/t/send-batch-of-multiline-events-to-http-output/86721 "2017-05-22T17:42:58Z")
Hi, I am new to Logstash world. Can some one help me how to write conf for below scenario. I have data collected from tcp and I output to http. Like below input { tcp { port =\>8777 codec =\> multiline { … --- ## [Java::ComSybaseJdbc3Jdbc::SybSQLException: ERROR=Attempt to insert duplicate row on line 1 of statement 'insert into netc.t2 values('NULL','NULL')': insert into netc.t2 values('NULL','NULL')](https://discuss.elastic.co/t/java-comsybasejdbc3jdbc-sybsqlexception-error-attempt-to-insert-duplicate-row-on-line-1-of-statement-insert-into-netc-t2-values-null-null-insert-into-netc-t2-values-null-null/86410)
**Author:** [@kabali12345](https://discuss.elastic.co/u/kabali12345)\ **Replies:** 7\ **Last updated:** [May 22, 2017, 4:18pm UTC](https://discuss.elastic.co/t/java-comsybasejdbc3jdbc-sybsqlexception-error-attempt-to-insert-duplicate-row-on-line-1-of-statement-insert-into-netc-t2-values-null-null-insert-into-netc-t2-values-null-null/86410 "2017-05-22T16:18:18Z")
Hi.......... Here i am trying to insert data from Elasticsearch to sybase with the help of jdbc but i am getting following error Java::ComSybaseJdbc3Jdbc::SybSQLException: ERROR=Attempt to insert duplicate row on line … --- ## [Empty fields in input - grok failure](https://discuss.elastic.co/t/empty-fields-in-input-grok-failure/86483)
**Author:** [@Nikhilpawar1985](https://discuss.elastic.co/u/Nikhilpawar1985)\ **Replies:** 3\ **Last updated:** [May 22, 2017, 2:55pm UTC](https://discuss.elastic.co/t/empty-fields-in-input-grok-failure/86483 "2017-05-22T14:55:09Z")
Hi , I am trying to use grok for input lines like ; INPUT DATA \[2017-05-15 00:00:07,751\] :|: INFO :|: dubprdsndlbfe33.dub.jabodo.com :|: 26baee2d30164c2083e380b40b2a3b15 :|: \[BT:CHROME, BV:55, BL:en, CC:C… --- ## [Logstash always tries to connect to ES](https://discuss.elastic.co/t/logstash-always-tries-to-connect-to-es/86188)
**Author:** [@John\_06](https://discuss.elastic.co/u/John_06)\ **Replies:** 5\ **Last updated:** [May 22, 2017, 3:57pm UTC](https://discuss.elastic.co/t/logstash-always-tries-to-connect-to-es/86188 "2017-05-22T15:57:48Z")
I am forwarding logs from Logstash to Kafka but Logstash always tries to connect to ES at http://localhost:9200/ \[2017-05-18T03:45:49,875\]\[INFO \]\[logstash.agent \] Successfully started Logstash API endpoint {:… --- ## [Logstash s3 region us-east-2 not supported?](https://discuss.elastic.co/t/logstash-s3-region-us-east-2-not-supported/86591)
**Author:** [@nielsen-at-cgt](https://discuss.elastic.co/u/nielsen-at-cgt)\ **Replies:** 3\ **Last updated:** [May 22, 2017, 2:47pm UTC](https://discuss.elastic.co/t/logstash-s3-region-us-east-2-not-supported/86591 "2017-05-22T14:47:07Z")
My logstash 5.4.0 lab is in AWS us-east-2. input { s3 { # This setting must be a \["us-east-1", "us-west-1", "us-west-2", "eu-central-1", "eu-west-1", "ap-southeast-1", "ap-southeast-2", "ap-northeast-1", "ap-nort… --- ## [Problem using jdbc streaming](https://discuss.elastic.co/t/problem-using-jdbc-streaming/86682)
**Author:** [@nano](https://discuss.elastic.co/u/nano)\ **Replies:** 0\ **Last updated:** [May 22, 2017, 2:16pm UTC](https://discuss.elastic.co/t/problem-using-jdbc-streaming/86682 "2017-05-22T14:16:18Z")
Hello please i try to use jdbc streaming but doesn't work for me this is how looks my configuration file: input{ jdbc{ statement =\> "select field1, field2, field3 from database1" ...... } } filter { jdbc\_st… --- ## [How does filter-dns handle multiple results?](https://discuss.elastic.co/t/how-does-filter-dns-handle-multiple-results/86677)
**Author:** [@jahlives](https://discuss.elastic.co/u/jahlives)\ **Replies:** 0\ **Last updated:** [May 22, 2017, 1:37pm UTC](https://discuss.elastic.co/t/how-does-filter-dns-handle-multiple-results/86677 "2017-05-22T13:37:57Z")
I'm currently useing filter-dns to perform rdns queries on a given field. Config looks as follows: dns { reverse =\> \[ "clientip" \] action =\> "append" } if ( "" in \[clientip\]\[1\] and \[clientip\]\[1\] != '1' ) { mutate … --- ## [Jump out of filter to output](https://discuss.elastic.co/t/jump-out-of-filter-to-output/86490)
**Author:** [@Jaxon\_Kochel](https://discuss.elastic.co/u/Jaxon_Kochel)\ **Replies:** 2\ **Last updated:** [May 22, 2017, 1:11pm UTC](https://discuss.elastic.co/t/jump-out-of-filter-to-output/86490 "2017-05-22T13:11:56Z")
Is there any way to jump out of the filter straight to output? I'm having an issue with log files with very similar names being picked up by multiple path =~, and its breaking my logging. is there anyway to jump out of t… --- ## [Multiple grok match don't work](https://discuss.elastic.co/t/multiple-grok-match-dont-work/86103)
**Author:** [@stefanocog](https://discuss.elastic.co/u/stefanocog)\ **Replies:** 10\ **Last updated:** [May 22, 2017, 12:32pm UTC](https://discuss.elastic.co/t/multiple-grok-match-dont-work/86103 "2017-05-22T12:32:30Z")
Hi, i have an ELK stack on docker and put logs througt "gelf", i need match logs by log format, i have two files (access.log and error.log, and one single tag "apache") i have this filter filter { if \[tag\] == "apac… --- ## [Consistent restarts of logstash with high CPU % recorded](https://discuss.elastic.co/t/consistent-restarts-of-logstash-with-high-cpu-recorded/86597)
**Author:** [@ImTheDeveloper](https://discuss.elastic.co/u/ImTheDeveloper)\ **Replies:** 2\ **Last updated:** [May 22, 2017, 8:21am UTC](https://discuss.elastic.co/t/consistent-restarts-of-logstash-with-high-cpu-recorded/86597 "2017-05-22T08:21:25Z")
Morning All, I've installed logstash and I am using a pipe to pull data from Postgres via JDBC into Elasticsearch. When running via systemctl e.g.: sudo systemctl start logstash.service I see that logstash starts and … --- ## [Add new field in logstash conf file](https://discuss.elastic.co/t/add-new-field-in-logstash-conf-file/85201)
**Author:** [@dinesht15](https://discuss.elastic.co/u/dinesht15)\ **Replies:** 0\ **Last updated:** [May 10, 2017, 8:19am UTC](https://discuss.elastic.co/t/add-new-field-in-logstash-conf-file/85201 "2017-05-10T08:19:02Z")
Hi, I am new to logstash and this is my first task with it. I wanted to add two new fields called "GMTDateTime" and "CSTDateTime" to use them in my kibana reporting. Same work i was doing in SQL query by GMTDa… --- ## [Execution order multiple S3 input plugins](https://discuss.elastic.co/t/execution-order-multiple-s3-input-plugins/86617)
**Author:** [@kbry](https://discuss.elastic.co/u/kbry)\ **Replies:** 0\ **Last updated:** [May 22, 2017, 7:56am UTC](https://discuss.elastic.co/t/execution-order-multiple-s3-input-plugins/86617 "2017-05-22T07:56:21Z")
Hi everyone, A question about the execution order of logstash with multiple S3 input plugins ( points to different buckets ). In such a scenario what is the expected behavior? Will logstash wait until each bucket is … --- ## [Cant I get the csv file from netflow data and send with UDP or TCP?](https://discuss.elastic.co/t/cant-i-get-the-csv-file-from-netflow-data-and-send-with-udp-or-tcp/86606)
**Author:** [@sangrho](https://discuss.elastic.co/u/sangrho)\ **Replies:** 0\ **Last updated:** [May 22, 2017, 6:54am UTC](https://discuss.elastic.co/t/cant-i-get-the-csv-file-from-netflow-data-and-send-with-udp-or-tcp/86606 "2017-05-22T06:54:00Z")
I wanna get the csv file from netflow data and send with UDP or TCP in output. My code is like below : input{ file{ path=\>"netflow.pcap" codec=\>netflow{ versions=\>\[5\] } } } output{ csv{ fi… --- ## [Logstash vs winlogbeats from external machine](https://discuss.elastic.co/t/logstash-vs-winlogbeats-from-external-machine/86102)
**Author:** [@ssi](https://discuss.elastic.co/u/ssi)\ **Replies:** 2\ **Last updated:** [May 22, 2017, 7:39am UTC](https://discuss.elastic.co/t/logstash-vs-winlogbeats-from-external-machine/86102 "2017-05-22T07:39:53Z")
I've been throught the posts aut have yet to find a solution. i've opend port 5044 on firewall on both sending client and receving server. I can telnet on port 5044 to the logstash server on the 5044 port. but i get … --- ## [Logstash config for multiple stacktrace formats in same log](https://discuss.elastic.co/t/logstash-config-for-multiple-stacktrace-formats-in-same-log/86608)
**Author:** [@gautamv](https://discuss.elastic.co/u/gautamv)\ **Replies:** 0\ **Last updated:** [May 22, 2017, 6:59am UTC](https://discuss.elastic.co/t/logstash-config-for-multiple-stacktrace-formats-in-same-log/86608 "2017-05-22T06:59:24Z")
Hi, I have a SystemError log which has two different formats for writing stacktraces. One is: \[12/26/16 17:41:58:237 GMT+05:30\] 0000010c SystemErr R com.ascd.qwer.common.SecurityViolationException: User Record … --- ## [Syslog is not consuming logs from rsyslog server](https://discuss.elastic.co/t/syslog-is-not-consuming-logs-from-rsyslog-server/86403)
**Author:** [@Mohan\_Mahajan](https://discuss.elastic.co/u/Mohan_Mahajan)\ **Replies:** 0\ **Last updated:** [May 19, 2017, 10:28am UTC](https://discuss.elastic.co/t/syslog-is-not-consuming-logs-from-rsyslog-server/86403 "2017-05-19T10:28:48Z")
I am having below configuration for rsyslog server to output it to logstash, $tail /etc/rsyslog.d/70-output.conf \*.\* @@logstash:5514 And in logstash.conf I have configured it as follows, i… --- ## [Incorrect permissions after rpm install (CentOS)](https://discuss.elastic.co/t/incorrect-permissions-after-rpm-install-centos/86607)
**Author:** [@LionelCons](https://discuss.elastic.co/u/LionelCons)\ **Replies:** 0\ **Last updated:** [May 22, 2017, 6:57am UTC](https://discuss.elastic.co/t/incorrect-permissions-after-rpm-install-centos/86607 "2017-05-22T06:57:51Z")
After having installed the logstash-5.4.0.rpm on a CentOS 7 machine, the permissions under /usr/share/logstash are incorrect. "rpm -V logstash" reports many warnings like: .....UG.. /usr/share/logstash This co… --- ## [Load database CSV](https://discuss.elastic.co/t/load-database-csv/86588)
**Author:** [@Kirtash](https://discuss.elastic.co/u/Kirtash)\ **Replies:** 3\ **Last updated:** [May 22, 2017, 6:12am UTC](https://discuss.elastic.co/t/load-database-csv/86588 "2017-05-22T06:12:40Z")
Hi guys! I have a relational database and now I want migrate the information of it to indexs in ElasticSearch. I have created 3 index using datasets but now I have a problem, how can I load a list using CSV? I have t… --- ## [Yet another \_dateparsefailure](https://discuss.elastic.co/t/yet-another--dateparsefailure/41214)
**Author:** [@yammy](https://discuss.elastic.co/u/yammy)\ **Replies:** 7\ **Last updated:** [May 22, 2017, 5:02am UTC](https://discuss.elastic.co/t/yet-another--dateparsefailure/41214 "2017-05-22T05:02:34Z")
I'm having trouble with a dateparse failure and hoping someone can spot the issue. Sample log entry: PSAPPSRV.45377 (947) 2016-01-31T01:00:18.707 GetCertificate Detected time zone is EST I can grok the data successfu… --- ## [Grok parsing errors](https://discuss.elastic.co/t/grok-parsing-errors/86584)
**Author:** [@Callahan](https://discuss.elastic.co/u/Callahan)\ **Replies:** 0\ **Last updated:** [May 21, 2017, 8:17pm UTC](https://discuss.elastic.co/t/grok-parsing-errors/86584 "2017-05-21T20:17:51Z")
Hi, I've been working with setting up Logstash to filter logs from our firewall and inject them into Elasticsearch. We already have other inputs up and running with success although this particular attempt is more comp… --- ## [Getting Data duplicated while parsing the logs using Logstash](https://discuss.elastic.co/t/getting-data-duplicated-while-parsing-the-logs-using-logstash/86138)
**Author:** [@Rajat\_Singh](https://discuss.elastic.co/u/Rajat_Singh)\ **Replies:** 2\ **Last updated:** [May 21, 2017, 4:57pm UTC](https://discuss.elastic.co/t/getting-data-duplicated-while-parsing-the-logs-using-logstash/86138 "2017-05-21T16:57:45Z")
Hello Folks, I am using Logstash for the purpose of parsing the logs to elasticsearch as their were 4 files each of 4 gb ,It took around one day it to parse all the value. so my first concern is that what things in can… --- ## [Netflow v9 Template (Logstash 5.4, Cisco 4500X, Netflow v9)](https://discuss.elastic.co/t/netflow-v9-template-logstash-5-4-cisco-4500x-netflow-v9/86581)
**Author:** [@profrowe](https://discuss.elastic.co/u/profrowe)\ **Replies:** 0\ **Last updated:** [May 21, 2017, 4:44pm UTC](https://discuss.elastic.co/t/netflow-v9-template-logstash-5-4-cisco-4500x-netflow-v9/86581 "2017-05-21T16:44:46Z")
Been scouring the forums and can't crack this one. I have a 4500X sending v9 Netflows with the following flow record defined (IOS is 152-4.E4): flow record elk match ipv4 tos match ipv4 protocol match ipv4 source addr… --- ## [Filter Plugin: Elasticsearch](https://discuss.elastic.co/t/filter-plugin-elasticsearch/44144)
**Author:** [@Christian16](https://discuss.elastic.co/u/Christian16)\ **Replies:** 6\ **Last updated:** [May 21, 2017, 3:39pm UTC](https://discuss.elastic.co/t/filter-plugin-elasticsearch/44144 "2017-05-21T15:39:51Z")
Hello, My main goal is to get a tag (TRANSFER ID) from a previous log event, and i would like to include this tag in my current log event. So I decided to use the "Elasticsearch Filter" plugin but I'm having trouble wi… --- ## [Exporting environment variables from startup.options doesn't work](https://discuss.elastic.co/t/exporting-environment-variables-from-startup-options-doesnt-work/86570)
**Author:** [@harelmoshe1](https://discuss.elastic.co/u/harelmoshe1)\ **Replies:** 0\ **Last updated:** [May 21, 2017, 2:02pm UTC](https://discuss.elastic.co/t/exporting-environment-variables-from-startup-options-doesnt-work/86570 "2017-05-21T14:02:16Z")
Hi, I'm trying to implement the following solution using chef, in which the startup.options file looks as follows: ... read -r -d '' PRESTART \<\< EOM export A=1 export B=2 export C=3 EOM ... Then, I'm starting the… --- ## [ELK 5.3.2 with X-Pack - Error Event in LogStash Log](https://discuss.elastic.co/t/elk-5-3-2-with-x-pack-error-event-in-logstash-log/86565)
**Author:** [@GJahchan](https://discuss.elastic.co/u/GJahchan)\ **Replies:** 0\ **Last updated:** [May 21, 2017, 10:39am UTC](https://discuss.elastic.co/t/elk-5-3-2-with-x-pack-error-event-in-logstash-log/86565 "2017-05-21T10:39:10Z")
Running ELK 5.3.2 on CentOS 7.3. I am seeing a stream of the following error events (once every ten seconds) which are seemingly due to X-Pack: \[2017-05-21T13:27:11,120\]\[ERROR\]\[logstash.outputs.elasticsearch\] Got a bad… --- ## [Logstash email output plugin in windows7](https://discuss.elastic.co/t/logstash-email-output-plugin-in-windows7/86548)
**Author:** [@Ho\_Shin](https://discuss.elastic.co/u/Ho_Shin)\ **Replies:** 2\ **Last updated:** [May 21, 2017, 2:44am UTC](https://discuss.elastic.co/t/logstash-email-output-plugin-in-windows7/86548 "2017-05-21T02:44:18Z")
Hi there, I want to use email output plugin in windows7. It seems that there is no email output available. There is no email in the logstash-plugin list I copied CMD Line below. When I try to install email plugin li… --- ## [Logstash agent to forward logs to IBM Websphere MQ](https://discuss.elastic.co/t/logstash-agent-to-forward-logs-to-ibm-websphere-mq/86085)
**Author:** [@satishkori](https://discuss.elastic.co/u/satishkori)\ **Replies:** 1\ **Last updated:** [May 21, 2017, 12:02am UTC](https://discuss.elastic.co/t/logstash-agent-to-forward-logs-to-ibm-websphere-mq/86085 "2017-05-21T00:02:31Z")
Hi , Has anyone come cross a scenarios to forward client server application logs using logstash agent to forward the logs to IBM Websphere MQ and then to another ELK server . Any idea if Logstash supports IBM Webspher… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=978) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=980)