# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=451 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 452 --- ## [Output with condition](https://discuss.elastic.co/t/output-with-condition/206505)
**Author:** [@sud0](https://discuss.elastic.co/u/sud0)\ **Replies:** 4\ **Last updated:** [November 5, 2019, 8:56pm UTC](https://discuss.elastic.co/t/output-with-condition/206505 "2019-11-05T20:56:48Z")
HI. I have two businesses that must be separated in two different indexes. I'm trying to separate them on the output: output { if \[region\] == "au" { elasticsearch { hosts =\> \["localhost:9200"\] … --- ## [Cisco ASA Parsing - Logstash vs Filebeat?](https://discuss.elastic.co/t/cisco-asa-parsing-logstash-vs-filebeat/206666)
**Author:** [@mgotechlock](https://discuss.elastic.co/u/mgotechlock)\ **Replies:** 0\ **Last updated:** [November 5, 2019, 7:01pm UTC](https://discuss.elastic.co/t/cisco-asa-parsing-logstash-vs-filebeat/206666 "2019-11-05T19:01:31Z")
This should be more obvious but I've spent hours searching and it's not. I just want to understand the best way(s) to get ASA logs to my internal logstash server and on to my AWS S3 output bucket. Ingesting Cisco ASA fi… --- ## [Grok parse failure despite pattern working in debugger](https://discuss.elastic.co/t/grok-parse-failure-despite-pattern-working-in-debugger/206665)
**Author:** [@mfisher](https://discuss.elastic.co/u/mfisher)\ **Replies:** 0\ **Last updated:** [November 5, 2019, 6:59pm UTC](https://discuss.elastic.co/t/grok-parse-failure-despite-pattern-working-in-debugger/206665 "2019-11-05T18:59:44Z")
I've been getting grok parse failures for these messages. "message": "\<164\>Nov 05 2019 12:46:04 NGFW1-IPICHIC : %ASA-4-434003: SFR requested to reset TCP connection from outside:13.249.87.36/443 to inside:38.142.127.155… --- ## [How to Correlate Logs](https://discuss.elastic.co/t/how-to-correlate-logs/206655)
**Author:** [@nuwancs](https://discuss.elastic.co/u/nuwancs)\ **Replies:** 0\ **Last updated:** [November 5, 2019, 5:45pm UTC](https://discuss.elastic.co/t/how-to-correlate-logs/206655 "2019-11-05T17:45:27Z")
Hi There, I'm trying to correlate requests and responses via logstash. what i am doing is keep the request in the logstash untill the response receive. then will emit the correlated event to elasticsearch. In this appr… --- ## [Getting warning messages in logstash for grok patterns](https://discuss.elastic.co/t/getting-warning-messages-in-logstash-for-grok-patterns/206589)
**Author:** [@satyam2593](https://discuss.elastic.co/u/satyam2593)\ **Replies:** 2\ **Last updated:** [November 5, 2019, 4:59pm UTC](https://discuss.elastic.co/t/getting-warning-messages-in-logstash-for-grok-patterns/206589 "2019-11-05T16:59:36Z")
I am getting the below warning messages in logstash logs and index is not creating for the same. Error:\[WARN \]\[org.logstash.execution.ShutdownWatcherExt\] {"inflight\_count"=\>0, "stalling\_threads\_info"=\>{"other"=\>\[{"threa… --- ## [Logstash - Glok and stdout -\> rubydebug doesn't work](https://discuss.elastic.co/t/logstash-glok-and-stdout-rubydebug-doesnt-work/206631)
**Author:** [@Andrii\_Fon](https://discuss.elastic.co/u/Andrii_Fon)\ **Replies:** 6\ **Last updated:** [November 5, 2019, 4:34pm UTC](https://discuss.elastic.co/t/logstash-glok-and-stdout-rubydebug-doesnt-work/206631 "2019-11-05T16:34:49Z")
Hi everyone, I'm pacing days on the ELK + filebeat to make it work with my Glok filter. I have started installing ELK + filebeat, then defined logstash.conf: input { file { path =\> "D:\\ElisticLogs\\MyLog.log… --- ## [Error when I'm launching logstash](https://discuss.elastic.co/t/error-when-im-launching-logstash/206561)
**Author:** [@lgllrd](https://discuss.elastic.co/u/lgllrd)\ **Replies:** 2\ **Last updated:** [November 5, 2019, 3:45pm UTC](https://discuss.elastic.co/t/error-when-im-launching-logstash/206561 "2019-11-05T15:45:09Z")
Hi, I'm trying to launch logstash with my .conf file but it tell me an error that I don't understand : OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be remov… --- ## [Filebeat -\> Logstash match metadata](https://discuss.elastic.co/t/filebeat-logstash-match-metadata/206444)
**Author:** [@imaad](https://discuss.elastic.co/u/imaad)\ **Replies:** 3\ **Last updated:** [November 5, 2019, 3:42pm UTC](https://discuss.elastic.co/t/filebeat-logstash-match-metadata/206444 "2019-11-05T15:42:31Z")
Hi, I've noticed filebeat sends some additional metadata for each generated event such as: log.file.path and process.name. I can access to log.file.path (\[log\]\[file\]\[path\]) but not to process.name (\[process\]\[name\]). ?? … --- ## [Parse logs with timezone](https://discuss.elastic.co/t/parse-logs-with-timezone/206628)
**Author:** [@kate1](https://discuss.elastic.co/u/kate1)\ **Replies:** 0\ **Last updated:** [November 5, 2019, 2:34pm UTC](https://discuss.elastic.co/t/parse-logs-with-timezone/206628 "2019-11-05T14:34:56Z")
I'm trying to parse some logs with 2 type \ \ \ \ \ and --- Mon Sep 16 11:30:20 MSK 2019 WSLoggerQueue Left: 0, Out: 4 --- … --- ## [Kibanaclient - Error when executing Kibana client request -nagios](https://discuss.elastic.co/t/kibanaclient-error-when-executing-kibana-client-request-nagios/206577)
**Author:** [@anusree\_arun](https://discuss.elastic.co/u/anusree_arun)\ **Replies:** 0\ **Last updated:** [November 5, 2019, 9:48am UTC](https://discuss.elastic.co/t/kibanaclient-error-when-executing-kibana-client-request-nagios/206577 "2019-11-05T09:48:06Z")
i am now using nagios module in my logstash . when i run the command /usr/share/logstash/bin/logstash --modules nagios --setup ---i got an error like as below Thread.exclusive is deprecated, use Thread::Mutex WARNIN… --- ## [MySQL log analysis with Filebeat Logstash and Elasticsearch](https://discuss.elastic.co/t/mysql-log-analysis-with-filebeat-logstash-and-elasticsearch/206576)
**Author:** [@Vladpov](https://discuss.elastic.co/u/Vladpov)\ **Replies:** 0\ **Last updated:** [November 5, 2019, 9:47am UTC](https://discuss.elastic.co/t/mysql-log-analysis-with-filebeat-logstash-and-elasticsearch/206576 "2019-11-05T09:47:34Z")
Hello everyone, I'm trying to analyze MySQL and wanted to ask what is the best way to do it? :smiley: I'm trying to ship logs by using Filebeat but have some problems with GROK Patterns to parsing mysql-slow-logs in a … --- ## [File input JSON with multiline codec plugin](https://discuss.elastic.co/t/file-input-json-with-multiline-codec-plugin/204669)
**Author:** [@Krypton8](https://discuss.elastic.co/u/Krypton8)\ **Replies:** 6\ **Last updated:** [November 5, 2019, 9:38am UTC](https://discuss.elastic.co/t/file-input-json-with-multiline-codec-plugin/204669 "2019-11-05T09:38:07Z")
Hello, I'm simply trying to parse my JSON fileds into events. The JSON file look like this: \[ { "field1": "value1", "field2": "value2", "field3": "value3", "field4": "value4", "field5": "value5", "field6"… --- ## [Elasticsearch to csv through logstash - basic question](https://discuss.elastic.co/t/elasticsearch-to-csv-through-logstash-basic-question/206528)
**Author:** [@TomasGreif](https://discuss.elastic.co/u/TomasGreif)\ **Replies:** 1\ **Last updated:** [November 5, 2019, 9:07am UTC](https://discuss.elastic.co/t/elasticsearch-to-csv-through-logstash-basic-question/206528 "2019-11-05T09:07:36Z")
I want to use logstash to process data stored in elasticsearch and write output to csv. Do I need to install logstash on the server where elastic runs or I can install it locally only and connect to the server? Or I can … --- ## [Kafka output plugin timeout exceptions](https://discuss.elastic.co/t/kafka-output-plugin-timeout-exceptions/206536)
**Author:** [@emilg02](https://discuss.elastic.co/u/emilg02)\ **Replies:** 0\ **Last updated:** [November 5, 2019, 7:28am UTC](https://discuss.elastic.co/t/kafka-output-plugin-timeout-exceptions/206536 "2019-11-05T07:28:50Z")
Hi all, I am using a Logstash instance to mirror data between two Kafkas, below is the current configuration I have: Logstash pipeline: input { kafka { id =\> "kafka-reader" boo… --- ## [ERROR: A plugin had an unrecoverable error. Will restart this plugin](https://discuss.elastic.co/t/error-a-plugin-had-an-unrecoverable-error-will-restart-this-plugin/206515)
**Author:** [@Sagar\_Mandal](https://discuss.elastic.co/u/Sagar_Mandal)\ **Replies:** 2\ **Last updated:** [November 5, 2019, 7:21am UTC](https://discuss.elastic.co/t/error-a-plugin-had-an-unrecoverable-error-will-restart-this-plugin/206515 "2019-11-05T07:21:27Z")
Hi sagar this side. I have installed all three Elastic search, kibana and logstash in my redhat, for practice purposes. I'm try to connect to oracle database in my network, and have used ojdbc7.jar for that, all my ser… --- ## [Cannot write to a field alias Error URGENT](https://discuss.elastic.co/t/cannot-write-to-a-field-alias-error-urgent/202825)
**Author:** [@khergner](https://discuss.elastic.co/u/khergner)\ **Replies:** 0\ **Last updated:** [October 9, 2019, 12:04pm UTC](https://discuss.elastic.co/t/cannot-write-to-a-field-alias-error-urgent/202825 "2019-10-09T12:04:27Z")
Hi everyone, I get the following error in logstash but ı don't understand why ı got an error. Can you help me for solution to problem? Elk & logstash & filebeat version 7.3.0 I use shared to below filebeat template … --- ## [Timeout executing grok and timeout\_millis not work](https://discuss.elastic.co/t/timeout-executing-grok-and-timeout-millis-not-work/206449)
**Author:** [@Luis\_Enrique\_Gastelu](https://discuss.elastic.co/u/Luis_Enrique_Gastelu)\ **Replies:** 4\ **Last updated:** [November 5, 2019, 12:56am UTC](https://discuss.elastic.co/t/timeout-executing-grok-and-timeout-millis-not-work/206449 "2019-11-05T00:56:04Z")
Hi i try to execute a grok but return text Timeout executing grok '\\\[(?\STUCK)\\\] %{DATA:STUCKMESSAGE}(\\n)\\\[\\n(%{DATA:Method} %{DATA:url} %{DATA:Protocol}\\n)(?\(.|\\n)\*)\\\]\\", (\\n|\\r|.)\*' against field … --- ## [Logstash: File Input does not work](https://discuss.elastic.co/t/logstash-file-input-does-not-work/206476)
**Author:** [@kcb](https://discuss.elastic.co/u/kcb)\ **Replies:** 2\ **Last updated:** [November 4, 2019, 9:20pm UTC](https://discuss.elastic.co/t/logstash-file-input-does-not-work/206476 "2019-11-04T21:20:37Z")
We have written a logstash script to process a simple ASCII file and export contents to stdout{}. We have verified the script works when the input is specified as stdin{} However, we cannot get the script to work when w… --- ## [Delete index before creating it](https://discuss.elastic.co/t/delete-index-before-creating-it/206185)
**Author:** [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Replies:** 1\ **Last updated:** [November 4, 2019, 8:32pm UTC](https://discuss.elastic.co/t/delete-index-before-creating-it/206185 "2019-11-04T20:32:41Z")
I have data that are coming daily. 90% of the data are same as previous day. 5% are new and 5% are gone. there is no way I can find out what is gone and what is new. this are list of dirs from one of our busy system whe… --- ## [Update Timezone](https://discuss.elastic.co/t/update-timezone/206475)
**Author:** [@Jim\_Thunder](https://discuss.elastic.co/u/Jim_Thunder)\ **Replies:** 0\ **Last updated:** [November 4, 2019, 8:20pm UTC](https://discuss.elastic.co/t/update-timezone/206475 "2019-11-04T20:20:52Z")
Hello! I have a @timestamp field that is created from two other fields. That works fine. Kibana also updates the timezone automatically (from what I read). That also works fine. I have a new field HOUR that has been c… --- ## [Logstash: ingest only message on elasticsearch](https://discuss.elastic.co/t/logstash-ingest-only-message-on-elasticsearch/206474)
**Author:** [@brianferrol](https://discuss.elastic.co/u/brianferrol)\ **Replies:** 1\ **Last updated:** [November 4, 2019, 8:28pm UTC](https://discuss.elastic.co/t/logstash-ingest-only-message-on-elasticsearch/206474 "2019-11-04T20:28:11Z")
Hi there! I'm trying to ingest a json file from AWS s3 to ElasticSearch. I've created an index with the mapping I need, but Logstash adds @timestamp , @version and puts the whole json file into message. How can I get o… --- ## [Separated conf file duplicates search](https://discuss.elastic.co/t/separated-conf-file-duplicates-search/206468)
**Author:** [@sud0](https://discuss.elastic.co/u/sud0)\ **Replies:** 2\ **Last updated:** [November 4, 2019, 7:47pm UTC](https://discuss.elastic.co/t/separated-conf-file-duplicates-search/206468 "2019-11-04T19:47:30Z")
Hi. I've created two conf files under /etc/logstash/conf.d. Because I have two businesses in two different countries, I want them to be displayed in two separately indexes. Index name nz-\* Index name au-\* However, … --- ## [Logstashs event parsing](https://discuss.elastic.co/t/logstashs-event-parsing/206206)
**Author:** [@funlearning321](https://discuss.elastic.co/u/funlearning321)\ **Replies:** 3\ **Last updated:** [November 4, 2019, 7:40pm UTC](https://discuss.elastic.co/t/logstashs-event-parsing/206206 "2019-11-04T19:40:01Z")
Hello, i am new to ELK. when i onboarded the below log file, it is going to "dead letter queue" in logstash because logstash couldn't able to process the events. I am not sure which type of plugin to use whether KV plu… --- ## [Collect tabular Jmx data?](https://discuss.elastic.co/t/collect-tabular-jmx-data/206463)
**Author:** [@Visuelastic](https://discuss.elastic.co/u/Visuelastic)\ **Replies:** 0\ **Last updated:** [November 4, 2019, 7:05pm UTC](https://discuss.elastic.co/t/collect-tabular-jmx-data/206463 "2019-11-04T19:05:59Z")
Does somebody know how to collect MBean data from a tabular typed attribute and visualize it into Kibana? --- ## [Basic Math in Logstash](https://discuss.elastic.co/t/basic-math-in-logstash/206452)
**Author:** [@Jim\_Thunder](https://discuss.elastic.co/u/Jim_Thunder)\ **Replies:** 3\ **Last updated:** [November 4, 2019, 6:52pm UTC](https://discuss.elastic.co/t/basic-math-in-logstash/206452 "2019-11-04T18:52:55Z")
All I want to do it subtract 6 from the "hour\_of\_day" field, because it's six hours ahead. Which from my understanding, it shouldn't be ahead b/c I used ruby. But whatever the case, it's wrong. Here's my code: ruby { … --- ## [Logstash stopping without any error](https://discuss.elastic.co/t/logstash-stopping-without-any-error/203602)
**Author:** [@Ayden](https://discuss.elastic.co/u/Ayden)\ **Replies:** 3\ **Last updated:** [November 4, 2019, 5:32pm UTC](https://discuss.elastic.co/t/logstash-stopping-without-any-error/203602 "2019-11-04T17:32:02Z")
Hello, I have two logstash pipelines (in docker containers) running with a jdbc input and elastic output. Both pipelines are set to run with a schedule every 1 minute. My problem is that sometimes the pipelines suddenl… --- ## [NFS issues](https://discuss.elastic.co/t/nfs-issues/206425)
**Author:** [@ansamHox](https://discuss.elastic.co/u/ansamHox)\ **Replies:** 0\ **Last updated:** [November 4, 2019, 2:59pm UTC](https://discuss.elastic.co/t/nfs-issues/206425 "2019-11-04T14:59:29Z")
My database team are exporting some documents couple of times per day and I read them as soon as possible and put them in Elasticsearch. Files (CSV) are mounted by NFS and I can see them on my local drive. Logstash is ru… --- ## [Broken JSON (Rsyslog -\> Logstash) - Message breaking in wrong place](https://discuss.elastic.co/t/broken-json-rsyslog-logstash-message-breaking-in-wrong-place/206424)
**Author:** [@iget-master](https://discuss.elastic.co/u/iget-master)\ **Replies:** 1\ **Last updated:** [November 4, 2019, 2:59pm UTC](https://discuss.elastic.co/t/broken-json-rsyslog-logstash-message-breaking-in-wrong-place/206424 "2019-11-04T14:59:21Z")
Hi guys, I have a setup where my rsyslog log messages are sent to Logstash in JSON format. My template in rsyslog is this: template(name="json-template" type="list") { constant(value="{") constant(value="\\… --- ## [Failed to parse date](https://discuss.elastic.co/t/failed-to-parse-date/206081)
**Author:** [@imaad](https://discuss.elastic.co/u/imaad)\ **Replies:** 4\ **Last updated:** [November 4, 2019, 2:58pm UTC](https://discuss.elastic.co/t/failed-to-parse-date/206081 "2019-11-04T14:58:48Z")
Hello guys, I failed to parse this date 31/Oct/2019:10:52:48 +0100 using the date plugin : date { match =\> \["date", "dd/MMM/YYYY:HH:mm:ss Z"\] } This is the error I got :slight\_smile: Oct 31 12:08:42 centos7 logs… --- ## [Parse json file with 100k/1 million records?](https://discuss.elastic.co/t/parse-json-file-with-100k-1-million-records/205659)
**Author:** [@Abdul\_Gaffar\_Shaikh](https://discuss.elastic.co/u/Abdul_Gaffar_Shaikh)\ **Replies:** 7\ **Last updated:** [November 4, 2019, 1:27pm UTC](https://discuss.elastic.co/t/parse-json-file-with-100k-1-million-records/205659 "2019-11-04T13:27:50Z")
i am trying to parse big json file , here is the sample 5 records in the file with the orignial file containing 100k {"dgdocs": \[{"doc id": "0", "docowner": "0", "documentType": "cte1\_0", "docver": "1", "ogsys": "fc", … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=450) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=452)