# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=674 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 675 --- ## [Grok multiple pattern and tag\_on\_failure for each pattern](https://discuss.elastic.co/t/grok-multiple-pattern-and-tag-on-failure-for-each-pattern/150816)
**Author:** [@sagark](https://discuss.elastic.co/u/sagark)\ **Replies:** 4\ **Last updated:** [October 5, 2018, 5:17pm UTC](https://discuss.elastic.co/t/grok-multiple-pattern-and-tag-on-failure-for-each-pattern/150816 "2018-10-05T17:17:38Z")
Hello, I am trying to use multiple grok pattern under single grok, but when any of the pattern matches, it should not add tag\_on\_failure. Or can I add tag\_on\_failure for each pattern? Configuration is as below: grok { … --- ## [MySQL connection error](https://discuss.elastic.co/t/mysql-connection-error/151199)
**Author:** [@emma1](https://discuss.elastic.co/u/emma1)\ **Replies:** 1\ **Last updated:** [October 5, 2018, 5:11pm UTC](https://discuss.elastic.co/t/mysql-connection-error/151199 "2018-10-05T17:11:54Z")
input { jdbc { jdbc\_driver\_library =\> "ELK/logstash-6.2.3/logstash-6.2.3/bin/mysql-connector-java-5.1.47-bin.jar" jdbc\_driver\_class =\> "com.mysql.jdbc.Driver" jdbc\_connection\_string =\> "jdbc:mysql://localhost:3306/cr… --- ## [Flatten complex json-array-structure (.har-file / http-archive)](https://discuss.elastic.co/t/flatten-complex-json-array-structure-har-file-http-archive/150768)
**Author:** [@cabp](https://discuss.elastic.co/u/cabp)\ **Replies:** 1\ **Last updated:** [October 5, 2018, 4:54pm UTC](https://discuss.elastic.co/t/flatten-complex-json-array-structure-har-file-http-archive/150768 "2018-10-05T16:54:05Z")
I am trying to "flatten" a nested array-structure given in a .HAR-File (httparchive). Incoming JSON (.har-File) { "log": { "version": "1.2", "creator": { "name": "mitmproxy har\_dump", "version": "0.1", "comment":… --- ## [Compiler does not respond](https://discuss.elastic.co/t/compiler-does-not-respond/151124)
**Author:** [@cmahan](https://discuss.elastic.co/u/cmahan)\ **Replies:** 1\ **Last updated:** [October 5, 2018, 4:52pm UTC](https://discuss.elastic.co/t/compiler-does-not-respond/151124 "2018-10-05T16:52:13Z")
bin/logstash -f first-pipeline.conf --config.reload.automatic after this command I am left hanging the compiler is stuck --- ## [LS 6.4.0 sincedb with converted records despite started empty](https://discuss.elastic.co/t/ls-6-4-0-sincedb-with-converted-records-despite-started-empty/151248)
**Author:** [@makibroshett](https://discuss.elastic.co/u/makibroshett)\ **Replies:** 0\ **Last updated:** [October 5, 2018, 4:39pm UTC](https://discuss.elastic.co/t/ls-6-4-0-sincedb-with-converted-records-despite-started-empty/151248 "2018-10-05T16:39:12Z")
Hello, I've started a pipeline in with file input in read mode, and configured sincedb\_clean\_after to 2 hours, and close\_older to 2 minutes (input files are added every minute). Initially the since db was empty. But si… --- ## [Netflow module launch errors, some SSL related(?)](https://discuss.elastic.co/t/netflow-module-launch-errors-some-ssl-related/149596)
**Author:** [@SmFs](https://discuss.elastic.co/u/SmFs)\ **Replies:** 12\ **Last updated:** [October 5, 2018, 4:28pm UTC](https://discuss.elastic.co/t/netflow-module-launch-errors-some-ssl-related/149596 "2018-10-05T16:28:43Z")
Was hoping someone might be able to advise. Running: latest versions of esearch, kibana, and logstash installed via repo plus Oracle Java 1.8.0\_181 on a Ubuntu Server 18.04 VM. I'm trying to set up an ELK stack for ne… --- ## [How to convert a timestamp(6) field into an integer?](https://discuss.elastic.co/t/how-to-convert-a-timestamp-6-field-into-an-integer/150923)
**Author:** [@OphyTe](https://discuss.elastic.co/u/OphyTe)\ **Replies:** 5\ **Last updated:** [October 5, 2018, 3:29pm UTC](https://discuss.elastic.co/t/how-to-convert-a-timestamp-6-field-into-an-integer/150923 "2018-10-05T15:29:57Z")
I get a timestamp(6) from an Oracle database with the input-jdbc plugin. This field is converted into Date when I put it in my elastic instance. But I need to have it in the UNIX\_MS format (= number of milliseconds since… --- ## [Logstash tcp input with tls stops responding](https://discuss.elastic.co/t/logstash-tcp-input-with-tls-stops-responding/150737)
**Author:** [@jgzurano](https://discuss.elastic.co/u/jgzurano)\ **Replies:** 1\ **Last updated:** [October 5, 2018, 3:10pm UTC](https://discuss.elastic.co/t/logstash-tcp-input-with-tls-stops-responding/150737 "2018-10-05T15:10:56Z")
I have fluentd 1.0 sending events to logstash 6.2.3 containers using tls. After a while (could be a few minutes or hours) logstash stops responding to some requests on tcp port 4000 then it stops responding completely. … --- ## [Custom filter Grok syntax help](https://discuss.elastic.co/t/custom-filter-grok-syntax-help/150388)
**Author:** [@hueyg](https://discuss.elastic.co/u/hueyg)\ **Replies:** 2\ **Last updated:** [October 5, 2018, 2:59pm UTC](https://discuss.elastic.co/t/custom-filter-grok-syntax-help/150388 "2018-10-05T14:59:02Z")
Most tutorials simply used the built in filters and leave it at that, but I am trying to parse an LDAP log and running into a few issues. The following post almost matches my issue, but need more clarification in genera… --- ## [Is there a better way to convert long to boolean](https://discuss.elastic.co/t/is-there-a-better-way-to-convert-long-to-boolean/151231)
**Author:** [@OphyTe](https://discuss.elastic.co/u/OphyTe)\ **Replies:** 0\ **Last updated:** [October 5, 2018, 2:55pm UTC](https://discuss.elastic.co/t/is-there-a-better-way-to-convert-long-to-boolean/151231 "2018-10-05T14:55:56Z")
I have a field where values are stored as long (0 for false and 1 for true). I wanted to store them as booleans but the boolean convert filter doesn't accept other type than string. The only way I found to do this is to… --- ## [Logstash Process Hangs - No Logging](https://discuss.elastic.co/t/logstash-process-hangs-no-logging/149136)
**Author:** [@AquaX](https://discuss.elastic.co/u/AquaX)\ **Replies:** 3\ **Last updated:** [October 5, 2018, 2:19pm UTC](https://discuss.elastic.co/t/logstash-process-hangs-no-logging/149136 "2018-10-05T14:19:16Z")
Hello everyone. I've got the strangest problem. I'm using multiple logstash pipelines to monitor different folders for log file input. The idea is to have a support person be able to drop a file in a website and have a… --- ## [Common filter(s) for multiple pipelines?](https://discuss.elastic.co/t/common-filter-s-for-multiple-pipelines/150889)
**Author:** [@boots18](https://discuss.elastic.co/u/boots18)\ **Replies:** 2\ **Last updated:** [October 5, 2018, 2:06pm UTC](https://discuss.elastic.co/t/common-filter-s-for-multiple-pipelines/150889 "2018-10-05T14:06:52Z")
I have multiple pipelines feeding into logstash (metricbeat, filebeat, winlogbeat, etc.) and each of them share some common filters. I am wondering if there is a way to have some filters in a location that each of the pi… --- ## [Extract domain from field](https://discuss.elastic.co/t/extract-domain-from-field/150541)
**Author:** [@elk2](https://discuss.elastic.co/u/elk2)\ **Replies:** 3\ **Last updated:** [October 5, 2018, 1:34pm UTC](https://discuss.elastic.co/t/extract-domain-from-field/150541 "2018-10-05T13:34:34Z")
I want to extract domain from email field that is like this: smtp.from\_mail : "news \" I would like to use grok or other useful plugin. --- ## [Import few data fields or complete fields from SQL?](https://discuss.elastic.co/t/import-few-data-fields-or-complete-fields-from-sql/151221)
**Author:** [@kumar3](https://discuss.elastic.co/u/kumar3)\ **Replies:** 0\ **Last updated:** [October 5, 2018, 1:38pm UTC](https://discuss.elastic.co/t/import-few-data-fields-or-complete-fields-from-sql/151221 "2018-10-05T13:38:35Z")
Hello, My SQL database has 100 data columns. Is it a good manner to import just few fields ( 5 approx ) for just the Search bar implementation or dump all data from SQL database to also use all related info for specific… --- ## ["LogStash::ConfigurationError"](https://discuss.elastic.co/t/logstash-configurationerror/151204)
**Author:** [@emma1](https://discuss.elastic.co/u/emma1)\ **Replies:** 1\ **Last updated:** [October 5, 2018, 1:31pm UTC](https://discuss.elastic.co/t/logstash-configurationerror/151204 "2018-10-05T13:31:28Z")
Here is my config file of logstash for sending data from MYSQL to Kibana input { jdbc { sincedb\_path =\> "/dev/null" ignore\_older =\> 0 jdbc\_driver\_library =\> "ELK/logstash-6.2.3/logstash-6.2.3/bin/mysql-connector-jav… --- ## [Improving logstash data resiliency](https://discuss.elastic.co/t/improving-logstash-data-resiliency/151217)
**Author:** [@NicoMoll](https://discuss.elastic.co/u/NicoMoll)\ **Replies:** 0\ **Last updated:** [October 5, 2018, 1:22pm UTC](https://discuss.elastic.co/t/improving-logstash-data-resiliency/151217 "2018-10-05T13:22:31Z")
The documentation on Logstash about persisten queues says: It does not handle permanent machine failures such as disk corruption, disk failure, and machine loss. The data persisted to disk is not replicated. So what … --- ## [Enable Logstash to send date as epoch in milliseconds](https://discuss.elastic.co/t/enable-logstash-to-send-date-as-epoch-in-milliseconds/151211)
**Author:** [@EricPSU](https://discuss.elastic.co/u/EricPSU)\ **Replies:** 0\ **Last updated:** [October 5, 2018, 1:05pm UTC](https://discuss.elastic.co/t/enable-logstash-to-send-date-as-epoch-in-milliseconds/151211 "2018-10-05T13:05:53Z")
I'm looking for the best way to have Logstash send dates as epoch in milliseconds. My Elasticsearch index has the field mapping below. "createdDate": { "type": "date", "format": "epoch\_millis" } If I post a do… --- ## [Help with gsub in mutate heading chars](https://discuss.elastic.co/t/help-with-gsub-in-mutate-heading-chars/151201)
**Author:** [@Anabella\_Cristaldi](https://discuss.elastic.co/u/Anabella_Cristaldi)\ **Replies:** 1\ **Last updated:** [October 5, 2018, 12:26pm UTC](https://discuss.elastic.co/t/help-with-gsub-in-mutate-heading-chars/151201 "2018-10-05T12:26:06Z")
Hi, I need to substract the headding 00 in some fields. I have tried something like mutate { gsub =\> \[ "from", "\[\\A00\\d+\]", "??????" \] } what do I have to put in the ??? in order … --- ## [Logstash configuration error logstash-6.4.1](https://discuss.elastic.co/t/logstash-configuration-error-logstash-6-4-1/151207)
**Author:** [@Nethaji](https://discuss.elastic.co/u/Nethaji)\ **Replies:** 1\ **Last updated:** [October 5, 2018, 12:05pm UTC](https://discuss.elastic.co/t/logstash-configuration-error-logstash-6-4-1/151207 "2018-10-05T12:05:14Z")
D:\\ELK\\logstash-6.4.1\\bin\>logstash -f D:\\ELK\\logstash-6.4.1\\config\\sample.conf i'm trying to run the above command it gives error like this, Sending Logstash logs to D:/ELK/logstash-6.4.1/logs which is now configured v… --- ## [Logstash service is running but the port is not responding](https://discuss.elastic.co/t/logstash-service-is-running-but-the-port-is-not-responding/150971)
**Author:** [@erkant](https://discuss.elastic.co/u/erkant)\ **Replies:** 2\ **Last updated:** [October 5, 2018, 11:29am UTC](https://discuss.elastic.co/t/logstash-service-is-running-but-the-port-is-not-responding/150971 "2018-10-05T11:29:32Z")
hi, when starting filebeat connect failed logstash. why not runnig port(5044)? Is the logstash running manually? --- ## [Sample project](https://discuss.elastic.co/t/sample-project/147945)
**Author:** [@Sripal](https://discuss.elastic.co/u/Sripal)\ **Replies:** 6\ **Last updated:** [October 5, 2018, 10:57am UTC](https://discuss.elastic.co/t/sample-project/147945 "2018-10-05T10:57:50Z")
Hi All, I am unable to add any json file into my elasticsearch. kindly help me no datas are reflecting to kibana. my logstash config file: input{ file{ path =\> "D:\\ELK-6.4\\logs.jsonl\\FL\_insurance\_sample.csv" start… --- ## [Document ID from logs](https://discuss.elastic.co/t/document-id-from-logs/151195)
**Author:** [@\_Derk](https://discuss.elastic.co/u/_Derk)\ **Replies:** 0\ **Last updated:** [October 5, 2018, 10:44am UTC](https://discuss.elastic.co/t/document-id-from-logs/151195 "2018-10-05T10:44:02Z")
I'm using the REST query API to retrieve certain logs from Logstash. For representation I just want to simply present the requested data as a (perma)link to the results on Kibana itself. The 'View single document' guides… --- ## [Logstash with error of read\_to\_eof](https://discuss.elastic.co/t/logstash-with-error-of-read-to-eof/150302)
**Author:** [@Chunbo\_Liao](https://discuss.elastic.co/u/Chunbo_Liao)\ **Replies:** 1\ **Last updated:** [October 5, 2018, 9:51am UTC](https://discuss.elastic.co/t/logstash-with-error-of-read-to-eof/150302 "2018-10-05T09:51:23Z")
Hi, When I run logstash 6.4.1, there are lots of follow error logs. \[ERROR\]\[filewatch.tailmode.handlers.grow\] read\_to\_eof: general error reading /home/11thone/caselog/logmonster/CAS-HUNAN-TEST/MME01/MMDU0/SubsHandlerSn… --- ## [Date Filter Doesn't Work](https://discuss.elastic.co/t/date-filter-doesnt-work/150689)
**Author:** [@stevetso](https://discuss.elastic.co/u/stevetso)\ **Replies:** 5\ **Last updated:** [October 5, 2018, 9:37am UTC](https://discuss.elastic.co/t/date-filter-doesnt-work/150689 "2018-10-05T09:37:52Z")
I have a message like this: 14/Jul/18 15:18:55 44.130.128.195 GET /url111 200 853 3 - - - Filter: filter { grok { match =\> { "message" =\> "%{MONTHDAY:day}/%{MONTH:month}/%{YEAR:year} %{TIME:time} %{IP:client\_ip} %{W… --- ## [Getting this error while installing Logstash in Windows](https://discuss.elastic.co/t/getting-this-error-while-installing-logstash-in-windows/151174)
**Author:** [@Tilak](https://discuss.elastic.co/u/Tilak)\ **Replies:** 0\ **Last updated:** [October 5, 2018, 8:49am UTC](https://discuss.elastic.co/t/getting-this-error-while-installing-logstash-in-windows/151174 "2018-10-05T08:49:58Z")
PS F:\\ELK\\logstash-6.2.4\\logstash-6.2.4\\bin\> .\\logstash.bat -f F:\\ELK\\logstash-6.2.4\\logstash-6.2.4\\config\\logstash-test.conf Sending Logstash's logs to F:/ELK/logstash-6.2.4/logstash-6.2.4/logs which is now configured … --- ## [Logstash elasticsearch output plugin upsert without document\_id](https://discuss.elastic.co/t/logstash-elasticsearch-output-plugin-upsert-without-document-id/151151)
**Author:** [@lasitha1991](https://discuss.elastic.co/u/lasitha1991)\ **Replies:** 1\ **Last updated:** [October 5, 2018, 8:25am UTC](https://discuss.elastic.co/t/logstash-elasticsearch-output-plugin-upsert-without-document-id/151151 "2018-10-05T08:25:04Z")
Hi, I want to upsert bunch of records to elasticsearch using logstash elasticsearch output plugin. But I can't use the document\_id for this purpose as I want to use \_update\_by\_query option. Eg: my document id is created… --- ## [Cisco ASA and logstash problem](https://discuss.elastic.co/t/cisco-asa-and-logstash-problem/150706)
**Author:** [@sles](https://discuss.elastic.co/u/sles)\ **Replies:** 5\ **Last updated:** [October 5, 2018, 6:16am UTC](https://discuss.elastic.co/t/cisco-asa-and-logstash-problem/150706 "2018-10-05T06:16:05Z")
Hello! I need to process logs from ASA, let's says this: Oct 2 09:22:41 192.168.42.129 %ASA-3-202010: NAT pool exhausted. Unable to create UDP connection from inside:192.168.222.24/47423 to outside:195.210.189.106/123 … --- ## ["query" that combines range and term in ElasticSearch plugin in Logstash input](https://discuss.elastic.co/t/query-that-combines-range-and-term-in-elasticsearch-plugin-in-logstash-input/151057)
**Author:** [@Dolph\_2709](https://discuss.elastic.co/u/Dolph_2709)\ **Replies:** 1\ **Last updated:** [October 4, 2018, 7:55pm UTC](https://discuss.elastic.co/t/query-that-combines-range-and-term-in-elasticsearch-plugin-in-logstash-input/151057 "2018-10-04T19:55:45Z")
Hello, I need a help with "query" syntax in Elasticsearch plugin. Currently my input looks like this (and it works OK): input { elasticsearch { hosts =\> \["1x.1x.1xx.1xx:9200"\] index =\> "worklight" scroll =\> "10m… --- ## [Left hanging when I'm trying to forward logs from s3](https://discuss.elastic.co/t/left-hanging-when-im-trying-to-forward-logs-from-s3/151117)
**Author:** [@cmahan](https://discuss.elastic.co/u/cmahan)\ **Replies:** 0\ **Last updated:** [October 4, 2018, 7:54pm UTC](https://discuss.elastic.co/t/left-hanging-when-im-trying-to-forward-logs-from-s3/151117 "2018-10-04T19:54:44Z")
My cursor is left hanging after I run the command bin/logstash -f first-pipeline.conf --config.reload.automatic. The console says that logstash server is started but no out-put about the logs or source. I tried to run… --- ## [Logstash running only localhost url](https://discuss.elastic.co/t/logstash-running-only-localhost-url/151102)
**Author:** [@Rahul\_Yadav](https://discuss.elastic.co/u/Rahul_Yadav)\ **Replies:** 1\ **Last updated:** [October 4, 2018, 7:34pm UTC](https://discuss.elastic.co/t/logstash-running-only-localhost-url/151102 "2018-10-04T19:34:59Z")
I have installed logstash, Kibana, elasticseach, metricbeat. How can I configure and static VM IP for logstash because bydefaul logstash running on localhost I want to use static IP instead of localhost, I want to ship… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=673) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=675)