# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=827 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 828 --- ## [Read path from another config file in Logstash configuration file](https://discuss.elastic.co/t/read-path-from-another-config-file-in-logstash-configuration-file/118439)
**Author:** [@RamyaGowda](https://discuss.elastic.co/u/RamyaGowda)\ **Replies:** 4\ **Last updated:** [February 7, 2018, 4:44am UTC](https://discuss.elastic.co/t/read-path-from-another-config-file-in-logstash-configuration-file/118439 "2018-02-07T04:44:25Z")
Hi I have a requirement that ,in input section i have to read file path location from another config file ,so that i can change the file location dynamically from the config file instead of going into logstash config fi… --- ## [Logstash CLOSE\_WAIT and Filbe beat I/0 timeout](https://discuss.elastic.co/t/logstash-close-wait-and-filbe-beat-i-0-timeout/118650)
**Author:** [@oraclept](https://discuss.elastic.co/u/oraclept)\ **Replies:** 2\ **Last updated:** [February 6, 2018, 10:30pm UTC](https://discuss.elastic.co/t/logstash-close-wait-and-filbe-beat-i-0-timeout/118650 "2018-02-06T22:30:11Z")
In Filebeat logs I am seeing Filebeat I/O timeout with logstash tcp connection going infinite CLOSE\_WAIT. until logstash stops processing. Dont see anything in logstashlogs. Logstash and filebeat both are 5.4 version.… --- ## [Grok expression for name value pairs](https://discuss.elastic.co/t/grok-expression-for-name-value-pairs/118703)
**Author:** [@Dushyanth\_Bn](https://discuss.elastic.co/u/Dushyanth_Bn)\ **Replies:** 1\ **Last updated:** [February 6, 2018, 9:39pm UTC](https://discuss.elastic.co/t/grok-expression-for-name-value-pairs/118703 "2018-02-06T21:39:19Z")
06 Feb 2018 13:11:24,640 INFO SaveMetrics:39 - Metrics :httpsessions.max -1 httpsessions.active 0 datasource.primary.active 0 datasource.primary.usage 0.0 datasource.hcarsessioninfo.active 0 datasource.hcarsessioninfo… --- ## [Logstash doesn't communicate with Elasticsearch](https://discuss.elastic.co/t/logstash-doesnt-communicate-with-elasticsearch/118447)
**Author:** [@hzarrabi](https://discuss.elastic.co/u/hzarrabi)\ **Replies:** 5\ **Last updated:** [February 6, 2018, 9:30pm UTC](https://discuss.elastic.co/t/logstash-doesnt-communicate-with-elasticsearch/118447 "2018-02-06T21:30:19Z")
Hi, "I'm novice in Elastic Stack !!! " My Elasticsearch and Kibana on Linux Ubuntu working well. I receive log from beats but my logstash seems to do not communicate with elasticsearch. Logstash on my server shoud rea… --- ## ["\_dateparsefailure" of UNIX time](https://discuss.elastic.co/t/dateparsefailure-of-unix-time/118683)
**Author:** [@ssasporta](https://discuss.elastic.co/u/ssasporta)\ **Replies:** 1\ **Last updated:** [February 6, 2018, 4:54pm UTC](https://discuss.elastic.co/t/dateparsefailure-of-unix-time/118683 "2018-02-06T16:54:32Z")
Hi, The input events from filebeat looks like that: {"@timestamp":"2018-02-06T16:12:59.892Z","beat":{"hostname":"velastic","name":"velastic","version":"5.4.1"},"fields":{"fields\_under\_root":true,"type":"uxform"},"input… --- ## [Index is not creating through logstash](https://discuss.elastic.co/t/index-is-not-creating-through-logstash/115742)
**Author:** [@uvarisubash](https://discuss.elastic.co/u/uvarisubash)\ **Replies:** 1\ **Last updated:** [February 6, 2018, 8:01pm UTC](https://discuss.elastic.co/t/index-is-not-creating-through-logstash/115742 "2018-02-06T20:01:08Z")
Hi , the index is not creating while doing through below command /usr/share/logstash/bin/logstash --path.settings=/etc/logstash/ --log.level=debug -f gesyslog.conf --path.data /usr/share/logstash/data1 the below er… --- ## [I dont want to restart plugin for logstash in case of unrecoverable error](https://discuss.elastic.co/t/i-dont-want-to-restart-plugin-for-logstash-in-case-of-unrecoverable-error/113761)
**Author:** [@Dimitry\_Khan](https://discuss.elastic.co/u/Dimitry_Khan)\ **Replies:** 2\ **Last updated:** [February 6, 2018, 7:39pm UTC](https://discuss.elastic.co/t/i-dont-want-to-restart-plugin-for-logstash-in-case-of-unrecoverable-error/113761 "2018-02-06T19:39:50Z")
I am processing events from elasticsearch with size =200. env\_thread\_id is just an env var. input { elasticsearch { hosts =\> "localhost:9200" index =\> "flexnet-perf-\*" query =\> '{"from":0,"size":200,"\_source":\["mess… --- ## [Match all Random IP Addresses](https://discuss.elastic.co/t/match-all-random-ip-addresses/118648)
**Author:** [@cjvermeulen](https://discuss.elastic.co/u/cjvermeulen)\ **Replies:** 3\ **Last updated:** [February 6, 2018, 6:11pm UTC](https://discuss.elastic.co/t/match-all-random-ip-addresses/118648 "2018-02-06T18:11:50Z")
Sorry if this a total rookie question but.. Scenario: I have random multiline log messages that have anywhere from 1 to 100 IP addresses in them. The IP's are sporadic and don't always follow a single pattern. Ie. They … --- ## [Import data from mysql to elasticsearch using logstash ,and the subobject are duplicate?](https://discuss.elastic.co/t/import-data-from-mysql-to-elasticsearch-using-logstash-and-the-subobject-are-duplicate/118698)
**Author:** [@wangbin](https://discuss.elastic.co/u/wangbin)\ **Replies:** 0\ **Last updated:** [February 6, 2018, 6:08pm UTC](https://discuss.elastic.co/t/import-data-from-mysql-to-elasticsearch-using-logstash-and-the-subobject-are-duplicate/118698 "2018-02-06T18:08:00Z")
This is the case: I want to import data use logstash from mysql to elasticsearch,the subobject data is duplicated.How can I filter the data? This is the error data in elasticsearch: error json: \[{ "\_index": "user",… --- ## [Logstash Configuration Issues](https://discuss.elastic.co/t/logstash-configuration-issues/116062)
**Author:** [@Bheeshma](https://discuss.elastic.co/u/Bheeshma)\ **Replies:** 2\ **Last updated:** [February 6, 2018, 4:28pm UTC](https://discuss.elastic.co/t/logstash-configuration-issues/116062 "2018-02-06T16:28:11Z")
Hi, I am new to this forum, trying to learn the ELK stack on my own . Thanks to ELK documentation on the website uploaded. i was able to install and configure Elastic Search, Kibana softwares however when trying to con… --- ## [Where can i get the logstash-codec-sflow plugin](https://discuss.elastic.co/t/where-can-i-get-the-logstash-codec-sflow-plugin/118520)
**Author:** [@bossi6of9](https://discuss.elastic.co/u/bossi6of9)\ **Replies:** 2\ **Last updated:** [February 6, 2018, 3:48pm UTC](https://discuss.elastic.co/t/where-can-i-get-the-logstash-codec-sflow-plugin/118520 "2018-02-06T15:48:34Z")
Hi, I need the logstash-codec-sflow but can't find it. My lab server doesn't have internet access, so I need to download it directly. Can someone point me to it? --- ## [Can’t start logstash after x-pack installation](https://discuss.elastic.co/t/can-t-start-logstash-after-x-pack-installation/115353)
**Author:** [@GSCully](https://discuss.elastic.co/u/GSCully)\ **Replies:** 17\ **Last updated:** [February 6, 2018, 3:43pm UTC](https://discuss.elastic.co/t/can-t-start-logstash-after-x-pack-installation/115353 "2018-02-06T15:43:26Z")
I have this issue now as well, I had logstash running and connecting without issue, after installing x-pack it is now looking at localhost for elastic which doesn't exist. Elastic 6.1.1, logstash 6.1.1 Windows 2012R2. … --- ## [Startup fails if path.data is a symlink](https://discuss.elastic.co/t/startup-fails-if-path-data-is-a-symlink/118669)
**Author:** [@jmb](https://discuss.elastic.co/u/jmb)\ **Replies:** 0\ **Last updated:** [February 6, 2018, 3:24pm UTC](https://discuss.elastic.co/t/startup-fails-if-path-data-is-a-symlink/118669 "2018-02-06T15:24:07Z")
There's a comment already in FileLockFactory.obtainLock() that the path traversal/create will fail in this case: Symptom in logs: \[2018-02-06T09:58:02,982\]\[FATAL\]\[logstash.runner \] An unexpected error occurr… --- ## [Parse Maven XML .pom files](https://discuss.elastic.co/t/parse-maven-xml-pom-files/116224)
**Author:** [@svdh](https://discuss.elastic.co/u/svdh)\ **Replies:** 3\ **Last updated:** [February 6, 2018, 1:49pm UTC](https://discuss.elastic.co/t/parse-maven-xml-pom-files/116224 "2018-02-06T13:49:57Z")
Hi all, I would like to create a logstash filter to get the project details out of the XML file. For most of the tags, this isn't a problem, because the tags have all the same structure like: \ \apac… --- ## [Unable to setup SSL connection from Filebeat to Logstash](https://discuss.elastic.co/t/unable-to-setup-ssl-connection-from-filebeat-to-logstash/118655)
**Author:** [@charan.gandra](https://discuss.elastic.co/u/charan.gandra)\ **Replies:** 0\ **Last updated:** [February 6, 2018, 1:44pm UTC](https://discuss.elastic.co/t/unable-to-setup-ssl-connection-from-filebeat-to-logstash/118655 "2018-02-06T13:44:39Z")
I've generated the necessary certificates on Logstash and copied the public key to Filebeat. However SSL connection is getting failed with the below error. Any help is greatly appreciated. Filebeat: 2018-02-06T13:24:3… --- ## [Can't see \_grokparsefailure](https://discuss.elastic.co/t/cant-see-grokparsefailure/118645)
**Author:** [@samiujan](https://discuss.elastic.co/u/samiujan)\ **Replies:** 0\ **Last updated:** [February 6, 2018, 12:37pm UTC](https://discuss.elastic.co/t/cant-see-grokparsefailure/118645 "2018-02-06T12:37:08Z")
I am running Logstash 6.1.3, Elasticsearch 6.1.3 and Filebeat 6.1.2 I have a grok pattern that looks like this (notice the fail text in the middle) grok { match =\> { "message" =\> \[ "^%{IPORHOST:clientip} \*… --- ## [How to parse the logs from docker to logstash](https://discuss.elastic.co/t/how-to-parse-the-logs-from-docker-to-logstash/118229)
**Author:** [@Abdur\_Raqeeb1](https://discuss.elastic.co/u/Abdur_Raqeeb1)\ **Replies:** 1\ **Last updated:** [February 6, 2018, 12:16pm UTC](https://discuss.elastic.co/t/how-to-parse-the-logs-from-docker-to-logstash/118229 "2018-02-06T12:16:14Z")
Need to parse the below logs from docker.Please check and let me know how to parse this below logs to my logstash via logstash forwarder ubuntu@host:~$ docker logs -f smap-archiver 2018-01-31 16:40:25+0530 \[-\] "IP" - … --- ## [Change default date format via Logstash to store in ES](https://discuss.elastic.co/t/change-default-date-format-via-logstash-to-store-in-es/118612)
**Author:** [@moriol](https://discuss.elastic.co/u/moriol)\ **Replies:** 1\ **Last updated:** [February 6, 2018, 10:42am UTC](https://discuss.elastic.co/t/change-default-date-format-via-logstash-to-store-in-es/118612 "2018-02-06T10:42:45Z")
I have some data being ingested via Logstash where a specific date field is stored as below in ES: "MyDate": "2018-01-17T10:11:59.982Z I would like to store this date in ES , as a date field in format as follows: "MyD… --- ## [Logstash recreating deleted Index in ES](https://discuss.elastic.co/t/logstash-recreating-deleted-index-in-es/118453)
**Author:** [@Manasa](https://discuss.elastic.co/u/Manasa)\ **Replies:** 2\ **Last updated:** [February 6, 2018, 10:12am UTC](https://discuss.elastic.co/t/logstash-recreating-deleted-index-in-es/118453 "2018-02-06T10:12:24Z")
I am pushing logstash output to ES and creating indices based on date and doc. I tried deleting one of today's index from ES using XDELETE and it got deleted. But it is getting recreated and all of today's data is also … --- ## [Ingesting Mainframe logs](https://discuss.elastic.co/t/ingesting-mainframe-logs/115087)
**Author:** [@Sajeew.Ganesh](https://discuss.elastic.co/u/Sajeew.Ganesh)\ **Replies:** 4\ **Last updated:** [February 6, 2018, 9:31am UTC](https://discuss.elastic.co/t/ingesting-mainframe-logs/115087 "2018-02-06T09:31:35Z")
Hi, Im looking to work on mainframe logs. i am looking for help. can anyone assist how to ingest mainframe logs to logstash. The sample logs are as below. N 0000000 T158 2017208 23:59:00.24 00000281 IEF1… --- ## [Custom patterns using patterns\_dir is not working](https://discuss.elastic.co/t/custom-patterns-using-patterns-dir-is-not-working/118572)
**Author:** [@ranganath\_nangineni](https://discuss.elastic.co/u/ranganath_nangineni)\ **Replies:** 2\ **Last updated:** [February 6, 2018, 8:52am UTC](https://discuss.elastic.co/t/custom-patterns-using-patterns-dir-is-not-working/118572 "2018-02-06T08:52:42Z")
I have the below custom patterns under /usr/share/logstash/patterns and /etc/logstash/patterns directories. These were created to accept HTTPDUSER in the COMBINEDPATCHELOG pattern. The pattern is as below USERNAME \[a-z… --- ## [Logstash Issue while runnig](https://discuss.elastic.co/t/logstash-issue-while-runnig/118198)
**Author:** [@saineshwar\_bageri](https://discuss.elastic.co/u/saineshwar_bageri)\ **Replies:** 6\ **Last updated:** [February 6, 2018, 8:36am UTC](https://discuss.elastic.co/t/logstash-issue-while-runnig/118198 "2018-02-06T08:36:24Z")
NameError: cannot link Java class org.logstash.RubyUtil org/logstash/RubyUtil : Unsupported major.minor version 52.0 method\_missing at org/jruby/javasupport/JavaPackage.java:259 \ at C:/logstash/logstash-cor… --- ## [Running a single pipeline in logstash](https://discuss.elastic.co/t/running-a-single-pipeline-in-logstash/118571)
**Author:** [@seanziee](https://discuss.elastic.co/u/seanziee)\ **Replies:** 0\ **Last updated:** [February 6, 2018, 7:23am UTC](https://discuss.elastic.co/t/running-a-single-pipeline-in-logstash/118571 "2018-02-06T07:23:28Z")
It would be useful in the new documentation for logstash pipelines if something was clarified. I'm not used to the whole pipeline logic. Where do I put the .conf file?! Some documentation says to put the .conf in /usr/sh… --- ## [Which version of logstash support for multiple which version of logstash can support multiple pipline?](https://discuss.elastic.co/t/which-version-of-logstash-support-for-multiple-which-version-of-logstash-can-support-multiple-pipline/118421)
**Author:** [@Robin\_Guo](https://discuss.elastic.co/u/Robin_Guo)\ **Replies:** 3\ **Last updated:** [February 6, 2018, 6:40am UTC](https://discuss.elastic.co/t/which-version-of-logstash-support-for-multiple-which-version-of-logstash-can-support-multiple-pipline/118421 "2018-02-06T06:40:18Z")
Hi ES team, I wonder know , which version of Logstash can support multiple pipline within a group of logstash server by now? Currently we used few beats on monitored side ,whatever metricbeat/filebeat/heartbeat or the… --- ## [Logstash messages to stride](https://discuss.elastic.co/t/logstash-messages-to-stride/116966)
**Author:** [@babeesh](https://discuss.elastic.co/u/babeesh)\ **Replies:** 3\ **Last updated:** [February 6, 2018, 5:52am UTC](https://discuss.elastic.co/t/logstash-messages-to-stride/116966 "2018-02-06T05:52:10Z")
Is it possible to redirect logs from ELK logserver to Atlassian Stride? Currently I am redirecting error logs to hipchat using plugin 'hipchat' in logstash configuration file. I need those error messages in Stride. But… --- ## [Help with logstash/netflow module](https://discuss.elastic.co/t/help-with-logstash-netflow-module/118515)
**Author:** [@bossi6of9](https://discuss.elastic.co/u/bossi6of9)\ **Replies:** 1\ **Last updated:** [February 6, 2018, 5:27am UTC](https://discuss.elastic.co/t/help-with-logstash-netflow-module/118515 "2018-02-06T05:27:17Z")
Hi, I installed/started the netflow module for logstash, but it runs in the foreground. When I broke out of it, my data stopped indexing. How can I start this module and make it run in the background automatically? --- ## [Logstash log files not in /var/log/logstash](https://discuss.elastic.co/t/logstash-log-files-not-in-var-log-logstash/118383)
**Author:** [@inhinyera16](https://discuss.elastic.co/u/inhinyera16)\ **Replies:** 2\ **Last updated:** [February 6, 2018, 2:59am UTC](https://discuss.elastic.co/t/logstash-log-files-not-in-var-log-logstash/118383 "2018-02-06T02:59:53Z")
Hi, Logstash put the logs in /usr/share/logstash/logs. How can we change it to put the logs in /var/logstash instead? Here's my yml config.reload.automatic: true path.config: /etc/logstash/conf.d/\*.conf path.data: /va… --- ## [Logstash -\> ssl-tcp -\> syslog-ng: Invalid frame header; header=''](https://discuss.elastic.co/t/logstash-ssl-tcp-syslog-ng-invalid-frame-header-header/118088)
**Author:** [@aogail](https://discuss.elastic.co/u/aogail)\ **Replies:** 2\ **Last updated:** [February 5, 2018, 11:47pm UTC](https://discuss.elastic.co/t/logstash-ssl-tcp-syslog-ng-invalid-frame-header-header/118088 "2018-02-05T23:47:44Z")
Hello, I am attempting to send log messages from logstash to an upstream syslog-ng server using the ssl-tcp protocol. The syslog-ng server is configured to receive messages using: syslog( port(6514) transport("tls"… --- ## [JSON parse failure](https://discuss.elastic.co/t/json-parse-failure/118356)
**Author:** [@Shyam\_Burla](https://discuss.elastic.co/u/Shyam_Burla)\ **Replies:** 2\ **Last updated:** [February 5, 2018, 10:56pm UTC](https://discuss.elastic.co/t/json-parse-failure/118356 "2018-02-05T22:56:55Z")
I trying to parse log message which is below.. { "records": \[ { "ActivityId": "XXX", "EventName": "XXX", "resourceId": "XXXX", "SubscriptionId": "XXXXX", "EventTimeString": "XXXXX", "EventPropertie… --- ## [Documentation Recommendation for Logstash Heap](https://discuss.elastic.co/t/documentation-recommendation-for-logstash-heap/118490)
**Author:** [@neu5ron](https://discuss.elastic.co/u/neu5ron)\ **Replies:** 2\ **Last updated:** [February 5, 2018, 10:40pm UTC](https://discuss.elastic.co/t/documentation-recommendation-for-logstash-heap/118490 "2018-02-05T22:40:14Z")
I believe in the 6.x version (compared to 5.x) for Logstash that when you use the memory queue it no longer just uses the available memory of the OS but rather what you have predefined in $LogstashDir/jvm.options Is it … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=826) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=828)