# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=707 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 708 --- ## [Input files already exist in folder \[solved\]](https://discuss.elastic.co/t/input-files-already-exist-in-folder-solved/143123)
**Author:** [@kimfut](https://discuss.elastic.co/u/kimfut)\ **Replies:** 6\ **Last updated:** [August 6, 2018, 11:53am UTC](https://discuss.elastic.co/t/input-files-already-exist-in-folder-solved/143123 "2018-08-06T11:53:30Z")
hello everyone , my problem (it's not a real problem) is when i for example modify logstash config to creat a new index pattern and this new index it's concern files that already contained into certain folder before con… --- ## [Logstash-filter-for multiple log formats](https://discuss.elastic.co/t/logstash-filter-for-multiple-log-formats/143145)
**Author:** [@dinesh1](https://discuss.elastic.co/u/dinesh1)\ **Replies:** 3\ **Last updated:** [August 6, 2018, 11:26am UTC](https://discuss.elastic.co/t/logstash-filter-for-multiple-log-formats/143145 "2018-08-06T11:26:31Z")
Hi, How to parse logs using filters in logstash, so that i will get fields in kibana Examples of my logs shown below 2018-08-06 00:00:03,828 \[ Logging.java:http-nio-9093-exec-141:50\] - Request "{"request":… --- ## [Xpath does not work, but it does in xmlspy](https://discuss.elastic.co/t/xpath-does-not-work-but-it-does-in-xmlspy/142918)
**Author:** [@JPelastic](https://discuss.elastic.co/u/JPelastic)\ **Replies:** 13\ **Last updated:** [August 6, 2018, 11:04am UTC](https://discuss.elastic.co/t/xpath-does-not-work-but-it-does-in-xmlspy/142918 "2018-08-06T11:04:09Z")
I have a wellformed XML in my DB-column. I get it out the DB with logstasj JDBC plugin. I then get an XML with escapecharacters in. In the XML filter I try to parse some stuff from it with an xpath. the xpath works in … --- ## [Couldn't find any filter plugin named 'uuid'](https://discuss.elastic.co/t/couldnt-find-any-filter-plugin-named-uuid/143144)
**Author:** [@Maheen\_Nasir](https://discuss.elastic.co/u/Maheen_Nasir)\ **Replies:** 1\ **Last updated:** [August 6, 2018, 11:03am UTC](https://discuss.elastic.co/t/couldnt-find-any-filter-plugin-named-uuid/143144 "2018-08-06T11:03:24Z")
I have upgraded my logstash from 5.5 version to 6.3 but now my configurations are not working. UUID plugin has stopped working. I am getting following errors: \[2018-08-06T15:34:12,513\]\[ERROR\]\[logstash.agent \]… --- ## [Errors After Filebeat Upgrade to 6.3.1](https://discuss.elastic.co/t/errors-after-filebeat-upgrade-to-6-3-1/142568)
**Author:** [@Or\_Arnon](https://discuss.elastic.co/u/Or_Arnon)\ **Replies:** 2\ **Last updated:** [August 6, 2018, 10:17am UTC](https://discuss.elastic.co/t/errors-after-filebeat-upgrade-to-6-3-1/142568 "2018-08-06T10:17:25Z")
Hi, After upgrading our Filebeat to 6.3.1, we get error on our Logstash servers that consume these logs from Kafka and insert to Elasticsearch (5.3) \[2018-08-01T09:19:04,028\]\[WARN \]\[logstash.outputs.elasticsearch\] Coul… --- ## [Combination of logfiles with time-only and datetime fields](https://discuss.elastic.co/t/combination-of-logfiles-with-time-only-and-datetime-fields/143048)
**Author:** [@carstengehling](https://discuss.elastic.co/u/carstengehling)\ **Replies:** 3\ **Last updated:** [August 6, 2018, 9:44am UTC](https://discuss.elastic.co/t/combination-of-logfiles-with-time-only-and-datetime-fields/143048 "2018-08-06T09:44:04Z")
On a single server, I have multiple logfiles, that I need to send to LogStash, the problem is, that logfiles from one application has a datetime column: 2018-06-29 13:44:22,569 \[26\] DEBUG yadayadayada and logfiles from… --- ## [Reading logs from URL](https://discuss.elastic.co/t/reading-logs-from-url/143089)
**Author:** [@Raghuveer\_SJ](https://discuss.elastic.co/u/Raghuveer_SJ)\ **Replies:** 9\ **Last updated:** [August 6, 2018, 9:39am UTC](https://discuss.elastic.co/t/reading-logs-from-url/143089 "2018-08-06T09:39:48Z")
Logstash setup should read from beat and http url so the configuration that am using is like : input { beats { port=\>5044 } http { method =\> "POST" host =\> "localhost" port =\> 8080 path =\> "/context/atest" } } D… --- ## [Cisco asa - grok failure probably](https://discuss.elastic.co/t/cisco-asa-grok-failure-probably/142897)
**Author:** [@trustbyte](https://discuss.elastic.co/u/trustbyte)\ **Replies:** 8\ **Last updated:** [August 6, 2018, 9:33am UTC](https://discuss.elastic.co/t/cisco-asa-grok-failure-probably/142897 "2018-08-06T09:33:46Z")
Hey guys! I am having a hard time with the logstash and a syslog file which gets info from multiple cisco routers/firewalls. syslog file is like this: Aug 3 00:00:03 host233.domain.com %ASA-6-305011: Built dynamic TC… --- ## [Cloudtrail import on ELK](https://discuss.elastic.co/t/cloudtrail-import-on-elk/141322)
**Author:** [@moi](https://discuss.elastic.co/u/moi)\ **Replies:** 2\ **Last updated:** [August 6, 2018, 8:28am UTC](https://discuss.elastic.co/t/cloudtrail-import-on-elk/141322 "2018-08-06T08:28:28Z")
Hi, This use case is classic I think but I try to import cloudtrail logs from a S3 bucket into a ELK stack but I have this problem : Each logfile is a bunch of events and the "Records" field isn't unserialized and the e… --- ## [S3 output - Corrupted gzip file on abnormal shutdown](https://discuss.elastic.co/t/s3-output-corrupted-gzip-file-on-abnormal-shutdown/143120)
**Author:** [@hpello](https://discuss.elastic.co/u/hpello)\ **Replies:** 0\ **Last updated:** [August 6, 2018, 8:19am UTC](https://discuss.elastic.co/t/s3-output-corrupted-gzip-file-on-abnormal-shutdown/143120 "2018-08-06T08:19:31Z")
Hi, I posted an issue on GitHub but I thought I might as well ask for some help here: Description: When logstash is shut down abnormally, the s3 files created in the temporary directory are saved, and automatically u… --- ## [Logstash exeption "=\>"undefined method \`\>' for nil:NilClass"](https://discuss.elastic.co/t/logstash-exeption-undefined-method-for-nil-nilclass/142859)
**Author:** [@M\_B\_I](https://discuss.elastic.co/u/M_B_I)\ **Replies:** 3\ **Last updated:** [August 6, 2018, 7:04am UTC](https://discuss.elastic.co/t/logstash-exeption-undefined-method-for-nil-nilclass/142859 "2018-08-06T07:04:42Z")
Hi all, I have a few filters. Some days I receive an errors : \[2018-08-03T07:38:53,905\]\[ERROR\]\[logstash.pipeline \] Exception in pipelineworker, the pipeline stopped processing new events, \_ please check your fil… --- ## [log4j:ERROR Could not connect to remote log4j server at Logstash-server-IP](https://discuss.elastic.co/t/log4j-error-could-not-connect-to-remote-log4j-server-at-logstash-server-ip/143094)
**Author:** [@Ankita\_Ghate](https://discuss.elastic.co/u/Ankita_Ghate)\ **Replies:** 3\ **Last updated:** [August 6, 2018, 6:52am UTC](https://discuss.elastic.co/t/log4j-error-could-not-connect-to-remote-log4j-server-at-logstash-server-ip/143094 "2018-08-06T06:52:44Z")
Hello, Below is the application log4j.xml file, \ \ \ An… --- ## [Does logstash jdbc connection work with Windows Authentication?](https://discuss.elastic.co/t/does-logstash-jdbc-connection-work-with-windows-authentication/141562)
**Author:** [@Palash\_Tichkule](https://discuss.elastic.co/u/Palash_Tichkule)\ **Replies:** 10\ **Last updated:** [August 6, 2018, 6:07am UTC](https://discuss.elastic.co/t/does-logstash-jdbc-connection-work-with-windows-authentication/141562 "2018-08-06T06:07:30Z")
How to use logstash JDBC connection with Windows Authentication. The server does not accepts username and password even if they are passed --- ## [Sqlite input plugin not working](https://discuss.elastic.co/t/sqlite-input-plugin-not-working/143073)
**Author:** [@arisbanach](https://discuss.elastic.co/u/arisbanach)\ **Replies:** 4\ **Last updated:** [August 6, 2018, 12:18am UTC](https://discuss.elastic.co/t/sqlite-input-plugin-not-working/143073 "2018-08-06T00:18:07Z")
Not sure what is wrong, but I am getting this output from the sqlite input plugin. Aug 05 19:54:55 ip-10-221-145-243 logstash\[22850\]: \[2018-08-05T19:54:55,346\]\[ERROR\]\[logstash.pipeline \] A plugin had an unr ecove… --- ## [How to configure Logstash conf file for multiples servers](https://discuss.elastic.co/t/how-to-configure-logstash-conf-file-for-multiples-servers/143057)
**Author:** [@gabberoid](https://discuss.elastic.co/u/gabberoid)\ **Replies:** 3\ **Last updated:** [August 5, 2018, 7:02pm UTC](https://discuss.elastic.co/t/how-to-configure-logstash-conf-file-for-multiples-servers/143057 "2018-08-05T19:02:13Z")
Hi. I still didn't recognize from the documentation how I can configure Logstash to get logs from multiple servers. I have 2 servers with filebeat (send mysql and syslogs) on each server and a dedicated server for Logsta… --- ## [How do I write parsers for multiple lines like this?](https://discuss.elastic.co/t/how-do-i-write-parsers-for-multiple-lines-like-this/141887)
**Author:** [@Blason](https://discuss.elastic.co/u/Blason)\ **Replies:** 14\ **Last updated:** [August 5, 2018, 7:28am UTC](https://discuss.elastic.co/t/how-do-i-write-parsers-for-multiple-lines-like-this/141887 "2018-08-05T07:28:24Z")
Hi Team, Can someone please help me here for creating parsers? Well my main confusion is and just need a hint how do I parse those multiple lines? ######################## \[2018-07-27 12:52:33\] \[3307\] \[http\_80\_tcp 336… --- ## [Custom Timestamp Grok Filter Design](https://discuss.elastic.co/t/custom-timestamp-grok-filter-design/141918)
**Author:** [@deepesh123](https://discuss.elastic.co/u/deepesh123)\ **Replies:** 1\ **Last updated:** [August 4, 2018, 7:30pm UTC](https://discuss.elastic.co/t/custom-timestamp-grok-filter-design/141918 "2018-08-04T19:30:20Z")
Please suggest a suitable grok filter for this pattern. I am unable to frame a custom timestamp pattern for this. ####\ --- ## [Logstash input file plugin reprocesses file from beginning whenever new line gets appended to log](https://discuss.elastic.co/t/logstash-input-file-plugin-reprocesses-file-from-beginning-whenever-new-line-gets-appended-to-log/142926)
**Author:** [@suresh\_bahuguna](https://discuss.elastic.co/u/suresh_bahuguna)\ **Replies:** 6\ **Last updated:** [August 4, 2018, 6:17pm UTC](https://discuss.elastic.co/t/logstash-input-file-plugin-reprocesses-file-from-beginning-whenever-new-line-gets-appended-to-log/142926 "2018-08-04T18:17:15Z")
Hi, I tried with both start\_position as beginning and end. Both behave in the same way. Only difference is, if there are existing logs and start\_position is set as end - logstash won't raise event until a new line gets … --- ## [Lumberjack Output Errors](https://discuss.elastic.co/t/lumberjack-output-errors/143019)
**Author:** [@FatalGlitch](https://discuss.elastic.co/u/FatalGlitch)\ **Replies:** 0\ **Last updated:** [August 4, 2018, 2:18pm UTC](https://discuss.elastic.co/t/lumberjack-output-errors/143019 "2018-08-04T14:18:31Z")
Trying to debug why the below errors are occurring. Logs seem to be flowing, but can't determine why this is repeating every 30 seconds or so \[2018-08-04T10:08:26,899\]\[ERROR\]\[logstash.outputs.lumberjack\] Client write er… --- ## [Error in logstash.conf](https://discuss.elastic.co/t/error-in-logstash-conf/141281)
**Author:** [@Yih\_Ashley](https://discuss.elastic.co/u/Yih_Ashley)\ **Replies:** 6\ **Last updated:** [August 4, 2018, 10:54am UTC](https://discuss.elastic.co/t/error-in-logstash-conf/141281 "2018-08-04T10:54:13Z")
Hello, I have a problem when running my logstash. Below is my configuration in logstash.conf input { file { path =\> \["/var/log/nsm/eve.json"\] codec =\> json type =\> "SuricataIDPS" } } filter { if \[type\] == "Sur… --- ## [DNS Filter And GeoIP can't see ip field](https://discuss.elastic.co/t/dns-filter-and-geoip-cant-see-ip-field/142690)
**Author:** [@Alper\_Mehmet\_Ozdemir](https://discuss.elastic.co/u/Alper_Mehmet_Ozdemir)\ **Replies:** 3\ **Last updated:** [August 4, 2018, 10:32am UTC](https://discuss.elastic.co/t/dns-filter-and-geoip-cant-see-ip-field/142690 "2018-08-04T10:32:45Z")
Hello, I am trying to parse netflow logs and would like to perform a DNS lookup and get GeoIP information for the source and destination IPs. However, I can't seem to get Logstash to see the fields that contain the IPs. … --- ## [Logstash doesn't load data until interrupted](https://discuss.elastic.co/t/logstash-doesnt-load-data-until-interrupted/142970)
**Author:** [@emanresu](https://discuss.elastic.co/u/emanresu)\ **Replies:** 2\ **Last updated:** [August 3, 2018, 9:09pm UTC](https://discuss.elastic.co/t/logstash-doesnt-load-data-until-interrupted/142970 "2018-08-03T21:09:37Z")
My cURL command in the input below invokes an https endpoint that will stream data until interrupted. My config below works but no data is loaded to Elasticsearch until I interrupt Logstash. Logstash keeps the data in me… --- ## [How to Look for Specific Tag in Filter Conditional](https://discuss.elastic.co/t/how-to-look-for-specific-tag-in-filter-conditional/142964)
**Author:** [@ruhrohshingo](https://discuss.elastic.co/u/ruhrohshingo)\ **Replies:** 2\ **Last updated:** [August 3, 2018, 7:45pm UTC](https://discuss.elastic.co/t/how-to-look-for-specific-tag-in-filter-conditional/142964 "2018-08-03T19:45:05Z")
I've been playing with building an ELK stack with Kafka between Filebeat and Logstash for a couple weeks. On hosts, I have Filebeat configured with a Tags property like so: filebeat.inputs: - type: log enabled: tr… --- ## [Failed to execute action](https://discuss.elastic.co/t/failed-to-execute-action/142960)
**Author:** [@Honda\_fred\_elk](https://discuss.elastic.co/u/Honda_fred_elk)\ **Replies:** 5\ **Last updated:** [August 3, 2018, 7:17pm UTC](https://discuss.elastic.co/t/failed-to-execute-action/142960 "2018-08-03T19:17:40Z")
Hello, Recently I upgraded ELK from 6.2.2 to 6.3.2. I have translate filter plugin for my logs processing filter which was working great in 6.2.2 but not in 6.3.2. I have 4 different conf files out of 4, 2 are working … --- ## [/dev/sda1 used from 6.2g 13% to 49g 100% in 4 days](https://discuss.elastic.co/t/dev-sda1-used-from-6-2g-13-to-49g-100-in-4-days/142952)
**Author:** [@databeata](https://discuss.elastic.co/u/databeata)\ **Replies:** 3\ **Last updated:** [August 3, 2018, 6:31pm UTC](https://discuss.elastic.co/t/dev-sda1-used-from-6-2g-13-to-49g-100-in-4-days/142952 "2018-08-03T18:31:15Z")
The /dev/sda1 used went from 6.2g 13% to 49g 100% in 4 days. The memory free went from 51g to 2g in 4 days. How do I manage it? 8/3: -- reboot -- /dev/sda1 6.2g 13% -- memory free 51g after 4 days. 8/7: -- /dev/… --- ## [Need help with restructuring the data using filters](https://discuss.elastic.co/t/need-help-with-restructuring-the-data-using-filters/142281)
**Author:** [@whoiscnu](https://discuss.elastic.co/u/whoiscnu)\ **Replies:** 3\ **Last updated:** [August 3, 2018, 5:30pm UTC](https://discuss.elastic.co/t/need-help-with-restructuring-the-data-using-filters/142281 "2018-08-03T17:30:40Z")
Hi Guys, Is there a way that we can reference field in a particular event in another event? Regards Srini --- ## [HTTP Poller in Logstash - input separation to multiple fields](https://discuss.elastic.co/t/http-poller-in-logstash-input-separation-to-multiple-fields/142910)
**Author:** [@pastechecker](https://discuss.elastic.co/u/pastechecker)\ **Replies:** 3\ **Last updated:** [August 3, 2018, 4:42pm UTC](https://discuss.elastic.co/t/http-poller-in-logstash-input-separation-to-multiple-fields/142910 "2018-08-03T16:42:21Z")
Hello, I have just started my adventure with Logstash. I would like to create new json in elasticsearch for every new line. By default whole file is loaded in a message field: Kibana output: { "\_index": "blacklists… --- ## [Dependency between Kibana and Logstash](https://discuss.elastic.co/t/dependency-between-kibana-and-logstash/142919)
**Author:** [@bahare\_hoseini](https://discuss.elastic.co/u/bahare_hoseini)\ **Replies:** 2\ **Last updated:** [August 3, 2018, 3:31pm UTC](https://discuss.elastic.co/t/dependency-between-kibana-and-logstash/142919 "2018-08-03T15:31:00Z")
Even when I stop Logstash, Kibana is up and runningwithout any change in data displayed by kibana. These 2 are independent or i'm stopping the wrong logstash? If they're independent, how could I check my changes in Logs… --- ## [Sonicwall (Syslog) + Logstash](https://discuss.elastic.co/t/sonicwall-syslog-logstash/141251)
**Author:** [@weyderfs](https://discuss.elastic.co/u/weyderfs)\ **Replies:** 2\ **Last updated:** [August 3, 2018, 3:13pm UTC](https://discuss.elastic.co/t/sonicwall-syslog-logstash/141251 "2018-08-03T15:13:20Z")
Hi, I research on web about how to configure Sonicwall send your syslogs to Logstash or Filebeat, practically did not find anything.. I would like if possible configure this? In side of SonicWall I already configured. … --- ## [Bad GELF performance](https://discuss.elastic.co/t/bad-gelf-performance/142933)
**Author:** [@hangstl](https://discuss.elastic.co/u/hangstl)\ **Replies:** 0\ **Last updated:** [August 3, 2018, 3:09pm UTC](https://discuss.elastic.co/t/bad-gelf-performance/142933 "2018-08-03T15:09:05Z")
I have a Graylog installation that can handle well over 25.000 msg/sec. I am now experimenting with setting up a dedicated Logstash cluster. So something like this: Filebeat (n hosts) -\> Logstash Cluster -\> Graylog Th… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=706) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=708)