# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1070 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1071 --- ## [Is there a GeoIP constraint?](https://discuss.elastic.co/t/is-there-a-geoip-constraint/67817)
**Author:** [@Dark\_Light](https://discuss.elastic.co/u/Dark_Light)\ **Replies:** 0\ **Last updated:** [December 2, 2016, 12:52am UTC](https://discuss.elastic.co/t/is-there-a-geoip-constraint/67817 "2016-12-02T00:52:52Z")
Wondering if the new GeoIP DB with Ver 5\* is constrained to only allowing 2 active GeoIP lookups in a Logstash Filter? I have a Logstash conf file processing Winlogbeats, Syslog, Squid Logs, Apache Logs and DNS logs. … --- ## [Elasticsearch Output Is Failing on action =\> "update" due to missing document?](https://discuss.elastic.co/t/elasticsearch-output-is-failing-on-action-update-due-to-missing-document/67807)
**Author:** [@Evan\_Ravenelle](https://discuss.elastic.co/u/Evan_Ravenelle)\ **Replies:** 0\ **Last updated:** [December 1, 2016, 8:43pm UTC](https://discuss.elastic.co/t/elasticsearch-output-is-failing-on-action-update-due-to-missing-document/67807 "2016-12-01T20:43:36Z")
I have the following input and output blocks: input { elasticsearch { index =\> "filebeat-\*" hosts =\> "localhost" query =\> '{ "query": { "dis\_max": { "queries": \[ { "match": { "tags": "\_grokparsefailure" … --- ## [Logstash isn't recognizing that a file has been updated](https://discuss.elastic.co/t/logstash-isnt-recognizing-that-a-file-has-been-updated/67800)
**Author:** [@TonyEnnis](https://discuss.elastic.co/u/TonyEnnis)\ **Replies:** 0\ **Last updated:** [December 1, 2016, 6:28pm UTC](https://discuss.elastic.co/t/logstash-isnt-recognizing-that-a-file-has-been-updated/67800 "2016-12-01T18:28:23Z")
I have a largish file (4G). Each hour, more rows are added. I can process the large file when logstash is started, but regardless of what I do, logstash will not notice updates to the file. The .sincedb\_ file has a pr… --- ## [Parsing XML files using LogStash](https://discuss.elastic.co/t/parsing-xml-files-using-logstash/25372)
**Author:** [@AjitGoel](https://discuss.elastic.co/u/AjitGoel)\ **Replies:** 13\ **Last updated:** [December 1, 2016, 8:17pm UTC](https://discuss.elastic.co/t/parsing-xml-files-using-logstash/25372 "2016-12-01T20:17:02Z")
I have the following logstash conf file: input { file { path =\> "C:\\Dashboard\\Elmah\\\*.xml" start\_position =\> "beginning" type =\> "error" codec =\> multiline { pattern =\> "^\<\\?error .\*\\\>" … --- ## [Add user and password and failed](https://discuss.elastic.co/t/add-user-and-password-and-failed/67525)
**Author:** [@robertchen117](https://discuss.elastic.co/u/robertchen117)\ **Replies:** 2\ **Last updated:** [December 1, 2016, 2:58pm UTC](https://discuss.elastic.co/t/add-user-and-password-and-failed/67525 "2016-12-01T14:58:10Z")
I followed this doc after I installed x-pack. And I update my logstash.conf with user and password. the error is: \[LogStash::Runner\] FATAL logstash.runner - The given configuration is invalid. Reason: Expected one of #… --- ## [Forwarding from beats to logstash](https://discuss.elastic.co/t/forwarding-from-beats-to-logstash/67776)
**Author:** [@bob-bza](https://discuss.elastic.co/u/bob-bza)\ **Replies:** 0\ **Last updated:** [December 1, 2016, 2:27pm UTC](https://discuss.elastic.co/t/forwarding-from-beats-to-logstash/67776 "2016-12-01T14:27:22Z")
I was new not ELK, sending logs from multiple beats to single logstash with same tag, but was not able to see all the beats hosts in kibana. for ex: i was sending 5beats to 1logstash with same tag name. but can see only … --- ## [Parsing of logs with multiline filter](https://discuss.elastic.co/t/parsing-of-logs-with-multiline-filter/67709)
**Author:** [@Rajat\_Singh](https://discuss.elastic.co/u/Rajat_Singh)\ **Replies:** 1\ **Last updated:** [December 1, 2016, 1:32pm UTC](https://discuss.elastic.co/t/parsing-of-logs-with-multiline-filter/67709 "2016-12-01T13:32:03Z")
i am java stack trace need to parse i am not getting the pattern to do the same i had tried different patterns for the same Current Date : Tue Sep 06 05:56:51 EDT 2016 | Current user : admin Exception during logging : j… --- ## [LS crashing when trying to read CSV with undefined method \`tv\_sec' error](https://discuss.elastic.co/t/ls-crashing-when-trying-to-read-csv-with-undefined-method-tv-sec-error/67079)
**Author:** [@fxiger](https://discuss.elastic.co/u/fxiger)\ **Replies:** 4\ **Last updated:** [December 1, 2016, 11:43am UTC](https://discuss.elastic.co/t/ls-crashing-when-trying-to-read-csv-with-undefined-method-tv-sec-error/67079 "2016-12-01T11:43:51Z")
Hi, i'm trying to read logifles (written on my own app) with csv format with logstash and output them to an ES index. i'm using: logstash 2.4.1 logstash-filter-csv (2.1.3) logstash-input-file (2.2.5) logstash-outpu… --- ## [Access Denied to ES](https://discuss.elastic.co/t/access-denied-to-es/61427)
**Author:** [@Allie\_Yang](https://discuss.elastic.co/u/Allie_Yang)\ **Replies:** 6\ **Last updated:** [December 1, 2016, 11:08am UTC](https://discuss.elastic.co/t/access-denied-to-es/61427 "2016-12-01T11:08:23Z")
I am trying to use logstash beta version 5.0 to parse data. I also installed Xpacks. However, below is the error code. How could i access ES server (beta version): \[2016-09-23T16:44:28,382\]\[INFO \]\[logstash.outputs.elas… --- ## [Bottleneck while inputting data into the elasticsearch](https://discuss.elastic.co/t/bottleneck-while-inputting-data-into-the-elasticsearch/67735)
**Author:** [@Mayank\_Agrawal](https://discuss.elastic.co/u/Mayank_Agrawal)\ **Replies:** 6\ **Last updated:** [December 1, 2016, 10:56am UTC](https://discuss.elastic.co/t/bottleneck-while-inputting-data-into-the-elasticsearch/67735 "2016-12-01T10:56:55Z")
Hi, I am using elasticsearch 5.0.1 and logstash 5.0.0. For 1GB data, logstash is easily parsing the logs in half an hr (approx) if I am not redirecting the output to ES. With ES in output, the time taken to parse the s… --- ## [Reading enterprise library logs using logstash](https://discuss.elastic.co/t/reading-enterprise-library-logs-using-logstash/67741)
**Author:** [@ealisha](https://discuss.elastic.co/u/ealisha)\ **Replies:** 0\ **Last updated:** [December 1, 2016, 10:27am UTC](https://discuss.elastic.co/t/reading-enterprise-library-logs-using-logstash/67741 "2016-12-01T10:27:31Z")
I have the following log file for which i want the logstash could parse out the necessary information. Can someone suggest me the filter conditions for the conf file that i should use? Timestamp: 9/23/2015 12:47:25 P… --- ## [Creating and using variables across the logstash config](https://discuss.elastic.co/t/creating-and-using-variables-across-the-logstash-config/66823)
**Author:** [@muhamadli302](https://discuss.elastic.co/u/muhamadli302)\ **Replies:** 6\ **Last updated:** [December 1, 2016, 9:03am UTC](https://discuss.elastic.co/t/creating-and-using-variables-across-the-logstash-config/66823 "2016-12-01T09:03:54Z")
Hi, I'm using LS 2.2, reading a CSV of events. I want it to be completely automatic, so in order to create a meaningful index name, I tried to "implant" the name I want to use, and then parse it and use it in the outpu… --- ## [Error on S3 plugin accessing ELB logs - Logstash version 5](https://discuss.elastic.co/t/error-on-s3-plugin-accessing-elb-logs-logstash-version-5/67663)
**Author:** [@Jimmy\_Etimos](https://discuss.elastic.co/u/Jimmy_Etimos)\ **Replies:** 4\ **Last updated:** [December 1, 2016, 4:50am UTC](https://discuss.elastic.co/t/error-on-s3-plugin-accessing-elb-logs-logstash-version-5/67663 "2016-12-01T04:50:56Z")
I am getting the following error while trying to read ELB logs from an S3 bucket. The instance running logstash is in same region a bucket and booted with a role that gives it full S3 access. Error: can't convert nil i… --- ## [Date match pattern with Z versus ZZ: any difference?](https://discuss.elastic.co/t/date-match-pattern-with-z-versus-zz-any-difference/67159)
**Author:** [@GrahamHannington](https://discuss.elastic.co/u/GrahamHannington)\ **Replies:** 1\ **Last updated:** [December 1, 2016, 2:34am UTC](https://discuss.elastic.co/t/date-match-pattern-with-z-versus-zz-any-difference/67159 "2016-12-01T02:34:27Z")
In brief: According to the docs, only ZZ matches +08:00. In practice, Z also matches. Why? In detail... From the Logstash 5.0 documentation for the match option of the date filter plugin: The date formats allowed are… --- ## [NetFlow v9 crashes on options data from Juniper SRX](https://discuss.elastic.co/t/netflow-v9-crashes-on-options-data-from-juniper-srx/67424)
**Author:** [@ktims](https://discuss.elastic.co/u/ktims)\ **Replies:** 1\ **Last updated:** [December 1, 2016, 2:26am UTC](https://discuss.elastic.co/t/netflow-v9-crashes-on-options-data-from-juniper-srx/67424 "2016-12-01T02:26:11Z")
The initial symptom I was seeing was logstash throwing NotImplementedError on receipt of the template packet from a Juniper SRX110 on 12.1X44-D45.2 (not sure if version matters, but this version seems to behave this way … --- ## [INFO: Windows, Logstash and could not find log4j2 configuration](https://discuss.elastic.co/t/info-windows-logstash-and-could-not-find-log4j2-configuration/67633)
**Author:** [@juliann](https://discuss.elastic.co/u/juliann)\ **Replies:** 1\ **Last updated:** [December 1, 2016, 1:56am UTC](https://discuss.elastic.co/t/info-windows-logstash-and-could-not-find-log4j2-configuration/67633 "2016-12-01T01:56:03Z")
I was getting a message others seem to get with a Windows installation: Could not find log4j2 configuration at path /logstash-5.0.1/config/log4j2.properties. Using default config which logs to console My logstash is … --- ## [Importing MSSQL data to Elasticsearch 5.0.0 via Logstash 5.0.0](https://discuss.elastic.co/t/importing-mssql-data-to-elasticsearch-5-0-0-via-logstash-5-0-0/67690)
**Author:** [@lakinen](https://discuss.elastic.co/u/lakinen)\ **Replies:** 0\ **Last updated:** [December 1, 2016, 12:26am UTC](https://discuss.elastic.co/t/importing-mssql-data-to-elasticsearch-5-0-0-via-logstash-5-0-0/67690 "2016-12-01T00:26:48Z")
Hi, I'm trying to import basic data from MSSQL table with 2 columns. Using logstash-input-jdbc built-in pluging I have managed to connect to database and run the simple SELECT query to fetch all data from the table. B… --- ## [Monit + logstash](https://discuss.elastic.co/t/monit-logstash/64553)
**Author:** [@rashthedude](https://discuss.elastic.co/u/rashthedude)\ **Replies:** 3\ **Last updated:** [November 30, 2016, 9:32pm UTC](https://discuss.elastic.co/t/monit-logstash/64553 "2016-11-30T21:32:44Z")
Hi there, I'm having a difficult time trying to monitor Logstash using monit. All I want to do is check if Logstash is either up/down and if it's processing requests. Can anyone point to me to a direction here? Rega… --- ## [Logstash 5.0.1 with kafka 0.9](https://discuss.elastic.co/t/logstash-5-0-1-with-kafka-0-9/67660)
**Author:** [@luis.carvalho](https://discuss.elastic.co/u/luis.carvalho)\ **Replies:** 0\ **Last updated:** [November 30, 2016, 6:07pm UTC](https://discuss.elastic.co/t/logstash-5-0-1-with-kafka-0-9/67660 "2016-11-30T18:07:40Z")
Hi! I have a logstash 5.0.1 installation, and I want to get data from my kafka 0.9, but logstash give me a erro. I already update the kafka plugin to 4.0.0 version. bin/logstash-plugin remove logstash-input-kafka bin… --- ## [Logstash-Kafka : each data pulled create new session on zookeeper](https://discuss.elastic.co/t/logstash-kafka-each-data-pulled-create-new-session-on-zookeeper/67652)
**Author:** [@Charlotte\_EGM](https://discuss.elastic.co/u/Charlotte_EGM)\ **Replies:** 0\ **Last updated:** [November 30, 2016, 4:52pm UTC](https://discuss.elastic.co/t/logstash-kafka-each-data-pulled-create-new-session-on-zookeeper/67652 "2016-11-30T16:52:07Z")
Hi, I have an ELK suite with logstash pulling data from Kafka. Kafka is getting data from Fiware Orion-Cygnus (based on Apache Flume). Everything is doing well for the 60 first data pushed. Then the connection with Cyg… --- ## [Logstash - How to Multiline correct](https://discuss.elastic.co/t/logstash-how-to-multiline-correct/67643)
**Author:** [@matze1992](https://discuss.elastic.co/u/matze1992)\ **Replies:** 1\ **Last updated:** [November 30, 2016, 4:35pm UTC](https://discuss.elastic.co/t/logstash-how-to-multiline-correct/67643 "2016-11-30T16:35:57Z")
I would like to analyse the following log. For this I have created a Grok-Filter in Logstash. My Input-Filter looks like this: file { path =\> "/etc/logstash/files.d/libview\*" codec =\> multil… --- ## [Gzipped csv input with exec input command =\> "gunzip -c \" gets OutOfMemoryError](https://discuss.elastic.co/t/gzipped-csv-input-with-exec-input-command-gunzip-c-file-gets-outofmemoryerror/67645)
**Author:** [@reuschling](https://discuss.elastic.co/u/reuschling)\ **Replies:** 0\ **Last updated:** [November 30, 2016, 4:05pm UTC](https://discuss.elastic.co/t/gzipped-csv-input-with-exec-input-command-gunzip-c-file-gets-outofmemoryerror/67645 "2016-11-30T16:05:48Z")
Hi there, I just trying to read a huge csv file (hundreds of gigabytes). It is gzipped. Since the csv input doesn't support gzipped files, as the csv output does, I try to get the content with an exec command. Neverthe… --- ## [File output plugin always outputs json](https://discuss.elastic.co/t/file-output-plugin-always-outputs-json/67630)
**Author:** [@gcsaba2](https://discuss.elastic.co/u/gcsaba2)\ **Replies:** 1\ **Last updated:** [November 30, 2016, 3:20pm UTC](https://discuss.elastic.co/t/file-output-plugin-always-outputs-json/67630 "2016-11-30T15:20:57Z")
I'm reading from a plain text file and want to write the same plain text. So I wrote this: output { file { path =\> .... codec =\> line { format =\> "%{message}" } } } Looks like the file plugin ignores the … --- ## [\[SOLVED\] Measuring duration without using elapsed](https://discuss.elastic.co/t/solved-measuring-duration-without-using-elapsed/67392)
**Author:** [@aeva\_assured](https://discuss.elastic.co/u/aeva_assured)\ **Replies:** 2\ **Last updated:** [November 30, 2016, 1:19pm UTC](https://discuss.elastic.co/t/solved-measuring-duration-without-using-elapsed/67392 "2016-11-30T13:19:48Z")
I've been using the elapsed plugin Github Link to determine the duration of events in my syslog parse and the plugin is not yet compatible with v5. I've subscribed to the GitHub to see when it gets updated, does anyone… --- ## [Using wildcard character in input](https://discuss.elastic.co/t/using-wildcard-character-in-input/67563)
**Author:** [@seanziee](https://discuss.elastic.co/u/seanziee)\ **Replies:** 2\ **Last updated:** [November 30, 2016, 1:13pm UTC](https://discuss.elastic.co/t/using-wildcard-character-in-input/67563 "2016-11-30T13:13:15Z")
Hi, I'm trying to use a wildcard character to get multiple folders in my email with the IMAP input but it doesn't seem to be valid. Am I missing something? Should it work? I have about 30 folders that I need to import … --- ## [Not able to parse XML string with logstash grok filter](https://discuss.elastic.co/t/not-able-to-parse-xml-string-with-logstash-grok-filter/67591)
**Author:** [@shamimhaider33](https://discuss.elastic.co/u/shamimhaider33)\ **Replies:** 1\ **Last updated:** [November 30, 2016, 12:20pm UTC](https://discuss.elastic.co/t/not-able-to-parse-xml-string-with-logstash-grok-filter/67591 "2016-11-30T12:20:01Z")
I am trying to parse the XML string data usign Grok but looks like I am doing something wrong, please help me on this. XML String Data : \\\ **Author:** [@Mike\_Clarke](https://discuss.elastic.co/u/Mike_Clarke)\ **Replies:** 2\ **Last updated:** [November 30, 2016, 11:56am UTC](https://discuss.elastic.co/t/divide-two-fields-and-store-in-a-new-field-to-be-visualized-in-kibana/67400 "2016-11-30T11:56:41Z") I would like to take two fields from a GROK filter and divide the two to get the number of events occurring in one second. ES, LS, FB, Kibana all 5.0. Something like this time\_per\_sel = enrichtime / totalsel. Input: 2… --- ## [Logstash plugin install network error](https://discuss.elastic.co/t/logstash-plugin-install-network-error/67131)
**Author:** [@txisme](https://discuss.elastic.co/u/txisme)\ **Replies:** 3\ **Last updated:** [November 30, 2016, 10:15am UTC](https://discuss.elastic.co/t/logstash-plugin-install-network-error/67131 "2016-11-30T10:15:12Z")
Hey, cant seem to install/update the beats plugin, keeps giving network error. I do have internet connection. ./bin/logstash-plugin update logstash-input-beats Updating logstash-input-beats Error Bundler::HTTPError, re… --- ## [Grok plugin and installation steps](https://discuss.elastic.co/t/grok-plugin-and-installation-steps/67369)
**Author:** [@xsallowed](https://discuss.elastic.co/u/xsallowed)\ **Replies:** 2\ **Last updated:** [November 30, 2016, 8:25am UTC](https://discuss.elastic.co/t/grok-plugin-and-installation-steps/67369 "2016-11-30T08:25:33Z")
Hi I am trying to use grok plugin for combinedapchelogs but I think it is not parsing logs . I wanted to ask if all the plugins (assuming grok is a logstash plugin) are installed as a bundle for logstash or you need to i… --- ## [Breaking changes in ruby filter plugin to ther version of 5.0.1](https://discuss.elastic.co/t/breaking-changes-in-ruby-filter-plugin-to-ther-version-of-5-0-1/67470)
**Author:** [@jamal](https://discuss.elastic.co/u/jamal)\ **Replies:** 3\ **Last updated:** [November 30, 2016, 1:29am UTC](https://discuss.elastic.co/t/breaking-changes-in-ruby-filter-plugin-to-ther-version-of-5-0-1/67470 "2016-11-30T01:29:52Z")
I upgrade logstash from 2.1.0 to 5.0.1 version i have error in filter-plugin-rubby. here is the code that i had in version 2.1.0 and work correctlly ruby { code =\> "require 'digest/md5'; … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1069) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1071)