# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1137 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1138 --- ## [Logstash error stack](https://discuss.elastic.co/t/logstash-error-stack/53561)
**Author:** [@sharon.c](https://discuss.elastic.co/u/sharon.c)\ **Replies:** 1\ **Last updated:** [June 21, 2016, 9:46pm UTC](https://discuss.elastic.co/t/logstash-error-stack/53561 "2016-06-21T21:46:27Z")
I am using logstash 2.2.4, elasticsearch 4.5, filebeat 1.1.1 Beats input: unhandled exception {:exception=\># Zlib::BufError: buffer error \>, :backtrace=\>\["org/jruby/ext/zlib/ZStream.java:134:in \`finish'", "org/jruby/ex… --- ## [Upgrade from 1.4.2 to 2.3.3](https://discuss.elastic.co/t/upgrade-from-1-4-2-to-2-3-3/53575)
**Author:** [@AGirin](https://discuss.elastic.co/u/AGirin)\ **Replies:** 2\ **Last updated:** [June 21, 2016, 10:31pm UTC](https://discuss.elastic.co/t/upgrade-from-1-4-2-to-2-3-3/53575 "2016-06-21T22:31:48Z")
We are testing upgrade from Logstash 1.4.2 to 2.3.3 and seeing a lot of this error in the logstash.log: {:timestamp=\>"2016-06-21T15:41:12.868000-0500", :message=\>"Ruby exception occurred: LogStash::Timestamp can't be c… --- ## [Logstash struck after Pipeline main started.. No output printed and nothing sent to elastic search](https://discuss.elastic.co/t/logstash-struck-after-pipeline-main-started-no-output-printed-and-nothing-sent-to-elastic-search/53559)
**Author:** [@sandeep\_vuppula](https://discuss.elastic.co/u/sandeep_vuppula)\ **Replies:** 3\ **Last updated:** [June 21, 2016, 10:25pm UTC](https://discuss.elastic.co/t/logstash-struck-after-pipeline-main-started-no-output-printed-and-nothing-sent-to-elastic-search/53559 "2016-06-21T22:25:24Z")
I am using logstash-2.3.3. when i run "logstash -f logstash-workday.conf" it is struck and no output is being displayed. C:\\logstash-2.3.3\\bin\>logstash -f logstash-workday.conf io/console not supported; tty will not be m… --- ## [How to account for different number of blank spaces in Log4j logs?](https://discuss.elastic.co/t/how-to-account-for-different-number-of-blank-spaces-in-log4j-logs/53580)
**Author:** [@ZillaG](https://discuss.elastic.co/u/ZillaG)\ **Replies:** 4\ **Last updated:** [June 21, 2016, 10:14pm UTC](https://discuss.elastic.co/t/how-to-account-for-different-number-of-blank-spaces-in-log4j-logs/53580 "2016-06-21T22:14:08Z")
I have the following log4j logs 2016-04-22 16:43:25,172 ERROR :SomeThread : 2 \[com.mycompany.SomeClass\] The Log Message. 2016-06-11 01:22:40,894 INFO :SomeThread1 : 0 \[com.mycompany.SomeClass1\] The Log message I hav… --- ## [Persistence Issue with Logstash](https://discuss.elastic.co/t/persistence-issue-with-logstash/53254)
**Author:** [@susrich0810](https://discuss.elastic.co/u/susrich0810)\ **Replies:** 2\ **Last updated:** [June 21, 2016, 2:48pm UTC](https://discuss.elastic.co/t/persistence-issue-with-logstash/53254 "2016-06-21T14:48:48Z")
I'm trying to create a script to run through my previous day's logs, and count occurrences of a particular field in the hash. There are hundreds of possible values that could be assigned to this particular key, including… --- ## [Where are data types for yaml file defined?](https://discuss.elastic.co/t/where-are-data-types-for-yaml-file-defined/53546)
**Author:** [@Fidel\_Netflow](https://discuss.elastic.co/u/Fidel_Netflow)\ **Replies:** 0\ **Last updated:** [June 21, 2016, 5:41pm UTC](https://discuss.elastic.co/t/where-are-data-types-for-yaml-file-defined/53546 "2016-06-21T17:41:46Z")
For instance, 8: - ip4\_addr \<--where is this defined - ipv4\_source\_address I ask because I am getting netflow records that include some extra mac address fields, and they are turning to junk before getting stashed in … --- ## [Using logstash to migrate data from postgres to ES](https://discuss.elastic.co/t/using-logstash-to-migrate-data-from-postgres-to-es/26829)
**Author:** [@Freddie](https://discuss.elastic.co/u/Freddie)\ **Replies:** 1\ **Last updated:** [June 21, 2016, 5:38pm UTC](https://discuss.elastic.co/t/using-logstash-to-migrate-data-from-postgres-to-es/26829 "2016-06-21T17:38:24Z")
I'm using the logstash logstash-input-jdbc plugin. logstash conf file as follows. input { jdbc { jdbc\_driver\_library =\> "/mars/services/ows/webapps/ows/WEB-INF/lib/postgresql-8.4-701.jdbc3.jar" jdbc\_driver\_cl… --- ## [Grok pattern for Names](https://discuss.elastic.co/t/grok-pattern-for-names/53476)
**Author:** [@sanju1323](https://discuss.elastic.co/u/sanju1323)\ **Replies:** 1\ **Last updated:** [June 21, 2016, 3:34pm UTC](https://discuss.elastic.co/t/grok-pattern-for-names/53476 "2016-06-21T15:34:16Z")
Hi, Can someone suggest grok pattern for the following names Mr. Rakesh Kumar Reddy Mr Harish Shankar Mr M. D. S. Reddy Mr. M D S Reddy Mr M.D.S. Reddy These are the names that are present in Name field. Here, … --- ## [Issues fetching Logstash data to index in Kibana](https://discuss.elastic.co/t/issues-fetching-logstash-data-to-index-in-kibana/53165)
**Author:** [@maltman](https://discuss.elastic.co/u/maltman)\ **Replies:** 3\ **Last updated:** [June 21, 2016, 3:23pm UTC](https://discuss.elastic.co/t/issues-fetching-logstash-data-to-index-in-kibana/53165 "2016-06-21T15:23:41Z")
I am having an issue getting my Logstash index to show up in Kibana. I have worked out all of the issues in my Logstash config and still nothing is showing up. I ran this command to start Logstash logstash agent -f C… --- ## [Logstash integration with Zabbix server](https://discuss.elastic.co/t/logstash-integration-with-zabbix-server/53509)
**Author:** [@Alan\_John](https://discuss.elastic.co/u/Alan_John)\ **Replies:** 1\ **Last updated:** [June 21, 2016, 2:55pm UTC](https://discuss.elastic.co/t/logstash-integration-with-zabbix-server/53509 "2016-06-21T14:55:29Z")
Hi, I am working on with ELK and Zabbix recently. I'm stuck now with the integration of Logstash with Zabbix server. I couldn't find a good documentation to follow other than the slides provided by you and some publ… --- ## [Logstash split fields](https://discuss.elastic.co/t/logstash-split-fields/53288)
**Author:** [@Nizar\_Khalifa](https://discuss.elastic.co/u/Nizar_Khalifa)\ **Replies:** 8\ **Last updated:** [June 21, 2016, 2:38pm UTC](https://discuss.elastic.co/t/logstash-split-fields/53288 "2016-06-21T14:38:16Z")
Actually im trying to split a field's request with & caracter kv { source =\> "request" field\_split =\> "&?" target =\> "params" } For example when the request is: toto=a&titi=b$tata=c I get this: … --- ## [Multiple logstash-forwarder on same server](https://discuss.elastic.co/t/multiple-logstash-forwarder-on-same-server/53477)
**Author:** [@YLombardi](https://discuss.elastic.co/u/YLombardi)\ **Replies:** 3\ **Last updated:** [June 21, 2016, 2:06pm UTC](https://discuss.elastic.co/t/multiple-logstash-forwarder-on-same-server/53477 "2016-06-21T14:06:34Z")
Hi. On my server, I have log file in json format and in plain text format. I want to separate them on two logstash input (one for with a json codec and one with plain codec). My logstash use two lumberjack input that l… --- ## [Grokparsefailure with NXlog and Logstash](https://discuss.elastic.co/t/grokparsefailure-with-nxlog-and-logstash/36210)
**Author:** [@Sandeep\_Krishnankutt](https://discuss.elastic.co/u/Sandeep_Krishnankutt)\ **Replies:** 3\ **Last updated:** [June 21, 2016, 1:36pm UTC](https://discuss.elastic.co/t/grokparsefailure-with-nxlog-and-logstash/36210 "2016-06-21T13:36:34Z")
I seem to have a strange(or not) issue with logstash where I am seeing random grokparsefailure. I am using NXLog to ship IIS logs from 5 of my webservers to Logstash. The NXlog config is as below Module xm\_json … --- ## [Is where condition supported with logstash-jdbc input plugin](https://discuss.elastic.co/t/is-where-condition-supported-with-logstash-jdbc-input-plugin/27170)
**Author:** [@rkhapre](https://discuss.elastic.co/u/rkhapre)\ **Replies:** 2\ **Last updated:** [August 12, 2015, 8:34am UTC](https://discuss.elastic.co/t/is-where-condition-supported-with-logstash-jdbc-input-plugin/27170 "2015-08-12T08:34:38Z")
Hi All I am not able to create index if i provide a complex sql query like with where condition and join condition But if i am providing simple "select \* from table', it is working well Please let me know if comple… --- ## [Kv plugin recursive with some names before the values](https://discuss.elastic.co/t/kv-plugin-recursive-with-some-names-before-the-values/53488)
**Author:** [@ImInLoveWithCacao](https://discuss.elastic.co/u/ImInLoveWithCacao)\ **Replies:** 0\ **Last updated:** [June 21, 2016, 11:51am UTC](https://discuss.elastic.co/t/kv-plugin-recursive-with-some-names-before-the-values/53488 "2016-06-21T11:51:26Z")
Hey, I'm currently trying to use the kv plugin on some log event. The part I want to parse with kv looks like this : key1 = value1, key2 = some\_function\_name(key21 = value21, key22 = value22), key3 = value3 So t… --- ## [ELK with Kafka](https://discuss.elastic.co/t/elk-with-kafka/53479)
**Author:** [@vienodp](https://discuss.elastic.co/u/vienodp)\ **Replies:** 0\ **Last updated:** [June 21, 2016, 10:10am UTC](https://discuss.elastic.co/t/elk-with-kafka/53479 "2016-06-21T10:10:40Z")
Hi , I am trying to introduce Kafka in ELK. I have 2 servers On first, configured ELK , filebeat and kafka input plugin On second, installed Kafka, logstash, kafka output plugin I am trying to push my second machine… --- ## [grokparseFailure](https://discuss.elastic.co/t/grokparsefailure/53467)
**Author:** [@supermario9](https://discuss.elastic.co/u/supermario9)\ **Replies:** 2\ **Last updated:** [June 21, 2016, 9:40am UTC](https://discuss.elastic.co/t/grokparsefailure/53467 "2016-06-21T09:40:47Z")
Hello, Log: 2016-05-15 00:00:01,305 fileUtils.py(167): INFO Finished removing hpsp\_msaLogRotate from the system. I get a grokparsefailure for the following: grok { match =\> {"message" =\> "%{TIMESTAMP\_ISO8601:timestam… --- ## [Best practises for logging from mobile devices](https://discuss.elastic.co/t/best-practises-for-logging-from-mobile-devices/53473)
**Author:** [@spaghetti0](https://discuss.elastic.co/u/spaghetti0)\ **Replies:** 0\ **Last updated:** [June 21, 2016, 9:31am UTC](https://discuss.elastic.co/t/best-practises-for-logging-from-mobile-devices/53473 "2016-06-21T09:31:51Z")
Hello, I'm setting up centralized login for my organization which supports both web and mobile clients. I've a question regarding the best architecture to use for this. I would like to avoid having unstructured logs henc… --- ## [Problem with Logstash and Influxdb](https://discuss.elastic.co/t/problem-with-logstash-and-influxdb/53471)
**Author:** [@srneodani](https://discuss.elastic.co/u/srneodani)\ **Replies:** 0\ **Last updated:** [June 21, 2016, 9:16am UTC](https://discuss.elastic.co/t/problem-with-logstash-and-influxdb/53471 "2016-06-21T09:16:45Z")
Hi folks, I read a mysql table by logstash and I want save output in influxdb. The problem I've found is that timestamp generated by logstash matches whit any of them, then when the data is saved in Influx only saved t… --- ## [Getting environment variables as array](https://discuss.elastic.co/t/getting-environment-variables-as-array/53468)
**Author:** [@Raul\_Uria](https://discuss.elastic.co/u/Raul_Uria)\ **Replies:** 1\ **Last updated:** [June 21, 2016, 8:57am UTC](https://discuss.elastic.co/t/getting-environment-variables-as-array/53468 "2016-06-21T08:57:32Z")
Hi, I´m trying to set PATH in file input plugin from env variables, but doesn´t seem to work. I run a "complex" logic to set files that I want to process with logstash before execute it, so I set a env variable with pa… --- ## [Logstash/ES date format issue](https://discuss.elastic.co/t/logstash-es-date-format-issue/33933)
**Author:** [@xtingray](https://discuss.elastic.co/u/xtingray)\ **Replies:** 6\ **Last updated:** [June 21, 2016, 8:24am UTC](https://discuss.elastic.co/t/logstash-es-date-format-issue/33933 "2016-06-21T08:24:43Z")
I have been playing with logstash filters, receiving log records with this format: 2015-11-05 17:37:01 This is a log message Using this filter: grok { match =\> { "message" =\> "%{DATE:date} %{TIME:time} %{GREE… --- ## [Logstash causing jvm to crash if kept running for long intervals](https://discuss.elastic.co/t/logstash-causing-jvm-to-crash-if-kept-running-for-long-intervals/53444)
**Author:** [@sameer\_srivastava](https://discuss.elastic.co/u/sameer_srivastava)\ **Replies:** 0\ **Last updated:** [June 21, 2016, 7:38am UTC](https://discuss.elastic.co/t/logstash-causing-jvm-to-crash-if-kept-running-for-long-intervals/53444 "2016-06-21T07:38:55Z")
We are running logstash (2.3.2) on few machine with windows 7. JDK version - 1.8.0\_31 This machine runs some other java processes also at random intervals (Using logstash to collect logs from these java processes). Off … --- ## [Lumberjack - LogStash SizedQueueTimeout TimeoutError](https://discuss.elastic.co/t/lumberjack-logstash-sizedqueuetimeout-timeouterror/53437)
**Author:** [@YLombardi](https://discuss.elastic.co/u/YLombardi)\ **Replies:** 0\ **Last updated:** [June 21, 2016, 7:21am UTC](https://discuss.elastic.co/t/lumberjack-logstash-sizedqueuetimeout-timeouterror/53437 "2016-06-21T07:21:52Z")
Hi. I use ELK with Logstash forwarder. Sometimes I have a lot of error in my logstash.log : Here is my logstash configuration : input { lumberjack { port =\> 5000 type =\> "logs" ssl\_certificate =\> "/etc/logstas… --- ## [Date and number format](https://discuss.elastic.co/t/date-and-number-format/53323)
**Author:** [@mat17](https://discuss.elastic.co/u/mat17)\ **Replies:** 1\ **Last updated:** [June 21, 2016, 6:00am UTC](https://discuss.elastic.co/t/date-and-number-format/53323 "2016-06-21T06:00:55Z")
Hello, I'm trying to integrate a csv files but i have troubles with date and number format. Files contain the information below 2016-06-01,xxxxx,AD,NA,4.31 .... I use the csv filter to collect datas then i try to add… --- ## [File Input is picky!](https://discuss.elastic.co/t/file-input-is-picky/53396)
**Author:** [@nsalt](https://discuss.elastic.co/u/nsalt)\ **Replies:** 1\ **Last updated:** [June 21, 2016, 5:57am UTC](https://discuss.elastic.co/t/file-input-is-picky/53396 "2016-06-21T05:57:07Z")
Hey team...Can anyone tell my why my logstash setup is reading one particular log file but not another in the same directory? If I set the path to the working file it starts reading the file and inputing to elasticsearch… --- ## [Get "Connection refused" error when I tried "Setting Up an Advanced Logstash Pipeline"](https://discuss.elastic.co/t/get-connection-refused-error-when-i-tried-setting-up-an-advanced-logstash-pipeline/53383)
**Author:** [@Kris-Li-lqx](https://discuss.elastic.co/u/Kris-Li-lqx)\ **Replies:** 1\ **Last updated:** [June 21, 2016, 5:56am UTC](https://discuss.elastic.co/t/get-connection-refused-error-when-i-tried-setting-up-an-advanced-logstash-pipeline/53383 "2016-06-21T05:56:15Z")
Hey guys, I'm following the tutorial but encountered an error saying : Connection refused {:class=\>"Manticore::SocketException", :level=\>:error} for "Setting Up an Advanced Logstash Pipeline". I just followed the off… --- ## [Create events in logstash?](https://discuss.elastic.co/t/create-events-in-logstash/53325)
**Author:** [@asp](https://discuss.elastic.co/u/asp)\ **Replies:** 1\ **Last updated:** [June 21, 2016, 5:54am UTC](https://discuss.elastic.co/t/create-events-in-logstash/53325 "2016-06-21T05:54:26Z")
Hi, Is it possible to create own / virtual events in logstash? I intent to do the following: I receive following log line: timestamp, value\_1, value\_2, value\_3 now I want to create 3 events out of it: Code: Select … --- ## [How to split the log data message ( log4j )](https://discuss.elastic.co/t/how-to-split-the-log-data-message-log4j/53123)
**Author:** [@venkat-accel](https://discuss.elastic.co/u/venkat-accel)\ **Replies:** 3\ **Last updated:** [June 21, 2016, 5:46am UTC](https://discuss.elastic.co/t/how-to-split-the-log-data-message-log4j/53123 "2016-06-21T05:46:34Z")
Hi, I have log4j data , i created config file and able to load the data in kibana but all the log data , each line is loaded as single filed called "message ". But i want to split each filed in the message as separ… --- ## [How to remove first line of a multiline message](https://discuss.elastic.co/t/how-to-remove-first-line-of-a-multiline-message/53156)
**Author:** [@markus](https://discuss.elastic.co/u/markus)\ **Replies:** 1\ **Last updated:** [June 21, 2016, 5:33am UTC](https://discuss.elastic.co/t/how-to-remove-first-line-of-a-multiline-message/53156 "2016-06-21T05:33:47Z")
hi, i am parsing a multiline log containg lots of garbage and in between pearls of xml data that I'd like to extract. my data looks like this: some Text I'd like to drop {} some Text with a token signaling XML Docum… --- ## [JMS Queue consume based on criteria](https://discuss.elastic.co/t/jms-queue-consume-based-on-criteria/53005)
**Author:** [@ckayay](https://discuss.elastic.co/u/ckayay)\ **Replies:** 1\ **Last updated:** [June 21, 2016, 5:31am UTC](https://discuss.elastic.co/t/jms-queue-consume-based-on-criteria/53005 "2016-06-21T05:31:47Z")
Hi All, Is it possible to consume a message in JMS queue from Logstash, based on the content of the message? I am trying to read messages which is greater than 5 min old from the JMS Queue with log stash. (by looking a… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1136) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1138)