# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1251 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1252 --- ## [Recursively Grok lines and streams](https://discuss.elastic.co/t/recursively-grok-lines-and-streams/25671)
**Author:** [@naisanza](https://discuss.elastic.co/u/naisanza)\ **Replies:** 0\ **Last updated:** [July 15, 2015, 7:53pm UTC](https://discuss.elastic.co/t/recursively-grok-lines-and-streams/25671 "2015-07-15T19:53:52Z")
I'm looking to use grok to parse through lines and streams of data. I'll explain how. Let's say I have a line of data of: 221.37.88.36.bc.googleusercontent.com,63.88.73.122,63.88.73.0,-,,-,Google Inc.,Mountain View,C… --- ## [Same field name with different types?](https://discuss.elastic.co/t/same-field-name-with-different-types/25742)
**Author:** [@brett](https://discuss.elastic.co/u/brett)\ **Replies:** 2\ **Last updated:** [July 16, 2015, 7:34pm UTC](https://discuss.elastic.co/t/same-field-name-with-different-types/25742 "2015-07-16T19:34:00Z")
Similar problem to this discussion, but I was hoping for some input on what the best solution would be. We're inputting json messages from multiple exchanges using the rabbitmq input plugin. Several of these types of m… --- ## [Logstash grok pattern for Wowza](https://discuss.elastic.co/t/logstash-grok-pattern-for-wowza/25641)
**Author:** [@gugansankar](https://discuss.elastic.co/u/gugansankar)\ **Replies:** 3\ **Last updated:** [July 16, 2015, 4:14pm UTC](https://discuss.elastic.co/t/logstash-grok-pattern-for-wowza/25641 "2015-07-16T16:14:04Z")
Can someone please share me the grok pattern for Wowza?. I have tried with below one. The pattern shows Ok on http://grokdebug.herokuapp.com/ site. But in logstash log , it went to "grokfailure section". I referred t… --- ## [Logstash grok timestamp from message: issue with multiple timestamp format](https://discuss.elastic.co/t/logstash-grok-timestamp-from-message-issue-with-multiple-timestamp-format/25717)
**Author:** [@sunilmchaudhari](https://discuss.elastic.co/u/sunilmchaudhari)\ **Replies:** 1\ **Last updated:** [July 16, 2015, 11:50am UTC](https://discuss.elastic.co/t/logstash-grok-timestamp-from-message-issue-with-multiple-timestamp-format/25717 "2015-07-16T11:50:13Z")
Hi I am taking out timestamp field from message and setting it as eventTime as new field. below is the code I am using. grok { match =\> { "message" =\> "%{TIMESTAMP\_ISO8601:eventTime}" } } date { match =\>… --- ## [Create field if it's not equal "Undefined"](https://discuss.elastic.co/t/create-field-if-its-not-equal-undefined/25718)
**Author:** [@Smasell](https://discuss.elastic.co/u/Smasell)\ **Replies:** 1\ **Last updated:** [July 16, 2015, 11:48am UTC](https://discuss.elastic.co/t/create-field-if-its-not-equal-undefined/25718 "2015-07-16T11:48:44Z")
Hi! I have logs: days\_in\_game: 198; pvp\_rating: 63; provider: Undefined; And i want to crete field "provider" in documents when it's not equal "Undefined". Can you write me filter config for this? THX! --- ## [How to handle Exceptions](https://discuss.elastic.co/t/how-to-handle-exceptions/25307)
**Author:** [@ABecker](https://discuss.elastic.co/u/ABecker)\ **Replies:** 9\ **Last updated:** [July 16, 2015, 10:46am UTC](https://discuss.elastic.co/t/how-to-handle-exceptions/25307 "2015-07-16T10:46:55Z")
Hello Logstash Community, in my logfiles I have different Exceptions from different Stack Traces. How can I handle singleline and multiline entries within one logfile? The normal logfile looks like: \<191\>Jul 10 13:… --- ## [Changing logstash @timestamp timezone](https://discuss.elastic.co/t/changing-logstash-timestamp-timezone/25711)
**Author:** [@Deb](https://discuss.elastic.co/u/Deb)\ **Replies:** 1\ **Last updated:** [July 16, 2015, 10:13am UTC](https://discuss.elastic.co/t/changing-logstash-timestamp-timezone/25711 "2015-07-16T10:13:39Z")
Hi, I am new to logstash and playing around with it. I have a simple set-up in which I am forwarding logs using logstash-forwarder to logstash. Both logstash and logstash-forwarder are in the same machine. My logstas… --- ## [Sincedb files not created](https://discuss.elastic.co/t/sincedb-files-not-created/25542)
**Author:** [@vilas](https://discuss.elastic.co/u/vilas)\ **Replies:** 5\ **Last updated:** [July 15, 2015, 6:12pm UTC](https://discuss.elastic.co/t/sincedb-files-not-created/25542 "2015-07-15T18:12:44Z")
Hi, The following is my sample Logstash configuration for file input: file { path =\> ".../logs/sample.log" start\_position =\> "beginning" type =\> "sample" … --- ## ["\ **Author:** [@Sriram\_Raviteja](https://discuss.elastic.co/u/Sriram_Raviteja)\ **Replies:** 0\ **Last updated:** [July 15, 2015, 5:35pm UTC](https://discuss.elastic.co/t/file-java-lang-unsupportedoperation-exception-when-indexing-data/25658 "2015-07-15T17:35:50Z") Hi, Im trying to index data through Logstash. which each document is of size 400kb. around 1500 documents. got the following error. after indexing some data. can any one help me . what this is about.. when left for … --- ## [How does "Keep\_empty\_captures" work](https://discuss.elastic.co/t/how-does-keep-empty-captures-work/25657)
**Author:** [@DavidL](https://discuss.elastic.co/u/DavidL)\ **Replies:** 0\ **Last updated:** [July 15, 2015, 5:34pm UTC](https://discuss.elastic.co/t/how-does-keep-empty-captures-work/25657 "2015-07-15T17:34:51Z")
I am trying to used this grok pattern %{Myfield:Path}\\t%{Myfield:Status}\\t\\t%{Myfield:model\_package} to parse out this two /folder/1/2\\tSuccess\\t\\tsometext, /folder/1/2\\t\\t\\tsometext, I turned on Keep\_Empty\_Capt… --- ## [Logstash Shipper](https://discuss.elastic.co/t/logstash-shipper/25649)
**Author:** [@mgirndt](https://discuss.elastic.co/u/mgirndt)\ **Replies:** 1\ **Last updated:** [July 15, 2015, 4:41pm UTC](https://discuss.elastic.co/t/logstash-shipper/25649 "2015-07-15T16:41:49Z")
I have logstash installed and logs are not being sent to redis on my elasticsearch server. My question is when starting logstash how do I tell it to use the shipper I created? --- ## [Install lastest version of logstash on centos 7](https://discuss.elastic.co/t/install-lastest-version-of-logstash-on-centos-7/25640)
**Author:** [@ukanabar](https://discuss.elastic.co/u/ukanabar)\ **Replies:** 1\ **Last updated:** [July 15, 2015, 2:57pm UTC](https://discuss.elastic.co/t/install-lastest-version-of-logstash-on-centos-7/25640 "2015-07-15T14:57:51Z")
Hello Friends, I am looking an article,web page or steps using which i can install logstash latest version and related components. We have IBM HTTP server Kibana 4.1.1 Logstash 1.5.2 Elasticsearch 1.6.0 Redis 3.0.2 Mai… --- ## [Geoip-Filter - Usage with GeoLite2 Databases?](https://discuss.elastic.co/t/geoip-filter-usage-with-geolite2-databases/25634)
**Author:** [@german23](https://discuss.elastic.co/u/german23)\ **Replies:** 0\ **Last updated:** [July 15, 2015, 1:59pm UTC](https://discuss.elastic.co/t/geoip-filter-usage-with-geolite2-databases/25634 "2015-07-15T13:59:58Z")
Hey guys, we are using the geoip filter of LS for quite a while with the GeoLite Database, as suggested in the documentation of LS. This works quite nicely, although now is a new "generation" of databases from Maxmin… --- ## [Create new field based on msg filed in logstash](https://discuss.elastic.co/t/create-new-field-based-on-msg-filed-in-logstash/25527)
**Author:** [@abathula](https://discuss.elastic.co/u/abathula)\ **Replies:** 10\ **Last updated:** [July 15, 2015, 1:42pm UTC](https://discuss.elastic.co/t/create-new-field-based-on-msg-filed-in-logstash/25527 "2015-07-15T13:42:51Z")
Hi All, Here i want to create a new field Invoice\_IID based on msg filed, contains Invoice\_IID value in log line msg filed. "msg" =\> "Finished Creating Parent Invoices for Invoice\_IID: 80000000-41fb-1638-cd42-ffff08… --- ## [Get timezone for the log event arriving in logstash](https://discuss.elastic.co/t/get-timezone-for-the-log-event-arriving-in-logstash/25624)
**Author:** [@DK\_3](https://discuss.elastic.co/u/DK_3)\ **Replies:** 1\ **Last updated:** [July 15, 2015, 1:40pm UTC](https://discuss.elastic.co/t/get-timezone-for-the-log-event-arriving-in-logstash/25624 "2015-07-15T13:40:29Z")
I'm using the date filter to create the @timestamp field based on a log\_timestamp field "log\_timestamp", "%{year}-%{month}-%{day} %{time},%{ms}" date { match =\> \[ "log\_timestamp", "YYYY-MM-dd HH:mm:ss,SSS" \] ti… --- ## [Replace the indexed data with new data](https://discuss.elastic.co/t/replace-the-indexed-data-with-new-data/25442)
**Author:** [@sanju1323](https://discuss.elastic.co/u/sanju1323)\ **Replies:** 10\ **Last updated:** [July 15, 2015, 11:50am UTC](https://discuss.elastic.co/t/replace-the-indexed-data-with-new-data/25442 "2015-07-15T11:50:29Z")
Hi, I have an Index named "sanjay\_data" and I want to replace the already existing data in the index with the new data. Instead of deleting the index and creating it again, is there any possibility? Please help me … --- ## [Does logstash support input file rename?](https://discuss.elastic.co/t/does-logstash-support-input-file-rename/25604)
**Author:** [@Jason\_Zheng](https://discuss.elastic.co/u/Jason_Zheng)\ **Replies:** 0\ **Last updated:** [July 15, 2015, 9:55am UTC](https://discuss.elastic.co/t/does-logstash-support-input-file-rename/25604 "2015-07-15T09:55:50Z")
Hi All, I have a question, when I setting input file of logstash.conf as following input { file { path =\> "/data/logs/count/\*" } } STEP 1. I put mesaage #1 to output1.log STEP 2. I rename output1.log to output2.log (… --- ## [Duplicate Message sent?](https://discuss.elastic.co/t/duplicate-message-sent/25494)
**Author:** [@Jason\_Zheng](https://discuss.elastic.co/u/Jason_Zheng)\ **Replies:** 3\ **Last updated:** [July 15, 2015, 9:50am UTC](https://discuss.elastic.co/t/duplicate-message-sent/25494 "2015-07-15T09:50:03Z")
Hi All, there are 5 hosts (logstash-forwarder \> redis \> logstash-center \> elasticsearch \< Kibana), when I push an message to logstash-forwarder, redis always receives 2 messages, I try to delete the keys of list of red… --- ## [Field reference syntax](https://discuss.elastic.co/t/field-reference-syntax/25589)
**Author:** [@eeveaud](https://discuss.elastic.co/u/eeveaud)\ **Replies:** 3\ **Last updated:** [July 15, 2015, 9:45am UTC](https://discuss.elastic.co/t/field-reference-syntax/25589 "2015-07-15T09:45:35Z")
when using the field reference syntax, is the "resolution" of the value associated to a specific field performed by LS or it is performed at plugin level ? for one of our specific needs we have to develop a filter plu… --- ## [Having problem enable logstash to talk to redis](https://discuss.elastic.co/t/having-problem-enable-logstash-to-talk-to-redis/25585)
**Author:** [@zpp](https://discuss.elastic.co/u/zpp)\ **Replies:** 3\ **Last updated:** [July 15, 2015, 8:44am UTC](https://discuss.elastic.co/t/having-problem-enable-logstash-to-talk-to-redis/25585 "2015-07-15T08:44:39Z")
Hi, i installed logstash as a shipper on a window box with the help of nssm, and configured as below: input { file { type =\> "autolog" path =\> "E:/elkcluster/log" } } output { redis { host =\> "xxx.xxx.xxx.xxx… --- ## [REST input for logstash?](https://discuss.elastic.co/t/rest-input-for-logstash/25580)
**Author:** [@sunilmchaudhari](https://discuss.elastic.co/u/sunilmchaudhari)\ **Replies:** 1\ **Last updated:** [July 15, 2015, 6:37am UTC](https://discuss.elastic.co/t/rest-input-for-logstash/25580 "2015-07-15T06:37:18Z")
Hello, We have replaced our old central logging with ELK. However, we have one client which used to talk with earlier service using REST template. Can we somehow, store xml ( coming from Rest post request) to redis or ra… --- ## [Logstash v1.5.2 crash](https://discuss.elastic.co/t/logstash-v1-5-2-crash/25579)
**Author:** [@judy](https://discuss.elastic.co/u/judy)\ **Replies:** 0\ **Last updated:** [July 15, 2015, 6:03am UTC](https://discuss.elastic.co/t/logstash-v1-5-2-crash/25579 "2015-07-15T06:03:36Z")
logstash plugin logstash was stopped due to a crash caused by the infinite. This workaround? A plugin had an unrecoverable error. Will restart this plugin. Plugin: \\["/usr1/s3/alpha/bz/\*"… --- ## [Multiple grok matching fileds](https://discuss.elastic.co/t/multiple-grok-matching-fileds/25523)
**Author:** [@abathula](https://discuss.elastic.co/u/abathula)\ **Replies:** 2\ **Last updated:** [July 15, 2015, 4:24am UTC](https://discuss.elastic.co/t/multiple-grok-matching-fileds/25523 "2015-07-15T04:24:46Z")
This is my log line: 2015-04-14 03:28:12,628 \[8\] DEBUG NCR.XE.Component.MessageHandler.HistoryEventMessageProcessor - Starting RentalTransactionMessageProcessor message type with the data \ **Author:** [@jcollie](https://discuss.elastic.co/u/jcollie)\ **Replies:** 4\ **Last updated:** [July 15, 2015, 4:20am UTC](https://discuss.elastic.co/t/grok-debugger-not-working/25530 "2015-07-15T04:20:59Z") Just tried to go to the Grok Debugger (https://grokdebug.herokuapp.com/) and it's giving me an application error... --- ## [Not able to config JDBC input](https://discuss.elastic.co/t/not-able-to-config-jdbc-input/25505)
**Author:** [@PavanRGowda](https://discuss.elastic.co/u/PavanRGowda)\ **Replies:** 2\ **Last updated:** [July 15, 2015, 4:04am UTC](https://discuss.elastic.co/t/not-able-to-config-jdbc-input/25505 "2015-07-15T04:04:47Z")
Hello everyone, below is my configuration - input { jdbc { jdbc\_driver\_class =\> "org.apache.derby.jdbc.EmbeddedDriver" jdbc\_connection\_string =\> "jdbc:derby://localhost:3306/test" jdbc\_user =\> "root" … --- ## [Logstash JDBC input configuration](https://discuss.elastic.co/t/logstash-jdbc-input-configuration/25535)
**Author:** [@PavanRGowda](https://discuss.elastic.co/u/PavanRGowda)\ **Replies:** 1\ **Last updated:** [July 15, 2015, 2:19am UTC](https://discuss.elastic.co/t/logstash-jdbc-input-configuration/25535 "2015-07-15T02:19:59Z")
I've googled a lot about JDBC input configuration but failed to find the suitable one. Please help me by posting the configuration or providing me a reference to the suitable content. --- ## [Logstash holding open deleted logs](https://discuss.elastic.co/t/logstash-holding-open-deleted-logs/25563)
**Author:** [@Brandon\_Metcalf](https://discuss.elastic.co/u/Brandon_Metcalf)\ **Replies:** 0\ **Last updated:** [July 14, 2015, 11:27pm UTC](https://discuss.elastic.co/t/logstash-holding-open-deleted-logs/25563 "2015-07-14T23:27:29Z")
Hi, we are using version 1.5.1-1 of logstash on centos 6 and logstash is holding open for days files that have been deleted by rotation. I've found a lot of discussion around this issue and logstash-forwarder, but nothi… --- ## [Logstash plugin flush](https://discuss.elastic.co/t/logstash-plugin-flush/25561)
**Author:** [@karthikbox](https://discuss.elastic.co/u/karthikbox)\ **Replies:** 0\ **Last updated:** [July 14, 2015, 11:17pm UTC](https://discuss.elastic.co/t/logstash-plugin-flush/25561 "2015-07-14T23:17:53Z")
Hi all, I've been writing a custom filter plugin for logstash. I have a cleanup method which needs to be called periodically (lets say 3 sec). I understand that this is achievable by overriding "flush" method in my plu… --- ## [CSV within CSV](https://discuss.elastic.co/t/csv-within-csv/25464)
**Author:** [@Khoa\_Nguyen1](https://discuss.elastic.co/u/Khoa_Nguyen1)\ **Replies:** 6\ **Last updated:** [July 14, 2015, 2:13pm UTC](https://discuss.elastic.co/t/csv-within-csv/25464 "2015-07-14T14:13:45Z")
My CSV records have fields that are CSV themselves (with different delimiter). For example: a,b,c,d1|d2|d3,e,f1|f2,g,h How do I write a logstash configuration file to handle this? Thanks --- ## [Logstash missing few lines /not detecting new files](https://discuss.elastic.co/t/logstash-missing-few-lines-not-detecting-new-files/25526)
**Author:** [@sugumar](https://discuss.elastic.co/u/sugumar)\ **Replies:** 0\ **Last updated:** [July 14, 2015, 2:03pm UTC](https://discuss.elastic.co/t/logstash-missing-few-lines-not-detecting-new-files/25526 "2015-07-14T14:03:59Z")
I am using Logstash 1.5.2 and elasticsearch 1.6.0 in windows 7 Below issues are could not address myself. 1. When i am using file input with wildcard option the newly added files are not getting processed. 2. If i m… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1250) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1252)