# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1234 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1235 --- ## [Deleting documents from couchdb doesn't delete it from ES index](https://discuss.elastic.co/t/deleting-documents-from-couchdb-doesnt-delete-it-from-es-index/29807)
**Author:** [@imad](https://discuss.elastic.co/u/imad)\ **Replies:** 8\ **Last updated:** [September 25, 2015, 7:45pm UTC](https://discuss.elastic.co/t/deleting-documents-from-couchdb-doesnt-delete-it-from-es-index/29807 "2015-09-25T19:45:07Z")
I wondering if there is anything extra that I need do to delete a document from ES index when its deleted from my couchdb. Here is how my config file looks like: input { couchdb\_changes { db =\> "users" ho… --- ## [Redis Input, Logstash 1.5.4](https://discuss.elastic.co/t/redis-input-logstash-1-5-4/29950)
**Author:** [@jlam](https://discuss.elastic.co/u/jlam)\ **Replies:** 3\ **Last updated:** [September 25, 2015, 4:33pm UTC](https://discuss.elastic.co/t/redis-input-logstash-1-5-4/29950 "2015-09-25T16:33:36Z")
There seems to be an issue with the redis input with logstash. From what I can tell, it is processing the redis lists but it is going slowly. The length of the lists keep growing. I haven't able to isolate the proble… --- ## [Non standard log lines](https://discuss.elastic.co/t/non-standard-log-lines/29942)
**Author:** [@treehouse](https://discuss.elastic.co/u/treehouse)\ **Replies:** 1\ **Last updated:** [September 25, 2015, 3:52pm UTC](https://discuss.elastic.co/t/non-standard-log-lines/29942 "2015-09-25T15:52:49Z")
I'm having some trouble with logstash grok filter. Specifically matching the log to a pattern. Say I have two log messages that looks something like this: 10.12.6.190 \[Thu, 24 Sep 2015 18:17:52 GMT\] 404 - 192.168.48.2… --- ## [Incorrect path being indexed](https://discuss.elastic.co/t/incorrect-path-being-indexed/29763)
**Author:** [@mburlington](https://discuss.elastic.co/u/mburlington)\ **Replies:** 3\ **Last updated:** [September 25, 2015, 1:35pm UTC](https://discuss.elastic.co/t/incorrect-path-being-indexed/29763 "2015-09-25T13:35:00Z")
Hi, I’m looking for help for an issue I have with Logsatsh 1.5.4. I’m using Logstash to read a number of log files from various Java services. The filenames are like this: AvailabilityAndFulfilment.log DocumentGene… --- ## [1.4.1 agent to 1.5.x consumer event/message compatibility?](https://discuss.elastic.co/t/1-4-1-agent-to-1-5-x-consumer-event-message-compatibility/30014)
**Author:** [@bitsofinfo](https://discuss.elastic.co/u/bitsofinfo)\ **Replies:** 1\ **Last updated:** [September 25, 2015, 1:33pm UTC](https://discuss.elastic.co/t/1-4-1-agent-to-1-5-x-consumer-event-message-compatibility/30014 "2015-09-25T13:33:44Z")
We have a lot of 1.4.1 agents deployed, they basically consume file events (lightly annotate them with a few fields) and then forward as-is onto Redis, and then are consumed by another 1.4.1 logstash instance that does h… --- ## [Logstash output s3 didn't work with condition](https://discuss.elastic.co/t/logstash-output-s3-didnt-work-with-condition/29905)
**Author:** [@vankhoa011](https://discuss.elastic.co/u/vankhoa011)\ **Replies:** 3\ **Last updated:** [September 25, 2015, 8:10am UTC](https://discuss.elastic.co/t/logstash-output-s3-didnt-work-with-condition/29905 "2015-09-25T08:10:50Z")
Hi all, I make a config for logstash , I want if the logs match logtype==request and env==prd , it will save to Bucket 1 If the logstash logtype==request and env==stg , i will save to Bucket2. However, there is some… --- ## [Drop filter](https://discuss.elastic.co/t/drop-filter/29951)
**Author:** [@treehouse](https://discuss.elastic.co/u/treehouse)\ **Replies:** 1\ **Last updated:** [September 25, 2015, 12:20am UTC](https://discuss.elastic.co/t/drop-filter/29951 "2015-09-25T00:20:54Z")
Newbie question here: but why would the drop filter have all of these settings like add\_field, add\_tag, etc if all it does is drop this particular event? I must be missing something here. thanks --- ## [Validating .conf file](https://discuss.elastic.co/t/validating-conf-file/29939)
**Author:** [@Didier](https://discuss.elastic.co/u/Didier)\ **Replies:** 1\ **Last updated:** [September 24, 2015, 8:14pm UTC](https://discuss.elastic.co/t/validating-conf-file/29939 "2015-09-24T20:14:14Z")
Hello, We wrote our first configuration file for a type of log and we would like to test out our code before moving along further. What is the procedure to test it out? Do I run logstash, and then run the a GET comm… --- ## [Question on "Multiple Connections for Logstash High Availability" diagram published in logstash documentation](https://discuss.elastic.co/t/question-on-multiple-connections-for-logstash-high-availability-diagram-published-in-logstash-documentation/29800)
**Author:** [@RajkumarV](https://discuss.elastic.co/u/RajkumarV)\ **Replies:** 9\ **Last updated:** [September 24, 2015, 7:06pm UTC](https://discuss.elastic.co/t/question-on-multiple-connections-for-logstash-high-availability-diagram-published-in-logstash-documentation/29800 "2015-09-24T19:06:15Z")
Hi Logstash Team Can you explain whether duplicate events or log messages will be processed if we implement the logstash stack as per HA architecture explained in 2nd diagram of "Multiple Connections for Logstash High … --- ## [Logdate field with format YYYY-MM-dd HH:mm:ss,SSS converted to 2015-01-01T00:33:33.000Z](https://discuss.elastic.co/t/logdate-field-with-format-yyyy-mm-dd-hh-mm-ss-sss-converted-to-2015-01-01t00-33-33-000z/29826)
**Author:** [@ushadatt](https://discuss.elastic.co/u/ushadatt)\ **Replies:** 3\ **Last updated:** [September 24, 2015, 2:29pm UTC](https://discuss.elastic.co/t/logdate-field-with-format-yyyy-mm-dd-hh-mm-ss-sss-converted-to-2015-01-01t00-33-33-000z/29826 "2015-09-24T14:29:39Z")
I am parsing a file with date format as YYYY-MM-dd HH:mm:ss.SSS but in Elastic, it is indexed as 2015-01-01T00:33:33.000Z with a T in between date and time. date{ match =\> \["Date", "yyyy-MM-dd HH:mm:ss" \] target =\> … --- ## [JSON Array in Logstash](https://discuss.elastic.co/t/json-array-in-logstash/25655)
**Author:** [@BZen](https://discuss.elastic.co/u/BZen)\ **Replies:** 2\ **Last updated:** [September 24, 2015, 2:23pm UTC](https://discuss.elastic.co/t/json-array-in-logstash/25655 "2015-09-24T14:23:52Z")
I have a field which look like this. "log":"\[{\\"id\\":\\"1233\\"}, ...some more json\] I want to have a field like id =\> 1233 I tried using json filter plugin but I am getting a parse error. What is the ideal way of d… --- ## [Logstash and flexlm log](https://discuss.elastic.co/t/logstash-and-flexlm-log/29766)
**Author:** [@el\_tunisiano38](https://discuss.elastic.co/u/el_tunisiano38)\ **Replies:** 4\ **Last updated:** [September 24, 2015, 2:05pm UTC](https://discuss.elastic.co/t/logstash-and-flexlm-log/29766 "2015-09-24T14:05:05Z")
Hi everyone, I am newbie at logstash and would like to parse a flexlm log file. The problem is the log is not easy to parse. (lmgrd) FlexNet Licensing blabla TIMESTAMP 06/12/2015 09:25:48 (MLM) OUT: "MATLAB" USER2@M… --- ## [Logstash error "undefined method \`close' for nil:NilClass"](https://discuss.elastic.co/t/logstash-error-undefined-method-close-for-nil-nilclass/29784)
**Author:** [@Navneet\_Mathpal](https://discuss.elastic.co/u/Navneet_Mathpal)\ **Replies:** 6\ **Last updated:** [September 24, 2015, 2:04pm UTC](https://discuss.elastic.co/t/logstash-error-undefined-method-close-for-nil-nilclass/29784 "2015-09-24T14:04:31Z")
Hi , I was running logstash in windows environment and it was running fine , but when I was trying the same thing in Linux machine with same version of logstash (1.5.4) it is showing an error that The error reported is… --- ## [Fingerprint unable to fingerprint @timestamp](https://discuss.elastic.co/t/fingerprint-unable-to-fingerprint-timestamp/29855)
**Author:** [@cpattonj](https://discuss.elastic.co/u/cpattonj)\ **Replies:** 3\ **Last updated:** [September 24, 2015, 11:34am UTC](https://discuss.elastic.co/t/fingerprint-unable-to-fingerprint-timestamp/29855 "2015-09-24T11:34:53Z")
With input lines like: {"count":10, "timestamp":"2015-09-22T00:00:00.000Z"} I am unable to fingerprint based off of the date extracted by my date filter. For fingerprint, it's exactly the same for all of my messages w… --- ## [Logstash ruby plugin to get the value from file and append](https://discuss.elastic.co/t/logstash-ruby-plugin-to-get-the-value-from-file-and-append/29895)
**Author:** [@Navneet\_Mathpal](https://discuss.elastic.co/u/Navneet_Mathpal)\ **Replies:** 2\ **Last updated:** [September 24, 2015, 6:39am UTC](https://discuss.elastic.co/t/logstash-ruby-plugin-to-get-the-value-from-file-and-append/29895 "2015-09-24T06:39:02Z")
Hi, I was trying to get the first line from the file and try to append the line into my logs,I am able to get the value from file but not able to append the value .. I am doing like if\[classname\] == "java.io.NotSeria… --- ## [Logstash\_forwarder connected to lostash-server-IP but never receive event](https://discuss.elastic.co/t/logstash-forwarder-connected-to-lostash-server-ip-but-never-receive-event/28675)
**Author:** [@soniaeli](https://discuss.elastic.co/u/soniaeli)\ **Replies:** 9\ **Last updated:** [September 24, 2015, 6:38am UTC](https://discuss.elastic.co/t/logstash-forwarder-connected-to-lostash-server-ip-but-never-receive-event/28675 "2015-09-24T06:38:50Z")
I installed elasticsearch, logstash, kibana, ngix and logstash-forwarder at same server to centralized logs. The log file (allapp.json) is a json file with logs entry like this: "{\\"timestamp\\":\\"2015-08-30 19:42:26.72… --- ## [Logstash not indexing anything](https://discuss.elastic.co/t/logstash-not-indexing-anything/24844)
**Author:** [@simonrisberg](https://discuss.elastic.co/u/simonrisberg)\ **Replies:** 26\ **Last updated:** [September 24, 2015, 5:40am UTC](https://discuss.elastic.co/t/logstash-not-indexing-anything/24844 "2015-09-24T05:40:37Z")
Hello! I thought I had Logstash all figured out but apparently I don't. It worked fine for me up until today. It indexed the log files I wanted to but today it just won't index anything at all and in the log I keep getti… --- ## [Poll: RFC 5424 vs. RFC 3164](https://discuss.elastic.co/t/poll-rfc-5424-vs-rfc-3164/29732)
**Author:** [@otisg](https://discuss.elastic.co/u/otisg)\ **Replies:** 2\ **Last updated:** [September 24, 2015, 3:04am UTC](https://discuss.elastic.co/t/poll-rfc-5424-vs-rfc-3164/29732 "2015-09-24T03:04:18Z")
Hi, Some of us here at Sematext debated the adoption of RFC 5424. So instead of guessing, we thought we'd conduct a 1-question poll \[:smile:\] Oh, and try to guess before peeking at the results! \[:smile:\] Thanks… --- ## [Parsing the Windows Message Field (Postprocessing after eventlog filter)](https://discuss.elastic.co/t/parsing-the-windows-message-field-postprocessing-after-eventlog-filter/29857)
**Author:** [@h2k](https://discuss.elastic.co/u/h2k)\ **Replies:** 1\ **Last updated:** [September 23, 2015, 6:59pm UTC](https://discuss.elastic.co/t/parsing-the-windows-message-field-postprocessing-after-eventlog-filter/29857 "2015-09-23T18:59:21Z")
I have used the eventlog filter to pull down Windows Event Logs via Logstash. This has worked handsomely, except for the Message (not message) field, which looks like this: Key file operation. Subject: Security ID: … --- ## [Testing Logstash filters within another Chef wrapper cookbook](https://discuss.elastic.co/t/testing-logstash-filters-within-another-chef-wrapper-cookbook/29471)
**Author:** [@hartfordfive](https://discuss.elastic.co/u/hartfordfive)\ **Replies:** 0\ **Last updated:** [September 17, 2015, 10:10am UTC](https://discuss.elastic.co/t/testing-logstash-filters-within-another-chef-wrapper-cookbook/29471 "2015-09-17T10:10:34Z")
My team and I have implemented a wrapper cookbook that uses the Logstash cookbook and we would like to know if there's a simple way we can test the filters in the Logstash config within Server Spec tests in our wrapper c… --- ## [I need to translate codes into descriptions](https://discuss.elastic.co/t/i-need-to-translate-codes-into-descriptions/29786)
**Author:** [@jjdepaul](https://discuss.elastic.co/u/jjdepaul)\ **Replies:** 2\ **Last updated:** [September 23, 2015, 3:51pm UTC](https://discuss.elastic.co/t/i-need-to-translate-codes-into-descriptions/29786 "2015-09-23T15:51:56Z")
We are processing log files that track how orders flow through the fulfillment workflow. One of the elements we capture is the status code, which is a 2-3 character acronym which translates to a meaningful description… --- ## [Date filter is applying an extra hour to the timestamp](https://discuss.elastic.co/t/date-filter-is-applying-an-extra-hour-to-the-timestamp/25524)
**Author:** [@DK\_3](https://discuss.elastic.co/u/DK_3)\ **Replies:** 6\ **Last updated:** [September 23, 2015, 2:27pm UTC](https://discuss.elastic.co/t/date-filter-is-applying-an-extra-hour-to-the-timestamp/25524 "2015-09-23T14:27:47Z")
It seems the locale or timezone I'm using for the date filter is applying an extra hour to the timestamp. The date filter is applied to the log\_timestamp field and results in an extra hour applied: I've tried both of… --- ## [Weird behavior with regexp conditionals](https://discuss.elastic.co/t/weird-behavior-with-regexp-conditionals/29804)
**Author:** [@spynxo](https://discuss.elastic.co/u/spynxo)\ **Replies:** 1\ **Last updated:** [September 23, 2015, 2:15pm UTC](https://discuss.elastic.co/t/weird-behavior-with-regexp-conditionals/29804 "2015-09-23T14:15:49Z")
Hi All, I am getting a weird behavior with regexp conditionals. I am using Logstash 1.5.4. The logs I am parsing should have the following format under normal conditions: Tue Sep 22 18:34:16 UTC 2015: Log Message I… --- ## [Not getting output as expected](https://discuss.elastic.co/t/not-getting-output-as-expected/29843)
**Author:** [@cpattonj](https://discuss.elastic.co/u/cpattonj)\ **Replies:** 2\ **Last updated:** [September 23, 2015, 1:53pm UTC](https://discuss.elastic.co/t/not-getting-output-as-expected/29843 "2015-09-23T13:53:01Z")
I am trying to establish a pipeline that reads in an entire file every time logstash is started and output it into stdout. I set my start\_position to beginning and sincedb\_path to /dev/null. It's my understanding that th… --- ## [At what point should I use a queue](https://discuss.elastic.co/t/at-what-point-should-i-use-a-queue/29415)
**Author:** [@michaellizhou](https://discuss.elastic.co/u/michaellizhou)\ **Replies:** 3\ **Last updated:** [September 23, 2015, 5:44am UTC](https://discuss.elastic.co/t/at-what-point-should-i-use-a-queue/29415 "2015-09-23T05:44:04Z")
I have been working on just an ELK stack with LSF. But I have come to the cross roads of do I need to put a queue in my system? I have only been on a test environment and do not even know what type of log traffic to expe… --- ## [Grok pattern Help](https://discuss.elastic.co/t/grok-pattern-help/29729)
**Author:** [@Didier](https://discuss.elastic.co/u/Didier)\ **Replies:** 3\ **Last updated:** [September 22, 2015, 7:01pm UTC](https://discuss.elastic.co/t/grok-pattern-help/29729 "2015-09-22T19:01:54Z")
Hello, I would need some help in generating a Grok pattern. I have the following log file. The lines alternate between a line with a timestamp and a message and another line with the log level and another message: Au… --- ## [How To: Back-fill Elasticsearch without losing data](https://discuss.elastic.co/t/how-to-back-fill-elasticsearch-without-losing-data/29793)
**Author:** [@cpattonj](https://discuss.elastic.co/u/cpattonj)\ **Replies:** 1\ **Last updated:** [September 22, 2015, 5:41pm UTC](https://discuss.elastic.co/t/how-to-back-fill-elasticsearch-without-losing-data/29793 "2015-09-22T17:41:45Z")
I want to import from my existing log file as a new file input like so to back-fill my old data: input { path =\> \[ '/path\_local\_to\_es/file'\] start\_position =\> 'beginning' } AND Continue to capture the tail of th… --- ## [Using logstash for parsing logs](https://discuss.elastic.co/t/using-logstash-for-parsing-logs/24329)
**Author:** [@chinmoyd](https://discuss.elastic.co/u/chinmoyd)\ **Replies:** 20\ **Last updated:** [September 22, 2015, 5:33pm UTC](https://discuss.elastic.co/t/using-logstash-for-parsing-logs/24329 "2015-09-22T17:33:46Z")
Hello, I am new to ELK, I have installed Logstash, Elastic search and Kibana. My use case is as below: Logstash will read the application log, parse it and send to Elastic Search. Can you please suggest whether I need … --- ## [Simple logstash for syslog and apache error](https://discuss.elastic.co/t/simple-logstash-for-syslog-and-apache-error/29788)
**Author:** [@lukas\_meier](https://discuss.elastic.co/u/lukas_meier)\ **Replies:** 3\ **Last updated:** [September 22, 2015, 5:29pm UTC](https://discuss.elastic.co/t/simple-logstash-for-syslog-and-apache-error/29788 "2015-09-22T17:29:18Z")
hello, im trying to get a logstash config file, that can parse syslogs and apachelogs too, the following conf shows up in a result: Error: Expected one of #, =\> at line 37, column 17 (byte 722) but i didnt changed … --- ## [No input via udp/syslog plugin](https://discuss.elastic.co/t/no-input-via-udp-syslog-plugin/29778)
**Author:** [@Tweek](https://discuss.elastic.co/u/Tweek)\ **Replies:** 5\ **Last updated:** [September 22, 2015, 2:17pm UTC](https://discuss.elastic.co/t/no-input-via-udp-syslog-plugin/29778 "2015-09-22T14:17:28Z")
My logstash-1.5.4 install does not collect any logs via the udp or the syslog input plugin anymore. I have the following config file: input { file{ path =\> "/opt/logs/tcpdump.log" } udp { port =\> "5… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1233) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1235)