# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1170 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1171 --- ## [Error - Manticore::SocketException: Connection refused](https://discuss.elastic.co/t/error-manticore-socketexception-connection-refused/46475)
**Author:** [@Pradyumna\_Joshi](https://discuss.elastic.co/u/Pradyumna_Joshi)\ **Replies:** 1\ **Last updated:** [April 6, 2016, 6:48am UTC](https://discuss.elastic.co/t/error-manticore-socketexception-connection-refused/46475 "2016-04-06T06:48:50Z")
I am trying to parse Bro IDS logs using ELK stack and getting this error: Connection refused {:class=\>"Manticore::SocketException", :level=\>:error} Connection refused {:class=\>"Manticore::SocketException", :level=\>:err… --- ## [Using Rsyslog for log fowarding](https://discuss.elastic.co/t/using-rsyslog-for-log-fowarding/46402)
**Author:** [@vivc](https://discuss.elastic.co/u/vivc)\ **Replies:** 2\ **Last updated:** [April 6, 2016, 5:49am UTC](https://discuss.elastic.co/t/using-rsyslog-for-log-fowarding/46402 "2016-04-06T05:49:31Z")
Is it a good option to use rsyslog to forward logs from server to logstash instead of filebeat. --- ## [Logstash 2.3 Twitter::Error::Unauthorized](https://discuss.elastic.co/t/logstash-2-3-twitter-error-unauthorized/46187)
**Author:** [@tellus83](https://discuss.elastic.co/u/tellus83)\ **Replies:** 3\ **Last updated:** [April 5, 2016, 11:13pm UTC](https://discuss.elastic.co/t/logstash-2-3-twitter-error-unauthorized/46187 "2016-04-05T23:13:17Z")
Do not work. input { twitter { consumer\_key =\> "" consumer\_secret =\> "" oauth\_token =\> "" oauth\_token\_secret =\> "" follows =\> "elastic" full\_tweet =\> true } } output { elasticsearch { hosts =\> \["localhost:… --- ## [S3 v4 signatures for logstash input plugin](https://discuss.elastic.co/t/s3-v4-signatures-for-logstash-input-plugin/46390)
**Author:** [@Abey\_Thomas](https://discuss.elastic.co/u/Abey_Thomas)\ **Replies:** 1\ **Last updated:** [April 5, 2016, 11:08pm UTC](https://discuss.elastic.co/t/s3-v4-signatures-for-logstash-input-plugin/46390 "2016-04-05T23:08:04Z")
Hi , I was trying to collect all my AWS config logs from S3 via logstash and could see it list the filenames fine but once listing is complete it starts to download the S3 file and that fails with an error "Error: Requ… --- ## [Cannot get date to match timestamp after xml parsing](https://discuss.elastic.co/t/cannot-get-date-to-match-timestamp-after-xml-parsing/46327)
**Author:** [@whyapenny](https://discuss.elastic.co/u/whyapenny)\ **Replies:** 2\ **Last updated:** [April 5, 2016, 8:33pm UTC](https://discuss.elastic.co/t/cannot-get-date-to-match-timestamp-after-xml-parsing/46327 "2016-04-05T20:33:13Z")
here is the xml field that comes in originally (xml parsing works fine, and this field shows up in kibana as entry.Timestamp): 2016-04-04T17:45:34.498-04:00 here is what the json looks like after xml parsing: "entr… --- ## [Check existence of a field and checking null value in field](https://discuss.elastic.co/t/check-existence-of-a-field-and-checking-null-value-in-field/24968)
**Author:** [@DavidL](https://discuss.elastic.co/u/DavidL)\ **Replies:** 3\ **Last updated:** [April 5, 2016, 8:28pm UTC](https://discuss.elastic.co/t/check-existence-of-a-field-and-checking-null-value-in-field/24968 "2016-04-05T20:28:18Z")
I have different types of logs generated by the software we user in one big file. I am trying to assign "log\_type" to different types of logs in my file. Since they have different number of columns(I used csv plugin to p… --- ## [After upgrade Logstash 2.3 tries to evaluate $1 enviroment-var in embedded ruby code](https://discuss.elastic.co/t/after-upgrade-logstash-2-3-tries-to-evaluate-1-enviroment-var-in-embedded-ruby-code/46064)
**Author:** [@kaem2111](https://discuss.elastic.co/u/kaem2111)\ **Replies:** 2\ **Last updated:** [April 5, 2016, 8:26pm UTC](https://discuss.elastic.co/t/after-upgrade-logstash-2-3-tries-to-evaluate-1-enviroment-var-in-embedded-ruby-code/46064 "2016-04-05T20:26:41Z")
Hallo, I am using ruby code in Logstash in the kind: ruby { code =\> " b = { 'K' =\> 1024.0, 'M' =\> 1048576.0, 'G' =\> 1073741824.0 } event\['z0'\].each do | k0, v0 | if (v0 =~ /^\[\\-\\+0-9\]/) … --- ## [Parsing logs for specific domain](https://discuss.elastic.co/t/parsing-logs-for-specific-domain/46445)
**Author:** [@himanshudas](https://discuss.elastic.co/u/himanshudas)\ **Replies:** 1\ **Last updated:** [April 5, 2016, 8:01pm UTC](https://discuss.elastic.co/t/parsing-logs-for-specific-domain/46445 "2016-04-05T20:01:40Z")
I want to parse HAPROXY logs for specific domain(below: abc refers to abc.com) using logstash.This is the filter that i am trying to use inside logstash input: input { filter { if \[message\] =~ "abc" { g… --- ## [Aggregate fields from different event](https://discuss.elastic.co/t/aggregate-fields-from-different-event/44078)
**Author:** [@rbkumar88](https://discuss.elastic.co/u/rbkumar88)\ **Replies:** 3\ **Last updated:** [April 5, 2016, 7:07pm UTC](https://discuss.elastic.co/t/aggregate-fields-from-different-event/44078 "2016-04-05T19:07:48Z")
Hi, Please help me setting the filter plugin for the following scenario. Say I have the following scenario INFO - 12345 - TASK\_START - start INFO - 12345 - SQL - sqlQuery1 - 12 INFO - 12345 - SQL - sqlQuery2 - 34 … --- ## [New java.exe process starts everytime I stop and run logstash](https://discuss.elastic.co/t/new-java-exe-process-starts-everytime-i-stop-and-run-logstash/45045)
**Author:** [@imad](https://discuss.elastic.co/u/imad)\ **Replies:** 5\ **Last updated:** [March 29, 2016, 10:01pm UTC](https://discuss.elastic.co/t/new-java-exe-process-starts-everytime-i-stop-and-run-logstash/45045 "2016-03-29T22:01:11Z")
Hi, Everytime I run logstash new jrube.exe and java.exe processes are starting up. when I stop logstsah, jrube.exe process finishes but java.exe stays. And upon starting logstash again another java process spawns. So … --- ## [Does it possible remove all nested fields except white-list?](https://discuss.elastic.co/t/does-it-possible-remove-all-nested-fields-except-white-list/46417)
**Author:** [@Alexander\_Popov](https://discuss.elastic.co/u/Alexander_Popov)\ **Replies:** 3\ **Last updated:** [April 5, 2016, 5:55pm UTC](https://discuss.elastic.co/t/does-it-possible-remove-all-nested-fields-except-white-list/46417 "2016-04-05T17:55:11Z")
source: { "message":"aaa" "req":{ "url":"..." ...... }, "string1":"value",, "stringN":"value" "int1":1, "intN":1, "object1":{}, "objectN":{}, } Can… --- ## [Logstash filter VMware access\_log](https://discuss.elastic.co/t/logstash-filter-vmware-access-log/46381)
**Author:** [@maher](https://discuss.elastic.co/u/maher)\ **Replies:** 1\ **Last updated:** [April 5, 2016, 5:28pm UTC](https://discuss.elastic.co/t/logstash-filter-vmware-access-log/46381 "2016-04-05T17:28:44Z")
HI, I am trying to parse an apache access\_log. The log format includes the VMware name and port so using the grok %{ COMBINEDAPACHELOG} I always have a wrong result. I was trying to use a custom filter without a us… --- ## [Logstash Filters - Grok](https://discuss.elastic.co/t/logstash-filters-grok/46384)
**Author:** [@ariceELK](https://discuss.elastic.co/u/ariceELK)\ **Replies:** 1\ **Last updated:** [April 5, 2016, 5:26pm UTC](https://discuss.elastic.co/t/logstash-filters-grok/46384 "2016-04-05T17:26:11Z")
Hi, With the latest releases and support to directly ship to Elasticsearch, is Grok still required to filter logs in Logstash or is there another method ? Thanks Aidan --- ## [TLV decoding in Logstash?](https://discuss.elastic.co/t/tlv-decoding-in-logstash/46414)
**Author:** [@paulkeogh](https://discuss.elastic.co/u/paulkeogh)\ **Replies:** 1\ **Last updated:** [April 5, 2016, 5:15pm UTC](https://discuss.elastic.co/t/tlv-decoding-in-logstash/46414 "2016-04-05T17:15:48Z")
Hi, I have an incoming data field that is TLV encoded as an ASCII Hex string, Is there an existing logstash filter or codec that I can use to unwrap this or am I looking at developing a custom filter ? Thanks, --- ## [Dropping / Mutating first part of log event](https://discuss.elastic.co/t/dropping-mutating-first-part-of-log-event/46063)
**Author:** [@joserose](https://discuss.elastic.co/u/joserose)\ **Replies:** 5\ **Last updated:** [April 5, 2016, 4:38pm UTC](https://discuss.elastic.co/t/dropping-mutating-first-part-of-log-event/46063 "2016-04-05T16:38:40Z")
Hi there I'm attempting to parse the following log entry: A, \[2016-01-27T11:49:29.702997 #90\] ANY -- : 2016-01-27 11:49:29 +0000 severity=INFO, {"method":"GET","path":"/404","format":"/","controller":"errors","acti… --- ## [Parsing custom Tomcat access log](https://discuss.elastic.co/t/parsing-custom-tomcat-access-log/46362)
**Author:** [@venediger](https://discuss.elastic.co/u/venediger)\ **Replies:** 3\ **Last updated:** [April 5, 2016, 2:18pm UTC](https://discuss.elastic.co/t/parsing-custom-tomcat-access-log/46362 "2016-04-05T14:18:28Z")
Hi, I have the following log line from tomcat access log: 192.123.120.11 (192.33.211.131) - USER123 \[23/Mar/2015:00:00:12 +0000\] GET /rest/url/v1/state?appStateOnly=true HTTP/1.1 200 111 7 rest.api.package:443 192.… --- ## [Logstash Out of memory](https://discuss.elastic.co/t/logstash-out-of-memory/28687)
**Author:** [@michaellizhou](https://discuss.elastic.co/u/michaellizhou)\ **Replies:** 22\ **Last updated:** [April 5, 2016, 12:51pm UTC](https://discuss.elastic.co/t/logstash-out-of-memory/28687 "2016-04-05T12:51:20Z")
Has anyone experience this when trying to run logstash with the jdbc input plugin ../bin/logstash -f logstash.conf Logstash startup completed Error: Your application used more memory than the safety cap of 500M. Speci… --- ## [How to feed the below Schema file in the ELK Stack and receive the output](https://discuss.elastic.co/t/how-to-feed-the-below-schema-file-in-the-elk-stack-and-receive-the-output/45903)
**Author:** [@dsuriyadeepan](https://discuss.elastic.co/u/dsuriyadeepan)\ **Replies:** 3\ **Last updated:** [April 5, 2016, 10:29am UTC](https://discuss.elastic.co/t/how-to-feed-the-below-schema-file-in-the-elk-stack-and-receive-the-output/45903 "2016-04-05T10:29:39Z")
Hi Folks , Have a pleasant day ahead. We have installed ELK stack with NX log in windows 2008 R2 server.Now,we need to analyse the following log structure in this stack.I am not a programmer.Could,some… --- ## [Failed to send event to Redis. IOError](https://discuss.elastic.co/t/failed-to-send-event-to-redis-ioerror/46349)
**Author:** [@diwertowski](https://discuss.elastic.co/u/diwertowski)\ **Replies:** 4\ **Last updated:** [April 5, 2016, 9:28am UTC](https://discuss.elastic.co/t/failed-to-send-event-to-redis-ioerror/46349 "2016-04-05T09:28:51Z")
Hi, today I got an Error, that I can not fix. My Setup (everything is on one local machine, so I can easily test it): Its a windows server with 16GB, 64-Bit Logstash Version 2.2.0 Kibana Version 4.4.1 Elasticsearch Vers… --- ## [LS 2.3.0-1 can't valid the logstash configuration](https://discuss.elastic.co/t/ls-2-3-0-1-cant-valid-the-logstash-configuration/46350)
**Author:** [@woodkan](https://discuss.elastic.co/u/woodkan)\ **Replies:** 1\ **Last updated:** [April 5, 2016, 8:44am UTC](https://discuss.elastic.co/t/ls-2-3-0-1-cant-valid-the-logstash-configuration/46350 "2016-04-05T08:44:04Z")
I just upgraded LS to 2.3.0-1. Then logstash failed to valid LS configuration with below errors. \[root@tmp-os7 logstash\]# /opt/logstash/bin/logstash -f /etc/logstash/confw.d/ -t An unexpected error occurred! {:error… --- ## [Logstash service stops every 2-3 days](https://discuss.elastic.co/t/logstash-service-stops-every-2-3-days/44826)
**Author:** [@meons](https://discuss.elastic.co/u/meons)\ **Replies:** 2\ **Last updated:** [April 5, 2016, 8:18am UTC](https://discuss.elastic.co/t/logstash-service-stops-every-2-3-days/44826 "2016-04-05T08:18:46Z")
Hi there, I have 2 Logstash servers that recieve logs from different systems such as Windows and Linux machines and some networks systems such Firewalls, Switches, etc. The problem is that every 2-3 days the logstash pro… --- ## [How to convert a json field to string by using ruby filter?](https://discuss.elastic.co/t/how-to-convert-a-json-field-to-string-by-using-ruby-filter/46206)
**Author:** [@abcfy2](https://discuss.elastic.co/u/abcfy2)\ **Replies:** 3\ **Last updated:** [April 5, 2016, 7:31am UTC](https://discuss.elastic.co/t/how-to-convert-a-json-field-to-string-by-using-ruby-filter/46206 "2016-04-05T07:31:27Z")
Hi all. I have a event like: {"@timestamp":"2016-04-03T08:17:15.202Z","host":"shifudao","type":"rtds","loglevel":"WARN","classname":"hawkeyes.rtds.gate.protocol.frametype.SamplerReportData","logdetail":{"message":"设备0… --- ## [Shutdown hooks for output codecs](https://discuss.elastic.co/t/shutdown-hooks-for-output-codecs/46333)
**Author:** [@garyL](https://discuss.elastic.co/u/garyL)\ **Replies:** 0\ **Last updated:** [April 5, 2016, 7:14am UTC](https://discuss.elastic.co/t/shutdown-hooks-for-output-codecs/46333 "2016-04-05T07:14:49Z")
I'm trying to write a batching codec, and need to flush the batch on shutdown. I've implemented the close method but that does no appear to get called. From what I can see of the pipeline code this only gets called on … --- ## [Grok custom filters](https://discuss.elastic.co/t/grok-custom-filters/46234)
**Author:** [@Giulio](https://discuss.elastic.co/u/Giulio)\ **Replies:** 3\ **Last updated:** [April 5, 2016, 7:13am UTC](https://discuss.elastic.co/t/grok-custom-filters/46234 "2016-04-05T07:13:32Z")
Hi I'm creating a custom filter for the log that I must store into the database. It work (more or less) but there are still a couple of problem (testing it on Grok Debugger) www.test.example.com 132.168.192.123 … --- ## [Logstash-filebeat error](https://discuss.elastic.co/t/logstash-filebeat-error/46315)
**Author:** [@Farouk\_Samu](https://discuss.elastic.co/u/Farouk_Samu)\ **Replies:** 5\ **Last updated:** [April 5, 2016, 6:48am UTC](https://discuss.elastic.co/t/logstash-filebeat-error/46315 "2016-04-05T06:48:45Z")
Can some one help me resolve this error root@logstash:/opt/logstash/bin# ./logstash -f logstash.conf Apr 04, 2016 7:53:07 PM org.elasticsearch.node.internal.InternalNode INFO: \[logstash-logstash-3557-12128\] version\[1.… --- ## [Grok filter cannot parse log lines that are AFTER a multiline event](https://discuss.elastic.co/t/grok-filter-cannot-parse-log-lines-that-are-after-a-multiline-event/46255)
**Author:** [@pantheo](https://discuss.elastic.co/u/pantheo)\ **Replies:** 7\ **Last updated:** [April 5, 2016, 5:43am UTC](https://discuss.elastic.co/t/grok-filter-cannot-parse-log-lines-that-are-after-a-multiline-event/46255 "2016-04-05T05:43:03Z")
Hello everyone, I have a really annoying problem. As I mention in the title if I have an single-line event after a multiline event Logstash just freezes and cannot parse the rest of the file. However, If that same sin… --- ## [Unrecognized token 'CONNECTION\_LOST](https://discuss.elastic.co/t/unrecognized-token-connection-lost/46319)
**Author:** [@Farouk\_Samu](https://discuss.elastic.co/u/Farouk_Samu)\ **Replies:** 1\ **Last updated:** [April 5, 2016, 4:00am UTC](https://discuss.elastic.co/t/unrecognized-token-connection-lost/46319 "2016-04-05T04:00:13Z")
This is my logstash config file, after running it I got the below error input { beats { port =\> 5044 ssl =\> true ssl\_certificate =\> "/etc/pki/tls/certs/logstash-forwarder.crt" ssl\_key =\> "/etc/pki/tls… --- ## [Unrecognized token 'CONNECTION\_LOST'](https://discuss.elastic.co/t/unrecognized-token-connection-lost/46317)
**Author:** [@Farouk\_Samu](https://discuss.elastic.co/u/Farouk_Samu)\ **Replies:** 1\ **Last updated:** [April 5, 2016, 4:00am UTC](https://discuss.elastic.co/t/unrecognized-token-connection-lost/46317 "2016-04-05T04:00:05Z")
Continuing the discussion from Logstash-filebeat error: thanks warkolm, Included protocol =\> http and I had this error again Trouble parsing json {:source=\>"rest", :raw=\>"CONNECTION\_LOST", :exception=\># at \[Sour… --- ## [Problem running on windows with embedded ES output](https://discuss.elastic.co/t/problem-running-on-windows-with-embedded-es-output/1730)
**Author:** [@nilsga](https://discuss.elastic.co/u/nilsga)\ **Replies:** 2\ **Last updated:** [April 5, 2016, 3:30am UTC](https://discuss.elastic.co/t/problem-running-on-windows-with-embedded-es-output/1730 "2016-04-05T03:30:57Z")
I'm trying to run logstash 1.5 with embedded ES, but I keep getting an error during startup: {:timestamp=\>"2015-06-02T15:55:02.275000+0200", :message=\>"The 'embedded =\> true' setting is enabled. This is not recommended… --- ## [Logstash 2.3 redis input not compatible with redis 2.4.10](https://discuss.elastic.co/t/logstash-2-3-redis-input-not-compatible-with-redis-2-4-10/46278)
**Author:** [@zombie101](https://discuss.elastic.co/u/zombie101)\ **Replies:** 2\ **Last updated:** [April 5, 2016, 12:15am UTC](https://discuss.elastic.co/t/logstash-2-3-redis-input-not-compatible-with-redis-2-4-10/46278 "2016-04-05T00:15:10Z")
Just a FYI for anyone else who hits this: I just upgraded to logstash 2.3 and my ELK pipeline broke because my indexers were not able to read from my redis instances, which were running redis 2.4.10 (Default RHEL 6 versi… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1169) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1171)