# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1246 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1247 --- ## [Multiple logstash-instances on 1 elasticsearch-cluster](https://discuss.elastic.co/t/multiple-logstash-instances-on-1-elasticsearch-cluster/27002)
**Author:** [@jarmoni](https://discuss.elastic.co/u/jarmoni)\ **Replies:** 4\ **Last updated:** [August 7, 2015, 10:57pm UTC](https://discuss.elastic.co/t/multiple-logstash-instances-on-1-elasticsearch-cluster/27002 "2015-08-07T22:57:38Z")
We are developing a distributed, docker-based server-application. We run several docker-containers on each host. The idea is to put a logstash-instance (as a docker-container) on each host which publishes to a single ela… --- ## [Begin collate or multiline with one conditional, but include more once running](https://discuss.elastic.co/t/begin-collate-or-multiline-with-one-conditional-but-include-more-once-running/27065)
**Author:** [@nekatreven](https://discuss.elastic.co/u/nekatreven)\ **Replies:** 1\ **Last updated:** [August 7, 2015, 8:31pm UTC](https://discuss.elastic.co/t/begin-collate-or-multiline-with-one-conditional-but-include-more-once-running/27065 "2015-08-07T20:31:28Z")
Hello, I'm planning to use the multiline filter or (the actual current plan) the collate filter to combine events into digests for emailing, similar to this example I found when searching around...: input { .... … --- ## [SQS input plugin fails with some messages](https://discuss.elastic.co/t/sqs-input-plugin-fails-with-some-messages/27059)
**Author:** [@weevilgenius](https://discuss.elastic.co/u/weevilgenius)\ **Replies:** 0\ **Last updated:** [August 7, 2015, 6:27pm UTC](https://discuss.elastic.co/t/sqs-input-plugin-fails-with-some-messages/27059 "2015-08-07T18:27:52Z")
I'm setting up a logstash pipeline using Amazon's SQS as the message handler. The flow looks like this: servers =\> receiver tier =\> SQS =\> filter tier =\> elasticsearch Logstash forwarder sends logs to a receiver tier… --- ## [Pattern or Filter to extract embedded JSON and Description in logfile](https://discuss.elastic.co/t/pattern-or-filter-to-extract-embedded-json-and-description-in-logfile/27043)
**Author:** [@Kevin\_B](https://discuss.elastic.co/u/Kevin_B)\ **Replies:** 0\ **Last updated:** [August 7, 2015, 11:30am UTC](https://discuss.elastic.co/t/pattern-or-filter-to-extract-embedded-json-and-description-in-logfile/27043 "2015-08-07T11:30:32Z")
Hi there We have an app that writes a log file stored in Oracle as a blob object ( I am already connected via logstash jdbc to extract latest log file data ) and I have stripped down to the best of my ability the messag… --- ## [Logstash-forwarder + SSL + HAProxy](https://discuss.elastic.co/t/logstash-forwarder-ssl-haproxy/27008)
**Author:** [@HelpComputer](https://discuss.elastic.co/u/HelpComputer)\ **Replies:** 1\ **Last updated:** [August 7, 2015, 10:48am UTC](https://discuss.elastic.co/t/logstash-forwarder-ssl-haproxy/27008 "2015-08-07T10:48:15Z")
Hello, I am trying to setup an ELK stack but I have a question about Logstash-forwarder and SSL. My setup looks like this: Logstash-forwarder -\> HAProxy (VM) -\> (2 VM's) Logstash/Redis -\> (2 VM's) Logstash Indexer -\> … --- ## [Read error looking for ack: EOF](https://discuss.elastic.co/t/read-error-looking-for-ack-eof/26880)
**Author:** [@anilmaddukuri](https://discuss.elastic.co/u/anilmaddukuri)\ **Replies:** 4\ **Last updated:** [August 7, 2015, 9:22am UTC](https://discuss.elastic.co/t/read-error-looking-for-ack-eof/26880 "2015-08-07T09:22:40Z")
I am new to ELK stack. My configuration contains logstash-forwarder pushing my server logs to logstash residing in a seperate instance. But recently i found two issues. Logstash-forwarder is able to connect to logstash… --- ## [Grok Pattern to get response\_time and app\_id from Pivotal CloudFoundry Loggregator](https://discuss.elastic.co/t/grok-pattern-to-get-response-time-and-app-id-from-pivotal-cloudfoundry-loggregator/26964)
**Author:** [@Amit\_Hora](https://discuss.elastic.co/u/Amit_Hora)\ **Replies:** 1\ **Last updated:** [August 7, 2015, 8:24am UTC](https://discuss.elastic.co/t/grok-pattern-to-get-response-time-and-app-id-from-pivotal-cloudfoundry-loggregator/26964 "2015-08-07T08:24:55Z")
Hi All, I am trying to get few fields from Logs of my app on Pivotal Cloud Foundry to analyze them using ELK but not able to get all the fields ,below is log from my APP 2015-08-03T09:51:15.000+00:00 \[RTR\] OUT mm1-s… --- ## [Is that possible to choose the informations to index in ElasticSearch?](https://discuss.elastic.co/t/is-that-possible-to-choose-the-informations-to-index-in-elasticsearch/26980)
**Author:** [@ELKApprentice](https://discuss.elastic.co/u/ELKApprentice)\ **Replies:** 2\ **Last updated:** [August 7, 2015, 8:20am UTC](https://discuss.elastic.co/t/is-that-possible-to-choose-the-informations-to-index-in-elasticsearch/26980 "2015-08-07T08:20:02Z")
Hello , (sorry in advance for my poor english i'm french and thanks in advance for reading my topic). I just started using the ELK stack this morning and i was wondering if you can help me with my problème. My logs f… --- ## [Special Characters in logs - how to escape them in logstash grok pattern](https://discuss.elastic.co/t/special-characters-in-logs-how-to-escape-them-in-logstash-grok-pattern/27023)
**Author:** [@shovik](https://discuss.elastic.co/u/shovik)\ **Replies:** 3\ **Last updated:** [August 7, 2015, 7:07am UTC](https://discuss.elastic.co/t/special-characters-in-logs-how-to-escape-them-in-logstash-grok-pattern/27023 "2015-08-07T07:07:46Z")
Hi, I am trying to analyse a log in logstash, here is one line from the log.. 11.111.111.111 - - \[13/Feb/2015:06:59:59 -0500\] "POST /myrequestURI HTTP/1.1" 200 780 the grok expression that i have written is: %{IP… --- ## [Fluentd sending to Logstash leads to lot of garbage data](https://discuss.elastic.co/t/fluentd-sending-to-logstash-leads-to-lot-of-garbage-data/26848)
**Author:** [@Zeal\_Vora](https://discuss.elastic.co/u/Zeal_Vora)\ **Replies:** 3\ **Last updated:** [August 7, 2015, 1:44am UTC](https://discuss.elastic.co/t/fluentd-sending-to-logstash-leads-to-lot-of-garbage-data/26848 "2015-08-07T01:44:16Z")
Hi We have fluentd which is sending logs from client server to our main ELK Server. In the Kibana, when i read the logs, there is lot of garbage data that comes along with the message. On the Logstash log, server w… --- ## [Http\_poller -\> graphite](https://discuss.elastic.co/t/http-poller-graphite/26736)
**Author:** [@Bruce\_Lysik](https://discuss.elastic.co/u/Bruce_Lysik)\ **Replies:** 15\ **Last updated:** [August 6, 2015, 11:06pm UTC](https://discuss.elastic.co/t/http-poller-graphite/26736 "2015-08-06T23:06:27Z")
Trying to use the new input-http\_poller. I've got an application which on a url, spits out json which looks like this in stdout/rubydebugger: { "metrics" =\> { "requests" =\> 12, "foo" =\> 2345, "bar" … --- ## [Parsing human readable log files with grok](https://discuss.elastic.co/t/parsing-human-readable-log-files-with-grok/26821)
**Author:** [@jjdepaul](https://discuss.elastic.co/u/jjdepaul)\ **Replies:** 18\ **Last updated:** [August 6, 2015, 9:05pm UTC](https://discuss.elastic.co/t/parsing-human-readable-log-files-with-grok/26821 "2015-08-06T21:05:20Z")
We have the following log sample file that was created in a human readable format. I'm interested in the lines in bold and want to capture the following event fields: timestamp (i.e. 7/17/15 18:50:59:616 GMT) status … --- ## [Grok pattern when last field can be multiline](https://discuss.elastic.co/t/grok-pattern-when-last-field-can-be-multiline/26906)
**Author:** [@FrankC](https://discuss.elastic.co/u/FrankC)\ **Replies:** 5\ **Last updated:** [August 6, 2015, 4:58pm UTC](https://discuss.elastic.co/t/grok-pattern-when-last-field-can-be-multiline/26906 "2015-08-06T16:58:25Z")
I am trying to come up with a solution for a log file that is pipe delimited with the last field could be a multiline. Example: a|b|this is multiline c|d|no mutliline Any pointers would be appreciated. Also, since … --- ## [Error when converting Date](https://discuss.elastic.co/t/error-when-converting-date/26306)
**Author:** [@EricK](https://discuss.elastic.co/u/EricK)\ **Replies:** 13\ **Last updated:** [August 6, 2015, 1:18pm UTC](https://discuss.elastic.co/t/error-when-converting-date/26306 "2015-08-06T13:18:57Z")
Trying to add date from field (start\_date) to @timestamp using the following filter: date { match =\> \[ "start\_date", "yyyy-MM-dd HH:mm:ss Z" \] } start\_date is in the format: 2015-07-26 11:54:43 +0100 I get t… --- ## [JSON events not being sent via redis, non-JSON events are](https://discuss.elastic.co/t/json-events-not-being-sent-via-redis-non-json-events-are/26901)
**Author:** [@deltadelta](https://discuss.elastic.co/u/deltadelta)\ **Replies:** 3\ **Last updated:** [August 6, 2015, 11:17am UTC](https://discuss.elastic.co/t/json-events-not-being-sent-via-redis-non-json-events-are/26901 "2015-08-06T11:17:51Z")
Hi, I'm in the process of getting an ELK stack set up at the request of the company I work for. So far, I'm very impressed! However, I've hit a bit of stumbling block whilst following the guides in the Logstash book. I… --- ## [Logstash complaining about no ES nodes being available](https://discuss.elastic.co/t/logstash-complaining-about-no-es-nodes-being-available/25939)
**Author:** [@falzm](https://discuss.elastic.co/u/falzm)\ **Replies:** 6\ **Last updated:** [August 6, 2015, 7:41am UTC](https://discuss.elastic.co/t/logstash-complaining-about-no-es-nodes-being-available/25939 "2015-08-06T07:41:35Z")
Hi there I have a weird problem with Logstash outputing to Elasticsearch: it keeps complaining in the logs about none of the configured Elasticsearch nodes being available (about once per second!), although it manages … --- ## [Run logs through logstash again](https://discuss.elastic.co/t/run-logs-through-logstash-again/26932)
**Author:** [@jeroentbt](https://discuss.elastic.co/u/jeroentbt)\ **Replies:** 2\ **Last updated:** [August 6, 2015, 7:07am UTC](https://discuss.elastic.co/t/run-logs-through-logstash-again/26932 "2015-08-06T07:07:31Z")
Is it possible to run logs that are in ElasticSearch through Logstash again, ideally replacing the new output in ES? How do you normally handle changes in the pipeline? Do you just leave the old stuff as is or are you … --- ## [Can't start Logstash in Windows](https://discuss.elastic.co/t/cant-start-logstash-in-windows/26878)
**Author:** [@Jemli\_Fathi](https://discuss.elastic.co/u/Jemli_Fathi)\ **Replies:** 9\ **Last updated:** [August 6, 2015, 2:46am UTC](https://discuss.elastic.co/t/cant-start-logstash-in-windows/26878 "2015-08-06T02:46:08Z")
When i execute logstash.bat i get this error : io/console not supported; tty will not be manipulated --- ## [Output file rotation](https://discuss.elastic.co/t/output-file-rotation/26921)
**Author:** [@mparthas](https://discuss.elastic.co/u/mparthas)\ **Replies:** 2\ **Last updated:** [August 5, 2015, 9:11pm UTC](https://discuss.elastic.co/t/output-file-rotation/26921 "2015-08-05T21:11:15Z")
Hi, I see that max\_size for the output file has not been implemented yet. I am finding a way where Logstash will write to an output file and close the file which then can serve as the indication that it is done with th… --- ## [Ship a completed logfile?](https://discuss.elastic.co/t/ship-a-completed-logfile/26910)
**Author:** [@Bruce\_Lysik](https://discuss.elastic.co/u/Bruce_Lysik)\ **Replies:** 4\ **Last updated:** [August 5, 2015, 8:48pm UTC](https://discuss.elastic.co/t/ship-a-completed-logfile/26910 "2015-08-05T20:48:12Z")
I have a use case where I want to take a complete file, and ship it once to logstash. (Not tail it.) Is there a shipper that does this? --- ## [What is the best way to parse a log file which has sections that I need to remember when reading following lines?](https://discuss.elastic.co/t/what-is-the-best-way-to-parse-a-log-file-which-has-sections-that-i-need-to-remember-when-reading-following-lines/26897)
**Author:** [@andr\_eas](https://discuss.elastic.co/u/andr_eas)\ **Replies:** 0\ **Last updated:** [August 5, 2015, 1:57pm UTC](https://discuss.elastic.co/t/what-is-the-best-way-to-parse-a-log-file-which-has-sections-that-i-need-to-remember-when-reading-following-lines/26897 "2015-08-05T13:57:40Z")
Hi everybody, I'm working on project where I need to parse a log file that is optimized for human consumpion and therefore difficult to parse. It looks like follows \[NODE\]-----\[DESCRIPTION\]-----\[DATE\] \[TIME\] \[PROCESS… --- ## [\_grokparsefailure\_syslog but grok filter perfectly from stdin](https://discuss.elastic.co/t/-grokparsefailure-syslog-but-grok-filter-perfectly-from-stdin/26551)
**Author:** [@BartoszZ](https://discuss.elastic.co/u/BartoszZ)\ **Replies:** 2\ **Last updated:** [August 5, 2015, 1:21pm UTC](https://discuss.elastic.co/t/-grokparsefailure-syslog-but-grok-filter-perfectly-from-stdin/26551 "2015-08-05T13:21:31Z")
Hi! I have little problem with Logstash 1.5.2 and grok filtering.. My Logstash get messages from syslog and grok have problems with one of them - it is adding grokparsefailuresyslog tag. When I put message to stdin eve… --- ## [Need help with logs](https://discuss.elastic.co/t/need-help-with-logs/26863)
**Author:** [@Smasell](https://discuss.elastic.co/u/Smasell)\ **Replies:** 2\ **Last updated:** [August 5, 2015, 9:59am UTC](https://discuss.elastic.co/t/need-help-with-logs/26863 "2015-08-05T09:59:11Z")
Hi there!!! I need to parse 2 field from: CLIENT\_LOG: User( ip: 109.132.225.96; user\_id: 90221; name: 'GGG'; days\_in\_game: 1; pvp\_rating: 0; provider: GooglePlay; device\_model: 'samsung SM-G900F';). event\_name: 'Mission\_… --- ## [Logstash not receiving server logs](https://discuss.elastic.co/t/logstash-not-receiving-server-logs/26858)
**Author:** [@karthick](https://discuss.elastic.co/u/karthick)\ **Replies:** 5\ **Last updated:** [August 5, 2015, 9:39am UTC](https://discuss.elastic.co/t/logstash-not-receiving-server-logs/26858 "2015-08-05T09:39:59Z")
Hi Am trying to config logstash server for my application server. I am not able to see the logs in the logstash. I need some ones help to check this configuration. --- ## [Installation steps for logstash](https://discuss.elastic.co/t/installation-steps-for-logstash/2656)
**Author:** [@Yatish\_Gopal](https://discuss.elastic.co/u/Yatish_Gopal)\ **Replies:** 10\ **Last updated:** [August 5, 2015, 8:33am UTC](https://discuss.elastic.co/t/installation-steps-for-logstash/2656 "2015-08-05T08:33:56Z")
Hello everybody, is their any complete documents with clear steps to install logstash, kibanan, elastic & nginx on centos. We are doing a POC but still not able to install the suite. Can any body point to the good docume… --- ## [Logstash 1.5.3 gem plugins](https://discuss.elastic.co/t/logstash-1-5-3-gem-plugins/26436)
**Author:** [@VALERIE](https://discuss.elastic.co/u/VALERIE)\ **Replies:** 9\ **Last updated:** [August 5, 2015, 8:23am UTC](https://discuss.elastic.co/t/logstash-1-5-3-gem-plugins/26436 "2015-08-05T08:23:48Z")
Hello, I am a newbie on ELK and recently I have installed LOGSTASH 1.5.3 on my ELASTICSEARCH 1.7 on one node on my WINDOWS LAPTOP. I need a plugin which is "logstash-filter-elasticsearch" ... found this on ruby.org, and… --- ## [Set permissions for file created using output plugin](https://discuss.elastic.co/t/set-permissions-for-file-created-using-output-plugin/26842)
**Author:** [@kkomv](https://discuss.elastic.co/u/kkomv)\ **Replies:** 1\ **Last updated:** [August 5, 2015, 5:59am UTC](https://discuss.elastic.co/t/set-permissions-for-file-created-using-output-plugin/26842 "2015-08-05T05:59:09Z")
Is there a way to set specific permissions for a file when it is created by file output plugin in logstash? output { if "ERROR" == \[@loglevel\]{ file { path =\> "/logs/consolidated-errors.log" } … --- ## [Selecting the right versions for ELK](https://discuss.elastic.co/t/selecting-the-right-versions-for-elk/23891)
**Author:** [@Stefan\_Meisner\_Larse](https://discuss.elastic.co/u/Stefan_Meisner_Larse)\ **Replies:** 4\ **Last updated:** [August 4, 2015, 8:35pm UTC](https://discuss.elastic.co/t/selecting-the-right-versions-for-elk/23891 "2015-08-04T20:35:15Z")
There used to be a table showing which versions of elasticsearch, logstash and kibana that worked together. I can´t find it anymore...Can I just assume that the newest versions will work together? /Stefan --- ## [Syslog input to gelf output not working](https://discuss.elastic.co/t/syslog-input-to-gelf-output-not-working/26825)
**Author:** [@cmusser](https://discuss.elastic.co/u/cmusser)\ **Replies:** 0\ **Last updated:** [August 4, 2015, 6:32pm UTC](https://discuss.elastic.co/t/syslog-input-to-gelf-output-not-working/26825 "2015-08-04T18:32:53Z")
Hi, I've got a logstash configuration I can't quite get to work. I'm attempting use a lumberjack input and a syslog input and ship the received messages to a file output and a GELF output. The GELF receiver is a Graylo… --- ## [Logstash Error with Twitter COnfig](https://discuss.elastic.co/t/logstash-error-with-twitter-config/26631)
**Author:** [@perkinsl](https://discuss.elastic.co/u/perkinsl)\ **Replies:** 2\ **Last updated:** [August 4, 2015, 3:30pm UTC](https://discuss.elastic.co/t/logstash-error-with-twitter-config/26631 "2015-08-04T15:30:23Z")
I've looked everywhere to find an answer to the below error but can only find a firewall issue (so tried outside the firewall) and the time stamp being different between the twitter server and my machine (machine timing … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1245) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1247)