# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1185 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1186 --- ## [How to parse "DTG string string ..."?](https://discuss.elastic.co/t/how-to-parse-dtg-string-string/43029)
**Author:** [@blake](https://discuss.elastic.co/u/blake)\ **Replies:** 5\ **Last updated:** [February 29, 2016, 6:53pm UTC](https://discuss.elastic.co/t/how-to-parse-dtg-string-string/43029 "2016-02-29T18:53:10Z")
I created a log with: 1. DTG 2. All users currently logged on. Sample log: Mon, 29 Feb 2016 17:28:01 +0000 bob fred xavier I can get the DTG with: filter { grok { match =\> { "message" =\> "%{DATESTAMP\_RFC2822:… --- ## [Logstash shipper throwing socket error](https://discuss.elastic.co/t/logstash-shipper-throwing-socket-error/42997)
**Author:** [@Kishore](https://discuss.elastic.co/u/Kishore)\ **Replies:** 1\ **Last updated:** [February 29, 2016, 6:19pm UTC](https://discuss.elastic.co/t/logstash-shipper-throwing-socket-error/42997 "2016-02-29T18:19:10Z")
When i am sending logstash events through logstash shipper to redis, getting below error. Kindly help me in fix this isse. exception=\>SocketError: initialize: name or service not known\> Other instances are throwing b… --- ## [Logstash refuses to start when 3 or more rabbitmq outputs are used. (message=\>"The error reported is: \\n)](https://discuss.elastic.co/t/logstash-refuses-to-start-when-3-or-more-rabbitmq-outputs-are-used-message-the-error-reported-is-n/42415)
**Author:** [@utilitynerd](https://discuss.elastic.co/u/utilitynerd)\ **Replies:** 3\ **Last updated:** [February 29, 2016, 5:39pm UTC](https://discuss.elastic.co/t/logstash-refuses-to-start-when-3-or-more-rabbitmq-outputs-are-used-message-the-error-reported-is-n/42415 "2016-02-29T17:39:46Z")
logstash 2.2.2 logstash-output-rabbitmq (3.0.7) After upgrading to logstash 2.2.0 and updating all plugins, my instance of logstash is refusing to start. The logstash log only contains: {:timestamp=\>"2016-02-22T09:39… --- ## [Elapsed Time Filter for calculating session Duration (elapsed\_time is very big)](https://discuss.elastic.co/t/elapsed-time-filter-for-calculating-session-duration-elapsed-time-is-very-big/32882)
**Author:** [@inderjeet26](https://discuss.elastic.co/u/inderjeet26)\ **Replies:** 6\ **Last updated:** [February 29, 2016, 3:22pm UTC](https://discuss.elastic.co/t/elapsed-time-filter-for-calculating-session-duration-elapsed-time-is-very-big/32882 "2016-02-29T15:22:33Z")
Hello folks, I am using Elapsed Filter Plugin for calculating the time between start\_session and session\_timeout events in my logs. Here are some events with Timestamps \[27/Apr/2015:10:41:45 -0500\] 1 ENG NORMAL TOU… --- ## [Logstash Skipping Events from Logcourier in Linux](https://discuss.elastic.co/t/logstash-skipping-events-from-logcourier-in-linux/43001)
**Author:** [@rajkamalkool6](https://discuss.elastic.co/u/rajkamalkool6)\ **Replies:** 0\ **Last updated:** [February 29, 2016, 1:03pm UTC](https://discuss.elastic.co/t/logstash-skipping-events-from-logcourier-in-linux/43001 "2016-02-29T13:03:08Z")
I have deployed 3 logstash configuration files in my server which is processing 3 different kinds of log files(.xml,.log and .txt). I am using 3 logurier files(version 1.6) to fetch the log files and logstash(version 1.5… --- ## [Recover only a part of a log with multiline codec](https://discuss.elastic.co/t/recover-only-a-part-of-a-log-with-multiline-codec/42824)
**Author:** [@Justin\_V](https://discuss.elastic.co/u/Justin_V)\ **Replies:** 3\ **Last updated:** [February 29, 2016, 12:56pm UTC](https://discuss.elastic.co/t/recover-only-a-part-of-a-log-with-multiline-codec/42824 "2016-02-29T12:56:17Z")
Hello again, I'm now having trouble with some particular logs. As you can see, these are "multiline" logs : the message is continuing in the next lines. But (as you can see too), they start again with the same patte… --- ## [Logstash with Google Pub/Sub](https://discuss.elastic.co/t/logstash-with-google-pub-sub/42952)
**Author:** [@Ilya\_Levin](https://discuss.elastic.co/u/Ilya_Levin)\ **Replies:** 1\ **Last updated:** [February 29, 2016, 9:48am UTC](https://discuss.elastic.co/t/logstash-with-google-pub-sub/42952 "2016-02-29T09:48:41Z")
Hi All I'm want to use logstash-input-google-cloud-pubsub as input for logstash 2.2.2. All my tries to run it have failed so far. Installing out of the box gives me an dependency error gcloud 0.6.1 requires Ruby 2.0 … --- ## [Beats plugin](https://discuss.elastic.co/t/beats-plugin/42835)
**Author:** [@mr\_rob](https://discuss.elastic.co/u/mr_rob)\ **Replies:** 2\ **Last updated:** [February 29, 2016, 9:39am UTC](https://discuss.elastic.co/t/beats-plugin/42835 "2016-02-29T09:39:44Z")
i setup syslog server A, with filebeats and then ELK server B. stuff gets sent from A to B, and for one moment yesterday appeared to make its way to kibana. I then had lunch and it never worked again! tcpdump shows… --- ## [Logstash HA with VIP?](https://discuss.elastic.co/t/logstash-ha-with-vip/42563)
**Author:** [@Kevin\_Ross](https://discuss.elastic.co/u/Kevin_Ross)\ **Replies:** 2\ **Last updated:** [February 29, 2016, 9:19am UTC](https://discuss.elastic.co/t/logstash-ha-with-vip/42563 "2016-02-29T09:19:54Z")
Hi, When I started I had 1 node with entire ELK stack on it which worked well for my security logging. Now I have a few though and what I have right now ELK is effectively on one node and there is other nodes in the ES… --- ## [How do we convert from a hash to json before event reaches the grok filter](https://discuss.elastic.co/t/how-do-we-convert-from-a-hash-to-json-before-event-reaches-the-grok-filter/29445)
**Author:** [@tmo\_bgc](https://discuss.elastic.co/u/tmo_bgc)\ **Replies:** 8\ **Last updated:** [February 29, 2016, 9:10am UTC](https://discuss.elastic.co/t/how-do-we-convert-from-a-hash-to-json-before-event-reaches-the-grok-filter/29445 "2016-02-29T09:10:57Z")
How do we convert from a hash to json before event reaches the grok filter. The JMX input plugin produces a Hash, but the Grok filter expects JSON. input { jmx { path =\> "c:/LogStash/jmxconf/" polling\_freq… --- ## [Testing and benchmarking log collection pipeline](https://discuss.elastic.co/t/testing-and-benchmarking-log-collection-pipeline/42942)
**Author:** [@gaspaio](https://discuss.elastic.co/u/gaspaio)\ **Replies:** 2\ **Last updated:** [February 29, 2016, 7:58am UTC](https://discuss.elastic.co/t/testing-and-benchmarking-log-collection-pipeline/42942 "2016-02-29T07:58:58Z")
I'm looking for tools, or a set of tools, to test and benchmark logstash+kafka+ES setups : - define test cases (log types, throughputs) - maintain a base of log examples - generate inputs for logstash at a given rate fr… --- ## [Logstash multiline help](https://discuss.elastic.co/t/logstash-multiline-help/42852)
**Author:** [@Roy\_K](https://discuss.elastic.co/u/Roy_K)\ **Replies:** 5\ **Last updated:** [February 29, 2016, 7:46am UTC](https://discuss.elastic.co/t/logstash-multiline-help/42852 "2016-02-29T07:46:42Z")
I am having an issue cobining this log into a multiline. Any suggestions on how I can use a specific timestamp is the identified to combine the line? Log Example (notice the timestamp is the same): \[Thr 47613344799040… --- ## [Logstash 2.2 Installation issue on Ubuntu 14.04 (APT) - First time](https://discuss.elastic.co/t/logstash-2-2-installation-issue-on-ubuntu-14-04-apt-first-time/42939)
**Author:** [@gabe](https://discuss.elastic.co/u/gabe)\ **Replies:** 6\ **Last updated:** [February 29, 2016, 7:13am UTC](https://discuss.elastic.co/t/logstash-2-2-installation-issue-on-ubuntu-14-04-apt-first-time/42939 "2016-02-29T07:13:20Z")
Hi All, This is my first time installing Logstash and have been doing my best to follow the documentation for APT (on Ubuntu). As an aside, I just finished installing InfluxDB, Telegraf, Kapacitor which had a similar… --- ## [Struggle in export the data from csv to elasticsearch](https://discuss.elastic.co/t/struggle-in-export-the-data-from-csv-to-elasticsearch/42211)
**Author:** [@Croos](https://discuss.elastic.co/u/Croos)\ **Replies:** 4\ **Last updated:** [February 29, 2016, 6:55am UTC](https://discuss.elastic.co/t/struggle-in-export-the-data-from-csv-to-elasticsearch/42211 "2016-02-29T06:55:28Z")
I have a csv file. It contains 13 columns and more than 150 000 rows. When I use the following config, There is no output on the logstash. input { file { path =\> "/salaries.csv" start\_position =\> "beginning… --- ## [Processing events on logstash-forwarder only 3.2k events/second](https://discuss.elastic.co/t/processing-events-on-logstash-forwarder-only-3-2k-events-second/42833)
**Author:** [@junior\_h](https://discuss.elastic.co/u/junior_h)\ **Replies:** 2\ **Last updated:** [February 29, 2016, 5:51am UTC](https://discuss.elastic.co/t/processing-events-on-logstash-forwarder-only-3-2k-events-second/42833 "2016-02-29T05:51:53Z")
I am using logstash-forwarder to send nginx access.log with flow: access.log -\> logstash-forwarder 0.4.0-\> logstash 1.5.4 -\> redis 3.0.1 The logstash only run for storing data to redis. Due to high activity on nginx, … --- ## [Logstash 2.2.2 date filter returning wrong day](https://discuss.elastic.co/t/logstash-2-2-2-date-filter-returning-wrong-day/42957)
**Author:** [@Stuart\_Donovan](https://discuss.elastic.co/u/Stuart_Donovan)\ **Replies:** 3\ **Last updated:** [February 28, 2016, 7:01pm UTC](https://discuss.elastic.co/t/logstash-2-2-2-date-filter-returning-wrong-day/42957 "2016-02-28T19:01:17Z")
This works fine with logstash 1.5.6 but with logstash 2.2.2 it sets @timestamp to the previous day?? echo "Mon Feb 22 22:20:41.144" | /opt/logstash/bin/logstash -e ' input { stdin {codec =\> line} } filter { date { ma… --- ## [Ready to throw logstash config file out the window -- help please :)](https://discuss.elastic.co/t/ready-to-throw-logstash-config-file-out-the-window-help-please/42951)
**Author:** [@timothybward](https://discuss.elastic.co/u/timothybward)\ **Replies:** 6\ **Last updated:** [February 28, 2016, 3:08pm UTC](https://discuss.elastic.co/t/ready-to-throw-logstash-config-file-out-the-window-help-please/42951 "2016-02-28T15:08:36Z")
I have the below file and I'm getting the completely useless error message: Error: Expected one of #, =\> at line 85, column 5 (byte 1787) after output { elasticsearch { if \_\_\_\_ File below: input { beats { … --- ## [Help to refine a grok parser](https://discuss.elastic.co/t/help-to-refine-a-grok-parser/42671)
**Author:** [@nzHillNet](https://discuss.elastic.co/u/nzHillNet)\ **Replies:** 2\ **Last updated:** [February 27, 2016, 11:23pm UTC](https://discuss.elastic.co/t/help-to-refine-a-grok-parser/42671 "2016-02-27T23:23:27Z")
Hi all, Logstash v2.2.2 I'm not that strong with grok and regex's, but have managed to create a working grok parser as follows: Log excerpt: 2016-02-25 19:06:23 msg 3/3 (4575 bytes) msgid 0000178e55c5dc45 from \ **Author:** [@tgdesrochers](https://discuss.elastic.co/u/tgdesrochers)\ **Replies:** 1\ **Last updated:** [February 27, 2016, 12:18pm UTC](https://discuss.elastic.co/t/general-config-question/42920 "2016-02-27T12:18:46Z") I have multiple logstash config files in my conf.d dir. My question is two fold: I see many people naming their config s with a preceding number. Is this because logstash will load them in numerical order? Should … --- ## [Realtime view of logs](https://discuss.elastic.co/t/realtime-view-of-logs/42899)
**Author:** [@Michael\_Korbakov](https://discuss.elastic.co/u/Michael_Korbakov)\ **Replies:** 2\ **Last updated:** [February 26, 2016, 11:17pm UTC](https://discuss.elastic.co/t/realtime-view-of-logs/42899 "2016-02-26T23:17:13Z")
Hi all. I'm wondering, is there any accepted solution to provide real time view of application logs collected through Logstash? Kibana is great for search and analysis but sometimes ability to view stream of events as … --- ## [New collectd types do not show up in logstash/ES](https://discuss.elastic.co/t/new-collectd-types-do-not-show-up-in-logstash-es/42727)
**Author:** [@stefws](https://discuss.elastic.co/u/stefws)\ **Replies:** 8\ **Last updated:** [February 26, 2016, 8:25pm UTC](https://discuss.elastic.co/t/new-collectd-types-do-not-show-up-in-logstash-es/42727 "2016-02-26T20:25:17Z")
I've created a collectd exec plugin with new collectd data types to send to logstash+ES, I get the ES documents but amputated without the new data type fields and wonder why. Hints appreciated, TIA. new collectd type: $… --- ## [Installing LogStash 2.2.2 failed on Windows Server 2012](https://discuss.elastic.co/t/installing-logstash-2-2-2-failed-on-windows-server-2012/42777)
**Author:** [@eso](https://discuss.elastic.co/u/eso)\ **Replies:** 5\ **Last updated:** [February 26, 2016, 7:09pm UTC](https://discuss.elastic.co/t/installing-logstash-2-2-2-failed-on-windows-server-2012/42777 "2016-02-26T19:09:55Z")
Hello, When I installed LogStash 2.2.2 (in Program Files folder), I got the following error messages: bin/plugin install logstash-input-beats Unable to download data from https://rubygems.org - SocketError: initi… --- ## [LS Indexer not grabbing messages from Rabbitmq](https://discuss.elastic.co/t/ls-indexer-not-grabbing-messages-from-rabbitmq/42784)
**Author:** [@misterjulien](https://discuss.elastic.co/u/misterjulien)\ **Replies:** 2\ **Last updated:** [February 26, 2016, 6:48pm UTC](https://discuss.elastic.co/t/ls-indexer-not-grabbing-messages-from-rabbitmq/42784 "2016-02-26T18:48:15Z")
My LS indexer won't get the messages in the rabbitmq queue. I can see that there are messages in the queue but they're not going anywhere. My Logstash server is setup as ... LS shipper -\> rabbitmq -\> LS indexer LS… --- ## [Newb Grok parsing failures, teach me how to fish?](https://discuss.elastic.co/t/newb-grok-parsing-failures-teach-me-how-to-fish/42758)
**Author:** [@batfasturd](https://discuss.elastic.co/u/batfasturd)\ **Replies:** 1\ **Last updated:** [February 26, 2016, 5:47pm UTC](https://discuss.elastic.co/t/newb-grok-parsing-failures-teach-me-how-to-fish/42758 "2016-02-26T17:47:18Z")
Ok, I am fighting with a logfile that I really need to parse from my Radiator server. My problem is that it does not output in a "standard" format and can change. I just want to scan and pull values into fields as it fin… --- ## [Replace Domain](https://discuss.elastic.co/t/replace-domain/42761)
**Author:** [@Juan\_Andres\_Ramirez](https://discuss.elastic.co/u/Juan_Andres_Ramirez)\ **Replies:** 4\ **Last updated:** [February 26, 2016, 2:56pm UTC](https://discuss.elastic.co/t/replace-domain/42761 "2016-02-26T14:56:04Z")
Hello Guys, In my config I have a filter to try replace domain1 by domain2, but I think it don't work. I recieved for example the server1.domain1.com and the same server with the domain2 and I don't want save t… --- ## [Metrics filter stopped working after upgrade to logstash 2.2.x](https://discuss.elastic.co/t/metrics-filter-stopped-working-after-upgrade-to-logstash-2-2-x/42336)
**Author:** [@tetlika](https://discuss.elastic.co/u/tetlika)\ **Replies:** 3\ **Last updated:** [February 26, 2016, 11:43am UTC](https://discuss.elastic.co/t/metrics-filter-stopped-working-after-upgrade-to-logstash-2-2-x/42336 "2016-02-26T11:43:01Z")
hi, have anyone faced issue that logstash metrics stopped working after upgrade to 2.2.x? it does not work both on 2.2.0 and 2.2.2 config is so (see below), also there is another ouptut section (to redis) in anothe… --- ## [S3 input plugin](https://discuss.elastic.co/t/s3-input-plugin/42694)
**Author:** [@aniamin](https://discuss.elastic.co/u/aniamin)\ **Replies:** 1\ **Last updated:** [February 25, 2016, 12:55pm UTC](https://discuss.elastic.co/t/s3-input-plugin/42694 "2016-02-25T12:55:03Z")
I want to send log of ELB to s3 bucket. ELB logs of different service will be in different directory of my main log bucket. When I tried to put that in my input s3 conf, I am not getting any log Here is my s3 input conf … --- ## [Need help to create current date based Index in LS](https://discuss.elastic.co/t/need-help-to-create-current-date-based-index-in-ls/42834)
**Author:** [@vikas\_gopal](https://discuss.elastic.co/u/vikas_gopal)\ **Replies:** 4\ **Last updated:** [February 26, 2016, 9:51am UTC](https://discuss.elastic.co/t/need-help-to-create-current-date-based-index-in-ls/42834 "2016-02-26T09:51:52Z")
Hi Experts, I am aware that I can use following to create date index . index =\> "prod-%{+YYYY.MM.dd}" but the problem is, it is picking up logs date rather current date, what I want is to create index on current date … --- ## [Logstash-input-mongodb plugin issues](https://discuss.elastic.co/t/logstash-input-mongodb-plugin-issues/42465)
**Author:** [@Weizhen\_Yan](https://discuss.elastic.co/u/Weizhen_Yan)\ **Replies:** 1\ **Last updated:** [February 26, 2016, 9:50am UTC](https://discuss.elastic.co/t/logstash-input-mongodb-plugin-issues/42465 "2016-02-26T09:50:23Z")
I'm using (logstash-input-mongodb) plugin right now, but it is really slow, I checked the logstash log, it seems that my data has too many nested list. Have no idea how to improve the performance, I only have 500MB da… --- ## [How to use Logstash filter condition to copy docs](https://discuss.elastic.co/t/how-to-use-logstash-filter-condition-to-copy-docs/42670)
**Author:** [@r.ganeshbabu](https://discuss.elastic.co/u/r.ganeshbabu)\ **Replies:** 9\ **Last updated:** [February 26, 2016, 9:08am UTC](https://discuss.elastic.co/t/how-to-use-logstash-filter-condition-to-copy-docs/42670 "2016-02-26T09:08:33Z")
Hi All, We are trying to copy docs from one index to another index using logstash. Our index (es\_item) doc count is nearly 194,556,883 with primary size of 567.7GB, the docs which are indexed to the es\_item has differ… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1184) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1186)