# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1260 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1261 --- ## [Several instances of Logstash](https://discuss.elastic.co/t/several-instances-of-logstash/1565)
**Author:** [@IlyaMatiborskiy](https://discuss.elastic.co/u/IlyaMatiborskiy)\ **Replies:** 2\ **Last updated:** [June 10, 2015, 2:20pm UTC](https://discuss.elastic.co/t/several-instances-of-logstash/1565 "2015-06-10T14:20:41Z")
Good Day, Colleagues! i'm getting to make two instances of Logstash. Actually two services with two different configs. I'm trying to do this, but the second service doesn't transfer data to kibana, although if i start it… --- ## [How to split array?](https://discuss.elastic.co/t/how-to-split-array/2325)
**Author:** [@Saket\_Kumar](https://discuss.elastic.co/u/Saket_Kumar)\ **Replies:** 0\ **Last updated:** [June 10, 2015, 1:35pm UTC](https://discuss.elastic.co/t/how-to-split-array/2325 "2015-06-10T13:35:16Z")
xpath =\> \[ #"response/data/latency","latency", "/response/data/run/firstView/videoFrames/frame\[\*\]/time/text()","Time\_FV", \] Output= "Time\_FV": \[ 0, 800, 1600, 1800, 7100 \], Would like split array before feedin… --- ## [Connection and configuration problems with logstash 1.5 zenoss input](https://discuss.elastic.co/t/connection-and-configuration-problems-with-logstash-1-5-zenoss-input/2314)
**Author:** [@mf2hd](https://discuss.elastic.co/u/mf2hd)\ **Replies:** 0\ **Last updated:** [June 10, 2015, 12:38pm UTC](https://discuss.elastic.co/t/connection-and-configuration-problems-with-logstash-1-5-zenoss-input/2314 "2015-06-10T12:38:46Z")
Hi all, I tried first IRC but it seems again to be serious case of "US channel" so nobody is awake when I am. But to the matter at hand: I've been trying to use zenoss input but configuration parameters does not seem … --- ## [Who could help me to find a conf file in Logstash for MySQL slowlog](https://discuss.elastic.co/t/who-could-help-me-to-find-a-conf-file-in-logstash-for-mysql-slowlog/2113)
**Author:** [@apaul](https://discuss.elastic.co/u/apaul)\ **Replies:** 1\ **Last updated:** [June 10, 2015, 12:27pm UTC](https://discuss.elastic.co/t/who-could-help-me-to-find-a-conf-file-in-logstash-for-mysql-slowlog/2113 "2015-06-10T12:27:40Z")
Hi I have a simple data with mysql slow query format as below. 2015-05-26 09:06:39^^^apaul\[apaul\] @ \[192.168.1.1\]^^^00:00:14^^^00:00:00^^^36772^^^2090463^^^mysql^^^select \* from general\_log The Separator is ^^^, … --- ## [What does the field in Grok "message" do?](https://discuss.elastic.co/t/what-does-the-field-in-grok-message-do/2253)
**Author:** [@michaellizhou](https://discuss.elastic.co/u/michaellizhou)\ **Replies:** 5\ **Last updated:** [June 10, 2015, 3:48am UTC](https://discuss.elastic.co/t/what-does-the-field-in-grok-message-do/2253 "2015-06-10T03:48:47Z")
Hi first post and new to the ELK system. So I am working on parsing through log4j logs and I cannot find what is the purpose of the "message" field. For example: filter { grok { match =\> {"message =\> I am so con… --- ## [Correct way to setup logstash](https://discuss.elastic.co/t/correct-way-to-setup-logstash/2270)
**Author:** [@abaykal](https://discuss.elastic.co/u/abaykal)\ **Replies:** 1\ **Last updated:** [June 10, 2015, 3:16am UTC](https://discuss.elastic.co/t/correct-way-to-setup-logstash/2270 "2015-06-10T03:16:47Z")
I am trying to use multiple input plugins for different functionalities. is it better to run a separate agent for each plugin instance or should I bundle it all into one? is there a preference? Thanks --- ## [How to avoid instantiating same object over and over](https://discuss.elastic.co/t/how-to-avoid-instantiating-same-object-over-and-over/2163)
**Author:** [@kyunam](https://discuss.elastic.co/u/kyunam)\ **Replies:** 1\ **Last updated:** [June 9, 2015, 10:20pm UTC](https://discuss.elastic.co/t/how-to-avoid-instantiating-same-object-over-and-over/2163 "2015-06-09T22:20:46Z")
I created a filter - some kind of converter. In the register method, I create an instance variable (i.e, @hexGrid = HexGrid.new(cellSize)) and use it in the filter method. In a configuration file, I chain my own filter… --- ## [Parse 1st Line of Multiple CSV files and set as Columns](https://discuss.elastic.co/t/parse-1st-line-of-multiple-csv-files-and-set-as-columns/2246)
**Author:** [@Wyatt\_Frelot](https://discuss.elastic.co/u/Wyatt_Frelot)\ **Replies:** 4\ **Last updated:** [June 9, 2015, 9:01pm UTC](https://discuss.elastic.co/t/parse-1st-line-of-multiple-csv-files-and-set-as-columns/2246 "2015-06-09T21:01:36Z")
Good afternoon all, I am new to using ELK. As I process CSVs using Logstash, I am unable to set "Columns" (in filter) dynamically based upon the first line in each CSV file (every CSV has different headers). I have t… --- ## [Logstash 1.5.0 json filter output to elasticsearch fails on nested json](https://discuss.elastic.co/t/logstash-1-5-0-json-filter-output-to-elasticsearch-fails-on-nested-json/2241)
**Author:** [@Justin\_Hochstetler](https://discuss.elastic.co/u/Justin_Hochstetler)\ **Replies:** 1\ **Last updated:** [June 9, 2015, 5:36pm UTC](https://discuss.elastic.co/t/logstash-1-5-0-json-filter-output-to-elasticsearch-fails-on-nested-json/2241 "2015-06-09T17:36:52Z")
Hi all, I am attempting to use logstash 1.5.0 to take in a json log file, parse via the json filter , and output to elasticsearch via the elasticsearch output with a protocol of 'http'. One of our two logs works oka… --- ## [Logstash 1.4.3 released](https://discuss.elastic.co/t/logstash-1-4-3-released/2238)
**Author:** [@suyograo](https://discuss.elastic.co/u/suyograo)\ **Replies:** 0\ **Last updated:** [June 9, 2015, 5:00pm UTC](https://discuss.elastic.co/t/logstash-1-4-3-released/2238 "2015-06-09T17:00:38Z")
Hello, We are pleased to announce the release of Logstash 1.4.3, an important bug fix release for the 1.4 series. Please check this blog for details: www.elastic.co/blog/logstash-1-4-3-released Download: https://www… --- ## [Error with Windows log](https://discuss.elastic.co/t/error-with-windows-log/1693)
**Author:** [@Sdclmb](https://discuss.elastic.co/u/Sdclmb)\ **Replies:** 4\ **Last updated:** [June 9, 2015, 2:50pm UTC](https://discuss.elastic.co/t/error-with-windows-log/1693 "2015-06-09T14:50:57Z")
hello, Let me explain the situation : I'm using 2 different server, one for my Rsyslog server and an onther for the ELK stack. I collect logs from Windows servers/computer on my Rsyslog server using Event to syslog. An… --- ## [Unknown setting for elasticsearch](https://discuss.elastic.co/t/unknown-setting-for-elasticsearch/2188)
**Author:** [@calle\_eman](https://discuss.elastic.co/u/calle_eman)\ **Replies:** 2\ **Last updated:** [June 9, 2015, 12:55pm UTC](https://discuss.elastic.co/t/unknown-setting-for-elasticsearch/2188 "2015-06-09T12:55:38Z")
I'm getting this error when trying to connect logstash to my ES cluster {:timestamp=\>"2015-06-09T07:23:11.996000+0000", :message=\>"Unknown setting 'aws\_region' for elasticsearch", :level=\>:error, :file=\>"logstash/config… --- ## [Reports - Old Data - Oh My](https://discuss.elastic.co/t/reports-old-data-oh-my/1947)
**Author:** [@bdunbar](https://discuss.elastic.co/u/bdunbar)\ **Replies:** 6\ **Last updated:** [June 9, 2015, 12:53pm UTC](https://discuss.elastic.co/t/reports-old-data-oh-my/1947 "2015-06-09T12:53:23Z")
We host 16 apache servers behind varnish. I have users (Marketing) who are not technical. They're not happy with the current stats software we have installed. I'm giving different webstats apps a try. I've got ELK ins… --- ## [XML field array question](https://discuss.elastic.co/t/xml-field-array-question/2184)
**Author:** [@Shubhangi\_Gaikwad](https://discuss.elastic.co/u/Shubhangi_Gaikwad)\ **Replies:** 2\ **Last updated:** [June 9, 2015, 11:39am UTC](https://discuss.elastic.co/t/xml-field-array-question/2184 "2015-06-09T11:39:42Z")
XML Source: "@version": "1", "@timestamp": "2015-06-08T11:10:30.296Z", "host": "localhost", "path": "/mnt/xmlpath/new13.xml", "tags": \[ "multiline" \], "response": { "statusCode": \[ … --- ## [Field conversion Query](https://discuss.elastic.co/t/field-conversion-query/2204)
**Author:** [@Saket\_Kumar](https://discuss.elastic.co/u/Saket_Kumar)\ **Replies:** 0\ **Last updated:** [June 9, 2015, 11:33am UTC](https://discuss.elastic.co/t/field-conversion-query/2204 "2015-06-09T11:33:06Z")
Parsed XML correctly using Logstash & output it to Elasticsearch... now from Kibana observed the field "response.data.run.firstView.videoFrames.frame.VisuallyComplete" has data type "String" Is there anyway to convert … --- ## [Local install only copies rb file to local\_gems](https://discuss.elastic.co/t/local-install-only-copies-rb-file-to-local-gems/2054)
**Author:** [@kyunam](https://discuss.elastic.co/u/kyunam)\ **Replies:** 5\ **Last updated:** [June 9, 2015, 7:21am UTC](https://discuss.elastic.co/t/local-install-only-copies-rb-file-to-local-gems/2054 "2015-06-09T07:21:59Z")
I followed "How to write a Logstash input plugin". When installed locally, only rb file is copied to local\_gems/..../logstash-input-something/lib/logstash/something.rb I had to copy manually all other files and folders… --- ## [How to add\_field and covert field type while parsing nested xml](https://discuss.elastic.co/t/how-to-add-field-and-covert-field-type-while-parsing-nested-xml/2134)
**Author:** [@Saket\_Kumar](https://discuss.elastic.co/u/Saket_Kumar)\ **Replies:** 5\ **Last updated:** [June 9, 2015, 6:13am UTC](https://discuss.elastic.co/t/how-to-add-field-and-covert-field-type-while-parsing-nested-xml/2134 "2015-06-09T06:13:59Z")
\ \ \ \ \...\ \...\ ..... Requirement is something like i would like add field with custom name add\_field =\> { … --- ## [New feature request: Docker files for Power platform (SLES, RHEL, Ubuntu)](https://discuss.elastic.co/t/new-feature-request-docker-files-for-power-platform-sles-rhel-ubuntu/2122)
**Author:** [@Christina\_Gandham](https://discuss.elastic.co/u/Christina_Gandham)\ **Replies:** 1\ **Last updated:** [June 9, 2015, 2:26am UTC](https://discuss.elastic.co/t/new-feature-request-docker-files-for-power-platform-sles-rhel-ubuntu/2122 "2015-06-09T02:26:38Z")
Hi, I have written dockerfile for building Logstash from source and to run the test cases. I have built and tested the source code available on Git successfully through the dockerfile for PPC64LE architecture. The dock… --- ## [Imap input filter - connection refused](https://discuss.elastic.co/t/imap-input-filter-connection-refused/2162)
**Author:** [@vilas](https://discuss.elastic.co/u/vilas)\ **Replies:** 4\ **Last updated:** [June 8, 2015, 10:40pm UTC](https://discuss.elastic.co/t/imap-input-filter-connection-refused/2162 "2015-06-08T22:40:51Z")
Hi, I am trying to use imap input filter. Using gmail account for testing purposes. Config File input { imap { host =\> "imap.gmail.com" user =\> "someuser@gmail.com" password =\> "somepassword" port =\> … --- ## [Logstash-forwarder queuing support](https://discuss.elastic.co/t/logstash-forwarder-queuing-support/2129)
**Author:** [@tlog](https://discuss.elastic.co/u/tlog)\ **Replies:** 3\ **Last updated:** [June 8, 2015, 8:14pm UTC](https://discuss.elastic.co/t/logstash-forwarder-queuing-support/2129 "2015-06-08T20:14:03Z")
Hello all, in https://github.com/elastic/logstash-forwarder/issues/18 we see the issue is closed, therefore I thought, redis support is now included in 0.4.0. But how can redis as target being configured? The config … --- ## [Error While Parsing CSV file](https://discuss.elastic.co/t/error-while-parsing-csv-file/2090)
**Author:** [@Vicky\_aquarian](https://discuss.elastic.co/u/Vicky_aquarian)\ **Replies:** 3\ **Last updated:** [June 8, 2015, 8:10pm UTC](https://discuss.elastic.co/t/error-while-parsing-csv-file/2090 "2015-06-08T20:10:14Z")
Hi, I'm using logstash ver logstash-1.5.0.rc2 and trying to parse log files containing lines like : 2015-02-03 06:10:38.6024|MEMSGSERVER|Trace|WcfMessageInspector|14|52e46324-a775-4a91-96d6-7efd3763cf88|END SERVICE RE… --- ## [Exact HOUR from date type field](https://discuss.elastic.co/t/exact-hour-from-date-type-field/2047)
**Author:** [@kyunam](https://discuss.elastic.co/u/kyunam)\ **Replies:** 2\ **Last updated:** [June 8, 2015, 7:34pm UTC](https://discuss.elastic.co/t/exact-hour-from-date-type-field/2047 "2015-06-08T19:34:20Z")
I have a date field like this - "orig\_dt" =\> "2008-09-20 17:51:00", How do I extract HOUR (=17) from it? Thanks! --- ## [How to discard entire lines of input](https://discuss.elastic.co/t/how-to-discard-entire-lines-of-input/2139)
**Author:** [@bdunbar](https://discuss.elastic.co/u/bdunbar)\ **Replies:** 3\ **Last updated:** [June 8, 2015, 7:30pm UTC](https://discuss.elastic.co/t/how-to-discard-entire-lines-of-input/2139 "2015-06-08T19:30:58Z")
I have input coming from eight Apache webservers. Included in these logs are a fair bit of inter-cluster traffic. I'd like to not include this in my results, as they aren't users, just the servers running a health chec… --- ## [Ruby Filter output to Field](https://discuss.elastic.co/t/ruby-filter-output-to-field/1876)
**Author:** [@jruybal](https://discuss.elastic.co/u/jruybal)\ **Replies:** 2\ **Last updated:** [June 8, 2015, 5:58pm UTC](https://discuss.elastic.co/t/ruby-filter-output-to-field/1876 "2015-06-08T17:58:41Z")
I'm testing a ruby filter to build a small list of hosts that are producing less than a certain number of events per minute. Is there a standard method to get this output back into the event json? Currently I'm tryin… --- ## [\[Zabbix output plugin\] zabbix\_key is missing](https://discuss.elastic.co/t/zabbix-output-plugin-zabbix-key-is-missing/2123)
**Author:** [@MrAV](https://discuss.elastic.co/u/MrAV)\ **Replies:** 1\ **Last updated:** [June 8, 2015, 3:55pm UTC](https://discuss.elastic.co/t/zabbix-output-plugin-zabbix-key-is-missing/2123 "2015-06-08T15:55:53Z")
Hi, I'm trying to parse a log-file with Logstash (1.5.0) and send output to Zabbix. Unfortunately, I'm receiving the following error: ←\[33mSkipping zabbix output; field referenced by quik.exp is missing {:level=\>:war… --- ## [When ctrl+c logstash, indexes in ES go into red state](https://discuss.elastic.co/t/when-ctrl-c-logstash-indexes-in-es-go-into-red-state/2147)
**Author:** [@rodrigomerino](https://discuss.elastic.co/u/rodrigomerino)\ **Replies:** 0\ **Last updated:** [June 8, 2015, 3:45pm UTC](https://discuss.elastic.co/t/when-ctrl-c-logstash-indexes-in-es-go-into-red-state/2147 "2015-06-08T15:45:29Z")
Hi all, I have a problem when using logstash to get tweets and inserting them into ES. When I stop logstash, via Ctrl+C or Kill (logstash stop doesn’t do anything) makes the indexes where the tweets are being stored to… --- ## [Upgraded to 1.5.0 from 1.4.2 and sincedbpath (file input) no longer happy](https://discuss.elastic.co/t/upgraded-to-1-5-0-from-1-4-2-and-sincedbpath-file-input-no-longer-happy/1355)
**Author:** [@anoban](https://discuss.elastic.co/u/anoban)\ **Replies:** 1\ **Last updated:** [June 8, 2015, 7:52am UTC](https://discuss.elastic.co/t/upgraded-to-1-5-0-from-1-4-2-and-sincedbpath-file-input-no-longer-happy/1355 "2015-06-08T07:52:02Z")
input file, sincedb\_path no happy with the previous config Logstash changes the permissions of the directory, and cannot then read the files. This is what I get after a restart on 1.5.0 \[elk@elksyd001 logstash-1.5… --- ## [About index template upon output from logstash to elasticsearch](https://discuss.elastic.co/t/about-index-template-upon-output-from-logstash-to-elasticsearch/1982)
**Author:** [@zkun](https://discuss.elastic.co/u/zkun)\ **Replies:** 3\ **Last updated:** [June 8, 2015, 6:56am UTC](https://discuss.elastic.co/t/about-index-template-upon-output-from-logstash-to-elasticsearch/1982 "2015-06-08T06:56:02Z")
Hi guys, recently I'm trying to setup multiple indices in ES for log storage of different services. And I noticed that I might need to set field value types for them respectively, which involves applying different templa… --- ## [Unknown setting 'user/password' for elasticsearch {:level=\>:error}](https://discuss.elastic.co/t/unknown-setting-user-password-for-elasticsearch-level-error/1959)
**Author:** [@Raman\_Saini](https://discuss.elastic.co/u/Raman_Saini)\ **Replies:** 2\ **Last updated:** [June 8, 2015, 6:11am UTC](https://discuss.elastic.co/t/unknown-setting-user-password-for-elasticsearch-level-error/1959 "2015-06-08T06:11:24Z")
Hi all, Please help to fix this issue. I am using elastic search with shield and logstash. When i try to run logstash i am getting below error Unknown setting 'user' for elasticsearch {:level=\>:error} Unknown setti… --- ## [Logstash-forwarder missing lines after log rotate](https://discuss.elastic.co/t/logstash-forwarder-missing-lines-after-log-rotate/2111)
**Author:** [@Laines](https://discuss.elastic.co/u/Laines)\ **Replies:** 2\ **Last updated:** [June 8, 2015, 3:59am UTC](https://discuss.elastic.co/t/logstash-forwarder-missing-lines-after-log-rotate/2111 "2015-06-08T03:59:56Z")
Hello, We have problems with the logstash-forwarder after the log rotate of massages & secure we are missing the first couple of messages. At least the first one and then sometimes the next 2-5. After that there isn't … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1259) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1261)