# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1202 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1203 --- ## [Logstash 2.1 fails to start after logstash-output-elasticsearch update](https://discuss.elastic.co/t/logstash-2-1-fails-to-start-after-logstash-output-elasticsearch-update/38330)
**Author:** [@techcheez](https://discuss.elastic.co/u/techcheez)\ **Replies:** 6\ **Last updated:** [January 14, 2016, 5:55am UTC](https://discuss.elastic.co/t/logstash-2-1-fails-to-start-after-logstash-output-elasticsearch-update/38330 "2016-01-14T05:55:39Z")
I just ran a bin/plugin update and the logstash-output-elasticsearch plugin was upgraded from 2.3.0 to 2.3.1. When I tried restarting logstash I get the following error: The error reported is: you might need to rein… --- ## [Logstash: Unable to create index in elasticsearch protected by Shield](https://discuss.elastic.co/t/logstash-unable-to-create-index-in-elasticsearch-protected-by-shield/39187)
**Author:** [@krushnat\_khawale](https://discuss.elastic.co/u/krushnat_khawale)\ **Replies:** 1\ **Last updated:** [January 14, 2016, 5:44am UTC](https://discuss.elastic.co/t/logstash-unable-to-create-index-in-elasticsearch-protected-by-shield/39187 "2016-01-14T05:44:40Z")
I've protected elasticsearch using SHIELD and elasticsearch is prompting me for credentials when trying to access using SENSE, or KIBANA as expected. But It is not allowing me to create index using logstash. It… --- ## [Creating Logstash Filter using Grok plugin](https://discuss.elastic.co/t/creating-logstash-filter-using-grok-plugin/38760)
**Author:** [@Kaufusihm](https://discuss.elastic.co/u/Kaufusihm)\ **Replies:** 6\ **Last updated:** [January 14, 2016, 12:25am UTC](https://discuss.elastic.co/t/creating-logstash-filter-using-grok-plugin/38760 "2016-01-14T00:25:32Z")
I'm trying to build a grok filter for my logstash filter config. This is a sample of a standard event I'm trying to tag. \[2016-01-05T23:59:23.352-0700\] \[glassfish 4.1\] \[INFO\] \[\] \[edu.utah.acs.student.classinformation… --- ## [File input breaks my logstash config](https://discuss.elastic.co/t/file-input-breaks-my-logstash-config/38909)
**Author:** [@CamJN](https://discuss.elastic.co/u/CamJN)\ **Replies:** 3\ **Last updated:** [January 13, 2016, 7:51pm UTC](https://discuss.elastic.co/t/file-input-breaks-my-logstash-config/38909 "2016-01-13T19:51:15Z")
I was previously using the stdin input plugin for testing and while startup was slow, it worked. When I switched to the file input startup takes ages and never actually completes (I don't get the startup completed messag… --- ## [Best way to ingest JSON files?](https://discuss.elastic.co/t/best-way-to-ingest-json-files/39136)
**Author:** [@CraigFoote](https://discuss.elastic.co/u/CraigFoote)\ **Replies:** 5\ **Last updated:** [January 13, 2016, 7:42pm UTC](https://discuss.elastic.co/t/best-way-to-ingest-json-files/39136 "2016-01-13T19:42:53Z")
I have 300 files to ingest. Each is pretty-printed JSON and contains one event. I tried using the file input and the multiline codec: input{ file{ path =\> "/tmp/ID\*.json" start\_position =\> "beginnin… --- ## [Logstash builds Recv-Q for minute long pauses](https://discuss.elastic.co/t/logstash-builds-recv-q-for-minute-long-pauses/38422)
**Author:** [@ggriffith](https://discuss.elastic.co/u/ggriffith)\ **Replies:** 5\ **Last updated:** [January 13, 2016, 7:14pm UTC](https://discuss.elastic.co/t/logstash-builds-recv-q-for-minute-long-pauses/38422 "2016-01-13T19:14:51Z")
Hello, We are sending syslog data over the wire using stunnel to encrypt, and when it gets to the logstash server (on Ubuntu 14.04), we periodically build up a big Recv-Q as the data moves out of the server-side stunne… --- ## [Duplicated logs from logstash after append logs](https://discuss.elastic.co/t/duplicated-logs-from-logstash-after-append-logs/39097)
**Author:** [@carmelom](https://discuss.elastic.co/u/carmelom)\ **Replies:** 6\ **Last updated:** [January 13, 2016, 4:26pm UTC](https://discuss.elastic.co/t/duplicated-logs-from-logstash-after-append-logs/39097 "2016-01-13T16:26:40Z")
When my java application is updating the logs, logstash is re-reading the file from the begning and kibana is displaing twice the same file. I tried with this configuration : input { file { path =\> "/home/ubuntu/ELK… --- ## [Syslog-ng directly into Redis from which a central Logstash indexer can read](https://discuss.elastic.co/t/syslog-ng-directly-into-redis-from-which-a-central-logstash-indexer-can-read/32590)
**Author:** [@ottignon](https://discuss.elastic.co/u/ottignon)\ **Replies:** 11\ **Last updated:** [January 13, 2016, 4:18pm UTC](https://discuss.elastic.co/t/syslog-ng-directly-into-redis-from-which-a-central-logstash-indexer-can-read/32590 "2016-01-13T16:18:44Z")
Hi, Currently doing a feasability test of the ELK stack and Redis solution as a replacement for Splunk. I've done much googling and lot of testing but it's hard to run wild in a DevOps enviroment where everything is po… --- ## [Logstash and OpenBSD format logs](https://discuss.elastic.co/t/logstash-and-openbsd-format-logs/39098)
**Author:** [@Devnull](https://discuss.elastic.co/u/Devnull)\ **Replies:** 3\ **Last updated:** [January 13, 2016, 1:08pm UTC](https://discuss.elastic.co/t/logstash-and-openbsd-format-logs/39098 "2016-01-13T13:08:56Z")
Am sending OpenBSD logs to logstash via a syslog input. Logstash however, appears to be having trouble digesting the OpenBSD format, it adds a "grokparsefailuresysloginput" tag. This is the message format : \<30\>Jan … --- ## [Email plugin (LS 2.1.1 windows) for Gmail](https://discuss.elastic.co/t/email-plugin-ls-2-1-1-windows-for-gmail/38972)
**Author:** [@john22122015](https://discuss.elastic.co/u/john22122015)\ **Replies:** 6\ **Last updated:** [January 13, 2016, 10:44am UTC](https://discuss.elastic.co/t/email-plugin-ls-2-1-1-windows-for-gmail/38972 "2016-01-13T10:44:37Z")
Hi colleagues! I have an issue with LS 2.1.1 windows, in email plugin for gmail configuration. Thus, the email is not sent. Despite of the email is not sent, the indices are created normally. When I iniciate logstash, … --- ## [How to Create a Watchlist/Lookup Tables](https://discuss.elastic.co/t/how-to-create-a-watchlist-lookup-tables/38228)
**Author:** [@praveen\_siem](https://discuss.elastic.co/u/praveen_siem)\ **Replies:** 5\ **Last updated:** [January 13, 2016, 10:23am UTC](https://discuss.elastic.co/t/how-to-create-a-watchlist-lookup-tables/38228 "2016-01-13T10:23:04Z")
Dear Team, Do anyone have idea on how do we create a watchlist in ELK. In other terms lookup tables. This specifies a range of values say 50 or 100 or more in a single file. And save it as .csv or .txt For Ex: there … --- ## [Grokdebugger works but not with logstash](https://discuss.elastic.co/t/grokdebugger-works-but-not-with-logstash/39010)
**Author:** [@tikejhya](https://discuss.elastic.co/u/tikejhya)\ **Replies:** 2\ **Last updated:** [January 13, 2016, 9:32am UTC](https://discuss.elastic.co/t/grokdebugger-works-but-not-with-logstash/39010 "2016-01-13T09:32:23Z")
my pattern seem to not work with logstash but works fine with grokdebbuger & consturctor. I thought its matter of order and so and and removed all other patterns except grok parser/shortcut. But that doesn't seem to he… --- ## [Read a gzip file with gzip\_lines codec](https://discuss.elastic.co/t/read-a-gzip-file-with-gzip-lines-codec/24558)
**Author:** [@dima\_eam](https://discuss.elastic.co/u/dima_eam)\ **Replies:** 11\ **Last updated:** [January 12, 2016, 9:42am UTC](https://discuss.elastic.co/t/read-a-gzip-file-with-gzip-lines-codec/24558 "2016-01-12T09:42:34Z")
Hi! I need to use Logstash for parsing \*.gz files. I've downloaded gzip\_lines gem and installed it with logstash-1.5.1\\bin\>plugin install logstash-codec-gzip\_lines-0.1.5.gem the output said "Installation succesful… --- ## [XPATH expression to parse a nested xml file](https://discuss.elastic.co/t/xpath-expression-to-parse-a-nested-xml-file/38941)
**Author:** [@Tarun\_Sant](https://discuss.elastic.co/u/Tarun_Sant)\ **Replies:** 2\ **Last updated:** [January 13, 2016, 8:04am UTC](https://discuss.elastic.co/t/xpath-expression-to-parse-a-nested-xml-file/38941 "2016-01-13T08:04:11Z")
Following is the input xml I want to parse. \ \ \ \ \ \ \ \test\_… --- ## [Store Data in a temporary Variable](https://discuss.elastic.co/t/store-data-in-a-temporary-variable/39056)
**Author:** [@shabinhashim](https://discuss.elastic.co/u/shabinhashim)\ **Replies:** 1\ **Last updated:** [January 13, 2016, 7:34am UTC](https://discuss.elastic.co/t/store-data-in-a-temporary-variable/39056 "2016-01-13T07:34:53Z")
HI All, I have the below filter in logstash if(\[orderstatuscode\]=="1"){ mutate{ add\_field =\> \["status","Accept"\] } } else if(\[orderstatuscode\]=="2"){ mutate{ ad… --- ## [Filter + if results in “can't convert Array into String” error](https://discuss.elastic.co/t/filter-if-results-in-cant-convert-array-into-string-error/38805)
**Author:** [@tebeka](https://discuss.elastic.co/u/tebeka)\ **Replies:** 6\ **Last updated:** [January 13, 2016, 6:19am UTC](https://discuss.elastic.co/t/filter-if-results-in-cant-convert-array-into-string-error/38805 "2016-01-13T06:19:54Z")
We have beaver shipping logs to logstash. Two log sources are nginx error logs and haproxy logs. beaver is adding a tag for the log type and we'd like to parse the log according to type. We wrote the following configurat… --- ## [Does Logstash 2.1.1 work well with Kafka 0.9.0?](https://discuss.elastic.co/t/does-logstash-2-1-1-work-well-with-kafka-0-9-0/38831)
**Author:** [@Kenny\_Qiao](https://discuss.elastic.co/u/Kenny_Qiao)\ **Replies:** 4\ **Last updated:** [January 13, 2016, 1:39am UTC](https://discuss.elastic.co/t/does-logstash-2-1-1-work-well-with-kafka-0-9-0/38831 "2016-01-13T01:39:47Z")
I plan to upgrade Kafka to 0.9.0 and wondering if logstash 2.1.1 could work well with Kafka 0.9.0. Does anyone have experience using logstash 2.1.1 with Kafka0.9.0 in production? Thanks. --- ## [Exception Handling](https://discuss.elastic.co/t/exception-handling/38989)
**Author:** [@matteocremolini](https://discuss.elastic.co/u/matteocremolini)\ **Replies:** 1\ **Last updated:** [January 12, 2016, 6:47pm UTC](https://discuss.elastic.co/t/exception-handling/38989 "2016-01-12T18:47:01Z")
Hi all !!!! I'm using this filter: date { match =\> \["My field","dd-MM-YYYY HH.mm.ss","ISO8601"\] target =\> \["My field"\] } I would like handle the runtime exception that can occur. For example a code structur… --- ## [Increasing logstash throughput for the file input and file output](https://discuss.elastic.co/t/increasing-logstash-throughput-for-the-file-input-and-file-output/38893)
**Author:** [@puneet035](https://discuss.elastic.co/u/puneet035)\ **Replies:** 11\ **Last updated:** [January 12, 2016, 5:00pm UTC](https://discuss.elastic.co/t/increasing-logstash-throughput-for-the-file-input-and-file-output/38893 "2016-01-12T17:00:47Z")
Dear all, How to increase logstash throughput for file input and file output plugins. I am using logstash on windows environment. My sample test produces 1.7GB of log file via stream writer, and I am moving these log m… --- ## [Trying to ingest gzipped csv logs](https://discuss.elastic.co/t/trying-to-ingest-gzipped-csv-logs/37885)
**Author:** [@naga1](https://discuss.elastic.co/u/naga1)\ **Replies:** 3\ **Last updated:** [January 12, 2016, 3:53pm UTC](https://discuss.elastic.co/t/trying-to-ingest-gzipped-csv-logs/37885 "2016-01-12T15:53:02Z")
Trying to ingest the CSV.gz logs on to elastic, is there any module to do that. For now the gunzipped files I was able to get to elastic using the logstash csv filter plugin. https://www.elastic.co/guide/en/logstash/c… --- ## [Logstash can not output redis data to ElasticSearch](https://discuss.elastic.co/t/logstash-can-not-output-redis-data-to-elasticsearch/38932)
**Author:** [@tctr715](https://discuss.elastic.co/u/tctr715)\ **Replies:** 2\ **Last updated:** [January 12, 2016, 12:13pm UTC](https://discuss.elastic.co/t/logstash-can-not-output-redis-data-to-elasticsearch/38932 "2016-01-12T12:13:40Z")
my logstash.conf: input { redis { host =\> "10.57.147.83" port =\> "6379" key =\> "logstash" data\_type =\> "list" codec =\> "json" type =\> "logstash-redis-demo" tags =\> \["logstashdemo"\] } } … --- ## [Grok Filter does not work with a newLine](https://discuss.elastic.co/t/grok-filter-does-not-work-with-a-newline/38951)
**Author:** [@golauty](https://discuss.elastic.co/u/golauty)\ **Replies:** 4\ **Last updated:** [January 12, 2016, 11:50am UTC](https://discuss.elastic.co/t/grok-filter-does-not-work-with-a-newline/38951 "2016-01-12T11:50:25Z")
I have following event in Logstash: "message": "\[The variable 'number' was not bound while trying to expand attribute 'text' with value '$(value)'.\]\\n2016-01-12 08:30:09,983 \[exec\]\[TestCase\] \ **Author:** [@karelstriegel](https://discuss.elastic.co/u/karelstriegel)\ **Replies:** 1\ **Last updated:** [January 12, 2016, 10:15am UTC](https://discuss.elastic.co/t/logstash-service-started-but-not-working-command-line-is-working-perfectly/38957 "2016-01-12T10:15:21Z") Dear, I have a logstash configuration that is working when I run logstash manually : sudo -u logstash /opt/logstash/bin/logstash agent -f /etc/logstash/conf.d But doesn't work when I'm running at as a service serv… --- ## [Catch SNMP traps from windows for every 5minutes](https://discuss.elastic.co/t/catch-snmp-traps-from-windows-for-every-5minutes/38846)
**Author:** [@Jansi](https://discuss.elastic.co/u/Jansi)\ **Replies:** 3\ **Last updated:** [January 12, 2016, 7:47am UTC](https://discuss.elastic.co/t/catch-snmp-traps-from-windows-for-every-5minutes/38846 "2016-01-12T07:47:14Z")
Hi, Can anyone please let me know how to collect the snmp traps from windows system using CentOS machine for every 5minutes using Logstash? Also i needs to know how to get trap by mentioning the OID using Logstash? … --- ## [Grok Filter with alternative](https://discuss.elastic.co/t/grok-filter-with-alternative/38858)
**Author:** [@golauty](https://discuss.elastic.co/u/golauty)\ **Replies:** 2\ **Last updated:** [January 12, 2016, 5:50am UTC](https://discuss.elastic.co/t/grok-filter-with-alternative/38858 "2016-01-12T05:50:18Z")
Hi, i have following GrokFilter extract: \[%{WORD:result}\]\\s+\[Warnings: %{WORD:WarningCount}\]|\[%{WORD:result}\]\\s\*\[Warnings: %{WORD:WarningCount}\]\\s\*\[Errors: %{WORD:ErrorCount}\]\\s\*\[Exceptions: %{WORD:ExceptionCount}\] F… --- ## [Undefined method \`close\` on Logstash](https://discuss.elastic.co/t/undefined-method-close-on-logstash/38934)
**Author:** [@tatdat](https://discuss.elastic.co/u/tatdat)\ **Replies:** 0\ **Last updated:** [January 12, 2016, 4:53am UTC](https://discuss.elastic.co/t/undefined-method-close-on-logstash/38934 "2016-01-12T04:53:48Z")
I Setup ELk : LS Forwarder -\> RabbitMQ -\> Logstash Proccessor -\> ES ES ver 2.1.1; Rabbit 3.6.1; LS newest. I have 2 LS-Forwarder (8Core, 8GB RAM), 2Rabbitmq (cluster HA - 8Core, 8GB RAM), 2 LS-Processer (8Core, 8GB RAM… --- ## [Logstash-output-jdbc plugin: Problems getting mysql example to work](https://discuss.elastic.co/t/logstash-output-jdbc-plugin-problems-getting-mysql-example-to-work/38921)
**Author:** [@jelliott](https://discuss.elastic.co/u/jelliott)\ **Replies:** 0\ **Last updated:** [January 11, 2016, 9:39pm UTC](https://discuss.elastic.co/t/logstash-output-jdbc-plugin-problems-getting-mysql-example-to-work/38921 "2016-01-11T21:39:16Z")
I'm playing around with diverting some documents to MySql. I thought I'd try out the logstash-output-jdbc plugin,but I'm having issues. The mysql example looks like this: input { stdin { } } output { jdbc … --- ## [Loading a CSV file, basic question](https://discuss.elastic.co/t/loading-a-csv-file-basic-question/38528)
**Author:** [@Christopher\_Curzon](https://discuss.elastic.co/u/Christopher_Curzon)\ **Replies:** 5\ **Last updated:** [January 11, 2016, 6:56pm UTC](https://discuss.elastic.co/t/loading-a-csv-file-basic-question/38528 "2016-01-11T18:56:11Z")
This is my first effort getting logstash to put data into Elasticsearch. So I think I must be missing something fairly obvious. I have a addr.csv file with records in this format: 369|90045|123 ABC ST|LOS ANGELES|CA … --- ## [Unable to start logstash-forwarder in centos 5](https://discuss.elastic.co/t/unable-to-start-logstash-forwarder-in-centos-5/38841)
**Author:** [@gugansankar](https://discuss.elastic.co/u/gugansankar)\ **Replies:** 1\ **Last updated:** [January 11, 2016, 2:39pm UTC](https://discuss.elastic.co/t/unable-to-start-logstash-forwarder-in-centos-5/38841 "2016-01-11T14:39:43Z")
I tried install logstash forwarder in one of my production server to monitor the application logs. When I start the logstash-forwarder , its show "Segment Fault Error". Could you please guide, does logstash-forwarder wo… --- ## [Can two logstash commands run in parallel to pull data in elastic search?](https://discuss.elastic.co/t/can-two-logstash-commands-run-in-parallel-to-pull-data-in-elastic-search/38790)
**Author:** [@maryam.tauheed](https://discuss.elastic.co/u/maryam.tauheed)\ **Replies:** 10\ **Last updated:** [January 11, 2016, 11:47am UTC](https://discuss.elastic.co/t/can-two-logstash-commands-run-in-parallel-to-pull-data-in-elastic-search/38790 "2016-01-11T11:47:52Z")
I am tring to execute this command. "bin/logstash -f new.conf \< a.log and bin/logstash -f new.conf \< b.log" In my config file of logstash: output { elasticsearch { action=\>"index" hosts =\> \["localhost:9200"\] … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1201) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1203)