# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1061 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1062 --- ## [Can't get logstash read json files to insert into ES](https://discuss.elastic.co/t/cant-get-logstash-read-json-files-to-insert-into-es/69809)
**Author:** [@francopp](https://discuss.elastic.co/u/francopp)\ **Replies:** 1\ **Last updated:** [December 22, 2016, 3:16pm UTC](https://discuss.elastic.co/t/cant-get-logstash-read-json-files-to-insert-into-es/69809 "2016-12-22T15:16:22Z")
I try to run a JSON file trough Logstash to index in Elasticsearch, but I can't seem to get it indexed. I don't get any error messages or anything. It says: "Logstash startup complete". My .conf file is set up like thi… --- ## [Multiple matches with a single regex](https://discuss.elastic.co/t/multiple-matches-with-a-single-regex/69811)
**Author:** [@jetnet](https://discuss.elastic.co/u/jetnet)\ **Replies:** 0\ **Last updated:** [December 22, 2016, 3:00pm UTC](https://discuss.elastic.co/t/multiple-matches-with-a-single-regex/69811 "2016-12-22T15:00:49Z")
Hi logstash users! is it possible to catch all matches (global match) with a single regex? E.g. given the following text, which gets processed by multiline plugin first: Caused by: a.b.c.Exception1: text1 Caused by… --- ## [S3 input plugin can't backup to same bucket using prefix](https://discuss.elastic.co/t/s3-input-plugin-cant-backup-to-same-bucket-using-prefix/69713)
**Author:** [@sura](https://discuss.elastic.co/u/sura)\ **Replies:** 1\ **Last updated:** [December 22, 2016, 3:08pm UTC](https://discuss.elastic.co/t/s3-input-plugin-cant-backup-to-same-bucket-using-prefix/69713 "2016-12-22T15:08:49Z")
In the s3 input plugin documentation , it says that it should be possible to backup to a different "folder" within the same bucket. This doesn't seem to work however. Any ideas? My input config: input { s3 { bu… --- ## [Query Relational DB based on a field value in Logstash?](https://discuss.elastic.co/t/query-relational-db-based-on-a-field-value-in-logstash/42215)
**Author:** [@RaviSidhu](https://discuss.elastic.co/u/RaviSidhu)\ **Replies:** 12\ **Last updated:** [December 22, 2016, 1:11pm UTC](https://discuss.elastic.co/t/query-relational-db-based-on-a-field-value-in-logstash/42215 "2016-12-22T13:11:57Z")
I can parse the log files using logstash. Now, I want to process further one of the fields just parsed. I want to gather additional information for that field. But the information is stored in a relational database. Is i… --- ## [Error file logstash configuration](https://discuss.elastic.co/t/error-file-logstash-configuration/52445)
**Author:** [@tantely](https://discuss.elastic.co/u/tantely)\ **Replies:** 3\ **Last updated:** [December 22, 2016, 1:09pm UTC](https://discuss.elastic.co/t/error-file-logstash-configuration/52445 "2016-12-22T13:09:46Z")
Hello everybody, I'm using logstash to connect with my database sql server the purpose is to send all data in my table to elasticsearch see below my logstash.conf input { jdbc { jdbc\_driver\_library =\> "c:\\program f… --- ## [Filter-CSV Plugin 5.0 - Columns are auto calculated or should be specified in the config file](https://discuss.elastic.co/t/filter-csv-plugin-5-0-columns-are-auto-calculated-or-should-be-specified-in-the-config-file/69754)
**Author:** [@Ajay1](https://discuss.elastic.co/u/Ajay1)\ **Replies:** 1\ **Last updated:** [December 22, 2016, 10:16am UTC](https://discuss.elastic.co/t/filter-csv-plugin-5-0-columns-are-auto-calculated-or-should-be-specified-in-the-config-file/69754 "2016-12-22T10:16:18Z")
I found Logstash - Filter csv Plugin interestingly but wondering whether i need to prepare a config file if the column header is different for the files which i like to see in Kibana. Is there any auto-intelligence to de… --- ## [Elastic Search index creating with logstash 5.1](https://discuss.elastic.co/t/elastic-search-index-creating-with-logstash-5-1/69520)
**Author:** [@shilong](https://discuss.elastic.co/u/shilong)\ **Replies:** 1\ **Last updated:** [December 22, 2016, 9:13am UTC](https://discuss.elastic.co/t/elastic-search-index-creating-with-logstash-5-1/69520 "2016-12-22T09:13:40Z")
We are migrating the old data to the new elasticsearch 5+ version. We find that elasticsearch in not automatically creating the index based on our logstash configuration. And always seeing the below error message, is the… --- ## [Hard to stash a log file with different occurrence of order for a field using Logstash](https://discuss.elastic.co/t/hard-to-stash-a-log-file-with-different-occurrence-of-order-for-a-field-using-logstash/69641)
**Author:** [@VijayKarthikeyan](https://discuss.elastic.co/u/VijayKarthikeyan)\ **Replies:** 6\ **Last updated:** [December 22, 2016, 7:07am UTC](https://discuss.elastic.co/t/hard-to-stash-a-log-file-with-different-occurrence-of-order-for-a-field-using-logstash/69641 "2016-12-22T07:07:21Z")
I am trying to stash a log file to elasticsearch using Logstash. I am facing a problem while doing this. If the log file has same kind of log lines like the below, \[12/Sep/2016:18:23:07\] VendorID=5037 Code=C AcctID=53… --- ## [Mutate filter throws NumberFormat Exception for no reason](https://discuss.elastic.co/t/mutate-filter-throws-numberformat-exception-for-no-reason/69597)
**Author:** [@rbrandstaedter](https://discuss.elastic.co/u/rbrandstaedter)\ **Replies:** 1\ **Last updated:** [December 22, 2016, 7:00am UTC](https://discuss.elastic.co/t/mutate-filter-throws-numberformat-exception-for-no-reason/69597 "2016-12-22T07:00:08Z")
I recently migrated from 3.1 to 5.1 and wanted to continue to use my logstash filter/output configuration. Since the event API changed I adopted those but soon realised that the mutate filter is suddenly throwing excpeti… --- ## [How to solve such a problem that multiple indexer access to incomplete data?](https://discuss.elastic.co/t/how-to-solve-such-a-problem-that-multiple-indexer-access-to-incomplete-data/69353)
**Author:** [@sockaddr\_in](https://discuss.elastic.co/u/sockaddr_in)\ **Replies:** 1\ **Last updated:** [December 22, 2016, 6:43am UTC](https://discuss.elastic.co/t/how-to-solve-such-a-problem-that-multiple-indexer-access-to-incomplete-data/69353 "2016-12-22T06:43:31Z")
If there are multiple indexer from redis access to data, resulting in each indexer data are not complete, I would like to ask how to solve this problem? Generally speaking, a logstash system can only be one indexer? So i… --- ## [Logstash cannot find mongodb output plugin](https://discuss.elastic.co/t/logstash-cannot-find-mongodb-output-plugin/69697)
**Author:** [@Jimmy\_Kang](https://discuss.elastic.co/u/Jimmy_Kang)\ **Replies:** 7\ **Last updated:** [December 21, 2016, 11:52pm UTC](https://discuss.elastic.co/t/logstash-cannot-find-mongodb-output-plugin/69697 "2016-12-21T23:52:11Z")
output { mongodb { uri =\> "url" collection =\> "test" database =\> "test"} Above is my logstash conf file. I have also installed the mongo output plugin and confirmed it is installed. Edit, so the plugin … --- ## [Logstash fails to load indian log records due to daylight savings change on host machine](https://discuss.elastic.co/t/logstash-fails-to-load-indian-log-records-due-to-daylight-savings-change-on-host-machine/69525)
**Author:** [@Matt\_Simcox](https://discuss.elastic.co/u/Matt_Simcox)\ **Replies:** 4\ **Last updated:** [December 21, 2016, 10:13pm UTC](https://discuss.elastic.co/t/logstash-fails-to-load-indian-log-records-due-to-daylight-savings-change-on-host-machine/69525 "2016-12-21T22:13:58Z")
I'm not sure if there's a way around this one. I have a concatenated year long log file provided by an indian client who do not yet use logstash. I have set up a test box in Sydney Australia to parse their logs to anal… --- ## [Using Multi Value in Zabbix Output](https://discuss.elastic.co/t/using-multi-value-in-zabbix-output/69707)
**Author:** [@singu](https://discuss.elastic.co/u/singu)\ **Replies:** 3\ **Last updated:** [December 21, 2016, 9:26pm UTC](https://discuss.elastic.co/t/using-multi-value-in-zabbix-output/69707 "2016-12-21T21:26:16Z")
Hi, I am new to logstash and looking for option to send application logs to Zabbix. I was able to setup the logstash configuration with zabbix\_key, but it is only getting single messages from the log and showing in Zabbi… --- ## [Bundle Update fails when updating snapshot](https://discuss.elastic.co/t/bundle-update-fails-when-updating-snapshot/69708)
**Author:** [@Jimmy\_Kang](https://discuss.elastic.co/u/Jimmy_Kang)\ **Replies:** 0\ **Last updated:** [December 21, 2016, 8:37pm UTC](https://discuss.elastic.co/t/bundle-update-fails-when-updating-snapshot/69708 "2016-12-21T20:37:04Z")
Getting this error when I am trying to update my snapshot to install a plugin. Bundler could not find compatible versions for gem "logstash": In Gemfile: logstash-output-sqs java was resolved to 0.1.0, which depen… --- ## [Custom Logstash input plugin](https://discuss.elastic.co/t/custom-logstash-input-plugin/69703)
**Author:** [@logstash.chirag](https://discuss.elastic.co/u/logstash.chirag)\ **Replies:** 0\ **Last updated:** [December 21, 2016, 7:32pm UTC](https://discuss.elastic.co/t/custom-logstash-input-plugin/69703 "2016-12-21T19:32:09Z")
Hi Team, I wanted to build custom Logstash input plugin, so I can call java classes from ruby code. I already installed jruby in my local system but unable to install gem locally. Could you please guide me or share a… --- ## [Output plugin for influxdb fails to flush items](https://discuss.elastic.co/t/output-plugin-for-influxdb-fails-to-flush-items/69687)
**Author:** [@rbrandstaedter](https://discuss.elastic.co/u/rbrandstaedter)\ **Replies:** 0\ **Last updated:** [December 21, 2016, 4:15pm UTC](https://discuss.elastic.co/t/output-plugin-for-influxdb-fails-to-flush-items/69687 "2016-12-21T16:15:17Z")
I've set up a environment wher logstash output plugin (4.0.0) sends data to influxdb (1.1) and for unknown reasons logstash suddenly stops sending data/flushing items: \[2016-12-21T16:00:26,810\]\[WARN \]\[logstash.outputs.… --- ## [\[Logstash 5.1.1\] Problem with logging](https://discuss.elastic.co/t/logstash-5-1-1-problem-with-logging/69663)
**Author:** [@Myllka](https://discuss.elastic.co/u/Myllka)\ **Replies:** 0\ **Last updated:** [December 21, 2016, 11:10am UTC](https://discuss.elastic.co/t/logstash-5-1-1-problem-with-logging/69663 "2016-12-21T11:10:24Z")
Hi, I updated this week from 5.0.0 to 5.1.1, put every configuration I made, and everything works fine, except the logging of Logstash. I compared with my backup of 5.0.0 and seems right, so I don't understand. In th… --- ## [UDP listener died. IOError: closed stream](https://discuss.elastic.co/t/udp-listener-died-ioerror-closed-stream/44009)
**Author:** [@Alexey\_Khudyakov](https://discuss.elastic.co/u/Alexey_Khudyakov)\ **Replies:** 2\ **Last updated:** [December 21, 2016, 10:27am UTC](https://discuss.elastic.co/t/udp-listener-died-ioerror-closed-stream/44009 "2016-12-21T10:27:50Z")
Dear community, I'm evaluating logstash and stuck on this problem with a UDP Input. The part of config input { udp { port =\> 9995 type =\> "netflow" } } Logstash version is 2.2.2 OS is Centos Linux 7, … --- ## [Logstash stops often](https://discuss.elastic.co/t/logstash-stops-often/69135)
**Author:** [@marcmga](https://discuss.elastic.co/u/marcmga)\ **Replies:** 2\ **Last updated:** [December 21, 2016, 8:57am UTC](https://discuss.elastic.co/t/logstash-stops-often/69135 "2016-12-21T08:57:03Z")
Hi, Since a long time ago my logstash is stopping. The log shows this error: {:timestamp=\>"2016-12-14T23:12:28.910000+0100", :message=\>"Exception in filterworker, the pipeline stopped processing new events, please che… --- ## [How to use logstash filter](https://discuss.elastic.co/t/how-to-use-logstash-filter/68671)
**Author:** [@JohnnyS](https://discuss.elastic.co/u/JohnnyS)\ **Replies:** 9\ **Last updated:** [December 21, 2016, 8:41am UTC](https://discuss.elastic.co/t/how-to-use-logstash-filter/68671 "2016-12-21T08:41:02Z")
Hi As I'm new to Logstash I just wanted to ask you some filter questions, probably you guys will be able to help me on this and get me starting... These are the logs I'm sending to Logstash (example): "message" … --- ## [How to drop a message](https://discuss.elastic.co/t/how-to-drop-a-message/69548)
**Author:** [@houfan](https://discuss.elastic.co/u/houfan)\ **Replies:** 8\ **Last updated:** [December 21, 2016, 8:31am UTC](https://discuss.elastic.co/t/how-to-drop-a-message/69548 "2016-12-21T08:31:34Z")
the logstash give me three message: "message" =\> "# administrator command: Prepare;" "message" =\> "# User@Host: gcenter\[gcenter\] @ \[192.168.168.100\] Id: 4882345700\\n# Schema: gcenter Last\_errno: 0 Killed: 0\\n# Q… --- ## [Logs from ES 2.2.0 to ES 5.0.2 with Logstash with date of indexing?](https://discuss.elastic.co/t/logs-from-es-2-2-0-to-es-5-0-2-with-logstash-with-date-of-indexing/69407)
**Author:** [@lifesaver13](https://discuss.elastic.co/u/lifesaver13)\ **Replies:** 2\ **Last updated:** [December 21, 2016, 8:23am UTC](https://discuss.elastic.co/t/logs-from-es-2-2-0-to-es-5-0-2-with-logstash-with-date-of-indexing/69407 "2016-12-21T08:23:12Z")
Hello everyone, I'm moving old logs of my ElasticSearch 2.2.0 to an higher version of ElasticSearch 5.0.2 using Logstash. I write the following configuration : input { elasticsearch { hosts =\> \["localhost:9200"… --- ## [Could not ingest data after installing xpack](https://discuss.elastic.co/t/could-not-ingest-data-after-installing-xpack/69638)
**Author:** [@ahsan](https://discuss.elastic.co/u/ahsan)\ **Replies:** 2\ **Last updated:** [December 21, 2016, 6:51am UTC](https://discuss.elastic.co/t/could-not-ingest-data-after-installing-xpack/69638 "2016-12-21T06:51:36Z")
\[Ruby-0-Thread-6: D:/kibanaAcc/install/five1/logstash-5.1.1/logstas h-5.1.1/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-5.4.0-java/li b/logstash/outputs/elasticsearch/http\_client/pool.rb:188\] WARN logst… --- ## [Https://www.elastic.co/guide/en/logstash/current/plugins-codecs-multiline.html](https://discuss.elastic.co/t/https-www-elastic-co-guide-en-logstash-current-plugins-codecs-multiline-html/69382)
**Author:** [@reubenpuketapu](https://discuss.elastic.co/u/reubenpuketapu)\ **Replies:** 3\ **Last updated:** [December 21, 2016, 6:38am UTC](https://discuss.elastic.co/t/https-www-elastic-co-guide-en-logstash-current-plugins-codecs-multiline-html/69382 "2016-12-21T06:38:02Z")
filter { multiline { type =\> "somefiletype" pattern =\> "\\\\$" what =\> "next" } } There is no explanation around what the "type" should be?? Just an update to the documentation. Is it possible to match on … --- ## [Logstash snmptrap chinese incorrect codes](https://discuss.elastic.co/t/logstash-snmptrap-chinese-incorrect-codes/69509)
**Author:** [@laywin](https://discuss.elastic.co/u/laywin)\ **Replies:** 3\ **Last updated:** [December 21, 2016, 6:24am UTC](https://discuss.elastic.co/t/logstash-snmptrap-chinese-incorrect-codes/69509 "2016-12-21T06:24:22Z")
snmaptrap test: logstash config: test result: I have tested many charsets,but have the same result! thanks in advance! --- ## [Strang things happened when logstash grok the log message](https://discuss.elastic.co/t/strang-things-happened-when-logstash-grok-the-log-message/69529)
**Author:** [@houfan](https://discuss.elastic.co/u/houfan)\ **Replies:** 6\ **Last updated:** [December 21, 2016, 1:57am UTC](https://discuss.elastic.co/t/strang-things-happened-when-logstash-grok-the-log-message/69529 "2016-12-21T01:57:07Z")
I use filebeat-1.2.2 to monitor the mysql-slow.log. when I add two slow query records in it, and the logstash only give me one : :disappointed\_relieved: I do it again,adding two more records into the mysql-slow.… --- ## [How to synchronize with oracle when data updated?](https://discuss.elastic.co/t/how-to-synchronize-with-oracle-when-data-updated/69381)
**Author:** [@Tao\_Zhang](https://discuss.elastic.co/u/Tao_Zhang)\ **Replies:** 5\ **Last updated:** [December 21, 2016, 12:48am UTC](https://discuss.elastic.co/t/how-to-synchronize-with-oracle-when-data-updated/69381 "2016-12-21T00:48:05Z")
I'm trying to synchronize oracle to Elasticsearch by logstash-input-jdbc .But I failed. Could you tell me how to do it? Env: windows 7 64bit (※Japanese) oracleDB 11g Elasticsearch 5.1.1 Logstash 5.1.1 Ex: synte… --- ## [\_grokparsefailure tag errors parsing IIS logs using FileBeat](https://discuss.elastic.co/t/-grokparsefailure-tag-errors-parsing-iis-logs-using-filebeat/69506)
**Author:** [@hp-adl](https://discuss.elastic.co/u/hp-adl)\ **Replies:** 2\ **Last updated:** [December 20, 2016, 10:27pm UTC](https://discuss.elastic.co/t/-grokparsefailure-tag-errors-parsing-iis-logs-using-filebeat/69506 "2016-12-20T22:27:17Z")
Hello, I've been struggling for 2 solid days on this and I just can't work out what keeps causing these \_grokparsefailure errors. I'm using the following services and versions for my ELK stack. FileBeat 5.1.1 (Windows… --- ## [Logstash 5 unexpected error occurred queue must be a writable](https://discuss.elastic.co/t/logstash-5-unexpected-error-occurred-queue-must-be-a-writable/68926)
**Author:** [@namobuddhaonion](https://discuss.elastic.co/u/namobuddhaonion)\ **Replies:** 3\ **Last updated:** [December 20, 2016, 9:17pm UTC](https://discuss.elastic.co/t/logstash-5-unexpected-error-occurred-queue-must-be-a-writable/68926 "2016-12-20T21:17:52Z")
I installed Logstash using the following guide (I also have Elasticseach and Kibana 5) on an Ubuntu 16.0x base. https://www.elastic.co/guide/en/logstash/current/getting-started-with-logstash.html Logstash appears to wo… --- ## [Logstash update broke redis](https://discuss.elastic.co/t/logstash-update-broke-redis/69289)
**Author:** [@xon](https://discuss.elastic.co/u/xon)\ **Replies:** 1\ **Last updated:** [December 20, 2016, 9:11pm UTC](https://discuss.elastic.co/t/logstash-update-broke-redis/69289 "2016-12-20T21:11:38Z")
Hi, I updated to Logstash version 5.1.1 and my previous configuration file doesnt work. The error message is: \[ERROR\]\[logstash.agent \] fetched an invalid config {:config=\>"input {\\n beats {\\n port =\> 50… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1060) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1062)