# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=892 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 893 --- ## [File Input Plugin Efficiency](https://discuss.elastic.co/t/file-input-plugin-efficiency/103371)
**Author:** [@CDR](https://discuss.elastic.co/u/CDR)\ **Replies:** 11\ **Last updated:** [October 13, 2017, 4:30pm UTC](https://discuss.elastic.co/t/file-input-plugin-efficiency/103371 "2017-10-13T16:30:32Z")
In my logstash.conf file I am utilizing the file input plugin. file{ path =\> "${PWD}/data/\*\*/admin\_ui\*.csv" start\_position =\> beginning ignore\_older =\> 0 sincedb\_path =\> "${PWD}/software\_files/logstash-5.6.1/logsta… --- ## [Logstash-plugin Installation Error](https://discuss.elastic.co/t/logstash-plugin-installation-error/103928)
**Author:** [@Terry\_Yeo](https://discuss.elastic.co/u/Terry_Yeo)\ **Replies:** 0\ **Last updated:** [October 13, 2017, 4:10pm UTC](https://discuss.elastic.co/t/logstash-plugin-installation-error/103928 "2017-10-13T16:10:36Z")
I tried to install the google pubsub input and elasticsearch output plugins after I install the logstash, however, it raised out the following error: Error Bundler::InstallError, retrying 1/10 An error occurred while in… --- ## [Event not displayed in Kibana, displayed in stdout](https://discuss.elastic.co/t/event-not-displayed-in-kibana-displayed-in-stdout/103921)
**Author:** [@zuokuok](https://discuss.elastic.co/u/zuokuok)\ **Replies:** 0\ **Last updated:** [October 13, 2017, 3:17pm UTC](https://discuss.elastic.co/t/event-not-displayed-in-kibana-displayed-in-stdout/103921 "2017-10-13T15:17:29Z")
I have seen this (only) thread about the same issue as me, but doesn't work for me. Could someone say what are the things to watch out for in the data? In which file should I look to find what was rejected? Are there… --- ## [Logstash not indexed to Elasticsearch](https://discuss.elastic.co/t/logstash-not-indexed-to-elasticsearch/103909)
**Author:** [@johntadd](https://discuss.elastic.co/u/johntadd)\ **Replies:** 0\ **Last updated:** [October 13, 2017, 1:53pm UTC](https://discuss.elastic.co/t/logstash-not-indexed-to-elasticsearch/103909 "2017-10-13T13:53:18Z")
Searching the forum, but I don't find any threads that actually fixes my problem. So posting the question here. The main problem is I don't find data processed by logstash gets indexed to elasticsearch. curl 'localhost… --- ## [Updating an index with logstash and elastic output 7.4.0 plugin fails, likely bug?](https://discuss.elastic.co/t/updating-an-index-with-logstash-and-elastic-output-7-4-0-plugin-fails-likely-bug/103894)
**Author:** [@falco\_rocks](https://discuss.elastic.co/u/falco_rocks)\ **Replies:** 0\ **Last updated:** [October 13, 2017, 12:29pm UTC](https://discuss.elastic.co/t/updating-an-index-with-logstash-and-elastic-output-7-4-0-plugin-fails-likely-bug/103894 "2017-10-13T12:29:51Z")
I am practicing with Logstash and I can't understand the source of this error. I want to modify an index by removing a field and renaming one. I'm on Logstash 5.6.2 Elastic output plugin is 7.4.0 This is an example of … --- ## [How to specify proxy\_url on Logstash with x-pack](https://discuss.elastic.co/t/how-to-specify-proxy-url-on-logstash-with-x-pack/103887)
**Author:** [@Giancarlo\_Rubio](https://discuss.elastic.co/u/Giancarlo_Rubio)\ **Replies:** 1\ **Last updated:** [October 13, 2017, 11:59am UTC](https://discuss.elastic.co/t/how-to-specify-proxy-url-on-logstash-with-x-pack/103887 "2017-10-13T11:59:38Z")
My Logstash is running behind a proxy server with all the logs successfully delivered to Elasticsearch Cloud. Now I'm trying to enable x-pack on logstash and looks like xpack is not following my proxy. Besides that I ca… --- ## [Survey: modules, data sources and ingestion experience](https://discuss.elastic.co/t/survey-modules-data-sources-and-ingestion-experience/100951)
**Author:** [@suyograo](https://discuss.elastic.co/u/suyograo)\ **Replies:** 1\ **Last updated:** [October 13, 2017, 10:53am UTC](https://discuss.elastic.co/t/survey-modules-data-sources-and-ingestion-experience/100951 "2017-10-13T10:53:04Z")
Modules (in Beats & Logstash) are built on the premise that simple things should be simple. They let you go from raw data to dashboards for common data sources in literally minutes. Run 1 command and instantly start expl… --- ## [Logstash v5.6.3 won't start if debug logging is on](https://discuss.elastic.co/t/logstash-v5-6-3-wont-start-if-debug-logging-is-on/103817)
**Author:** [@sgladkovich](https://discuss.elastic.co/u/sgladkovich)\ **Replies:** 2\ **Last updated:** [October 13, 2017, 8:12am UTC](https://discuss.elastic.co/t/logstash-v5-6-3-wont-start-if-debug-logging-is-on/103817 "2017-10-13T08:12:55Z")
Hello, I am running Logstash on Ubuntu 14.04 in AWS. Here are my config details: $ grep -E -v '^#|^$' /etc/logstash/logstash.yml node.name: logstash path.data: /var/lib/logstash path.config: /etc/logstash/conf.d confi… --- ## [Can't write data into graphite from database query](https://discuss.elastic.co/t/cant-write-data-into-graphite-from-database-query/103711)
**Author:** [@olivier974](https://discuss.elastic.co/u/olivier974)\ **Replies:** 1\ **Last updated:** [October 13, 2017, 7:55am UTC](https://discuss.elastic.co/t/cant-write-data-into-graphite-from-database-query/103711 "2017-10-13T07:55:19Z")
hello, I'm trying to extract some data from an oracle db containing KPIs to our graphing tool which is based on graphite. I'm can easily get the data from the DB with the logstash-input-jdbc plugin { "instance" =\> "1… --- ## [Logstash input jdbc not working](https://discuss.elastic.co/t/logstash-input-jdbc-not-working/103707)
**Author:** [@Navinder](https://discuss.elastic.co/u/Navinder)\ **Replies:** 5\ **Last updated:** [October 13, 2017, 5:47am UTC](https://discuss.elastic.co/t/logstash-input-jdbc-not-working/103707 "2017-10-13T05:47:37Z")
Hi, I am using logstash jdbc input, but it is throwing this error below. The given configuration is invalid. Reason: Expected one of #, ", ', } at line 2, column 9 (byte 18) after input { jdbc { {:level=\>:fatal} Here… --- ## [Logstash A to Logstash B](https://discuss.elastic.co/t/logstash-a-to-logstash-b/103477)
**Author:** [@Sjaak01](https://discuss.elastic.co/u/Sjaak01)\ **Replies:** 3\ **Last updated:** [October 13, 2017, 5:14am UTC](https://discuss.elastic.co/t/logstash-a-to-logstash-b/103477 "2017-10-13T05:14:21Z")
Hi, How can I ship data collected by Logstash A (client) to Logstash B (server)? I want to collect and process some data locally on a remote client before sending it to the server for further processing. I'm testing u… --- ## [Logstash netflow cisco router and wlc](https://discuss.elastic.co/t/logstash-netflow-cisco-router-and-wlc/103819)
**Author:** [@VamPikmin](https://discuss.elastic.co/u/VamPikmin)\ **Replies:** 0\ **Last updated:** [October 13, 2017, 4:24am UTC](https://discuss.elastic.co/t/logstash-netflow-cisco-router-and-wlc/103819 "2017-10-13T04:24:49Z")
Hi I have been ingesting netflow v9 data from a cisco router without major problems so thought I'd do the same with the wireless controller but I have come across some issues. I added another listener on a different po… --- ## [Logstash error netflow plugin](https://discuss.elastic.co/t/logstash-error-netflow-plugin/103809)
**Author:** [@bendersky](https://discuss.elastic.co/u/bendersky)\ **Replies:** 0\ **Last updated:** [October 13, 2017, 2:41am UTC](https://discuss.elastic.co/t/logstash-error-netflow-plugin/103809 "2017-10-13T02:41:37Z")
Hello Everyone, Netflow plugin from RPM version 5.6.3-1 give error on incoming connection Hardware: ASA5525, 8192 MB RAM, CPU Lynnfield 2394 MHz, 1 CPU (4 cores) Cisco Adaptive Security Appliance Software Version 9.… --- ## [Logstash is not creating new index in elastic cloud](https://discuss.elastic.co/t/logstash-is-not-creating-new-index-in-elastic-cloud/103774)
**Author:** [@sbade](https://discuss.elastic.co/u/sbade)\ **Replies:** 18\ **Last updated:** [October 12, 2017, 10:28pm UTC](https://discuss.elastic.co/t/logstash-is-not-creating-new-index-in-elastic-cloud/103774 "2017-10-12T22:28:15Z")
I' ve got a pretty simple configuration to setup. what I need to do is create a new index for each of my servers in my elastic cloud setup. my output is elasticsearch { hosts =\> "https://XXXXXXXXX.us-east-1.aws.found… --- ## [Explanation/Description of metrics reported by LS monitoring API](https://discuss.elastic.co/t/explanation-description-of-metrics-reported-by-ls-monitoring-api/102611)
**Author:** [@rbrandstaedter](https://discuss.elastic.co/u/rbrandstaedter)\ **Replies:** 1\ **Last updated:** [October 12, 2017, 10:09pm UTC](https://discuss.elastic.co/t/explanation-description-of-metrics-reported-by-ls-monitoring-api/102611 "2017-10-12T22:09:55Z")
Is there a documentation what the metrics reported by the monitoring API mean? Some of them are self-explanatory others aren't. e.g. what is queue\_push\_duration\_in\_millis" : 21 duration\_in\_millis Thanks! --- ## [Can I access the \_score for events from an elasticsearch input in logstash?](https://discuss.elastic.co/t/can-i-access-the-score-for-events-from-an-elasticsearch-input-in-logstash/102873)
**Author:** [@Badger](https://discuss.elastic.co/u/Badger)\ **Replies:** 5\ **Last updated:** [October 12, 2017, 9:16pm UTC](https://discuss.elastic.co/t/can-i-access-the-score-for-events-from-an-elasticsearch-input-in-logstash/102873 "2017-10-12T21:16:02Z")
logstash is pulling data out of elasticsearch using the following query query =\> '{ "\_source": false, "query": { "function\_score": { "query": { "query\_string": { "query": "\*" } }, "random\_score": {}… --- ## [Logstash not able to send bulk requests to ES](https://discuss.elastic.co/t/logstash-not-able-to-send-bulk-requests-to-es/103585)
**Author:** [@sudi\_2611](https://discuss.elastic.co/u/sudi_2611)\ **Replies:** 0\ **Last updated:** [October 11, 2017, 5:21pm UTC](https://discuss.elastic.co/t/logstash-not-able-to-send-bulk-requests-to-es/103585 "2017-10-11T17:21:28Z")
Hi All, Please help i have no idea why i am getting the below error in logstash logs Logstash is not able to send data to ES, because of which i am not able to get the entire data. I am using a single node setup logs… --- ## [\[solved\] Host values differ between plugins on the same host](https://discuss.elastic.co/t/solved-host-values-differ-between-plugins-on-the-same-host/103456)
**Author:** [@jratliff](https://discuss.elastic.co/u/jratliff)\ **Replies:** 4\ **Last updated:** [October 12, 2017, 6:38pm UTC](https://discuss.elastic.co/t/solved-host-values-differ-between-plugins-on-the-same-host/103456 "2017-10-12T18:38:05Z")
I'm trying out Elastic Stack. I setup a filebeat to ship my apache access logs and a syslog input on logstash to get the syslog data from the host. The logs end up in the right place, but the syslog shows the host as th… --- ## [Logstash unable to parse multiline](https://discuss.elastic.co/t/logstash-unable-to-parse-multiline/103587)
**Author:** [@dbElastic](https://discuss.elastic.co/u/dbElastic)\ **Replies:** 2\ **Last updated:** [October 12, 2017, 4:23pm UTC](https://discuss.elastic.co/t/logstash-unable-to-parse-multiline/103587 "2017-10-12T16:23:38Z")
I have the conf file as below: input { file { path =\> "E:/Logs/A\_logs.log" start\_position =\> "beginning" codec =\> multiline { pattern =\> "^\\\[%{TIMESTAMP\_ISO8601}\\\] " negate =\> true what =\> "previous" } type =\> … --- ## [Redis input plugin high CPU](https://discuss.elastic.co/t/redis-input-plugin-high-cpu/103752)
**Author:** [@danuuk](https://discuss.elastic.co/u/danuuk)\ **Replies:** 0\ **Last updated:** [October 12, 2017, 2:54pm UTC](https://discuss.elastic.co/t/redis-input-plugin-high-cpu/103752 "2017-10-12T14:54:35Z")
Hello! We faced some issues with our Redis & Logstash setup. syslog\_data --\> |-\>rsyslog-\>redis-\>logstash-\>| --\> elastic' RSyslog & Redis & Logstash -- are installed on one physical server (8x cores,). I try to imp… --- ## [Cannot find protobuf codec](https://discuss.elastic.co/t/cannot-find-protobuf-codec/103626)
**Author:** [@michaelharris](https://discuss.elastic.co/u/michaelharris)\ **Replies:** 1\ **Last updated:** [October 12, 2017, 2:49pm UTC](https://discuss.elastic.co/t/cannot-find-protobuf-codec/103626 "2017-10-12T14:49:18Z")
I am trying to install the protobuf codec for logstash, but the logstash plugin utility can't find it. I need an offline pack as the Linux machine logstash is running on has no access outside our intranet. I am running … --- ## [Grep multiline log](https://discuss.elastic.co/t/grep-multiline-log/103664)
**Author:** [@Vivek\_Nigam](https://discuss.elastic.co/u/Vivek_Nigam)\ **Replies:** 0\ **Last updated:** [October 12, 2017, 7:29am UTC](https://discuss.elastic.co/t/grep-multiline-log/103664 "2017-10-12T07:29:42Z")
i am new to logstash, Its two line log i want to extract following fields in BOLD & exclude other contents & grep date from file name, how this can achieve through grok filter, could you please help me on this, thanks. … --- ## [Dates dates dates](https://discuss.elastic.co/t/dates-dates-dates/103035)
**Author:** [@dorj1234](https://discuss.elastic.co/u/dorj1234)\ **Replies:** 3\ **Last updated:** [October 12, 2017, 1:56pm UTC](https://discuss.elastic.co/t/dates-dates-dates/103035 "2017-10-12T13:56:30Z")
Thanks in advance sorry it's a bit long. Trying not to be frustrated here, but the handling of dates is not behaving as documented. Below is a pretty complicated conf file, able to handle multiple .csv formats I have 2… --- ## [First pipeline issue in logstash config](https://discuss.elastic.co/t/first-pipeline-issue-in-logstash-config/103720)
**Author:** [@prachi1](https://discuss.elastic.co/u/prachi1)\ **Replies:** 1\ **Last updated:** [October 12, 2017, 1:47pm UTC](https://discuss.elastic.co/t/first-pipeline-issue-in-logstash-config/103720 "2017-10-12T13:47:02Z")
After creating basic lostash.conf file and running the command to verify it ..I get the error as follows - logstash.conf input { beats { port =\> "5044" } } The filter part of this file is commented out to indicat… --- ## [How to create multiple events in logstash to push multiple request in elasticsearch](https://discuss.elastic.co/t/how-to-create-multiple-events-in-logstash-to-push-multiple-request-in-elasticsearch/103732)
**Author:** [@ajit](https://discuss.elastic.co/u/ajit)\ **Replies:** 1\ **Last updated:** [October 12, 2017, 1:36pm UTC](https://discuss.elastic.co/t/how-to-create-multiple-events-in-logstash-to-push-multiple-request-in-elasticsearch/103732 "2017-10-12T13:36:30Z")
I have logstash configuration. I want to take data from one event manipulate the data and create multiple events and push that events to elasticsearch. e.g. { "activity" : "meeting", "clientName" : "ABC/XYZ/LMN" "Du… --- ## [Using response http\_poller](https://discuss.elastic.co/t/using-response-http-poller/103668)
**Author:** [@abrejuin](https://discuss.elastic.co/u/abrejuin)\ **Replies:** 1\ **Last updated:** [October 12, 2017, 1:20pm UTC](https://discuss.elastic.co/t/using-response-http-poller/103668 "2017-10-12T13:20:41Z")
Hello, I want to be able to add some information to Elastic from an API using http\_poller. I began with this : input { http\_poller { urls =\> { "test" =\> { method =\> get url =\> "https://dashboard.meraki.com/api/v0… --- ## [Increament the nested Json in date field](https://discuss.elastic.co/t/increament-the-nested-json-in-date-field/103694)
**Author:** [@Vignesh\_Kumar](https://discuss.elastic.co/u/Vignesh_Kumar)\ **Replies:** 4\ **Last updated:** [October 12, 2017, 1:05pm UTC](https://discuss.elastic.co/t/increament-the-nested-json-in-date-field/103694 "2017-10-12T13:05:57Z")
Have the below field in date field which need to be incremented for length of json, match =\> \["\[Capture\]\[0\]\[TimeStamp\]", "yyyyMMddHHmmss"\] In this \[0\] has to increamented. please help --- ## [Ship logs to logstash from local machine using filebeat](https://discuss.elastic.co/t/ship-logs-to-logstash-from-local-machine-using-filebeat/103682)
**Author:** [@prachi1](https://discuss.elastic.co/u/prachi1)\ **Replies:** 3\ **Last updated:** [October 12, 2017, 12:46pm UTC](https://discuss.elastic.co/t/ship-logs-to-logstash-from-local-machine-using-filebeat/103682 "2017-10-12T12:46:36Z")
How to ship logs to logstash from local machine using filebeat on windows. Can I get to know a detailed procedure of how to do this task apart from the documentation. How do I verify the changes and ensure it's working. … --- ## [Error: Expected one of #, ", ', }](https://discuss.elastic.co/t/error-expected-one-of/103504)
**Author:** [@Navinder](https://discuss.elastic.co/u/Navinder)\ **Replies:** 5\ **Last updated:** [October 12, 2017, 11:59am UTC](https://discuss.elastic.co/t/error-expected-one-of/103504 "2017-10-12T11:59:04Z")
Hi all, I am facing a problem with logstash. Here's the config file - input { file { path =\> "/home/app/estest.csv" start\_position =\> "beginning" } } filter { } output { elasticsearch { hosts =\> "http://my… --- ## [Multiple logstash conf](https://discuss.elastic.co/t/multiple-logstash-conf/103693)
**Author:** [@ramesh205](https://discuss.elastic.co/u/ramesh205)\ **Replies:** 3\ **Last updated:** [October 12, 2017, 11:36am UTC](https://discuss.elastic.co/t/multiple-logstash-conf/103693 "2017-10-12T11:36:25Z")
Hi, I created two conf files one to read data from db and store in index(sysjob-index) and another to read data from log file and store in index(jupiterindex3) and i ran both conf file bin\>logstash -f logstash\*.conf. … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=891) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=893)