# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=900 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 901 --- ## ["lexicographical order and then parsed as a single config file"](https://discuss.elastic.co/t/lexicographical-order-and-then-parsed-as-a-single-config-file/102393)
**Author:** [@antwanvanderstap](https://discuss.elastic.co/u/antwanvanderstap)\ **Replies:** 1\ **Last updated:** [October 2, 2017, 2:01am UTC](https://discuss.elastic.co/t/lexicographical-order-and-then-parsed-as-a-single-config-file/102393 "2017-10-02T02:01:45Z")
Anyone out there that can point me to the config file so I can look at it. I have a few .conf files in my /etc/logstash/conf.d/ directory and it is my understanding that running 'sudo ./bin/logstash --path.config /etc/l… --- ## [Rabbotmq to Elasticsearch](https://discuss.elastic.co/t/rabbotmq-to-elasticsearch/102339)
**Author:** [@ericd](https://discuss.elastic.co/u/ericd)\ **Replies:** 1\ **Last updated:** [October 1, 2017, 9:58pm UTC](https://discuss.elastic.co/t/rabbotmq-to-elasticsearch/102339 "2017-10-01T21:58:27Z")
I have a problem with transporting filtered logs to Elasticsearch. This is an overview: syslog device -\> syslog-ng -\> rabbitmq -\> logstash input (no filters) -\> logstash elasticsearch output Elastic stack is v5.6 In … --- ## [Error trying to run logstash as service on Ubuntu 14](https://discuss.elastic.co/t/error-trying-to-run-logstash-as-service-on-ubuntu-14/102317)
**Author:** [@brw314](https://discuss.elastic.co/u/brw314)\ **Replies:** 1\ **Last updated:** [October 1, 2017, 9:32pm UTC](https://discuss.elastic.co/t/error-trying-to-run-logstash-as-service-on-ubuntu-14/102317 "2017-10-01T21:32:51Z")
Hi, I installed logstash on Ubuntu 14; sudo apt-cache policy logstash logstash: Installed: 1:5.6.2-1 However, I have be following the tutorial by elastic https://www.elastic.co/guide/en/logstash/current/first-event… --- ## [Logstash with sql server configuration](https://discuss.elastic.co/t/logstash-with-sql-server-configuration/102343)
**Author:** [@bharathpbk](https://discuss.elastic.co/u/bharathpbk)\ **Replies:** 1\ **Last updated:** [October 1, 2017, 9:29pm UTC](https://discuss.elastic.co/t/logstash-with-sql-server-configuration/102343 "2017-10-01T21:29:20Z")
input { jdbc { jdbc\_driver\_library =\> "D:\\Data\\sqljdbc\_6.0\\enu\\jre8\\sqljdbc42.jar" jdbc\_driver\_class =\> "com.microsoft.sqlserver.jdbc.SQLServerDriver" jdbc\_connection\_string =\> "jdbc:sqlserver://xxxxxxx:1433;database… --- ## [POST form data with Logstash http\_poller input plug-in](https://discuss.elastic.co/t/post-form-data-with-logstash-http-poller-input-plug-in/102362)
**Author:** [@Invigor](https://discuss.elastic.co/u/Invigor)\ **Replies:** 1\ **Last updated:** [October 1, 2017, 3:03am UTC](https://discuss.elastic.co/t/post-form-data-with-logstash-http-poller-input-plug-in/102362 "2017-10-01T03:03:43Z")
Hi, I'm trying to use the http\_poller input plug-in to recreate the curl statement below. curl --user xps\_client: --basic --data "username=admin&password=admin&scope=XPS\_API\_ALL&grant\_type=password" http://xps.invigori… --- ## [Logstash Issue with netflow modul (5.6)](https://discuss.elastic.co/t/logstash-issue-with-netflow-modul-5-6/100554)
**Author:** [@lumi](https://discuss.elastic.co/u/lumi)\ **Replies:** 10\ **Last updated:** [October 1, 2017, 12:53am UTC](https://discuss.elastic.co/t/logstash-issue-with-netflow-modul-5-6/100554 "2017-10-01T00:53:33Z")
Hello I installed Logstash 5.6 and X-pack. Now i try to to configure it as it is described in the link: I configured it in /etc/logstash/logstash.yml modules: - name: netflow #var.elasticsearch.hosts: "ht… --- ## [5.6.1 and 5.6.2 installing issues](https://discuss.elastic.co/t/5-6-1-and-5-6-2-installing-issues/102321)
**Author:** [@bdillon1](https://discuss.elastic.co/u/bdillon1)\ **Replies:** 2\ **Last updated:** [September 30, 2017, 3:59pm UTC](https://discuss.elastic.co/t/5-6-1-and-5-6-2-installing-issues/102321 "2017-09-30T15:59:40Z")
Hello, I am making this post after being let down by every other solution I could find online. I did try first. When trying to run my .conf file, I am told Logstash cannot locate jruby, yet the jruby folder is there. I… --- ## [Logstash - convert %{\[@metadata\] variables before sending output to ES?](https://discuss.elastic.co/t/logstash-convert-metadata-variables-before-sending-output-to-es/102348)
**Author:** [@safehouse](https://discuss.elastic.co/u/safehouse)\ **Replies:** 0\ **Last updated:** [September 30, 2017, 2:12pm UTC](https://discuss.elastic.co/t/logstash-convert-metadata-variables-before-sending-output-to-es/102348 "2017-09-30T14:12:53Z")
Hey All, I recently started using metricbeat on some remote servers and collecting the data on my central ES cluster. I'm sending the metricbeat data on each server using the logstash output to the remote ES server over… --- ## [Logstash - selectively applying filter](https://discuss.elastic.co/t/logstash-selectively-applying-filter/102226)
**Author:** [@rs\_ela](https://discuss.elastic.co/u/rs_ela)\ **Replies:** 6\ **Last updated:** [September 30, 2017, 1:19pm UTC](https://discuss.elastic.co/t/logstash-selectively-applying-filter/102226 "2017-09-30T13:19:29Z")
Hi there I am sending logs from my nginx instance to ELK via Filebeat. The problem is that nginx access logs are json, but error logs are not and Logstash reports a lot of parsing errors which then swamp the log file on… --- ## [Auto\_flush\_interval in multiline codec plugin errors out](https://discuss.elastic.co/t/auto-flush-interval-in-multiline-codec-plugin-errors-out/99345)
**Author:** [@amruth](https://discuss.elastic.co/u/amruth)\ **Replies:** 7\ **Last updated:** [September 29, 2017, 8:23pm UTC](https://discuss.elastic.co/t/auto-flush-interval-in-multiline-codec-plugin-errors-out/99345 "2017-09-29T20:23:46Z")
Hi, I am using auto\_flush\_interval in multiline codec plugin. It works fine for all the events but when it reaches the end of file, it is throwing grokparsefailure error. And when I remove auto\_flush\_interval, there are… --- ## [Duplicate documents after Logstash restart](https://discuss.elastic.co/t/duplicate-documents-after-logstash-restart/102175)
**Author:** [@Absinthe](https://discuss.elastic.co/u/Absinthe)\ **Replies:** 0\ **Last updated:** [September 28, 2017, 7:44pm UTC](https://discuss.elastic.co/t/duplicate-documents-after-logstash-restart/102175 "2017-09-28T19:44:12Z")
Using Windows7 Using Logstash 5.4.1 input: { azurewadtable { add\_field =\> { "input\_origin" =\> "MyServerLogs" } storage\_account\_name =\> "MyDeploy" storage\_sas\_token =\> "xxxxxxxxxx" table\_name =\> "WadLogsTable" co… --- ## [Missing fields Filebeat -\> Logstash -\> Elasticsearch](https://discuss.elastic.co/t/missing-fields-filebeat-logstash-elasticsearch/101747)
**Author:** [@Matt\_Shields](https://discuss.elastic.co/u/Matt_Shields)\ **Replies:** 2\ **Last updated:** [September 29, 2017, 2:37pm UTC](https://discuss.elastic.co/t/missing-fields-filebeat-logstash-elasticsearch/101747 "2017-09-29T14:37:41Z")
I'm using Filebeat on Windows to send some custom logs to a custom index on an Elasticsearch server, but I'm sending them through Logstash so I can apply a custom filter. Previously I tried sending them directly from Fi… --- ## [How to add new file type called ForwardedEvents in logstash eventlog input plugin?](https://discuss.elastic.co/t/how-to-add-new-file-type-called-forwardedevents-in-logstash-eventlog-input-plugin/102264)
**Author:** [@sam281](https://discuss.elastic.co/u/sam281)\ **Replies:** 0\ **Last updated:** [September 29, 2017, 1:34pm UTC](https://discuss.elastic.co/t/how-to-add-new-file-type-called-forwardedevents-in-logstash-eventlog-input-plugin/102264 "2017-09-29T13:34:40Z")
Hi, Is there anyway we can use ForwardedEvents in logstash with eventlog input plugin? Windows Os Event log are in encrypted format, because of which the file input is not working i believe, I tried something like belo… --- ## [JSON File Input](https://discuss.elastic.co/t/json-file-input/100916)
**Author:** [@Pratiksha\_Tewary](https://discuss.elastic.co/u/Pratiksha_Tewary)\ **Replies:** 6\ **Last updated:** [September 29, 2017, 1:33pm UTC](https://discuss.elastic.co/t/json-file-input/100916 "2017-09-29T13:33:43Z")
Hi, I am just trying to take in a JSON file and spit it out to the stdout. The pipeline starts and nothing happens after that. There are no error messages and no other movements. I have the following in my configuration… --- ## [Logstash conditional in output](https://discuss.elastic.co/t/logstash-conditional-in-output/101897)
**Author:** [@Pratiksha\_Tewary](https://discuss.elastic.co/u/Pratiksha_Tewary)\ **Replies:** 5\ **Last updated:** [September 29, 2017, 1:29pm UTC](https://discuss.elastic.co/t/logstash-conditional-in-output/101897 "2017-09-29T13:29:34Z")
Hi, I wanted to know if it is possible to carry out an action based on a conditional in the output section of my logstash configuration file. I am parsing a file into CSV format and want to add a header. I do this by ad… --- ## [Elasticsearch filter plugin - how to query data](https://discuss.elastic.co/t/elasticsearch-filter-plugin-how-to-query-data/102258)
**Author:** [@Cristian](https://discuss.elastic.co/u/Cristian)\ **Replies:** 0\ **Last updated:** [September 29, 2017, 12:51pm UTC](https://discuss.elastic.co/t/elasticsearch-filter-plugin-how-to-query-data/102258 "2017-09-29T12:51:23Z")
Hi All, I ingest forex data into an index called forex1. The data received looks like this per minute. I do a split on candles: { "@timestamp" =\> 2017-09-29T12:25:00.434Z, "granularity" =\> "M1", "@versi… --- ## [Lostash becoming unresponsive (?)](https://discuss.elastic.co/t/lostash-becoming-unresponsive/102242)
**Author:** [@pkaramol](https://discuss.elastic.co/u/pkaramol)\ **Replies:** 9\ **Last updated:** [September 29, 2017, 12:05pm UTC](https://discuss.elastic.co/t/lostash-becoming-unresponsive/102242 "2017-09-29T12:05:13Z")
I am trying to start logstash via sudo systemctl start logstash It seems to succeed (service status now is active), however nothing gets written in logstash's own logs, under /var/log/logstash/logstash-plain.log Until… --- ## [Java-Error after installing a new Logstash-server](https://discuss.elastic.co/t/java-error-after-installing-a-new-logstash-server/102253)
**Author:** [@JR\_FFM](https://discuss.elastic.co/u/JR_FFM)\ **Replies:** 0\ **Last updated:** [September 29, 2017, 11:42am UTC](https://discuss.elastic.co/t/java-error-after-installing-a-new-logstash-server/102253 "2017-09-29T11:42:42Z")
Hello, today I've installed a new debian-server with logstash 5.6.2 out of the elasticsearch-apt-source. When I start logstash with debugging-logging, I get following error-message in /var/log/logstash/logstash-plain.l… --- ## [Netflow Plugin does not decode all data-templates](https://discuss.elastic.co/t/netflow-plugin-does-not-decode-all-data-templates/102245)
**Author:** [@HenryTheSir](https://discuss.elastic.co/u/HenryTheSir)\ **Replies:** 0\ **Last updated:** [September 29, 2017, 10:37am UTC](https://discuss.elastic.co/t/netflow-plugin-does-not-decode-all-data-templates/102245 "2017-09-29T10:37:57Z")
Hi, Logstash version 5.6 The data-templates send from netscaler are not all recognized by netflow plugin. Some data templates are used, some not. Templates IDs: 0|264: 0|265: 0|266: 0|267: 0|269: 0|270: 0|272: … --- ## [Ingest Kubernetes kube\_state\_metrics data](https://discuss.elastic.co/t/ingest-kubernetes-kube-state-metrics-data/102247)
**Author:** [@ronidaho](https://discuss.elastic.co/u/ronidaho)\ **Replies:** 0\ **Last updated:** [September 29, 2017, 10:54am UTC](https://discuss.elastic.co/t/ingest-kubernetes-kube-state-metrics-data/102247 "2017-09-29T10:54:59Z")
I have a Kubernetes cluster that includes an instance of kube\_state\_metrics for reporting cluster information. I will use httpbeat to query the metrics endpoint and push data to Logstash. The text format for data from ku… --- ## [Auditing Windows file acccess](https://discuss.elastic.co/t/auditing-windows-file-acccess/101438)
**Author:** [@admlko](https://discuss.elastic.co/u/admlko)\ **Replies:** 1\ **Last updated:** [September 29, 2017, 10:47am UTC](https://discuss.elastic.co/t/auditing-windows-file-acccess/101438 "2017-09-29T10:47:55Z")
Hi, This didn't really fit into any of the categories, so I'll post it here. I'm trying to audit windows file share accesses, but it seems to be quite hard, at least for me. I have found numerous resources which indic… --- ## [Use logstash to "read" from index when ingesting new data to calculate a field](https://discuss.elastic.co/t/use-logstash-to-read-from-index-when-ingesting-new-data-to-calculate-a-field/100693)
**Author:** [@Cristian](https://discuss.elastic.co/u/Cristian)\ **Replies:** 2\ **Last updated:** [September 29, 2017, 8:47am UTC](https://discuss.elastic.co/t/use-logstash-to-read-from-index-when-ingesting-new-data-to-calculate-a-field/100693 "2017-09-29T08:47:19Z")
Hi All, Sorry for the bad heading. I have a question. Is it possible to read from an index when ingesting data to calculate a field? I have some daily data that I ingest into an index. Now I want to use one of the fiel… --- ## [Nested Json to parse](https://discuss.elastic.co/t/nested-json-to-parse/101494)
**Author:** [@gius78](https://discuss.elastic.co/u/gius78)\ **Replies:** 8\ **Last updated:** [September 29, 2017, 7:10am UTC](https://discuss.elastic.co/t/nested-json-to-parse/101494 "2017-09-29T07:10:36Z")
Hi all again, I know this topic has been discussed in tha past, but I am a newbie in this world and could not be able to apply to my case the solutions provided in the old discussions. I have to parse a log like this: … --- ## [Logstash-5.6.2-1 logstash.service failed to load: No such file or directory](https://discuss.elastic.co/t/logstash-5-6-2-1-logstash-service-failed-to-load-no-such-file-or-directory/102172)
**Author:** [@scch](https://discuss.elastic.co/u/scch)\ **Replies:** 2\ **Last updated:** [September 29, 2017, 6:40am UTC](https://discuss.elastic.co/t/logstash-5-6-2-1-logstash-service-failed-to-load-no-such-file-or-directory/102172 "2017-09-29T06:40:43Z")
Hi , I have installed Logstash-5.6.2-1 through yum repo.. on centOS 7. it installed successfully. however I am unable to start the service. sudo systemctl start logstash. it gives following error. Failed to start log… --- ## [Indexing logs in json format](https://discuss.elastic.co/t/indexing-logs-in-json-format/102072)
**Author:** [@daaakeee](https://discuss.elastic.co/u/daaakeee)\ **Replies:** 2\ **Last updated:** [September 29, 2017, 3:40am UTC](https://discuss.elastic.co/t/indexing-logs-in-json-format/102072 "2017-09-29T03:40:15Z")
Hi guys! I am trying to index log in json format: {"interests": "dota", "car": null, "surname": "Zhussipnazar", "name": "Daulet", "age": 20} But it gives me error, \_jsonparsefailure , because logstash cant handle val… --- ## [How To Output to Amazon Elasticsearch Service](https://discuss.elastic.co/t/how-to-output-to-amazon-elasticsearch-service/101247)
**Author:** [@mdanner](https://discuss.elastic.co/u/mdanner)\ **Replies:** 5\ **Last updated:** [September 28, 2017, 8:06pm UTC](https://discuss.elastic.co/t/how-to-output-to-amazon-elasticsearch-service/101247 "2017-09-28T20:06:13Z")
Hello, I'm setting up my first ELK stack on AWS. I installed Logstash 5.6.0 on an EC2 instance running Ubuntu 16.04.3 LTS. It was installed using the instructions for Debian/RPM. The input side of things is running fine… --- ## [HTTP\_Poller send output to elasticsearch](https://discuss.elastic.co/t/http-poller-send-output-to-elasticsearch/100674)
**Author:** [@Sharivari](https://discuss.elastic.co/u/Sharivari)\ **Replies:** 6\ **Last updated:** [September 28, 2017, 5:38pm UTC](https://discuss.elastic.co/t/http-poller-send-output-to-elasticsearch/100674 "2017-09-28T17:38:45Z")
Hey everyone, for testcases I am requesting some urls and want to send the result to my elasticsearch. this is my config file: input { http\_poller { urls =\> { "service1" =\> "http://192.168.99.100:9200/servi… --- ## [Logstash HTTP\_Poller input plugin not sending output to Elasticsearch](https://discuss.elastic.co/t/logstash-http-poller-input-plugin-not-sending-output-to-elasticsearch/101193)
**Author:** [@VIJAYP](https://discuss.elastic.co/u/VIJAYP)\ **Replies:** 3\ **Last updated:** [September 28, 2017, 5:37pm UTC](https://discuss.elastic.co/t/logstash-http-poller-input-plugin-not-sending-output-to-elasticsearch/101193 "2017-09-28T17:37:53Z")
Hello, I'm trying to log IBM Liberty server's metrics to Elasticsearch for dashboard purposes. Here's my logstash config file: input { http\_poller { urls =\> { "service1" =\> "https://dc2-svzsys01.xxxx.com:11… --- ## [Terminal command to download log files through ubuntu from remote end not working](https://discuss.elastic.co/t/terminal-command-to-download-log-files-through-ubuntu-from-remote-end-not-working/102150)
**Author:** [@nicedee](https://discuss.elastic.co/u/nicedee)\ **Replies:** 0\ **Last updated:** [September 28, 2017, 5:02pm UTC](https://discuss.elastic.co/t/terminal-command-to-download-log-files-through-ubuntu-from-remote-end-not-working/102150 "2017-09-28T17:02:12Z")
i am not able to download the log files from remote end to local. first approach: i tried to use the reversed syntax order of uploading a file onto remote end with my private key got from amazon EC2. the uploading ter… --- ## [Logstash Elastic Search JDBC very slow](https://discuss.elastic.co/t/logstash-elastic-search-jdbc-very-slow/102145)
**Author:** [@Wu\_Chun\_Wa](https://discuss.elastic.co/u/Wu_Chun_Wa)\ **Replies:** 0\ **Last updated:** [September 28, 2017, 4:01pm UTC](https://discuss.elastic.co/t/logstash-elastic-search-jdbc-very-slow/102145 "2017-09-28T16:01:06Z")
hi, my dataset limited to 400k. And, i run the following setting. around 20 seconds to import 10k records. How can i optimize the config? Version docker-elastic search: 5.4.1 docker-logstash: 5.4.1 Config input { jdb… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=899) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=901)