# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=872 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 873 --- ## [Insert longitude and latitude to logstash config file](https://discuss.elastic.co/t/insert-longitude-and-latitude-to-logstash-config-file/107615)
**Author:** [@ahmedcharef](https://discuss.elastic.co/u/ahmedcharef)\ **Replies:** 21\ **Last updated:** [November 20, 2017, 6:41am UTC](https://discuss.elastic.co/t/insert-longitude-and-latitude-to-logstash-config-file/107615 "2017-11-20T06:41:06Z")
Hello, I have CSV file with two geolocation columns: cell\_easting cell\_northing 26.1541 66.48703 26.161 66.49312 26.166 66.49182 I would like to insert them as a markers into a map Here my config file: \> input … --- ## [Applying joins to the data through logstash](https://discuss.elastic.co/t/applying-joins-to-the-data-through-logstash/108320)
**Author:** [@devil\_srj7](https://discuss.elastic.co/u/devil_srj7)\ **Replies:** 2\ **Last updated:** [November 20, 2017, 6:15am UTC](https://discuss.elastic.co/t/applying-joins-to-the-data-through-logstash/108320 "2017-11-20T06:15:48Z")
I need to apply joins to my documents which are in RDMS while importing them to elasticserch through logstash. --- ## [How to handle erroring logs, that just stick around](https://discuss.elastic.co/t/how-to-handle-erroring-logs-that-just-stick-around/108307)
**Author:** [@cybergoof](https://discuss.elastic.co/u/cybergoof)\ **Replies:** 0\ **Last updated:** [November 20, 2017, 2:44am UTC](https://discuss.elastic.co/t/how-to-handle-erroring-logs-that-just-stick-around/108307 "2017-11-20T02:44:06Z")
We have been running Logstash in production for a while. When I restart Logstash, it seems that logstash is attempting to resend Logs that have errors in them "Could not index event to ElasticSearch", caused by problems… --- ## [How to put logstash in debug logging mode?](https://discuss.elastic.co/t/how-to-put-logstash-in-debug-logging-mode/108303)
**Author:** [@Jose\_Ramierez](https://discuss.elastic.co/u/Jose_Ramierez)\ **Replies:** 1\ **Last updated:** [November 20, 2017, 1:15am UTC](https://discuss.elastic.co/t/how-to-put-logstash-in-debug-logging-mode/108303 "2017-11-20T01:15:01Z")
We are chasing a problem where logstash stops listening, and starts logging this over and over: \[2017-11-17T13:57:44,949\]\[ERROR\]\[logstash.outputs.elasticsearch\] Encountered a retryable error. Will Retry with exponenti… --- ## [Trouble with using LS to filter based on tag](https://discuss.elastic.co/t/trouble-with-using-ls-to-filter-based-on-tag/108241)
**Author:** [@mackwage](https://discuss.elastic.co/u/mackwage)\ **Replies:** 1\ **Last updated:** [November 19, 2017, 7:06pm UTC](https://discuss.elastic.co/t/trouble-with-using-ls-to-filter-based-on-tag/108241 "2017-11-19T19:06:40Z")
Greetings all- I am trying to route different log types to different indices. Since v6 removed 'type', I have Filebeat adding tags for the various log sources. One of the log sources is Cowrie and is the one I will use … --- ## [Dependency Conflicts while creating Pubsub output plugin](https://discuss.elastic.co/t/dependency-conflicts-while-creating-pubsub-output-plugin/108269)
**Author:** [@NirKa](https://discuss.elastic.co/u/NirKa)\ **Replies:** 0\ **Last updated:** [November 19, 2017, 7:03am UTC](https://discuss.elastic.co/t/dependency-conflicts-while-creating-pubsub-output-plugin/108269 "2017-11-19T07:03:21Z")
Hi, I'm trying to create a custom plugin for publishing events to google pubsub. Currently I'm using: Logstash 6.0.0 Trying to use Google::cloud::Pubsub API I was able to build the gem successfully. In the gemspec … --- ## [Logstash-input-jmx plugin installation Error](https://discuss.elastic.co/t/logstash-input-jmx-plugin-installation-error/108273)
**Author:** [@kant.gaurav](https://discuss.elastic.co/u/kant.gaurav)\ **Replies:** 0\ **Last updated:** [November 19, 2017, 7:46am UTC](https://discuss.elastic.co/t/logstash-input-jmx-plugin-installation-error/108273 "2017-11-19T07:46:41Z")
when I am trying to install logstash plugin for JMX, I am getting error for logstash-core, then I tried installing logstash-code using gem then I am facing error "No such file or directory - C:/elk/logstash-6.0.0/vendor/… --- ## [How to monitor that logstash has stopped listening on tcp listener?](https://discuss.elastic.co/t/how-to-monitor-that-logstash-has-stopped-listening-on-tcp-listener/108259)
**Author:** [@Jose\_Ramierez](https://discuss.elastic.co/u/Jose_Ramierez)\ **Replies:** 0\ **Last updated:** [November 19, 2017, 3:38am UTC](https://discuss.elastic.co/t/how-to-monitor-that-logstash-has-stopped-listening-on-tcp-listener/108259 "2017-11-19T03:38:46Z")
Logstash 5.6.4, on CentOS We are exploring the logstash monitoring system. Specifically, we see logstash run fine for awhile, but then stop listening on TCP, and the logstash log (/var/log/logstash/logstash-plain.log) … --- ## [Rsyslog to logstash help](https://discuss.elastic.co/t/rsyslog-to-logstash-help/108254)
**Author:** [@ajhstn](https://discuss.elastic.co/u/ajhstn)\ **Replies:** 3\ **Last updated:** [November 19, 2017, 12:11am UTC](https://discuss.elastic.co/t/rsyslog-to-logstash-help/108254 "2017-11-19T00:11:36Z")
Hello, I am seeking advice regarding getting multiple syslog devices into logstash/elasticsearch. I currently send syslog messages from multiple devices to a central rsyslog server on port 514. Because the messages ar… --- ## [Elasticsearch - Logstash communication problem](https://discuss.elastic.co/t/elasticsearch-logstash-communication-problem/108217)
**Author:** [@homardboy](https://discuss.elastic.co/u/homardboy)\ **Replies:** 10\ **Last updated:** [November 18, 2017, 10:12pm UTC](https://discuss.elastic.co/t/elasticsearch-logstash-communication-problem/108217 "2017-11-18T22:12:48Z")
Hi, I discovered Logstash, elasticsearch and kibana few days ago, and i'm now trying to have a kibana Dashboard of my Squid's log from Pfsense, but i got some issues ... The logs from Squid are from the web trafic of m… --- ## [Netflow Setup Broken?](https://discuss.elastic.co/t/netflow-setup-broken/108024)
**Author:** [@Hank42](https://discuss.elastic.co/u/Hank42)\ **Replies:** 2\ **Last updated:** [November 18, 2017, 10:11pm UTC](https://discuss.elastic.co/t/netflow-setup-broken/108024 "2017-11-18T22:11:29Z")
Succesfully configured logstash 5.6 with netflow setup for remote Kibana and remote Elasticsearch server as follows: bin/logstash --path.settings /etc/logstash --modules netflow --setup -M netflow.var.input.udp.port=7… --- ## [Periodically Query And Process Only New Records](https://discuss.elastic.co/t/periodically-query-and-process-only-new-records/108240)
**Author:** [@panda2004](https://discuss.elastic.co/u/panda2004)\ **Replies:** 1\ **Last updated:** [November 18, 2017, 4:46pm UTC](https://discuss.elastic.co/t/periodically-query-and-process-only-new-records/108240 "2017-11-18T16:46:21Z")
Hello everybody, Background: In our system events arrives in stream. In most cases, we just need to index (Add) the events to new or existsing documnts (Upsert), however there is one event in which we need to update ex… --- ## [Logstash 6 - Multiple Pipelines for One Input](https://discuss.elastic.co/t/logstash-6-multiple-pipelines-for-one-input/107929)
**Author:** [@MarcusCaepio](https://discuss.elastic.co/u/MarcusCaepio)\ **Replies:** 9\ **Last updated:** [November 18, 2017, 4:35pm UTC](https://discuss.elastic.co/t/logstash-6-multiple-pipelines-for-one-input/107929 "2017-11-18T16:35:20Z")
Hi all, Currently in Logstash 5.x I have one Input (Beats) about which I get many different kinds of logs. To grok this logs, I have many config files with filter like "if \[type\] == "yxz....." defined. The results i… --- ## [String split](https://discuss.elastic.co/t/string-split/108117)
**Author:** [@Shekhar\_Pandey](https://discuss.elastic.co/u/Shekhar_Pandey)\ **Replies:** 2\ **Last updated:** [November 18, 2017, 1:22pm UTC](https://discuss.elastic.co/t/string-split/108117 "2017-11-18T13:22:02Z")
I use the following syntax to split a string...but its not working... filter { split { field =\> \["title"," "\] } } But its not working......I want to split a string and want to store as a list. For example: He is a… --- ## [Problem with multiple mapping types in 6.0. Not sure why](https://discuss.elastic.co/t/problem-with-multiple-mapping-types-in-6-0-not-sure-why/108231)
**Author:** [@mbsj](https://discuss.elastic.co/u/mbsj)\ **Replies:** 6\ **Last updated:** [November 18, 2017, 10:05am UTC](https://discuss.elastic.co/t/problem-with-multiple-mapping-types-in-6-0-not-sure-why/108231 "2017-11-18T10:05:00Z")
I have previously commented on another topic with a similar issue, but as that was beats related, and fixed by updates there, and mine is not, i a, creating this topic. I am getting errors due to multiple mapping types,… --- ## [Greedydata doesn't work](https://discuss.elastic.co/t/greedydata-doesnt-work/108201)
**Author:** [@iamthealex](https://discuss.elastic.co/u/iamthealex)\ **Replies:** 1\ **Last updated:** [November 18, 2017, 7:05am UTC](https://discuss.elastic.co/t/greedydata-doesnt-work/108201 "2017-11-18T07:05:26Z")
https://pastebin.com/W0w8UqLm and pasted here for your ease of reference: what is in gstatus.log 2017-11-17 14:16:43.317157 {"volume\_summary" : "u" } what is in my configuration input { exec { command =\> "cat /hom… --- ## [Could not config test logstash](https://discuss.elastic.co/t/could-not-config-test-logstash/108153)
**Author:** [@Makra](https://discuss.elastic.co/u/Makra)\ **Replies:** 2\ **Last updated:** [November 18, 2017, 6:59am UTC](https://discuss.elastic.co/t/could-not-config-test-logstash/108153 "2017-11-18T06:59:20Z")
Hi I have an issue where i cannot config test logstash configuration file, Looks like a Java issue but could not figured out. JAVA\_HOME and java path is set correctly. I m using CentOS 7 and logstash 5 $ /usr/share/l… --- ## [Help newcomer: Payloads received but no output](https://discuss.elastic.co/t/help-newcomer-payloads-received-but-no-output/108178)
**Author:** [@tehowe](https://discuss.elastic.co/u/tehowe)\ **Replies:** 2\ **Last updated:** [November 18, 2017, 6:47am UTC](https://discuss.elastic.co/t/help-newcomer-payloads-received-but-no-output/108178 "2017-11-18T06:47:06Z")
Hello! I'm trying to get a single filebeat to logstash test connection up and running in my environment and have been for some time. I'd been struggling with what I thought was an encryption problem for a while on 5.6 u… --- ## [Logstash clone resulting in literal "%{\[@metadata\]\[type\]}" index names!](https://discuss.elastic.co/t/logstash-clone-resulting-in-literal-metadata-type-index-names/108218)
**Author:** [@excalq](https://discuss.elastic.co/u/excalq)\ **Replies:** 1\ **Last updated:** [November 18, 2017, 3:34am UTC](https://discuss.elastic.co/t/logstash-clone-resulting-in-literal-metadata-type-index-names/108218 "2017-11-18T03:34:24Z")
I'm using Logstash 5.5.1 and ElasticSearch 5.6.4. I'm ingesting Filebeat log data, and I'm seeing a very strange result when using the logstash-filter-clone. The intended indexes (app\_stats-2017.11.17) are missing, but I… --- ## [Encountered a retryable error in Logstash 5.6.3](https://discuss.elastic.co/t/encountered-a-retryable-error-in-logstash-5-6-3/105409)
**Author:** [@lwolf86](https://discuss.elastic.co/u/lwolf86)\ **Replies:** 1\ **Last updated:** [November 17, 2017, 9:00pm UTC](https://discuss.elastic.co/t/encountered-a-retryable-error-in-logstash-5-6-3/105409 "2017-11-17T21:00:49Z")
I'm using logstash 5.6.3 with filebeat input and output to elasticsearch. Dealing huge amount of logs, filebeat reads about 100GB logs. In the begining , everything works fine, but after about 30 minutes. Logstash get th… --- ## [Deprecated document\_type in Logstash 6](https://discuss.elastic.co/t/deprecated-document-type-in-logstash-6/107814)
**Author:** [@Maekee](https://discuss.elastic.co/u/Maekee)\ **Replies:** 3\ **Last updated:** [November 17, 2017, 8:00pm UTC](https://discuss.elastic.co/t/deprecated-document-type-in-logstash-6/107814 "2017-11-17T20:00:49Z")
Hello, See in the breaking Changes (Breaking changes | Logstash Reference \[8.11\] | Elastic) that: "The default document\_type has changed from logs to doc for consistency with Beats" What does this mean for me? I am… --- ## [Logstash - Aggregate Filter](https://discuss.elastic.co/t/logstash-aggregate-filter/108026)
**Author:** [@kazul33](https://discuss.elastic.co/u/kazul33)\ **Replies:** 2\ **Last updated:** [November 17, 2017, 7:33pm UTC](https://discuss.elastic.co/t/logstash-aggregate-filter/108026 "2017-11-17T19:33:01Z")
I'm working with auditd and I'm trying to take multiple events, with the same auditID, and convert it (insert a new document) into one document. The end goal would be to have the original documents be submitted, but the… --- ## [Logstash Aggregate multi levels](https://discuss.elastic.co/t/logstash-aggregate-multi-levels/108169)
**Author:** [@ythalorossy](https://discuss.elastic.co/u/ythalorossy)\ **Replies:** 0\ **Last updated:** [November 17, 2017, 4:55pm UTC](https://discuss.elastic.co/t/logstash-aggregate-multi-levels/108169 "2017-11-17T16:55:13Z")
Hi all, So, I am trying do a simple aggregation using plugins-filters-aggregate. I extract data from relation database using plugin-jdbc and the result is: P\_ID P\_DESC PROD\_I P\_ID PROD\_DESC PROD\_ID SPROD\_DESC COUN… --- ## [Problem running logstash-oss 6 docker image](https://discuss.elastic.co/t/problem-running-logstash-oss-6-docker-image/108173)
**Author:** [@Kelly\_Goedert](https://discuss.elastic.co/u/Kelly_Goedert)\ **Replies:** 0\ **Last updated:** [November 17, 2017, 5:08pm UTC](https://discuss.elastic.co/t/problem-running-logstash-oss-6-docker-image/108173 "2017-11-17T17:08:07Z")
Hi, I created a docker compose file to run the oss 6.0 images of elasticsearch, kibana and logstash. Kibana and Elasticsearch are running fine, with logstash I get an error: logstash\_1 | \[2017-11-17T17:01:37,266… --- ## [Can I specify the pipeline ID in an http request to logstash that uses http input?](https://discuss.elastic.co/t/can-i-specify-the-pipeline-id-in-an-http-request-to-logstash-that-uses-http-input/108030)
**Author:** [@troy](https://discuss.elastic.co/u/troy)\ **Replies:** 2\ **Last updated:** [November 17, 2017, 4:40pm UTC](https://discuss.elastic.co/t/can-i-specify-the-pipeline-id-in-an-http-request-to-logstash-that-uses-http-input/108030 "2017-11-17T16:40:05Z")
My current logstash instance has a single pipeline that uses http input to receive an array of JSON objects. I upgraded to logstash 6.0 and would like to use multiple pipelines. Is there some way for me to specify the pi… --- ## [Conditional, check multiple values for a field](https://discuss.elastic.co/t/conditional-check-multiple-values-for-a-field/108131)
**Author:** [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Replies:** 3\ **Last updated:** [November 17, 2017, 4:35pm UTC](https://discuss.elastic.co/t/conditional-check-multiple-values-for-a-field/108131 "2017-11-17T16:35:06Z")
Hello, I have a pipeline on logstash where I receive messages from network devices (firewalls), parse the message using grok patterns and store them in elasticsearch. I have three different models which I parse some ki… --- ## [Missing event - file input plugin / NFS](https://discuss.elastic.co/t/missing-event-file-input-plugin-nfs/107838)
**Author:** [@franck.lefebure](https://discuss.elastic.co/u/franck.lefebure)\ **Replies:** 19\ **Last updated:** [November 17, 2017, 4:29pm UTC](https://discuss.elastic.co/t/missing-event-file-input-plugin-nfs/107838 "2017-11-17T16:29:40Z")
Hi, We have a problem with our Logstash Installation and I 'm a little stuck. Initially, our logstash install was a v2.4 four nodes install. We recently spent some days to migrate towards 5.6.4 with some hope the probl… --- ## [Schedule job for elasticsearch](https://discuss.elastic.co/t/schedule-job-for-elasticsearch/108079)
**Author:** [@prabhu\_pandurangan](https://discuss.elastic.co/u/prabhu_pandurangan)\ **Replies:** 3\ **Last updated:** [November 17, 2017, 4:10pm UTC](https://discuss.elastic.co/t/schedule-job-for-elasticsearch/108079 "2017-11-17T16:10:57Z")
Hi Team i would like to setup schedule job for elasticsearch input plugin? how to do it? --- ## [Logstash v6 JMS input plugin error](https://discuss.elastic.co/t/logstash-v6-jms-input-plugin-error/107951)
**Author:** [@moriol](https://discuss.elastic.co/u/moriol)\ **Replies:** 1\ **Last updated:** [November 17, 2017, 1:41pm UTC](https://discuss.elastic.co/t/logstash-v6-jms-input-plugin-error/107951 "2017-11-17T13:41:10Z")
I am using Logstash v.6 and trying to get ActiveMQ to work as an input. I am trying to use jms-input plugin. My Activem MQ is on a remote Windows machine, whereas Logstash is on a Centos machine. I have two issues: 1)… --- ## [grokParseFailure but works on grokdebug tester](https://discuss.elastic.co/t/grokparsefailure-but-works-on-grokdebug-tester/108101)
**Author:** [@richard-lee-bjss](https://discuss.elastic.co/u/richard-lee-bjss)\ **Replies:** 2\ **Last updated:** [November 17, 2017, 12:25pm UTC](https://discuss.elastic.co/t/grokparsefailure-but-works-on-grokdebug-tester/108101 "2017-11-17T12:25:12Z")
Hi, I'm on Logstash 5.4.3 (the latest version to support the community NewRelic plugin), shipping IIS logs from filebeat. I've got log entries which pass the grok filter at grokdebug.herokuapp.com, but fail to parse on … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=871) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=873)