# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=819 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 820 --- ## [Unable to understand below indices can someone please help?](https://discuss.elastic.co/t/unable-to-understand-below-indices-can-someone-please-help/120363)
**Author:** [@Blason](https://discuss.elastic.co/u/Blason)\ **Replies:** 1\ **Last updated:** [February 18, 2018, 9:25pm UTC](https://discuss.elastic.co/t/unable-to-understand-below-indices-can-someone-please-help/120363 "2018-02-18T21:25:35Z")
Hi team, I am seeing below indices made index =\> "logstash-bro-%{+YYYY.MM.dd}" index =\> "logstash-switch-%{+YYYY.MM.dd}" index =\> "logstash-import-%{+YYYY.MM.dd}" index =\> "logstash-flow-%{+YYYY.MM.dd}" index… --- ## [Connection refused to webhdfs](https://discuss.elastic.co/t/connection-refused-to-webhdfs/120012)
**Author:** [@ssasporta](https://discuss.elastic.co/u/ssasporta)\ **Replies:** 2\ **Last updated:** [February 18, 2018, 7:51pm UTC](https://discuss.elastic.co/t/connection-refused-to-webhdfs/120012 "2018-02-18T19:51:55Z")
Hi, The Hadoop is up: vpwrk1@vpelastic:\[/home/vpwrk1/var/log/logstash\]$ ps -ef | grep hadoop root 7711 1 0 16:32 ? 00:00:04 /usr/java/jdk1.8.0\_74/bin/java -Dproc\_datanode -Xmx1000m -Djava.net.preferIP… --- ## [Memory leak in Logstash 6.0.0](https://discuss.elastic.co/t/memory-leak-in-logstash-6-0-0/120095)
**Author:** [@vineet](https://discuss.elastic.co/u/vineet)\ **Replies:** 8\ **Last updated:** [February 18, 2018, 6:02pm UTC](https://discuss.elastic.co/t/memory-leak-in-logstash-6-0-0/120095 "2018-02-18T18:02:54Z")
Hello, I am looking for help with Logstash 6.0.0. Winlogbeat is sending logs to Logstash 6.0.0 which is running on a 8 GB machine with 4 GB min/max heap size. Over a period of a few hours, Logstash uses up all the memor… --- ## [Trouble with exec output plugin](https://discuss.elastic.co/t/trouble-with-exec-output-plugin/120321)
**Author:** [@arisbanach](https://discuss.elastic.co/u/arisbanach)\ **Replies:** 2\ **Last updated:** [February 18, 2018, 3:15pm UTC](https://discuss.elastic.co/t/trouble-with-exec-output-plugin/120321 "2018-02-18T15:15:42Z")
I'm trying to get started with the exec output plugin but nothing i try seems to work. I currently have just the simplest config setting possible for it: output { # stdout { codec =\> rubydebug } … --- ## [Logstash Data Filter](https://discuss.elastic.co/t/logstash-data-filter/120317)
**Author:** [@Bheeshma](https://discuss.elastic.co/u/Bheeshma)\ **Replies:** 1\ **Last updated:** [February 18, 2018, 2:28pm UTC](https://discuss.elastic.co/t/logstash-data-filter/120317 "2018-02-18T14:28:46Z")
Hi, I have application log file with user log in and logout details , number of users logged in to the application and the status of the application in the below format recorded in App.log file. \[Tue Oct 17 12:15:01 20… --- ## [The "sonic-traffic" index pattern does not contain any of the following field types: geo\_point](https://discuss.elastic.co/t/the-sonic-traffic-index-pattern-does-not-contain-any-of-the-following-field-types-geo-point/119849)
**Author:** [@legolas\_bilbao](https://discuss.elastic.co/u/legolas_bilbao)\ **Replies:** 1\ **Last updated:** [February 18, 2018, 2:10pm UTC](https://discuss.elastic.co/t/the-sonic-traffic-index-pattern-does-not-contain-any-of-the-following-field-types-geo-point/119849 "2018-02-18T14:10:44Z")
Good morning, I've come back to test Elasticsearch and its works like i like but the option of geoip doesnt work. Logstash filter file if \[type\]=="sonicwall"{ kv { exclude\_keys =\> \[ "… --- ## [SOLVED! CentOs 7 minimal - Syslog received on port 5000 but not after a redirect fw rule from 514](https://discuss.elastic.co/t/solved-centos-7-minimal-syslog-received-on-port-5000-but-not-after-a-redirect-fw-rule-from-514/120345)
**Author:** [@QuizzyRascal](https://discuss.elastic.co/u/QuizzyRascal)\ **Replies:** 0\ **Last updated:** [February 18, 2018, 11:11am UTC](https://discuss.elastic.co/t/solved-centos-7-minimal-syslog-received-on-port-5000-but-not-after-a-redirect-fw-rule-from-514/120345 "2018-02-18T11:11:14Z")
Hi, I'm running a minimal centos 7 setup and am trying to receive syslog into Elk. I have installed ELK with defaults and all three services are up. Using port 5000 for logstash I have started to receive Syslog - sendi… --- ## [java.lang.OutOfMemoryError: Java heap space when transferring data from jdbc to elasticsearch via logstash](https://discuss.elastic.co/t/java-lang-outofmemoryerror-java-heap-space-when-transferring-data-from-jdbc-to-elasticsearch-via-logstash/120067)
**Author:** [@famo](https://discuss.elastic.co/u/famo)\ **Replies:** 0\ **Last updated:** [February 15, 2018, 8:32pm UTC](https://discuss.elastic.co/t/java-lang-outofmemoryerror-java-heap-space-when-transferring-data-from-jdbc-to-elasticsearch-via-logstash/120067 "2018-02-15T20:32:58Z")
I have a huge postgres database with 20 million rows and i want to transfer it to elasticsearch via logstash . I followed the advice mentioned here and I test it for a simple database with 300 rows and all things worked … --- ## [Error on starting logstash with twitter input plugin](https://discuss.elastic.co/t/error-on-starting-logstash-with-twitter-input-plugin/119873)
**Author:** [@AmirHossein\_Ma](https://discuss.elastic.co/u/AmirHossein_Ma)\ **Replies:** 2\ **Last updated:** [February 18, 2018, 6:00am UTC](https://discuss.elastic.co/t/error-on-starting-logstash-with-twitter-input-plugin/119873 "2018-02-18T06:00:54Z")
Hi I will launch Logstash on the basis of this tutorial, but I encounter the following error I use current version of logstash on windows server 2008. C:\\elk\\logstash\>.\\bin\\logstash -f logstash.conf Sending Logstash'… --- ## [Timestamp replaced](https://discuss.elastic.co/t/timestamp-replaced/120054)
**Author:** [@Pororo](https://discuss.elastic.co/u/Pororo)\ **Replies:** 17\ **Last updated:** [February 18, 2018, 12:03am UTC](https://discuss.elastic.co/t/timestamp-replaced/120054 "2018-02-18T00:03:40Z")
Hi, There is one attribute in my json file called "ts" for each item in the file. {"name" : "David", "ts" : 2018-2-15 13:20:00 +0000", .....} And if I directly import this json to ES, the "@timestamp" will be the exac… --- ## [Logstash Configuration file is not working its giving error](https://discuss.elastic.co/t/logstash-configuration-file-is-not-working-its-giving-error/119911)
**Author:** [@Krunal\_kalaria](https://discuss.elastic.co/u/Krunal_kalaria)\ **Replies:** 4\ **Last updated:** [February 17, 2018, 8:45pm UTC](https://discuss.elastic.co/t/logstash-configuration-file-is-not-working-its-giving-error/119911 "2018-02-17T20:45:34Z")
Hello All, I am using Elastic Stack 6.2.0. Elasticsearch and Kibana working fine but When i run any logstash config file then its giving following error: WARNING: Could not find logstash.yml which is typically located… --- ## [Winlogbeat file output -\> logstash -\> Kibana](https://discuss.elastic.co/t/winlogbeat-file-output-logstash-kibana/120128)
**Author:** [@wdaburu](https://discuss.elastic.co/u/wdaburu)\ **Replies:** 4\ **Last updated:** [February 17, 2018, 7:01pm UTC](https://discuss.elastic.co/t/winlogbeat-file-output-logstash-kibana/120128 "2018-02-17T19:01:55Z")
Hi, I've come upon the trouble of making logstash parsing the winlogbeat file that i created from other computer. Additional info: The computer that generate winlogbeat doesn't have any LAN or WAN connection so no logsta… --- ## [How to parse a logs which is in docker to logstash forwarder](https://discuss.elastic.co/t/how-to-parse-a-logs-which-is-in-docker-to-logstash-forwarder/119413)
**Author:** [@Abdur\_Raqeeb1](https://discuss.elastic.co/u/Abdur_Raqeeb1)\ **Replies:** 4\ **Last updated:** [February 17, 2018, 4:42pm UTC](https://discuss.elastic.co/t/how-to-parse-a-logs-which-is-in-docker-to-logstash-forwarder/119413 "2018-02-17T16:42:59Z")
Please let me know how to parse the logs which is in docker and i need to parse it on logstash forwarder. Kindly note have installed logstash forwarder on the server not on an docker container --- ## [Logstash ruby filter](https://discuss.elastic.co/t/logstash-ruby-filter/119978)
**Author:** [@dangal](https://discuss.elastic.co/u/dangal)\ **Replies:** 2\ **Last updated:** [February 17, 2018, 5:31am UTC](https://discuss.elastic.co/t/logstash-ruby-filter/119978 "2018-02-17T05:31:49Z")
Hi all, how pass an url in the ruby filter plugin so that i can do http request i appreciate quick response Thank you --- ## [Default value of the type field](https://discuss.elastic.co/t/default-value-of-the-type-field/119280)
**Author:** [@snuffkin](https://discuss.elastic.co/u/snuffkin)\ **Replies:** 2\ **Last updated:** [February 17, 2018, 4:46am UTC](https://discuss.elastic.co/t/default-value-of-the-type-field/119280 "2018-02-17T04:46:40Z")
Elasticsearch Reference says "The preferred type name is \_doc". https://www.elastic.co/guide/en/elasticsearch/reference/current/removal-of-types.html#\_schedule\_for\_removal\_of\_mapping\_types But, logstash-output-elastics… --- ## [Cant access aws elasticsearch instance](https://discuss.elastic.co/t/cant-access-aws-elasticsearch-instance/120273)
**Author:** [@red888](https://discuss.elastic.co/u/red888)\ **Replies:** 0\ **Last updated:** [February 17, 2018, 12:02am UTC](https://discuss.elastic.co/t/cant-access-aws-elasticsearch-instance/120273 "2018-02-17T00:02:37Z")
So I confirmed I can curl the enpoint successfully but logstash cant access: output { elasticsearch { hosts =\> "https://search-blahbluahbluah.eu-central-1.es.amazonaws.com/" timeout =\> 120 index =\> "sdffsdfsdfsdfsdf… --- ## [Unable to create index properly in elastic search](https://discuss.elastic.co/t/unable-to-create-index-properly-in-elastic-search/120087)
**Author:** [@Arkapravo\_Das](https://discuss.elastic.co/u/Arkapravo_Das)\ **Replies:** 3\ **Last updated:** [February 16, 2018, 11:53pm UTC](https://discuss.elastic.co/t/unable-to-create-index-properly-in-elastic-search/120087 "2018-02-16T23:53:13Z")
Hi All, I am using elastic search 1.7 and logstash 1.5 basic version\[no plugin installed\]. I am trying to create a document to elastic search ,getting input from stdin Here is my pipeline config: input{ stdin{} } o… --- ## [Import aggregate data for each record](https://discuss.elastic.co/t/import-aggregate-data-for-each-record/120271)
**Author:** [@akkash.bansal](https://discuss.elastic.co/u/akkash.bansal)\ **Replies:** 0\ **Last updated:** [February 16, 2018, 10:57pm UTC](https://discuss.elastic.co/t/import-aggregate-data-for-each-record/120271 "2018-02-16T22:57:40Z")
Hi, I am trying to achieve following search record - { "hits": { "total": 148178, "max\_score": 1.0, "hits": \[ { "\_index": "ofc", "\_type": "pers\_desc", "\_id": "141234", "\_score": 1.0, "\_routing": "141234", "\_… --- ## [Import aggregated data](https://discuss.elastic.co/t/import-aggregated-data/120062)
**Author:** [@Scott.Jensen](https://discuss.elastic.co/u/Scott.Jensen)\ **Replies:** 4\ **Last updated:** [February 16, 2018, 9:56pm UTC](https://discuss.elastic.co/t/import-aggregated-data/120062 "2018-02-16T21:56:59Z")
Hi All, I'm trying to import a csv that was exported out of Oracle. The way the CSV is exported has a "count" of similar items. I want to "de"aggregate the data so that in Elasticsearch I can do timebased analysis on… --- ## [Run xml filter on fields created in ruby](https://discuss.elastic.co/t/run-xml-filter-on-fields-created-in-ruby/119945)
**Author:** [@ssasporta](https://discuss.elastic.co/u/ssasporta)\ **Replies:** 2\ **Last updated:** [February 16, 2018, 9:52pm UTC](https://discuss.elastic.co/t/run-xml-filter-on-fields-created-in-ruby/119945 "2018-02-16T21:52:59Z")
While handling a huge log, I am running ruby code filer and according to some rules, I am creating new fields to the event: Object1, Object2, etc.... . The Number of Objects is according to the input in the event, but … --- ## [Logstash container cant read jvm.options correctly](https://discuss.elastic.co/t/logstash-container-cant-read-jvm-options-correctly/120255)
**Author:** [@red888](https://discuss.elastic.co/u/red888)\ **Replies:** 3\ **Last updated:** [February 16, 2018, 9:17pm UTC](https://discuss.elastic.co/t/logstash-container-cant-read-jvm-options-correctly/120255 "2018-02-16T21:17:34Z")
here is my dockerfile: FROM docker.elastic.co/logstash/logstash:6.2.1 SHELL \["/bin/bash", "-c"\] COPY ./jvm.options /etc/logstash/jvm.options Copied my jvm options right from https://github.com/elastic/logstash/blob/m… --- ## [Events being stuck in a mutade](https://discuss.elastic.co/t/events-being-stuck-in-a-mutade/120231)
**Author:** [@danielmotaleite](https://discuss.elastic.co/u/danielmotaleite)\ **Replies:** 0\ **Last updated:** [February 16, 2018, 6:16pm UTC](https://discuss.elastic.co/t/events-being-stuck-in-a-mutade/120231 "2018-02-16T18:16:21Z")
I notice that sometime my logstash (5.6, i can't upgrade yet to 6.x) performance start to drop and if i let it continue, it will start to get a very erratic behavior... i suspect that the java garbage collector is always… --- ## [Remove white spaces from fields - Logstash](https://discuss.elastic.co/t/remove-white-spaces-from-fields-logstash/120177)
**Author:** [@tag\_v](https://discuss.elastic.co/u/tag_v)\ **Replies:** 7\ **Last updated:** [February 16, 2018, 6:44pm UTC](https://discuss.elastic.co/t/remove-white-spaces-from-fields-logstash/120177 "2018-02-16T18:44:07Z")
Message line: key1=val1, key2=val2, key3=val3 Below is my logstash filter: filter { grok { "match" =\> { "message" =\> "%{SYSLOGTIMESTAMP:msg\_timestamp} %{GREEDYDATA:kvFields}" } } kv { source =\> "kvFields" field… --- ## [Insert tags input beats in logstash not work](https://discuss.elastic.co/t/insert-tags-input-beats-in-logstash-not-work/120225)
**Author:** [@gonzauy](https://discuss.elastic.co/u/gonzauy)\ **Replies:** 0\ **Last updated:** [February 16, 2018, 5:24pm UTC](https://discuss.elastic.co/t/insert-tags-input-beats-in-logstash-not-work/120225 "2018-02-16T17:24:06Z")
I need make multiple input in logstash and then multiple output. I have this configuration and I need if the events in on port 5055 tag with "docker" and then add a field and output to elasticsearch. The command tags =\>… --- ## [S3 output leaving temp files still open after being deleted](https://discuss.elastic.co/t/s3-output-leaving-temp-files-still-open-after-being-deleted/120224)
**Author:** [@James\_Harr](https://discuss.elastic.co/u/James_Harr)\ **Replies:** 0\ **Last updated:** [February 16, 2018, 5:23pm UTC](https://discuss.elastic.co/t/s3-output-leaving-temp-files-still-open-after-being-deleted/120224 "2018-02-16T17:23:27Z")
Hi, I've been having issues with the S3 output plugin leaving files open after they're deleted. This is causing disk space to fill up because space is not entirely freed until the file is closed. Once the process is kill… --- ## [Logstash Unable to Parse Response](https://discuss.elastic.co/t/logstash-unable-to-parse-response/120208)
**Author:** [@NaveenKY](https://discuss.elastic.co/u/NaveenKY)\ **Replies:** 1\ **Last updated:** [February 16, 2018, 3:34pm UTC](https://discuss.elastic.co/t/logstash-unable-to-parse-response/120208 "2018-02-16T15:34:39Z")
I am trying to call one of the remote endpoint which is http endpoint. Service is returning the valid JSON. But when i integrate that in http\_poller, logstash is not able to parse the response. Would really appriciate i… --- ## [Trigger logstash config loading remotely](https://discuss.elastic.co/t/trigger-logstash-config-loading-remotely/120205)
**Author:** [@mvienne](https://discuss.elastic.co/u/mvienne)\ **Replies:** 1\ **Last updated:** [February 16, 2018, 3:35pm UTC](https://discuss.elastic.co/t/trigger-logstash-config-loading-remotely/120205 "2018-02-16T15:35:53Z")
I am using Logstash to load data from a PostgreSQL database to Elasticsearch. I would like to be able to trigger this process remotely from a different machine. Is there a way to do that with Logstash by itself, or wha… --- ## [Kibana is missing some logs however I can see that line of information is present in the logs on the server](https://discuss.elastic.co/t/kibana-is-missing-some-logs-however-i-can-see-that-line-of-information-is-present-in-the-logs-on-the-server/119997)
**Author:** [@imarjun](https://discuss.elastic.co/u/imarjun)\ **Replies:** 7\ **Last updated:** [February 16, 2018, 2:28pm UTC](https://discuss.elastic.co/t/kibana-is-missing-some-logs-however-i-can-see-that-line-of-information-is-present-in-the-logs-on-the-server/119997 "2018-02-16T14:28:19Z")
I am unable to view a particular log in Kibana interface however that line of log is present in the log file of the application server. I am searching wih the GUID parameter from below line, but unable to get it in the… --- ## [Is it possible to make a gem be used as an input and output plugin?](https://discuss.elastic.co/t/is-it-possible-to-make-a-gem-be-used-as-an-input-and-output-plugin/120195)
**Author:** [@smbullet](https://discuss.elastic.co/u/smbullet)\ **Replies:** 0\ **Last updated:** [February 16, 2018, 2:23pm UTC](https://discuss.elastic.co/t/is-it-possible-to-make-a-gem-be-used-as-an-input-and-output-plugin/120195 "2018-02-16T14:23:37Z")
I'm trying to debug a plugin that's attempting to sublcass LogStash::Outputs::ElasticSearch and overrides the submit() method. It also tries to act as an input plugin and overrides the run() method so it can feed events … --- ## [Kv filter plugin](https://discuss.elastic.co/t/kv-filter-plugin/119969)
**Author:** [@antwan](https://discuss.elastic.co/u/antwan)\ **Replies:** 3\ **Last updated:** [February 16, 2018, 11:58am UTC](https://discuss.elastic.co/t/kv-filter-plugin/119969 "2018-02-16T11:58:56Z")
Hello, I am playing around with logstash filters to parse logs from auditd. Its a challenge to write a filter to catch them all, without going crazy with conditional logics. What I am trying to accomplish is to send da… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=818) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=820)