# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=815 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 816 --- ## [Issue in Geo\_point data type](https://discuss.elastic.co/t/issue-in-geo-point-data-type/120941)
**Author:** [@Prakash\_Punj](https://discuss.elastic.co/u/Prakash_Punj)\ **Replies:** 0\ **Last updated:** [February 21, 2018, 10:00pm UTC](https://discuss.elastic.co/t/issue-in-geo-point-data-type/120941 "2018-02-21T22:00:31Z")
Below are the steps I am following to store geo\_point a.) Created index : PUT chicago\_crime b.) Geo point mapping: PUT chicago\_crime/\_mapping/my\_type { "my\_type": { "properties": { "location": { "type": "geo\_point… --- ## [Monitoring pipeline fails x-pack for logstash](https://discuss.elastic.co/t/monitoring-pipeline-fails-x-pack-for-logstash/121098)
**Author:** [@ecamero2](https://discuss.elastic.co/u/ecamero2)\ **Replies:** 2\ **Last updated:** [February 22, 2018, 9:02pm UTC](https://discuss.elastic.co/t/monitoring-pipeline-fails-x-pack-for-logstash/121098 "2018-02-22T21:02:13Z")
logstash monitoring never shows up. I see the below over and over again in the log file: \[2018-02-22T12:15:59,993\]\[INFO \]\[logstash.pipeline \] Starting pipeline {:pipeline\_id=\>".monitoring-logstash", "pipeline.wor… --- ## [Installing x-pack on logstash 5.6.6 fails](https://discuss.elastic.co/t/installing-x-pack-on-logstash-5-6-6-fails/121117)
**Author:** [@zubinr](https://discuss.elastic.co/u/zubinr)\ **Replies:** 0\ **Last updated:** [February 22, 2018, 7:20pm UTC](https://discuss.elastic.co/t/installing-x-pack-on-logstash-5-6-6-fails/121117 "2018-02-22T19:20:33Z")
I am trying to install x-pack on logstash 5.6.6 /usr/share/logstash/bin/./logstash-plugin install x-pack Error: Exception thrown by the agent : java.rmi.server.ExportException: Port already in use: 2026; nested exceptio… --- ## [Is there a way to give each mutate task a friendly name?](https://discuss.elastic.co/t/is-there-a-way-to-give-each-mutate-task-a-friendly-name/120960)
**Author:** [@hassanbabaie](https://discuss.elastic.co/u/hassanbabaie)\ **Replies:** 2\ **Last updated:** [February 22, 2018, 6:17pm UTC](https://discuss.elastic.co/t/is-there-a-way-to-give-each-mutate-task-a-friendly-name/120960 "2018-02-22T18:17:27Z")
I've searched and not really found a clear example: I have a lot of mutate tasks and I'm trying to understand where I am having some issues with log processing. However when I run the local node stat command: curl -XG… --- ## [Logstash config test hangs when Logstash is running w/ DLQ enabled](https://discuss.elastic.co/t/logstash-config-test-hangs-when-logstash-is-running-w-dlq-enabled/121043)
**Author:** [@Evesy](https://discuss.elastic.co/u/Evesy)\ **Replies:** 1\ **Last updated:** [February 22, 2018, 5:30pm UTC](https://discuss.elastic.co/t/logstash-config-test-hangs-when-logstash-is-running-w-dlq-enabled/121043 "2018-02-22T17:30:27Z")
Hi, Logstash: 5.6.1 Additional plugins: x-pack (5.6.1) logstash-filter-prune (3.0.3) I'm observing some strange behaviour when running Logstash config test when an instance of Logstash is already running with the DL… --- ## [Codec to convert from EBICS to UTF8](https://discuss.elastic.co/t/codec-to-convert-from-ebics-to-utf8/121101)
**Author:** [@Srikanth\_FT](https://discuss.elastic.co/u/Srikanth_FT)\ **Replies:** 0\ **Last updated:** [February 22, 2018, 5:26pm UTC](https://discuss.elastic.co/t/codec-to-convert-from-ebics-to-utf8/121101 "2018-02-22T17:26:43Z")
Hi Does any one how do i convert EBICS (https://en.wikipedia.org/wiki/Electronic\_Banking\_Internet\_Communication\_Standard) data to human readable data (UTF-8) My source data is in EBICS. I want to decode the data to UT… --- ## [Configuring Logstash to Use Dead Letter Queues](https://discuss.elastic.co/t/configuring-logstash-to-use-dead-letter-queues/120892)
**Author:** [@alexus](https://discuss.elastic.co/u/alexus)\ **Replies:** 8\ **Last updated:** [February 22, 2018, 4:00am UTC](https://discuss.elastic.co/t/configuring-logstash-to-use-dead-letter-queues/120892 "2018-02-22T04:00:09Z")
Per Configuring Logstash to Use Dead Letter Queues: Dead letter queues are disabled by default. To enable dead letter queues, set the dead\_letter\_queue\_enable option in the logstash.yml settings file: dead\_letter\_queu… --- ## [Logstash shutdown stalled](https://discuss.elastic.co/t/logstash-shutdown-stalled/121086)
**Author:** [@stevesimpson](https://discuss.elastic.co/u/stevesimpson)\ **Replies:** 0\ **Last updated:** [February 22, 2018, 4:19pm UTC](https://discuss.elastic.co/t/logstash-shutdown-stalled/121086 "2018-02-22T16:19:14Z")
Hi I've attempted to restart logstash and now find the following error: \[2018-02-22T15:54:41,710\]\[INFO \]\[logstash.pipeline \] Pipeline terminated {"pipeline.id"=\>".monitoring-logstash"} \[2018-02-22T15:54:46,200\]\[… --- ## [Issue with split filter plugin since 6.x](https://discuss.elastic.co/t/issue-with-split-filter-plugin-since-6-x/120795)
**Author:** [@asp](https://discuss.elastic.co/u/asp)\ **Replies:** 2\ **Last updated:** [February 22, 2018, 4:13pm UTC](https://discuss.elastic.co/t/issue-with-split-filter-plugin-since-6-x/120795 "2018-02-22T16:13:55Z")
Hi all, I am experiencing very strange issues with the parsing of on of our logs since version 6.x. The issue is the same for 6.1.0, 6.1.2 and 6.2.0. Other 6.x. versions have not been tested. We have a logfile with fo… --- ## [Apache NiFI to Logstash](https://discuss.elastic.co/t/apache-nifi-to-logstash/121066)
**Author:** [@sm00thindian](https://discuss.elastic.co/u/sm00thindian)\ **Replies:** 3\ **Last updated:** [February 22, 2018, 4:12pm UTC](https://discuss.elastic.co/t/apache-nifi-to-logstash/121066 "2018-02-22T16:12:49Z")
I am looking to securely send data from my dynamic workflow tool Apache NiFi to Logstash. What are effective methods of accomplishing this? -krw --- ## [\_grokparsefailure shows always](https://discuss.elastic.co/t/grokparsefailure-shows-always/121014)
**Author:** [@chianingwang](https://discuss.elastic.co/u/chianingwang)\ **Replies:** 4\ **Last updated:** [February 22, 2018, 3:55pm UTC](https://discuss.elastic.co/t/grokparsefailure-shows-always/121014 "2018-02-22T15:55:50Z")
Hi I got the input string as below " Object update sweep completed on /srv/node/d3 in 0.00s seconds:, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 0 errors (pid: 26739)" And my grok pattern is as below. grok { … --- ## [Logs being tagged with IP instead of hostname](https://discuss.elastic.co/t/logs-being-tagged-with-ip-instead-of-hostname/121076)
**Author:** [@spiffytech](https://discuss.elastic.co/u/spiffytech)\ **Replies:** 0\ **Last updated:** [February 22, 2018, 3:49pm UTC](https://discuss.elastic.co/t/logs-being-tagged-with-ip-instead-of-hostname/121076 "2018-02-22T15:49:31Z")
I have some logs collected with logspout+logspout-logstash which are stored with host: x.x.x.x instead of host: foo.example.com. Where does the host field come from? Does Logstash generate it, or is it fed in from the co… --- ## [ELK installation completed with latest version 6.x.x on linux servers But i am not able to access webpage](https://discuss.elastic.co/t/elk-installation-completed-with-latest-version-6-x-x-on-linux-servers-but-i-am-not-able-to-access-webpage/116487)
**Author:** [@Panakaj\_Patel](https://discuss.elastic.co/u/Panakaj_Patel)\ **Replies:** 19\ **Last updated:** [February 22, 2018, 2:56pm UTC](https://discuss.elastic.co/t/elk-installation-completed-with-latest-version-6-x-x-on-linux-servers-but-i-am-not-able-to-access-webpage/116487 "2018-02-22T14:56:37Z")
All services are up and running no issue with services and also with below test. \[root@xxx ~\]# curl http://localhost:9200 { "name" : "\_ZCxA3a", "cluster\_name" : "elasticsearch", "cluster\_uuid" : "2N1OSkYjSMmP3Nh8nYZ… --- ## [Behaviour of logstash restart](https://discuss.elastic.co/t/behaviour-of-logstash-restart/121022)
**Author:** [@Kasun\_Siyambalapitiy](https://discuss.elastic.co/u/Kasun_Siyambalapitiy)\ **Replies:** 3\ **Last updated:** [February 22, 2018, 12:15pm UTC](https://discuss.elastic.co/t/behaviour-of-logstash-restart/121022 "2018-02-22T12:15:09Z")
I have a logstash setup configured for publishing events from a file (once a new record gets added into a log file) Currently the logstash process has stopped on my instance. What happens if I restart logstash, does it w… --- ## [How to delete an index from logstash output](https://discuss.elastic.co/t/how-to-delete-an-index-from-logstash-output/121017)
**Author:** [@Efrain\_Gonzalez](https://discuss.elastic.co/u/Efrain_Gonzalez)\ **Replies:** 6\ **Last updated:** [February 22, 2018, 11:01am UTC](https://discuss.elastic.co/t/how-to-delete-an-index-from-logstash-output/121017 "2018-02-22T11:01:49Z")
Hi, From logstash, if there is a specific event, I need to delete all documents in an index or delete the entire index (both options works for me, so the more efficient the better) I don't find a way to do that, some c… --- ## [7y8ui9o](https://discuss.elastic.co/t/7y8ui9o/121025)
**Author:** [@Eswar\_Kumar\_Musiboin](https://discuss.elastic.co/u/Eswar_Kumar_Musiboin)\ **Replies:** 1\ **Last updated:** [February 22, 2018, 10:51am UTC](https://discuss.elastic.co/t/7y8ui9o/121025 "2018-02-22T10:51:44Z")
Continuing the discussion from Ruby filter loop through fields in logstash 5?: --- ## [Logstash split fields into new fields(field1, field2, field3, field4, field5)](https://discuss.elastic.co/t/logstash-split-fields-into-new-fields-field1-field2-field3-field4-field5/120840)
**Author:** [@moketemokoena](https://discuss.elastic.co/u/moketemokoena)\ **Replies:** 14\ **Last updated:** [February 22, 2018, 9:05am UTC](https://discuss.elastic.co/t/logstash-split-fields-into-new-fields-field1-field2-field3-field4-field5/120840 "2018-02-22T09:05:18Z")
Hi I am importing data from mssql database into Elasticsearch. I am trying to split field "my\_message" that is structured as follows: "{1:Mokete}{2:Mokoena}{3:{888:Logstash}}{4:\\n:20:Testing\\n:LG:Logstash\\n:ES:Elastic… --- ## [Logstash unable to input into Elasticsearch](https://discuss.elastic.co/t/logstash-unable-to-input-into-elasticsearch/121013)
**Author:** [@petegriggs](https://discuss.elastic.co/u/petegriggs)\ **Replies:** 0\ **Last updated:** [February 22, 2018, 9:03am UTC](https://discuss.elastic.co/t/logstash-unable-to-input-into-elasticsearch/121013 "2018-02-22T09:03:03Z")
Hi, Just upgraded to Elasticsearch and Logstash 6.2.0, everything was going well however logstash now can't put data in to elasticsearch. We are using this for IDS traffic the error we are getting is: \[2018-02-21T14:11… --- ## [Logstash multiline pattern : continuous newline](https://discuss.elastic.co/t/logstash-multiline-pattern-continuous-newline/120993)
**Author:** [@byungik](https://discuss.elastic.co/u/byungik)\ **Replies:** 8\ **Last updated:** [February 22, 2018, 8:12am UTC](https://discuss.elastic.co/t/logstash-multiline-pattern-continuous-newline/120993 "2018-02-22T08:12:58Z")
This is my DB logs(CUBRID Slow query log). '16-09-26 14:26:31.564 (35) execute\_all srv\_h\_id 14 DELETE FROM MSG\_RCV\_CC WHERE RCV\_ID = ? 16-09-26 14:26:31.564 (35) bind 1 : VARCHAR (9)11282366 16-09-26 14:26:33.987 (35)… --- ## [Plugin Install fails\`gem install logstash-core -v '6.1.3'\`](https://discuss.elastic.co/t/plugin-install-fails-gem-install-logstash-core-v-6-1-3/118006)
**Author:** [@sarlacpit](https://discuss.elastic.co/u/sarlacpit)\ **Replies:** 15\ **Last updated:** [February 21, 2018, 4:25pm UTC](https://discuss.elastic.co/t/plugin-install-fails-gem-install-logstash-core-v-6-1-3/118006 "2018-02-21T16:25:04Z")
I am having problems installing plugins: I am running ELK 1.6.3 with an aim to get Netflow data showing in Kibana on Ubuntu 16.04LTS - When trying to install numerous plugins I am getting an error: /usr/share/logstash… --- ## [Interger field showing as string](https://discuss.elastic.co/t/interger-field-showing-as-string/120988)
**Author:** [@sreejiths](https://discuss.elastic.co/u/sreejiths)\ **Replies:** 7\ **Last updated:** [February 22, 2018, 7:36am UTC](https://discuss.elastic.co/t/interger-field-showing-as-string/120988 "2018-02-22T07:36:33Z")
GROK grok { match =\> \[ # IOS "message", "%{IP:host\_nagios}: Nagios-Log device\_id=%{WORD:hostname} rtt=%{NUMBER:RTT} avgSD=%{NUMBER:SDLATENCY} avgDS=% {NUMBER:DSLATENCY} syslog\_sev\_level=%{INT:syslo… --- ## [Elasticsearch filter query not working](https://discuss.elastic.co/t/elasticsearch-filter-query-not-working/120998)
**Author:** [@ranganath\_nangineni](https://discuss.elastic.co/u/ranganath_nangineni)\ **Replies:** 0\ **Last updated:** [February 22, 2018, 7:22am UTC](https://discuss.elastic.co/t/elasticsearch-filter-query-not-working/120998 "2018-02-22T07:22:53Z")
Hi, The below config file is workingfine but the same query when I used it in my logstash config , it is throwing some error. Working file: input { elasticsearch { "hosts" =\> "XYZ.com:9200" "index" =\> "test-m-docs" … --- ## [Vertical Scalability of Logstash in AWS - ECS](https://discuss.elastic.co/t/vertical-scalability-of-logstash-in-aws-ecs/120935)
**Author:** [@Jay1](https://discuss.elastic.co/u/Jay1)\ **Replies:** 1\ **Last updated:** [February 22, 2018, 7:06am UTC](https://discuss.elastic.co/t/vertical-scalability-of-logstash-in-aws-ecs/120935 "2018-02-22T07:06:06Z")
Hi Everyone , I am trying to implement ELK stack on AWS environment and planning to run logstash as Docker Container on ECS . The incoming traffic to logstash are from Filebeat installed on around 20+ App servers and I… --- ## [Increase .gz compression in file output (s3 output is better)?](https://discuss.elastic.co/t/increase-gz-compression-in-file-output-s3-output-is-better/120994)
**Author:** [@Sjaak01](https://discuss.elastic.co/u/Sjaak01)\ **Replies:** 0\ **Last updated:** [February 22, 2018, 7:00am UTC](https://discuss.elastic.co/t/increase-gz-compression-in-file-output-s3-output-is-better/120994 "2018-02-22T07:00:41Z")
Hi, Is it possible to increase the compression ratio for .gz files in the file output plugin? I noticed the s3 output plugin can get up three times smaller file sizes with the exact same data, it would be great if the s… --- ## [Python log Logstash grok filter](https://discuss.elastic.co/t/python-log-logstash-grok-filter/120753)
**Author:** [@Data-drone](https://discuss.elastic.co/u/Data-drone)\ **Replies:** 3\ **Last updated:** [February 22, 2018, 6:50am UTC](https://discuss.elastic.co/t/python-log-logstash-grok-filter/120753 "2018-02-22T06:50:04Z")
I have a filter setup: filter { grok { match =\> \[ "message", "%{TIMESTAMP\_ISO8601:timestamp} - %{DATA:module} - %{LOGLEVEL:Level} - %{GREEDYDATA:logmessage}" \] } date { timezone =\> "Australia… --- ## [Multipipeline on Windows Server](https://discuss.elastic.co/t/multipipeline-on-windows-server/120545)
**Author:** [@wwalker](https://discuss.elastic.co/u/wwalker)\ **Replies:** 1\ **Last updated:** [February 22, 2018, 6:31am UTC](https://discuss.elastic.co/t/multipipeline-on-windows-server/120545 "2018-02-22T06:31:37Z")
Anybody have any luck getting this to work on Server 2012/2016? Below is my configuration and trace log Pipelines.yml - pipeline.id: DMARC path.config: "C:/logstash/config/dmarcpipeline.cfg" config.reload.auto… --- ## [CSV filter parsing errors](https://discuss.elastic.co/t/csv-filter-parsing-errors/120789)
**Author:** [@ranganath\_nangineni](https://discuss.elastic.co/u/ranganath_nangineni)\ **Replies:** 2\ **Last updated:** [February 22, 2018, 6:04am UTC](https://discuss.elastic.co/t/csv-filter-parsing-errors/120789 "2018-02-22T06:04:59Z")
Hi I am getting the below errors while parsing using CSV filter \[2018-02-21T08:30:48,657\]\[WARN \]\[logstash.filters.csv \] Error parsing csv {:field=\>"filename", :source=\>\["CNT415653", "CNT415653"\], :exception=\>#\ **Author:** [@elasticheart](https://discuss.elastic.co/u/elasticheart)\ **Replies:** 0\ **Last updated:** [February 21, 2018, 12:53pm UTC](https://discuss.elastic.co/t/why-avro-message-becomes-nil-in-logstash/120839 "2018-02-21T12:53:38Z") Hi, I am using ELK GA 6.0.0. I am pushing an avro message into Kafka. Below is the avsc; { "type": "record", "name": "AvroTestVO", "namespace": "com.home.user.kafka.avro", "doc": "Test logs AVRO Schema", "fields":… --- ## [Logstash with elasticsearch input and output keep looping results](https://discuss.elastic.co/t/logstash-with-elasticsearch-input-and-output-keep-looping-results/115031)
**Author:** [@layla](https://discuss.elastic.co/u/layla)\ **Replies:** 14\ **Last updated:** [February 22, 2018, 3:59am UTC](https://discuss.elastic.co/t/logstash-with-elasticsearch-input-and-output-keep-looping-results/115031 "2018-02-22T03:59:16Z")
I would like to reindex and filter my log again. What I get the information from Internet is using the logstash to filter the data again. I tried and it can really split my data into different fields, however, the data k… --- ## [Logstash hangs](https://discuss.elastic.co/t/logstash-hangs/120958)
**Author:** [@GSCully](https://discuss.elastic.co/u/GSCully)\ **Replies:** 0\ **Last updated:** [February 22, 2018, 1:31am UTC](https://discuss.elastic.co/t/logstash-hangs/120958 "2018-02-22T01:31:32Z")
Often logstash hangs until I hit enter in the console, Version 6.1.1 Logstash on Windows 2012R2 4G RAM 4 CPU - all traffic is HTTPS I have run into a problem where the logstash hangs until I press enter in the console … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=814) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=816)