# Logstash

**URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=565

[Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md)

**Page:** 566

---

## [Trapping mapper\_parsing\_exception original messages](https://discuss.elastic.co/t/trapping-mapper-parsing-exception-original-messages/176975)

<div class="topic-metadata">

**Author:** [@jeffkirk1](https://discuss.elastic.co/u/jeffkirk1)\
**Replies:** 2\
**Last updated:** [April 16, 2019, 12:25am UTC](https://discuss.elastic.co/t/trapping-mapper-parsing-exception-original-messages/176975 "2019-04-16T00:25:14Z")

</div>

Hi there. I am debugging a Logstash filter pipeline that occasionally throws an Elasticsearch indexing error. Long story short is that I am trying to parse a log message from an application into key-value pairs using a …

---

## [Install logstash plugin via RPM?](https://discuss.elastic.co/t/install-logstash-plugin-via-rpm/176686)

<div class="topic-metadata">

**Author:** [@jcaballero](https://discuss.elastic.co/u/jcaballero)\
**Replies:** 2\
**Last updated:** [April 15, 2019, 10:00pm UTC](https://discuss.elastic.co/t/install-logstash-plugin-via-rpm/176686 "2019-04-15T22:00:10Z")

</div>

Hi, for some reasons, it would be better for me to install the logstash-filter-prune plugin via RPM. I have been able to generate the RPM from the gem file, using command gem2rpm, with some modifications in the resulti…

---

## [Filebeat input from external source](https://discuss.elastic.co/t/filebeat-input-from-external-source/176966)

<div class="topic-metadata">

**Author:** [@etsai7](https://discuss.elastic.co/u/etsai7)\
**Replies:** 1\
**Last updated:** [April 15, 2019, 8:04pm UTC](https://discuss.elastic.co/t/filebeat-input-from-external-source/176966 "2019-04-15T20:04:05Z")

</div>

Hello, I am looking to use an input from either a tcp or udp port. Is this one of the input plugins that comes with the logstash packages, or does it have to be installed as a separate plugin? Thanks, Eric

---

## [Logstash Filtering split](https://discuss.elastic.co/t/logstash-filtering-split/176956)

<div class="topic-metadata">

**Author:** [@Elk\_huh](https://discuss.elastic.co/u/Elk_huh)\
**Replies:** 3\
**Last updated:** [April 15, 2019, 8:03pm UTC](https://discuss.elastic.co/t/logstash-filtering-split/176956 "2019-04-15T20:03:28Z")

</div>

How would I filter "fileWriteEvent/username": "domain\\bob132" so that I can create a new field ( basically taking out hte domain\\) Username: bob132

---

## [What filter i need?](https://discuss.elastic.co/t/what-filter-i-need/176954)

<div class="topic-metadata">

**Author:** [@OlegInishev](https://discuss.elastic.co/u/OlegInishev)\
**Replies:** 1\
**Last updated:** [April 15, 2019, 7:22pm UTC](https://discuss.elastic.co/t/what-filter-i-need/176954 "2019-04-15T19:22:30Z")

</div>

Hi evryone! I have next log example \[time\_field\] key1=val1 key2=val2 key3=val3 ACTION\_TYPE key4=val4 key5=val5 I want cut ACTION\_TYPE from message and add him in new field, and then use KV-filter for key-value. I use …

---

## [Logstash equivalent of FluentD “\<secondary\>” tag](https://discuss.elastic.co/t/logstash-equivalent-of-fluentd-secondary-tag/176964)

<div class="topic-metadata">

**Author:** [@cyxop](https://discuss.elastic.co/u/cyxop)\
**Replies:** 0\
**Last updated:** [April 15, 2019, 7:21pm UTC](https://discuss.elastic.co/t/logstash-equivalent-of-fluentd-secondary-tag/176964 "2019-04-15T19:21:13Z")

</div>

I am using Logstash to ship data to Kafka queue. If Kafka primary cluster is down, I want to switch to secondary Kafka cluster. I know that Fluentd 3.4+ can route traffic to another cluster with "" tag. How can I achiev…

---

## [Logstash filter section](https://discuss.elastic.co/t/logstash-filter-section/176941)

<div class="topic-metadata">

**Author:** [@Michalis\_Kyprianou](https://discuss.elastic.co/u/Michalis_Kyprianou)\
**Replies:** 1\
**Last updated:** [April 15, 2019, 6:59pm UTC](https://discuss.elastic.co/t/logstash-filter-section/176941 "2019-04-15T18:59:23Z")

</div>

Hi all i'm just very curious to have you input on the below quotation. find my logstash conf file input { beats { port =\> "5044" client\_inactivity\_timeout =\> "600" ssl =\> false } } filter { if \[fileset\]\[module…

---

## [Parsing log with comma separated key value pairs](https://discuss.elastic.co/t/parsing-log-with-comma-separated-key-value-pairs/176942)

<div class="topic-metadata">

**Author:** [@Abhilash\_B](https://discuss.elastic.co/u/Abhilash_B)\
**Replies:** 2\
**Last updated:** [April 15, 2019, 6:52pm UTC](https://discuss.elastic.co/t/parsing-log-with-comma-separated-key-value-pairs/176942 "2019-04-15T18:52:43Z")

</div>

Hi all, I have a log file some like this. 2019-01-30T06:01:02.551-07:00;82828;10400035;INFO;Measure vehicle detected;System Vehicle detected;"CAL\_CHKS=,CAL\_CPPX=0,CAL\_CPPY=0,CAM\_CUSR=CAM\_CUSR,CAM\_NUMB=1,CAM\_NUS2…

---

## [Grok debug](https://discuss.elastic.co/t/grok-debug/176947)

<div class="topic-metadata">

**Author:** [@OlegInishev](https://discuss.elastic.co/u/OlegInishev)\
**Replies:** 2\
**Last updated:** [April 15, 2019, 6:32pm UTC](https://discuss.elastic.co/t/grok-debug/176947 "2019-04-15T18:32:39Z")

</div>

Hi I have problem with my grok filter. When i adding new pattern in MATCH section ~ 80% message lost. I check my filter in grok debugger - it's fine How i can debug (step by step) my filter? Can i see how grok work …

---

## [When I convert start\_time : string to date it does not work](https://discuss.elastic.co/t/when-i-convert-start-time-string-to-date-it-does-not-work/176935)

<div class="topic-metadata">

**Author:** [@angham](https://discuss.elastic.co/u/angham)\
**Replies:** 1\
**Last updated:** [April 15, 2019, 4:37pm UTC](https://discuss.elastic.co/t/when-i-convert-start-time-string-to-date-it-does-not-work/176935 "2019-04-15T16:37:10Z")

</div>

help plz when I convert start\_time : string to date it does not work this is date in the log file : 2019-04-15 15:49:02,988 filter{ grok { match =\> \[ "message", "%{TIMESTAMP\_ISO8601:start\_time}\\s%{LOGLEVEL:level} \[%…

---

## [Nginx & Kibana 7.0.0 setup woes](https://discuss.elastic.co/t/nginx-kibana-7-0-0-setup-woes/176931)

<div class="topic-metadata">

**Author:** [@Callahan](https://discuss.elastic.co/u/Callahan)\
**Replies:** 1\
**Last updated:** [April 15, 2019, 4:28pm UTC](https://discuss.elastic.co/t/nginx-kibana-7-0-0-setup-woes/176931 "2019-04-15T16:28:50Z")

</div>

Hi, I'm trying to set up a nginx reverse proxy to sit in front of my Kibana box. Both instances (nginx and kibana) are running in a docker environment on one host. The kibana container is named "kibana" - the nginx cont…

---

## [Enabling X-Pack in Logstash 7 gives me this error](https://discuss.elastic.co/t/enabling-x-pack-in-logstash-7-gives-me-this-error/176671)

<div class="topic-metadata">

**Author:** [@teamg](https://discuss.elastic.co/u/teamg)\
**Replies:** 9\
**Last updated:** [April 15, 2019, 2:21pm UTC](https://discuss.elastic.co/t/enabling-x-pack-in-logstash-7-gives-me-this-error/176671 "2019-04-15T14:21:53Z")

</div>

I have a single node ELK 7 stack and I want Kibana to display Logstash in Monitoring. I enabled X-Pack in /etc/logstash/logstash.yml and restart logstash. I see this in /var/log: Apr 12 11:21:28 nocptc-elk logstash: …

---

## [Logstash 7 install error Ubuntu 18.04](https://discuss.elastic.co/t/logstash-7-install-error-ubuntu-18-04/176917)

<div class="topic-metadata">

**Author:** [@lolmcbride](https://discuss.elastic.co/u/lolmcbride)\
**Replies:** 2\
**Last updated:** [April 15, 2019, 2:16pm UTC](https://discuss.elastic.co/t/logstash-7-install-error-ubuntu-18-04/176917 "2019-04-15T14:16:14Z")

</div>

I'm trying to install the ELK stack on a fresh Ubuntu 18.04 server. Elasticsearch and kibana are now working and I've moved onto Logstash but i get the following error when trying to install using apt. I am following the…

---

## [Issue in casting uuid](https://discuss.elastic.co/t/issue-in-casting-uuid/176921)

<div class="topic-metadata">

**Author:** [@Anup\_Kurup](https://discuss.elastic.co/u/Anup_Kurup)\
**Replies:** 0\
**Last updated:** [April 15, 2019, 2:09pm UTC](https://discuss.elastic.co/t/issue-in-casting-uuid/176921 "2019-04-15T14:09:11Z")

</div>

Hi, I have a cassandra table with id column having datatype as uuid. Now i want to select records from the table but it is giving error as uuid is unidentified in jdbc (i am using cassandrajdbc1.1.jar) . So I tried cas…

---

## [Logstash pulls all records from postgres db not as per join](https://discuss.elastic.co/t/logstash-pulls-all-records-from-postgres-db-not-as-per-join/176895)

<div class="topic-metadata">

**Author:** [@rhari](https://discuss.elastic.co/u/rhari)\
**Replies:** 0\
**Last updated:** [April 15, 2019, 11:52am UTC](https://discuss.elastic.co/t/logstash-pulls-all-records-from-postgres-db-not-as-per-join/176895 "2019-04-15T11:52:33Z")

</div>

Here is my logstash.conf file input { jdbc { # Postgres jdbc connection string to our database, nnm jdbc\_connection\_string =\> "jdbc:postgresql://localhost:5432/nnm" # The user we wish to execute our statement as jd…

---

## [Need to get filenames at a particular directory without reading the data inside files](https://discuss.elastic.co/t/need-to-get-filenames-at-a-particular-directory-without-reading-the-data-inside-files/176188)

<div class="topic-metadata">

**Author:** [@shruti51](https://discuss.elastic.co/u/shruti51)\
**Replies:** 3\
**Last updated:** [April 15, 2019, 11:24am UTC](https://discuss.elastic.co/t/need-to-get-filenames-at-a-particular-directory-without-reading-the-data-inside-files/176188 "2019-04-15T11:24:15Z")

</div>

Hi, I am trying to read the file names without reading the content inside file. I am not able to get the desired result using filebeat. The logstash is: input { \> beats { \> port =\> 5044 \> } \> } filter { if …

---

## [Logstash don't read lines from imput file SAMBA](https://discuss.elastic.co/t/logstash-dont-read-lines-from-imput-file-samba/176742)

<div class="topic-metadata">

**Author:** [@joansa](https://discuss.elastic.co/u/joansa)\
**Replies:** 2\
**Last updated:** [April 15, 2019, 10:47am UTC](https://discuss.elastic.co/t/logstash-dont-read-lines-from-imput-file-samba/176742 "2019-04-15T10:47:43Z")

</div>

Hi. I've a IDS Suricata ina a raspberry-pi, and I install logstash, elasticsearch and kibana in my Windows PC. My idea is show from a SAMBA directory the suricata eve.json log to logstash. So, When I run it, logst…

---

## [Generate Input Plugin](https://discuss.elastic.co/t/generate-input-plugin/176863)

<div class="topic-metadata">

**Author:** [@Jasonespo](https://discuss.elastic.co/u/Jasonespo)\
**Replies:** 0\
**Last updated:** [April 15, 2019, 9:00am UTC](https://discuss.elastic.co/t/generate-input-plugin/176863 "2019-04-15T09:00:55Z")

</div>

Hi all, I haven't found any documentation on this. Are there any parameters you can add to the generate input plugin so that it sends X amount of logs / second or minute. I am going to assume the default (as below), se…

---

## [JDBC - Using parameter for both SQL statement and last\_run\_metadata\_path](https://discuss.elastic.co/t/jdbc-using-parameter-for-both-sql-statement-and-last-run-metadata-path/176860)

<div class="topic-metadata">

**Author:** [@saif3r](https://discuss.elastic.co/u/saif3r)\
**Replies:** 0\
**Last updated:** [April 15, 2019, 8:40am UTC](https://discuss.elastic.co/t/jdbc-using-parameter-for-both-sql-statement-and-last-run-metadata-path/176860 "2019-04-15T08:40:25Z")

</div>

Hello, I'm querying several different databases with the exact same sql statement. The only difference between them is the database name and sql\_last\_value based on ID. Is it possible to use one jdbc input so that it wi…

---

## [6.7.1 - Unexpected Log Destination](https://discuss.elastic.co/t/6-7-1-unexpected-log-destination/176650)

<div class="topic-metadata">

**Author:** [@dawiro](https://discuss.elastic.co/u/dawiro)\
**Replies:** 2\
**Last updated:** [April 15, 2019, 8:14am UTC](https://discuss.elastic.co/t/6-7-1-unexpected-log-destination/176650 "2019-04-15T08:14:24Z")

</div>

Hi, I have installed logstash 6.7.1 and while logs are being written to /var/log/logstash, they're also being written to syslog. How do I disable this behaviour? OS is ubuntu bionic... Regards, D

---

## [Geoip.location not showing in Kibana](https://discuss.elastic.co/t/geoip-location-not-showing-in-kibana/175130)

<div class="topic-metadata">

**Author:** [@miles2smiles](https://discuss.elastic.co/u/miles2smiles)\
**Replies:** 5\
**Last updated:** [April 15, 2019, 6:54am UTC](https://discuss.elastic.co/t/geoip-location-not-showing-in-kibana/175130 "2019-04-15T06:54:33Z")

</div>

I am trying to create tile map but I am getting error " No Compatible Fields: The prod-\* index pattern does not contain any of the following field types: geo\_point" Below is my logstash config file. input { rabbitmq …

---

## [Logstash stopped working (sincedb is not updated) after upgrade from 6.2.4 –\> 6.4.0 (Update 6.5 doesn't work also)](https://discuss.elastic.co/t/logstash-stopped-working-sincedb-is-not-updated-after-upgrade-from-6-2-4-6-4-0-update-6-5-doesnt-work-also/161335)

<div class="topic-metadata">

**Author:** [@RonGros](https://discuss.elastic.co/u/RonGros)\
**Replies:** 67\
**Last updated:** [April 15, 2019, 6:33am UTC](https://discuss.elastic.co/t/logstash-stopped-working-sincedb-is-not-updated-after-upgrade-from-6-2-4-6-4-0-update-6-5-doesnt-work-also/161335 "2019-04-15T06:33:31Z")

</div>

Hi, In continuation of the following topic which is now closed: We found no solution for that. moreover - we just tried installing Logstash 6.5 on a new machine and we get the same results - the logstash runs, but sin…

---

## [Logstash 7.0 document\_type deprecated setting warning](https://discuss.elastic.co/t/logstash-7-0-document-type-deprecated-setting-warning/176804)

<div class="topic-metadata">

**Author:** [@VamPikmin](https://discuss.elastic.co/u/VamPikmin)\
**Replies:** 0\
**Last updated:** [April 15, 2019, 12:43am UTC](https://discuss.elastic.co/t/logstash-7-0-document-type-deprecated-setting-warning/176804 "2019-04-15T00:43:14Z")

</div>

I've upgraded to 7.0 today and I've been seeing this for a while now in my logstash log. Can someone please tell me what I should do to fix this? \[ 2019-04-15T10:37:11,050\]\[WARN \]\[logstash.outputs.elasticsearch\] You a…

---

## [Logstash - XML Filter not working properly](https://discuss.elastic.co/t/logstash-xml-filter-not-working-properly/176758)

<div class="topic-metadata">

**Author:** [@miguelfreitas93](https://discuss.elastic.co/u/miguelfreitas93)\
**Replies:** 14\
**Last updated:** [April 14, 2019, 10:07pm UTC](https://discuss.elastic.co/t/logstash-xml-filter-not-working-properly/176758 "2019-04-14T22:07:47Z")

</div>

Hi, I'm trying to parse a very complex XML with nested arrays into logstash, but for some reason XML filter only parses right 2 objects, discarding many of them. Here below my configuration for Logstash: input { …

---

## [Grok filter for different inputs](https://discuss.elastic.co/t/grok-filter-for-different-inputs/176797)

<div class="topic-metadata">

**Author:** [@Mysterious\_Ze](https://discuss.elastic.co/u/Mysterious_Ze)\
**Replies:** 0\
**Last updated:** [April 14, 2019, 7:59pm UTC](https://discuss.elastic.co/t/grok-filter-for-different-inputs/176797 "2019-04-14T19:59:26Z")

</div>

Hello On one of my web server, i have also installed fail2ban. My question is :slight\_smile: how to configure filebeat for several inputs on my logstash, how to assign appropriate grok filters for apache and the fail…

---

## [Logstash pipeline using elasticsearch filter experiencing low performance](https://discuss.elastic.co/t/logstash-pipeline-using-elasticsearch-filter-experiencing-low-performance/172739)

<div class="topic-metadata">

**Author:** [@JeroenK](https://discuss.elastic.co/u/JeroenK)\
**Replies:** 1\
**Last updated:** [April 14, 2019, 2:58pm UTC](https://discuss.elastic.co/t/logstash-pipeline-using-elasticsearch-filter-experiencing-low-performance/172739 "2019-04-14T14:58:59Z")

</div>

Hi there, I'm probably having some infrastructural problem, but I sure can use some help in finding out what I need to look at and fix. I'm trying to 'enrich' data from one index with data from another index using logs…

---

## [Loop on list topic](https://discuss.elastic.co/t/loop-on-list-topic/176762)

<div class="topic-metadata">

**Author:** [@arp220](https://discuss.elastic.co/u/arp220)\
**Replies:** 1\
**Last updated:** [April 14, 2019, 12:13pm UTC](https://discuss.elastic.co/t/loop-on-list-topic/176762 "2019-04-14T12:13:30Z")

</div>

Hi. I have 30 topics kafka from server1 and I want to send all these topics to server2. my input config is: input { kafka { bootstrap\_servers =\> "192.168.1.101:9092" group\_id =\> "all\_topic" …

---

## [Websocket input plugin](https://discuss.elastic.co/t/websocket-input-plugin/176744)

<div class="topic-metadata">

**Author:** [@Mohamed\_eshaftri](https://discuss.elastic.co/u/Mohamed_eshaftri)\
**Replies:** 0\
**Last updated:** [April 13, 2019, 5:46pm UTC](https://discuss.elastic.co/t/websocket-input-plugin/176744 "2019-04-13T17:46:17Z")

</div>

hello, i need to use Websocket input plugin for indexing json data in to elasticsearch. this my code in javascript. \<script\> var ws = new WebSocket(" wss://ws.blockchain.info/inv"); ws.onopen = functio…

---

## [Netflow in Logstash cannot output to kibana - not sure correctly installed netflow](https://discuss.elastic.co/t/netflow-in-logstash-cannot-output-to-kibana-not-sure-correctly-installed-netflow/176735)

<div class="topic-metadata">

**Author:** [@Pradeepan](https://discuss.elastic.co/u/Pradeepan)\
**Replies:** 0\
**Last updated:** [April 13, 2019, 12:19pm UTC](https://discuss.elastic.co/t/netflow-in-logstash-cannot-output-to-kibana-not-sure-correctly-installed-netflow/176735 "2019-04-13T12:19:45Z")

</div>

Hi Techs Installed Logstash , Elsaticsearch and Kibana. Have a Kibana web is good. Installed netflow in logstash and cannot see any outputs in Kibana. I am not that good for linux....Please help me

---

## [%{POSINT} doesn't match 0 (zero)](https://discuss.elastic.co/t/posint-doesnt-match-0-zero/176733)

<div class="topic-metadata">

**Author:** [@elasticSmash](https://discuss.elastic.co/u/elasticSmash)\
**Replies:** 0\
**Last updated:** [April 13, 2019, 11:22am UTC](https://discuss.elastic.co/t/posint-doesnt-match-0-zero/176733 "2019-04-13T11:22:05Z")

</div>

I spent about 15 minutes trying to figure out why my match pattern didn't match only to realize that the core pattern %{POSINT} doesn't match 0. This was a bit unexpected and I can already hear the arguments as to why i…

[Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=564)

[Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=566)