# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=340 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 341 --- ## [Getting GeoIP field for NGINX Logs](https://discuss.elastic.co/t/getting-geoip-field-for-nginx-logs/237903)
**Author:** [@jaysbeekay](https://discuss.elastic.co/u/jaysbeekay)\ **Replies:** 1\ **Last updated:** [June 20, 2020, 9:45am UTC](https://discuss.elastic.co/t/getting-geoip-field-for-nginx-logs/237903 "2020-06-20T09:45:22Z")
Hi, I am having trouble trying to get the location from NGINX logs to be shown as a GeoIP (instead of a float) within Kibana to visualise on a dashboard I am currently filebeats to ship logs from my NGINX server to Log… --- ## [Logstash.inputs.tcp: Error in Netty pipeline: java.io.IOException: Connection reset by peer](https://discuss.elastic.co/t/logstash-inputs-tcp-error-in-netty-pipeline-java-io-ioexception-connection-reset-by-peer/237868)
**Author:** [@Kushal\_Zamkade1](https://discuss.elastic.co/u/Kushal_Zamkade1)\ **Replies:** 0\ **Last updated:** [June 19, 2020, 8:24pm UTC](https://discuss.elastic.co/t/logstash-inputs-tcp-error-in-netty-pipeline-java-io-ioexception-connection-reset-by-peer/237868 "2020-06-19T20:24:08Z")
Getting this error frequently during using TCP input plugin in latest logstash version 7.8 and tcp plugin version 6.0.5 --- ## [Logstash restarts frequently](https://discuss.elastic.co/t/logstash-restarts-frequently/237836)
**Author:** [@avinjos](https://discuss.elastic.co/u/avinjos)\ **Replies:** 3\ **Last updated:** [June 19, 2020, 6:25pm UTC](https://discuss.elastic.co/t/logstash-restarts-frequently/237836 "2020-06-19T18:25:20Z")
I have installed Logstash from RPM and when I stared it with the default config, I see it is restarting frequently. --- ## [Elasticsearch Same document repeating multiple times with filebeat to logstash output](https://discuss.elastic.co/t/elasticsearch-same-document-repeating-multiple-times-with-filebeat-to-logstash-output/237833)
**Author:** [@Bhanu\_Praveen](https://discuss.elastic.co/u/Bhanu_Praveen)\ **Replies:** 0\ **Last updated:** [June 19, 2020, 4:30pm UTC](https://discuss.elastic.co/t/elasticsearch-same-document-repeating-multiple-times-with-filebeat-to-logstash-output/237833 "2020-06-19T16:30:23Z")
Hello, Configured Filebeat to parse inputs from log file: filebeat.inputs: type: log enabled: true recursive\_glob.enabled: true close\_inactive: 1h close\_removed: true clean\_inactive: 5h clean\_removed: true ign… --- ## [Logstash Error: LoadError: no such file to load -- pg](https://discuss.elastic.co/t/logstash-error-loaderror-no-such-file-to-load-pg/237831)
**Author:** [@Arnav\_Parashar](https://discuss.elastic.co/u/Arnav_Parashar)\ **Replies:** 2\ **Last updated:** [June 19, 2020, 4:16pm UTC](https://discuss.elastic.co/t/logstash-error-loaderror-no-such-file-to-load-pg/237831 "2020-06-19T16:16:56Z")
Hi, I am trying to sync a postgres database with elastic search. My file is the following: input { jdbc { jdbc\_driver\_library =\> "~/logstash/postgresql-42.2.14.jar" jdbc\_driver\_class =\> "org.p… --- ## [Same log file contains different format of logs data. How can I parse it?](https://discuss.elastic.co/t/same-log-file-contains-different-format-of-logs-data-how-can-i-parse-it/237742)
**Author:** [@TUSHAR\_Sinha](https://discuss.elastic.co/u/TUSHAR_Sinha)\ **Replies:** 8\ **Last updated:** [June 19, 2020, 2:49pm UTC](https://discuss.elastic.co/t/same-log-file-contains-different-format-of-logs-data-how-can-i-parse-it/237742 "2020-06-19T14:49:19Z")
Hi all, I want to parse the logfile which has a different pattern in almost every line. The main difference is actually the length of each line. Suppose my logs have a named variable, in first log it may contain three n… --- ## [Logstash elasticsearch filter causing high CPU usage](https://discuss.elastic.co/t/logstash-elasticsearch-filter-causing-high-cpu-usage/237693)
**Author:** [@Andrew22](https://discuss.elastic.co/u/Andrew22)\ **Replies:** 2\ **Last updated:** [June 19, 2020, 2:17pm UTC](https://discuss.elastic.co/t/logstash-elasticsearch-filter-causing-high-cpu-usage/237693 "2020-06-19T14:17:26Z")
Hello, like the title says I am using the elasticsearch filter in logstash and it is causing high CPU usage. I am wondering if there is a way to cache the looked up results so that it may reduce the CPU usage I am seein… --- ## [Logstash start successfully but no result](https://discuss.elastic.co/t/logstash-start-successfully-but-no-result/237611)
**Author:** [@Katia](https://discuss.elastic.co/u/Katia)\ **Replies:** 18\ **Last updated:** [June 19, 2020, 12:55pm UTC](https://discuss.elastic.co/t/logstash-start-successfully-but-no-result/237611 "2020-06-19T12:55:18Z")
Hello, I really need help importing my log files. If I create a file right now, this will be imported successfully, but if I try to import a file that already exists since about 11 hours, logstash will not show any out… --- ## [Kerberos support in Jdbc input plugin](https://discuss.elastic.co/t/kerberos-support-in-jdbc-input-plugin/237801)
**Author:** [@shubhamshd99](https://discuss.elastic.co/u/shubhamshd99)\ **Replies:** 0\ **Last updated:** [June 19, 2020, 12:05pm UTC](https://discuss.elastic.co/t/kerberos-support-in-jdbc-input-plugin/237801 "2020-06-19T12:05:47Z")
Hi Guys, Earlier jdbc input plugin did not support kerberos authentication!! So has kerberos support for database connection through jdbc input plugin been added or not?? --- ## [How to make output to two index using logstash](https://discuss.elastic.co/t/how-to-make-output-to-two-index-using-logstash/237783)
**Author:** [@Mohan\_vel](https://discuss.elastic.co/u/Mohan_vel)\ **Replies:** 1\ **Last updated:** [June 19, 2020, 11:25am UTC](https://discuss.elastic.co/t/how-to-make-output-to-two-index-using-logstash/237783 "2020-06-19T11:25:29Z")
Hi, I have established connection from filebeat to logstash with below configuration. It is working fine. input { beats { port =\> 5044 } } filter { } output { … --- ## [Netflow and Syslog router data point to Logstash or to an intermediate Syslog-NG server?](https://discuss.elastic.co/t/netflow-and-syslog-router-data-point-to-logstash-or-to-an-intermediate-syslog-ng-server/237640)
**Author:** [@jelocabral](https://discuss.elastic.co/u/jelocabral)\ **Replies:** 3\ **Last updated:** [June 19, 2020, 7:34am UTC](https://discuss.elastic.co/t/netflow-and-syslog-router-data-point-to-logstash-or-to-an-intermediate-syslog-ng-server/237640 "2020-06-19T07:34:55Z")
Hi people, please I need to know this: If I want to view in Kibana the Netflow and Syslog data from a Cisco router, I need to know which of the following choices are correct: I have to send the Netflow and Syslog dat… --- ## [Dateparse failure using the date filter](https://discuss.elastic.co/t/dateparse-failure-using-the-date-filter/237692)
**Author:** [@marco2005](https://discuss.elastic.co/u/marco2005)\ **Replies:** 2\ **Last updated:** [June 19, 2020, 9:34am UTC](https://discuss.elastic.co/t/dateparse-failure-using-the-date-filter/237692 "2020-06-19T09:34:22Z")
I have a log line Thu Jun 18 08:01:42 CEST 2020|{"timestamp":1592460102635,"caller":"test.customer@customer1","action":"LOGIN","message":"ID000066 User \[test.customer@customer1\] logged in successfully."} This is my fil… --- ## [Filtering and output the first item to elastic](https://discuss.elastic.co/t/filtering-and-output-the-first-item-to-elastic/237630)
**Author:** [@kileou78](https://discuss.elastic.co/u/kileou78)\ **Replies:** 4\ **Last updated:** [June 19, 2020, 8:51am UTC](https://discuss.elastic.co/t/filtering-and-output-the-first-item-to-elastic/237630 "2020-06-19T08:51:59Z")
Hi everybody I'm collecting using an API and filebeat a list of computer's status OK or KO I would like to store on elastic a global status for this group of computer My list is sorted by state, "0" for OK and "2" for… --- ## [How logstash receive multiple topics from kafka](https://discuss.elastic.co/t/how-logstash-receive-multiple-topics-from-kafka/237605)
**Author:** [@Lan\_Lynn](https://discuss.elastic.co/u/Lan_Lynn)\ **Replies:** 2\ **Last updated:** [June 19, 2020, 6:51am UTC](https://discuss.elastic.co/t/how-logstash-receive-multiple-topics-from-kafka/237605 "2020-06-19T06:51:47Z")
I'm trying to use logstash to receive data from kafka. I'm having this configurations in Kafka below, two topics and one groupID. "DeviceTopic": "device", //Topic for device-gruoped messages "TaskTopic": "task", //Top… --- ## [Logstash doesn't start](https://discuss.elastic.co/t/logstash-doesnt-start/236919)
**Author:** [@Uzzi](https://discuss.elastic.co/u/Uzzi)\ **Replies:** 50\ **Last updated:** [June 19, 2020, 6:47am UTC](https://discuss.elastic.co/t/logstash-doesnt-start/236919 "2020-06-19T06:47:45Z")
Hi,I'm debugging Logstash istance: /usr/share/logstash/bin/logstash --config.test\_and\_exit --path.settings --config.debug -f /etc/logstash/conf.d/01-wazuh.conf ---\> Config Validation Result: OK. Exiting Logstash /us… --- ## [\_dateparsefailure csv date time YYYY-MM-dd HH:mm:ss](https://discuss.elastic.co/t/dateparsefailure-csv-date-time-yyyy-mm-dd-hhss/237725)
**Author:** [@gonzapata](https://discuss.elastic.co/u/gonzapata)\ **Replies:** 0\ **Last updated:** [June 19, 2020, 3:53am UTC](https://discuss.elastic.co/t/dateparsefailure-csv-date-time-yyyy-mm-dd-hhss/237725 "2020-06-19T03:53:55Z")
Hi, I manage to parse message field from a csv to several fields from filebeat to logstash, I want to match a date format (like 2020-06-18 22:36:21) but when I use date and match filter it give me \_dateparsefailure I tr… --- ## [Logstash Hangs After Too Many 400's on Bulk Requests to ES](https://discuss.elastic.co/t/logstash-hangs-after-too-many-400s-on-bulk-requests-to-es/237612)
**Author:** [@scott\_stash](https://discuss.elastic.co/u/scott_stash)\ **Replies:** 2\ **Last updated:** [June 19, 2020, 2:53am UTC](https://discuss.elastic.co/t/logstash-hangs-after-too-many-400s-on-bulk-requests-to-es/237612 "2020-06-19T02:53:35Z")
Intermittently I ran into an issue where Logstash will hang. I think what is happening is that for some reason I start logging "Encountered a returnable error. Will Retry with exponential backoff" and the code is 400. … --- ## [Assitance required in a Grok pattern](https://discuss.elastic.co/t/assitance-required-in-a-grok-pattern/237388)
**Author:** [@Saravana37](https://discuss.elastic.co/u/Saravana37)\ **Replies:** 4\ **Last updated:** [June 19, 2020, 2:50am UTC](https://discuss.elastic.co/t/assitance-required-in-a-grok-pattern/237388 "2020-06-19T02:50:28Z")
Hello All , I am facing some challenges in building the below Grok pattern. %{WORD:Server}%{SPACE}%{WORD:Component}%{SPACE}%{NOTSPACE:ComponentStatus}%{SPACE}%{WORD:CG\_ALIAS}%{SPACE}%{WORD:CompStartMode}%{SPACE}%{NUMBE… --- ## [Stopping logstash always enters failed status](https://discuss.elastic.co/t/stopping-logstash-always-enters-failed-status/237713)
**Author:** [@azteker](https://discuss.elastic.co/u/azteker)\ **Replies:** 0\ **Last updated:** [June 19, 2020, 1:28am UTC](https://discuss.elastic.co/t/stopping-logstash-always-enters-failed-status/237713 "2020-06-19T01:28:01Z")
I stop logstash with "systemctl stop logstash" This is what "systemctl status logstash" returns ● logstash.service - logstash Loaded: loaded (/etc/systemd/system/logstash.service; enabled; vendor preset: disabled) Ac… --- ## [Add geo\_shape polygon coordinates with Logstash](https://discuss.elastic.co/t/add-geo-shape-polygon-coordinates-with-logstash/237651)
**Author:** [@Attic](https://discuss.elastic.co/u/Attic)\ **Replies:** 1\ **Last updated:** [June 18, 2020, 9:38pm UTC](https://discuss.elastic.co/t/add-geo-shape-polygon-coordinates-with-logstash/237651 "2020-06-18T21:38:39Z")
Hello everyone, I'm using the JDBC connector to read IDs from a DB. These IDs are spots around the world that mark the center of a polygon. Is it possible to bind the polygon edge coodinates to a spot-id using Logstas… --- ## [Parse timestamp using date filter plugin](https://discuss.elastic.co/t/parse-timestamp-using-date-filter-plugin/237641)
**Author:** [@marco2005](https://discuss.elastic.co/u/marco2005)\ **Replies:** 7\ **Last updated:** [June 18, 2020, 5:36pm UTC](https://discuss.elastic.co/t/parse-timestamp-using-date-filter-plugin/237641 "2020-06-18T17:36:00Z")
I have a log line: test-process 2020/06/04 13:35:42 process.go:646: INFO: GetTest: X-Request-Id: 14242dqdq2aA, status: HTTP 200 OK Below is my filter: filter { dissect { mapping =\> { … --- ## [Grok Data in curly braces](https://discuss.elastic.co/t/grok-data-in-curly-braces/237680)
**Author:** [@marco2005](https://discuss.elastic.co/u/marco2005)\ **Replies:** 1\ **Last updated:** [June 18, 2020, 4:52pm UTC](https://discuss.elastic.co/t/grok-data-in-curly-braces/237680 "2020-06-18T16:52:46Z")
I have section in the log in curly braces Thu Jun 18 08:01:42 CEST 2020|{"timestamp":1592460102635,"caller":"test.customer@customer1","action":"LOGIN","message":"ID000066 User \[test.customer@customer1\] logged in success… --- ## [Logstash Syntax :: Copy String from One Field into a New Field](https://discuss.elastic.co/t/logstash-syntax-copy-string-from-one-field-into-a-new-field/237661)
**Author:** [@redapplesonly](https://discuss.elastic.co/u/redapplesonly)\ **Replies:** 2\ **Last updated:** [June 18, 2020, 3:29pm UTC](https://discuss.elastic.co/t/logstash-syntax-copy-string-from-one-field-into-a-new-field/237661 "2020-06-18T15:29:58Z")
Hello Logstash Zen Masters, In my data, I have a field named FieldA which contains a string. Later in my filter{} section, I need to access and copy that string into a newly-created field. I've tried a lot of variatio… --- ## [Logstash filter pipeline Dynamic if on fields](https://discuss.elastic.co/t/logstash-filter-pipeline-dynamic-if-on-fields/237581)
**Author:** [@vaclav1](https://discuss.elastic.co/u/vaclav1)\ **Replies:** 1\ **Last updated:** [June 18, 2020, 3:15pm UTC](https://discuss.elastic.co/t/logstash-filter-pipeline-dynamic-if-on-fields/237581 "2020-06-18T15:15:28Z")
Hello, is there any way how to match any field while constructing if in filter pipeline, for example: Tried but it seems it matches half of the fields, and half it does not.. if \[coad\]\[queueStatus\]\[queueByTypes\]\[\*… --- ## [ELK without logstash](https://discuss.elastic.co/t/elk-without-logstash/237119)
**Author:** [@YanisALBIK](https://discuss.elastic.co/u/YanisALBIK)\ **Replies:** 6\ **Last updated:** [June 18, 2020, 2:59pm UTC](https://discuss.elastic.co/t/elk-without-logstash/237119 "2020-06-18T14:59:41Z")
Hi, My team and me are currenlty working on a project to allow our client to view some dashboards in Kibana and we would like to get some elements to validate the architecture we define. We would like to know at which … --- ## [Logstash Syntax :: Check Boolian Value within a Field?](https://discuss.elastic.co/t/logstash-syntax-check-boolian-value-within-a-field/237530)
**Author:** [@redapplesonly](https://discuss.elastic.co/u/redapplesonly)\ **Replies:** 3\ **Last updated:** [June 18, 2020, 2:57pm UTC](https://discuss.elastic.co/t/logstash-syntax-check-boolian-value-within-a-field/237530 "2020-06-18T14:57:19Z")
Hi Logstash Ninjas, In my filter{} section, I think I am creating a field with a Boolean value in it: if SOME CONDITION { mutate { add\_field =\> { "MyFlag" =\> 'true' } convert =\> { "MyFlag" =\> "boolean" } } … --- ## [How to parse nested json through logstash](https://discuss.elastic.co/t/how-to-parse-nested-json-through-logstash/237290)
**Author:** [@Rahul\_rr](https://discuss.elastic.co/u/Rahul_rr)\ **Replies:** 7\ **Last updated:** [June 18, 2020, 2:55pm UTC](https://discuss.elastic.co/t/how-to-parse-nested-json-through-logstash/237290 "2020-06-18T14:55:39Z")
I'm getting the data from sqs queue as a nested json and parsing through logstash. i tried some approaches but it didn't work. Below is my sample nested json. { "service\_event": { "id": 166175, "equipment": { "id": … --- ## [Help with Logstash plug-in development](https://discuss.elastic.co/t/help-with-logstash-plug-in-development/236937)
**Author:** [@byoungman](https://discuss.elastic.co/u/byoungman)\ **Replies:** 1\ **Last updated:** [June 18, 2020, 1:15pm UTC](https://discuss.elastic.co/t/help-with-logstash-plug-in-development/236937 "2020-06-18T13:15:24Z")
I am trying to develop a new logstash plugin in my development Elasticsearch stack environment. I am new to this so this is all a learning experience for me. I will try to be as clear and concise as I can be including as… --- ## [Replace substring and concat rest](https://discuss.elastic.co/t/replace-substring-and-concat-rest/237622)
**Author:** [@ksremo](https://discuss.elastic.co/u/ksremo)\ **Replies:** 2\ **Last updated:** [June 18, 2020, 12:52pm UTC](https://discuss.elastic.co/t/replace-substring-and-concat-rest/237622 "2020-06-18T12:52:27Z")
I have DNS Logs looks like (14)xxxxxxxxxxxx(11)keydelivery(13)mediaservices(7)windows(3)net(0) (4)mediaservices(7)windows(3)net(0) (3)windows.com(0) I want to remove all (...) and replace this with a dot "." if (..)… --- ## [RabbitMQ output plugin](https://discuss.elastic.co/t/rabbitmq-output-plugin/236641)
**Author:** [@Marat](https://discuss.elastic.co/u/Marat)\ **Replies:** 1\ **Last updated:** [June 18, 2020, 12:44pm UTC](https://discuss.elastic.co/t/rabbitmq-output-plugin/236641 "2020-06-18T12:44:32Z")
I am a new user for Logstash, and I want to use Logstash's RabbitMQ output plugin to feed data to RabbitMQ, schema is pretty simple: Nginx -(input)-\>Logstash -(output)-\> RabbitMQ cat /etc/logstash/conf.d/10-nginx-f… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=339) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=341)