# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=740 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 741 --- ## [Problem with parsing multiline filter plugin](https://discuss.elastic.co/t/problem-with-parsing-multiline-filter-plugin/135906)
**Author:** [@RamyaGowda](https://discuss.elastic.co/u/RamyaGowda)\ **Replies:** 16\ **Last updated:** [June 15, 2018, 11:21am UTC](https://discuss.elastic.co/t/problem-with-parsing-multiline-filter-plugin/135906 "2018-06-15T11:21:52Z")
Hi i have to parse netstat ("Network Statistics") output using logstash sample entries will be like bellow TIME: 16:14:30.14 Active Connections Proto Local Address Foreign Address State TCP 0.… --- ## [No logs generated after upgrade to 6.3.0.1](https://discuss.elastic.co/t/no-logs-generated-after-upgrade-to-6-3-0-1/136036)
**Author:** [@sru](https://discuss.elastic.co/u/sru)\ **Replies:** 0\ **Last updated:** [June 15, 2018, 6:24am UTC](https://discuss.elastic.co/t/no-logs-generated-after-upgrade-to-6-3-0-1/136036 "2018-06-15T06:24:25Z")
since the upgrade from 6.2 to 6.3 there are no log entries in /var/log/logstash/logstash-plain.log Logpath in /etc/logstash/logstash.yml path.logs: /var/log/logstash with the new queue.drain:true in the same file fou… --- ## [How to pass selected fileds from an XML source to a file/message](https://discuss.elastic.co/t/how-to-pass-selected-fileds-from-an-xml-source-to-a-file-message/135943)
**Author:** [@FBRA](https://discuss.elastic.co/u/FBRA)\ **Replies:** 8\ **Last updated:** [June 15, 2018, 8:22am UTC](https://discuss.elastic.co/t/how-to-pass-selected-fileds-from-an-xml-source-to-a-file-message/135943 "2018-06-15T08:22:32Z")
I'm parsing an XML file and would like to create message from some of the fields: Here is the part from the config file: filter { xml { source =\> "inxml" store\_xml =\> true target =\> "message" xpath =\> \[ "/QMGRD… --- ## [Logstash is not parsing the data base on config](https://discuss.elastic.co/t/logstash-is-not-parsing-the-data-base-on-config/135930)
**Author:** [@Pradana](https://discuss.elastic.co/u/Pradana)\ **Replies:** 10\ **Last updated:** [June 15, 2018, 6:56am UTC](https://discuss.elastic.co/t/logstash-is-not-parsing-the-data-base-on-config/135930 "2018-06-15T06:56:21Z")
I have below config : filter{ if \[type\] == "log" { grok { match =\> { "message" =\> "%{TIMESTAMP\_ISO8601:logdate} %{LOGLEVEL:debugtype} %{DATA:source} %{TIMESTAMP\_ISO8601:smsdate},%{WORD:sourceaddr},%{NUMBER:addrton},%… --- ## [Logstash Parsing](https://discuss.elastic.co/t/logstash-parsing/136010)
**Author:** [@jorgep92](https://discuss.elastic.co/u/jorgep92)\ **Replies:** 1\ **Last updated:** [June 15, 2018, 4:34am UTC](https://discuss.elastic.co/t/logstash-parsing/136010 "2018-06-15T04:34:03Z")
Hello, I have a question, I am using ELK. In logstash, I have created my own parsings logs and I want to ask you if I can re-process the previous logs but using my new parsings. Thank you very much --- ## [Logstash errors on filebeat received apache logs](https://discuss.elastic.co/t/logstash-errors-on-filebeat-received-apache-logs/135808)
**Author:** [@ocabj](https://discuss.elastic.co/u/ocabj)\ **Replies:** 6\ **Last updated:** [June 15, 2018, 1:46am UTC](https://discuss.elastic.co/t/logstash-errors-on-filebeat-received-apache-logs/135808 "2018-06-15T01:46:01Z")
After an upgrade from 6.2 to 6.3, I am now getting indexing errors for apache logs. \[2018-06-13T16:12:18,782\]\[WARN \]\[logstash.outputs.elasticsearch\] Could not index event to Elasticsearch. {:status=\>400, :action=\>\["inde… --- ## [New to the stack, http logstash input question](https://discuss.elastic.co/t/new-to-the-stack-http-logstash-input-question/135981)
**Author:** [@mcarpenterjr](https://discuss.elastic.co/u/mcarpenterjr)\ **Replies:** 3\ **Last updated:** [June 15, 2018, 1:34am UTC](https://discuss.elastic.co/t/new-to-the-stack-http-logstash-input-question/135981 "2018-06-15T01:34:08Z")
Completely new to the elasticsearch-stack, having some trouble understanding where and the format logstash plugins are configured. I was able to install the logstash\_http\_input plugin but I am having trouble understandi… --- ## [Attempting to create a logstash plugin - running into major issues](https://discuss.elastic.co/t/attempting-to-create-a-logstash-plugin-running-into-major-issues/136011)
**Author:** [@crazyphil](https://discuss.elastic.co/u/crazyphil)\ **Replies:** 0\ **Last updated:** [June 14, 2018, 11:38pm UTC](https://discuss.elastic.co/t/attempting-to-create-a-logstash-plugin-running-into-major-issues/136011 "2018-06-14T23:38:27Z")
Looking for several pointers here, as I'm trying to create a logstash codec plugin to solve a problem in one of our data streams. First, the data stream in question has two or 4 "binary data" bytes that it mixes in with… --- ## [Parsing KV where value is a list](https://discuss.elastic.co/t/parsing-kv-where-value-is-a-list/134831)
**Author:** [@thegrockq](https://discuss.elastic.co/u/thegrockq)\ **Replies:** 3\ **Last updated:** [June 14, 2018, 9:33pm UTC](https://discuss.elastic.co/t/parsing-kv-where-value-is-a-list/134831 "2018-06-14T21:33:06Z")
I have a log I can mostly parse w/ the KV filter. I have a few fields that aren't parsing correctly though as the values are a list instead of a single value. For example: ..., rand\_key: rand\_value, IP Address: (192.168… --- ## [Send one event in one packet](https://discuss.elastic.co/t/send-one-event-in-one-packet/135898)
**Author:** [@Raul\_Uria](https://discuss.elastic.co/u/Raul_Uria)\ **Replies:** 4\ **Last updated:** [June 14, 2018, 8:01pm UTC](https://discuss.elastic.co/t/send-one-event-in-one-packet/135898 "2018-06-14T20:01:14Z")
Hi all, I´m trying to use logstash to collect some data from a legacy shipper using tcp connection. Each event is in a tcp packet. I want to filter some features and output the result event, but I need to put just one… --- ## [Logstash Log: Could not retrieve remote IP address for beats input](https://discuss.elastic.co/t/logstash-log-could-not-retrieve-remote-ip-address-for-beats-input/133280)
**Author:** [@Maekee](https://discuss.elastic.co/u/Maekee)\ **Replies:** 2\ **Last updated:** [June 14, 2018, 7:05pm UTC](https://discuss.elastic.co/t/logstash-log-could-not-retrieve-remote-ip-address-for-beats-input/133280 "2018-06-14T19:05:55Z")
Hi, Need help! All of a sudden i started getting these in my Logstash log... ALOT!! Like 2 every millisecond I see that events are coming in from my beat agents, but not in the pace i usually see. Have not changed an… --- ## [Duplicate entries in elasticsearch with fingerprint and uuid](https://discuss.elastic.co/t/duplicate-entries-in-elasticsearch-with-fingerprint-and-uuid/135909)
**Author:** [@christian\_ha](https://discuss.elastic.co/u/christian_ha)\ **Replies:** 3\ **Last updated:** [June 14, 2018, 6:42pm UTC](https://discuss.elastic.co/t/duplicate-entries-in-elasticsearch-with-fingerprint-and-uuid/135909 "2018-06-14T18:42:54Z")
Hello together, i try to send syslog data from rsyslod via json to logstash and this is working nearly as expected. Unfortunately i can see every event twice in elasticsearch. First i double checked if logstash send th… --- ## [This driver is not configured for integrated authentication](https://discuss.elastic.co/t/this-driver-is-not-configured-for-integrated-authentication/135940)
**Author:** [@Sourav\_Saha](https://discuss.elastic.co/u/Sourav_Saha)\ **Replies:** 2\ **Last updated:** [June 14, 2018, 5:59pm UTC](https://discuss.elastic.co/t/this-driver-is-not-configured-for-integrated-authentication/135940 "2018-06-14T17:59:16Z")
I am trying to connect SQL SERVER in window authentication mode from logstash and my connection string is given below: input { jdbc { jdbc\_driver\_class =\> "com.microsoft.sqlserver.jdbc.SQLServerDriver" jdbc\_driver\_li… --- ## [Create filter with this content .help me](https://discuss.elastic.co/t/create-filter-with-this-content-help-me/135965)
**Author:** [@ngocky](https://discuss.elastic.co/u/ngocky)\ **Replies:** 1\ **Last updated:** [June 14, 2018, 5:40pm UTC](https://discuss.elastic.co/t/create-filter-with-this-content-help-me/135965 "2018-06-14T17:40:17Z")
\#Software: Microsoft Internet Information Services 10.0 #Version: 1.0 #Date: 2018-06-10 09:58:55 #Fields: date time c-ip cs-username s-ip s-port cs-method cs-uri-stem sc-status sc-win32-status sc-substatus x-session x… --- ## [How to extract json out of an event?](https://discuss.elastic.co/t/how-to-extract-json-out-of-an-event/134684)
**Author:** [@amruth](https://discuss.elastic.co/u/amruth)\ **Replies:** 8\ **Last updated:** [June 14, 2018, 4:28pm UTC](https://discuss.elastic.co/t/how-to-extract-json-out-of-an-event/134684 "2018-06-14T16:28:27Z")
Hi, Here is my sample event, \<6\>Jun 5 12:12:45 172.25.29.92 doppler\[10\]: time="2018-06-05T16:12:46Z" level=info msg="{ "Level": "Info", "Timestamp": "2018-06-05 14:12:45.1195", "Message": "sample message" } " app\_id=44… --- ## [Help to understand GROK fields in expression](https://discuss.elastic.co/t/help-to-understand-grok-fields-in-expression/135781)
**Author:** [@DHN71](https://discuss.elastic.co/u/DHN71)\ **Replies:** 3\ **Last updated:** [June 14, 2018, 3:29pm UTC](https://discuss.elastic.co/t/help-to-understand-grok-fields-in-expression/135781 "2018-06-14T15:29:44Z")
Sample Data: 2018-05-22 15:34:58.068 UTC INFO name=StationRecord.unAssociatedGauge, value=32 The GROK expression needs to write everything after "name=" to a new field called "record". If I only use the expression: (… --- ## [Remove fields that match regex](https://discuss.elastic.co/t/remove-fields-that-match-regex/132830)
**Author:** [@M\_B\_I](https://discuss.elastic.co/u/M_B_I)\ **Replies:** 12\ **Last updated:** [June 14, 2018, 3:14pm UTC](https://discuss.elastic.co/t/remove-fields-that-match-regex/132830 "2018-06-14T15:14:49Z")
Hi! Could someone tell me , can I delete a few fields that match regex expression? I get snmp traps and in my filter I set all information that I need to a specific fields, so I don't need anymore fields that starts wi… --- ## [Logstash Refusing Connections After Uninstalling xpack](https://discuss.elastic.co/t/logstash-refusing-connections-after-uninstalling-xpack/135892)
**Author:** [@jarnold](https://discuss.elastic.co/u/jarnold)\ **Replies:** 2\ **Last updated:** [June 14, 2018, 2:05pm UTC](https://discuss.elastic.co/t/logstash-refusing-connections-after-uninstalling-xpack/135892 "2018-06-14T14:05:17Z")
Hello all, I've recently uninstalled xpack from my stack as my trial had expired - Elasticsearch & kibana are running fine, but logstash isn't receiving any data. Was working fine before I removed xpack. Trying a curl … --- ## [Create Mappings for DB output columns in Logstash configurations file](https://discuss.elastic.co/t/create-mappings-for-db-output-columns-in-logstash-configurations-file/135917)
**Author:** [@GangamPrashanth](https://discuss.elastic.co/u/GangamPrashanth)\ **Replies:** 1\ **Last updated:** [June 14, 2018, 2:04pm UTC](https://discuss.elastic.co/t/create-mappings-for-db-output-columns-in-logstash-configurations-file/135917 "2018-06-14T14:04:48Z")
Hello everyone, I'm new to logstash and elastic search.Please help me out on the below We are working on loading data from SQL Server database onto elasticsearch.(Version 6.2.4) The below is the simple configuration i… --- ## [How to store logs to json files that can be read in later](https://discuss.elastic.co/t/how-to-store-logs-to-json-files-that-can-be-read-in-later/135599)
**Author:** [@sanketshinde](https://discuss.elastic.co/u/sanketshinde)\ **Replies:** 4\ **Last updated:** [June 14, 2018, 1:38pm UTC](https://discuss.elastic.co/t/how-to-store-logs-to-json-files-that-can-be-read-in-later/135599 "2018-06-14T13:38:51Z")
I am sourcing some documents from a remote cluster and storing them in files on a local machine. I would like to store the documents as a valid json that I can later read in, say for example, in python. I used the file… --- ## [Send logstash events to log file problem](https://discuss.elastic.co/t/send-logstash-events-to-log-file-problem/135908)
**Author:** [@dyl](https://discuss.elastic.co/u/dyl)\ **Replies:** 1\ **Last updated:** [June 14, 2018, 12:10pm UTC](https://discuss.elastic.co/t/send-logstash-events-to-log-file-problem/135908 "2018-06-14T12:10:23Z")
Hey, I use logstash to parse filebeat events and then send these logs to a .txt file This is my pipeline : input { beats { port =\> 5044 } } filter { grok { match =\> { "message" =\> \[ "%{WORD:TYPE};%{D… --- ## [Jdbc Static Loader Schedule Giving Error](https://discuss.elastic.co/t/jdbc-static-loader-schedule-giving-error/135905)
**Author:** [@MariumHassan](https://discuss.elastic.co/u/MariumHassan)\ **Replies:** 0\ **Last updated:** [June 14, 2018, 11:21am UTC](https://discuss.elastic.co/t/jdbc-static-loader-schedule-giving-error/135905 "2018-06-14T11:21:39Z")
Hi, I am getting an error when I try to add loader\_schedule setting in my configuration file. The script works fine without this setting but adding this gives me an error shown below: \[Capture\] Following is my configu… --- ## [Logstash-input-jms and solace issue](https://discuss.elastic.co/t/logstash-input-jms-and-solace-issue/133899)
**Author:** [@sachin\_frayne](https://discuss.elastic.co/u/sachin_frayne)\ **Replies:** 1\ **Last updated:** [June 4, 2018, 8:58pm UTC](https://discuss.elastic.co/t/logstash-input-jms-and-solace-issue/133899 "2018-06-04T20:58:07Z")
I am trying to connect the logstash jms input plugin to a Solace PubSub topic with no luck. The resources I have used: Solace PubSub Docker in amazon linux: https://docs.solace.com/Solace-SW-Broker-Set-Up/Docker-Conta… --- ## [Multi line Handling of Java Exception Stacktrace](https://discuss.elastic.co/t/multi-line-handling-of-java-exception-stacktrace/135885)
**Author:** [@Sandeep\_Singh](https://discuss.elastic.co/u/Sandeep_Singh)\ **Replies:** 1\ **Last updated:** [June 14, 2018, 10:00am UTC](https://discuss.elastic.co/t/multi-line-handling-of-java-exception-stacktrace/135885 "2018-06-14T10:00:17Z")
Hi, We are using the ELK stack to process logs from Docker containers using Filebeat as the log pusher. And we are using json-file default logger for Docker. I have tried putting patterns in filebeat configuration, in… --- ## [Logstash : pipeline is running but not generating any output file](https://discuss.elastic.co/t/logstash-pipeline-is-running-but-not-generating-any-output-file/135871)
**Author:** [@priya1](https://discuss.elastic.co/u/priya1)\ **Replies:** 1\ **Last updated:** [June 14, 2018, 9:21am UTC](https://discuss.elastic.co/t/logstash-pipeline-is-running-but-not-generating-any-output-file/135871 "2018-06-14T09:21:31Z")
Hi Team, Please find the below details : input{ file { path =\> "C:\\Users\\priya\_shukla\\Desktop\\SAPLogsERR\*.log" start\_position =\> "beginning" sincedb\_path =\> "NUL" } } filter { grok { match =\> \[ "message", "%{D… --- ## [Logstash connection refused (using searchguard + monitoring x pack)](https://discuss.elastic.co/t/logstash-connection-refused-using-searchguard-monitoring-x-pack/135874)
**Author:** [@kimfut](https://discuss.elastic.co/u/kimfut)\ **Replies:** 1\ **Last updated:** [June 14, 2018, 8:59am UTC](https://discuss.elastic.co/t/logstash-connection-refused-using-searchguard-monitoring-x-pack/135874 "2018-06-14T08:59:11Z")
hello everyone , i am using elk stack via docker , for security access i use searchguard tool , then for monitor my stack i added x pack plugin with making security and other service that not free as false ( xpack.mon… --- ## [Could not split in multiline Logs in Logstash](https://discuss.elastic.co/t/could-not-split-in-multiline-logs-in-logstash/135681)
**Author:** [@umang\_bhargava](https://discuss.elastic.co/u/umang_bhargava)\ **Replies:** 3\ **Last updated:** [June 14, 2018, 8:38am UTC](https://discuss.elastic.co/t/could-not-split-in-multiline-logs-in-logstash/135681 "2018-06-14T08:38:15Z")
Greetings, I am trying to create filter for my log file. My log file is : =-=-=-=-=-=-=-=- Timestamp: Thursday, April 19, 2018 2:48:54 AM Message: ID: 7 An exception of type 'System.Exception' occurred and was caught.… --- ## [Logstash: jdbc input plugin with MongoDb](https://discuss.elastic.co/t/logstash-jdbc-input-plugin-with-mongodb/135866)
**Author:** [@KamalHr](https://discuss.elastic.co/u/KamalHr)\ **Replies:** 0\ **Last updated:** [June 14, 2018, 8:01am UTC](https://discuss.elastic.co/t/logstash-jdbc-input-plugin-with-mongodb/135866 "2018-06-14T08:01:58Z")
I'm trying to configure a pipeline between MongoDb and Elasticsearch using logstash and the jdbc input plugin. I tried to use Unity Mongo Jdbc driver, and dbschema's driver but nothing works, and I got the error: Seque… --- ## [Logstash is not connecting to elasticsearch over ssl](https://discuss.elastic.co/t/logstash-is-not-connecting-to-elasticsearch-over-ssl/135852)
**Author:** [@ashok9177](https://discuss.elastic.co/u/ashok9177)\ **Replies:** 0\ **Last updated:** [June 14, 2018, 7:41am UTC](https://discuss.elastic.co/t/logstash-is-not-connecting-to-elasticsearch-over-ssl/135852 "2018-06-14T07:41:01Z")
Hello, I have installed elk setup 6.2.2 including x-pack, I have configured elasticsearch to use SSL since the problem started my logstash is not connecting to elasticsearch, it always trying to connect localhost:9200 ev… --- ## [Creating fields from grok](https://discuss.elastic.co/t/creating-fields-from-grok/135824)
**Author:** [@bobbytan](https://discuss.elastic.co/u/bobbytan)\ **Replies:** 3\ **Last updated:** [June 14, 2018, 6:57am UTC](https://discuss.elastic.co/t/creating-fields-from-grok/135824 "2018-06-14T06:57:39Z")
I have used grok to pull fields from logs as shown below: filter { grok { match =\> {"message"=\> "%{DATE\_EU:logdate:date} %{TIME:logtime:date} %{GREEDYDATA}" } } } However the logdate and logtime fields are shown in… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=739) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=741)