# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=439 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 440 --- ## [Logstash Doesnt Connect to ES Cluster after enabling Xpack](https://discuss.elastic.co/t/logstash-doesnt-connect-to-es-cluster-after-enabling-xpack/206800)
**Author:** [@sudhirjunonikar](https://discuss.elastic.co/u/sudhirjunonikar)\ **Replies:** 1\ **Last updated:** [November 28, 2019, 9:43am UTC](https://discuss.elastic.co/t/logstash-doesnt-connect-to-es-cluster-after-enabling-xpack/206800 "2019-11-28T09:43:16Z")
Hi, Please help to identify the configuration issue of Logstash, the Stack version is 7.4.0 Logstash stopped working after enabling xpack. It is three node cluster and it is up. Logstash Debug shows following error, … --- ## [Logstash not updating Data as schedule](https://discuss.elastic.co/t/logstash-not-updating-data-as-schedule/209462)
**Author:** [@saifrehman](https://discuss.elastic.co/u/saifrehman)\ **Replies:** 0\ **Last updated:** [November 26, 2019, 10:04am UTC](https://discuss.elastic.co/t/logstash-not-updating-data-as-schedule/209462 "2019-11-26T10:04:08Z")
@magnusbaeck , I m continuously migrating data from MySQL to elastic search through scheduling via logstash. Everything is working fine and data is indexed in ES. But when i update 4 to 5 records it will update 3 and re… --- ## [Cant parse correctly](https://discuss.elastic.co/t/cant-parse-correctly/209671)
**Author:** [@Alexandros888](https://discuss.elastic.co/u/Alexandros888)\ **Replies:** 2\ **Last updated:** [November 28, 2019, 7:14am UTC](https://discuss.elastic.co/t/cant-parse-correctly/209671 "2019-11-28T07:14:24Z")
Hello, I want to parse the following message: \<155\>##preprod\_AISAMSWEB##: 2019-11-25 16:08:34,876 \[ERROR\] QT\_WebEvents - System.InvalidOperationException: The requested Performance Counter is not a custom counter, it h… --- ## [Converting row to column from logstash](https://discuss.elastic.co/t/converting-row-to-column-from-logstash/209795)
**Author:** [@Sagar\_Mandal](https://discuss.elastic.co/u/Sagar_Mandal)\ **Replies:** 0\ **Last updated:** [November 28, 2019, 6:08am UTC](https://discuss.elastic.co/t/converting-row-to-column-from-logstash/209795 "2019-11-28T06:08:18Z")
I've some data in linear row format and i want to use some filter in my logstash config file so that i can get specific data and convert that into column format. Any solution would do great, Thanks --- ## [Using multiple pipeline ERROR](https://discuss.elastic.co/t/using-multiple-pipeline-error/209672)
**Author:** [@Sagar\_Mandal](https://discuss.elastic.co/u/Sagar_Mandal)\ **Replies:** 2\ **Last updated:** [November 28, 2019, 5:56am UTC](https://discuss.elastic.co/t/using-multiple-pipeline-error/209672 "2019-11-28T05:56:14Z")
Hi, I'm trying to use two configurations file, with two different pipelines as shown below. but i'm not able to fetch data from 2nd config file which is from filebeat. and the config is shown below. I've comment… --- ## [Multiple filebeat servers to configure single logstash](https://discuss.elastic.co/t/multiple-filebeat-servers-to-configure-single-logstash/209740)
**Author:** [@Raavi\_Arun](https://discuss.elastic.co/u/Raavi_Arun)\ **Replies:** 5\ **Last updated:** [November 28, 2019, 4:43am UTC](https://discuss.elastic.co/t/multiple-filebeat-servers-to-configure-single-logstash/209740 "2019-11-28T04:43:19Z")
Hi, I am new to ELK. I am configured single web servers to single logstash in production. Now i am add filebeat in different web servers but i don't know how to configure all web server access log into single logstash. … --- ## [Can't running mutiple JDBC](https://discuss.elastic.co/t/cant-running-mutiple-jdbc/209778)
**Author:** [@sist](https://discuss.elastic.co/u/sist)\ **Replies:** 0\ **Last updated:** [November 28, 2019, 2:13am UTC](https://discuss.elastic.co/t/cant-running-mutiple-jdbc/209778 "2019-11-28T02:13:12Z")
I'm trying to import data from multiple database including mysql and sqlserver to es. With single JDBC it works just fine, but with multiple JDBC it doesn't work and gives no error message. Is this caused by using differ… --- ## [Filter winlog.event\_id in wineventlog](https://discuss.elastic.co/t/filter-winlog-event-id-in-wineventlog/206507)
**Author:** [@lusynda](https://discuss.elastic.co/u/lusynda)\ **Replies:** 3\ **Last updated:** [November 28, 2019, 2:01am UTC](https://discuss.elastic.co/t/filter-winlog-event-id-in-wineventlog/206507 "2019-11-28T02:01:52Z")
Dear all I am trying to filter the window event\_id i want to remove the message field and add my own field But i have try 3 method i can think of like this \[winlog\]\[event\_id\] == 4624 winlog.event\_id == 4624 \[event\_i… --- ## [CSV parsing meaning get rid of logstash csv plugin?](https://discuss.elastic.co/t/csv-parsing-meaning-get-rid-of-logstash-csv-plugin/209770)
**Author:** [@arnaudlemaignen](https://discuss.elastic.co/u/arnaudlemaignen)\ **Replies:** 0\ **Last updated:** [November 27, 2019, 10:39pm UTC](https://discuss.elastic.co/t/csv-parsing-meaning-get-rid-of-logstash-csv-plugin/209770 "2019-11-27T22:39:14Z")
Well I have an issue which is coming with a solution :slight\_smile: but I would like to know if someone has a better idea I am runnging logstash csv plugin 3.0.8 I had to treat unlimited different kind of CSV with diff… --- ## ["\_dateparsefailure" due to Timestamp Cisco IOS](https://discuss.elastic.co/t/dateparsefailure-due-to-timestamp-cisco-ios/208989)
**Author:** [@mfisher](https://discuss.elastic.co/u/mfisher)\ **Replies:** 6\ **Last updated:** [November 27, 2019, 7:41pm UTC](https://discuss.elastic.co/t/dateparsefailure-due-to-timestamp-cisco-ios/208989 "2019-11-27T19:41:33Z")
I used the conf file from here and the pattern below in a pattern file referenced by the conf file. .conf grok section grok { # There are a couple of custom patterns associated with this filter. patterns\_dir =\> \[ "… --- ## [NOt able to discover logtash in kibana](https://discuss.elastic.co/t/not-able-to-discover-logtash-in-kibana/209597)
**Author:** [@anjana1](https://discuss.elastic.co/u/anjana1)\ **Replies:** 2\ **Last updated:** [November 27, 2019, 7:14pm UTC](https://discuss.elastic.co/t/not-able-to-discover-logtash-in-kibana/209597 "2019-11-27T19:14:26Z")
I am using version 7.3 for all elastic search, kibana, logstash and I have an APM agent installed on my application. I am trying to create a file beat and send the input using logstash to elastic search. below is the l… --- ## [Issue with date format](https://discuss.elastic.co/t/issue-with-date-format/209598)
**Author:** [@ShaneLillie\_RAD](https://discuss.elastic.co/u/ShaneLillie_RAD)\ **Replies:** 7\ **Last updated:** [November 27, 2019, 6:46pm UTC](https://discuss.elastic.co/t/issue-with-date-format/209598 "2019-11-27T18:46:12Z")
We're using this date filter: filter { date { … --- ## [Where is \_dateparsefailure?](https://discuss.elastic.co/t/where-is-dateparsefailure/209355)
**Author:** [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Replies:** 6\ **Last updated:** [November 27, 2019, 5:25pm UTC](https://discuss.elastic.co/t/where-is-dateparsefailure/209355 "2019-11-27T17:25:58Z")
I have many column csv files with few thousands lines getting \_dateparsefailure how do I find out which field is throwing this error? I understand that it is one of the date field is doing this. but which one? I have… --- ## [Add json indies another json when uploading data to ES](https://discuss.elastic.co/t/add-json-indies-another-json-when-uploading-data-to-es/209734)
**Author:** [@Anuwa\_A](https://discuss.elastic.co/u/Anuwa_A)\ **Replies:** 0\ **Last updated:** [November 27, 2019, 5:15pm UTC](https://discuss.elastic.co/t/add-json-indies-another-json-when-uploading-data-to-es/209734 "2019-11-27T17:15:06Z")
When uploading data to ES i need to have a json structure like "Countries" : { "Asia" : { "name" : "Srilanka" }, "Africa" : { "name" : "Kenya" } } Aggregate inside Aggregate. is it possible to do this … --- ## [Unable to convert string to integer in log stash in XML](https://discuss.elastic.co/t/unable-to-convert-string-to-integer-in-log-stash-in-xml/209622)
**Author:** [@gangireddy\_l](https://discuss.elastic.co/u/gangireddy_l)\ **Replies:** 1\ **Last updated:** [November 27, 2019, 4:46pm UTC](https://discuss.elastic.co/t/unable-to-convert-string-to-integer-in-log-stash-in-xml/209622 "2019-11-27T16:46:39Z")
Hi Team, I am unable to convert data type in logstash, my configurations are shown below. Logstash filter{ if ("ccn" in \[tags\]) { xml { store\_xml =\> false source =\> "message" xpath =\> \[ "/mv/moid/text()", "ccnc… --- ## [Logstash : ERROR RollingFileManager](https://discuss.elastic.co/t/logstash-error-rollingfilemanager/209660)
**Author:** [@Nabil777](https://discuss.elastic.co/u/Nabil777)\ **Replies:** 1\ **Last updated:** [November 27, 2019, 4:42pm UTC](https://discuss.elastic.co/t/logstash-error-rollingfilemanager/209660 "2019-11-27T16:42:32Z")
Hello, Logstash worked normally until this morning I don't understand why it doesn't work anymore here's the error parts of the logs, it seems that it's related to the log4j2.properties (wich i never modified) and some … --- ## [How to override @timestamp with your log timestamp in logstash](https://discuss.elastic.co/t/how-to-override-timestamp-with-your-log-timestamp-in-logstash/209627)
**Author:** [@SoniyaGupta](https://discuss.elastic.co/u/SoniyaGupta)\ **Replies:** 1\ **Last updated:** [November 27, 2019, 4:39pm UTC](https://discuss.elastic.co/t/how-to-override-timestamp-with-your-log-timestamp-in-logstash/209627 "2019-11-27T16:39:54Z")
I want to override logstash @timestamp: Nov 27, 2019 @ 12:22:22.566 with field (custom field) ts:Wed 27 Nov 2019 6:52:21:514 --- ## [Logstash encoding issue](https://discuss.elastic.co/t/logstash-encoding-issue/209719)
**Author:** [@shiv\_khullar](https://discuss.elastic.co/u/shiv_khullar)\ **Replies:** 0\ **Last updated:** [November 27, 2019, 3:46pm UTC](https://discuss.elastic.co/t/logstash-encoding-issue/209719 "2019-11-27T15:46:14Z")
Hi, I am having issue with logstash receiving extra characters when the message is being sent to logstash. My logstash.conf file looks like - input { stdin { codec =\> plain { charset=\>"UTF-8" } } tcp { port =\> 456… --- ## [Logstash 5.x to 7.x - Gzip Files](https://discuss.elastic.co/t/logstash-5-x-to-7-x-gzip-files/209718)
**Author:** [@MaikLinnemann](https://discuss.elastic.co/u/MaikLinnemann)\ **Replies:** 0\ **Last updated:** [November 27, 2019, 3:42pm UTC](https://discuss.elastic.co/t/logstash-5-x-to-7-x-gzip-files/209718 "2019-11-27T15:42:11Z")
Hi, is there a change in file input for logstash 7.x? I struggle about an issue with reading gzip files. in my old version 5.x i could give like: input { file { path =\> "/var/log/packetfilter.\*" } } And it reads a… --- ## [InvalidFrameProtocolException: Invalid version of beats protocol: 10](https://discuss.elastic.co/t/invalidframeprotocolexception-invalid-version-of-beats-protocol-10/209716)
**Author:** [@sgreszcz](https://discuss.elastic.co/u/sgreszcz)\ **Replies:** 0\ **Last updated:** [November 27, 2019, 3:19pm UTC](https://discuss.elastic.co/t/invalidframeprotocolexception-invalid-version-of-beats-protocol-10/209716 "2019-11-27T15:19:11Z")
I'm trying to send data from filebeat version 5.6.4 (amd64), libbeat 5.6.4 to Logstash 7.4.2. I've tested the configuration sending filebeat version 7.4.2 from another server and I can see the data flowing OK from fileb… --- ## [\_grokparsefailure because of time match](https://discuss.elastic.co/t/grokparsefailure-because-of-time-match/209585)
**Author:** [@04Konst](https://discuss.elastic.co/u/04Konst)\ **Replies:** 4\ **Last updated:** [November 27, 2019, 1:22pm UTC](https://discuss.elastic.co/t/grokparsefailure-because-of-time-match/209585 "2019-11-27T13:22:55Z")
Hi, I am facing an issue while transferiang a data with converted time. In the logstash output everithing looks fine, but in Kibana I can see \_grokparsefailure. After few test I found out, that when I am deleting the "d… --- ## [After upgrade 5.7 logstash to 6.0 unable to get some logs](https://discuss.elastic.co/t/after-upgrade-5-7-logstash-to-6-0-unable-to-get-some-logs/209514)
**Author:** [@shadrach1985](https://discuss.elastic.co/u/shadrach1985)\ **Replies:** 0\ **Last updated:** [November 26, 2019, 1:00pm UTC](https://discuss.elastic.co/t/after-upgrade-5-7-logstash-to-6-0-unable-to-get-some-logs/209514 "2019-11-26T13:00:50Z")
After upgrade 5.7 logstash to 6.0 unable to get some logs in kibana I am getting below error in logstash error log \[2019-11-26T12:17:21,720\]\[WARN \]\[logstash.outputs.elasticsearch\] Could not index event to Elasticsearch… --- ## [How to remove duplicate values?](https://discuss.elastic.co/t/how-to-remove-duplicate-values/209665)
**Author:** [@ja123](https://discuss.elastic.co/u/ja123)\ **Replies:** 0\ **Last updated:** [November 27, 2019, 10:58am UTC](https://discuss.elastic.co/t/how-to-remove-duplicate-values/209665 "2019-11-27T10:58:00Z")
I've got serveral docs with same "index" and many duplicate values in elasticsearch: { "\_index" : "bahnlog-29\_11\_2019", "\_type" : "\_doc", "\_id" : "l7YfrG4Baw1619OK5ufn", "\_score" : 1.0, "\_source" : { "@timestamp"… --- ## [Parsing a varying length line with custom grok filter, only returns the first field](https://discuss.elastic.co/t/parsing-a-varying-length-line-with-custom-grok-filter-only-returns-the-first-field/209533)
**Author:** [@anar](https://discuss.elastic.co/u/anar)\ **Replies:** 4\ **Last updated:** [November 27, 2019, 11:52am UTC](https://discuss.elastic.co/t/parsing-a-varying-length-line-with-custom-grok-filter-only-returns-the-first-field/209533 "2019-11-27T11:52:43Z")
Hello, I'm trying to set up a custom grok filter for my data input, but when I test it in Kibana's Grok Debugger, I only get the value for the first field (field1). I'm using grok instead of the csv parser because after… --- ## [I want to change @timestamp value with my message timestamp value](https://discuss.elastic.co/t/i-want-to-change-timestamp-value-with-my-message-timestamp-value/209608)
**Author:** [@SoniyaGupta](https://discuss.elastic.co/u/SoniyaGupta)\ **Replies:** 4\ **Last updated:** [November 27, 2019, 10:53am UTC](https://discuss.elastic.co/t/i-want-to-change-timestamp-value-with-my-message-timestamp-value/209608 "2019-11-27T10:53:09Z")
--- ## [How to create a pipeline in elasticsearch cloud](https://discuss.elastic.co/t/how-to-create-a-pipeline-in-elasticsearch-cloud/209650)
**Author:** [@Sajeeva\_Lakmal](https://discuss.elastic.co/u/Sajeeva_Lakmal)\ **Replies:** 0\ **Last updated:** [November 27, 2019, 10:02am UTC](https://discuss.elastic.co/t/how-to-create-a-pipeline-in-elasticsearch-cloud/209650 "2019-11-27T10:02:58Z")
I'm evaluating elasticsearch cloud and I tried creating a pipeline which consumes data from a data source and saves it to ES. I'm not sure how to enable the pipeline once created. There is no option to enable this or de… --- ## [After upgrade from 6.8.4 to 7.4.1 no data is sent to ES](https://discuss.elastic.co/t/after-upgrade-from-6-8-4-to-7-4-1-no-data-is-sent-to-es/206821)
**Author:** [@MichelV](https://discuss.elastic.co/u/MichelV)\ **Replies:** 6\ **Last updated:** [November 27, 2019, 8:42am UTC](https://discuss.elastic.co/t/after-upgrade-from-6-8-4-to-7-4-1-no-data-is-sent-to-es/206821 "2019-11-27T08:42:40Z")
I'm preparing an upgrade of ELK for a customer, moving from 6.2.0 to 7.4.1. The upgrade of ELK to 6.8.4 worked fine, however after upgrading to 7.4.1 logstash no longer sends data to ES. ES and Kibana are both on 7.4.1 … --- ## [Substring record basing on numerical index](https://discuss.elastic.co/t/substring-record-basing-on-numerical-index/209286)
**Author:** [@sam.cook](https://discuss.elastic.co/u/sam.cook)\ **Replies:** 2\ **Last updated:** [November 27, 2019, 8:12am UTC](https://discuss.elastic.co/t/substring-record-basing-on-numerical-index/209286 "2019-11-27T08:12:04Z")
Hi everyone, my logstash config file is as follows input { jdbc { jdbc\_connection\_string =\> "Connection String" jdbc\_driver\_class =\> "com.microsoft.sqlserver.jdbc.SQLServerDriver" jdbc\_user =\> "User" jdbc\_driv… --- ## [JDBC want to connect to db as windows user](https://discuss.elastic.co/t/jdbc-want-to-connect-to-db-as-windows-user/209628)
**Author:** [@kolten](https://discuss.elastic.co/u/kolten)\ **Replies:** 0\ **Last updated:** [November 27, 2019, 7:57am UTC](https://discuss.elastic.co/t/jdbc-want-to-connect-to-db-as-windows-user/209628 "2019-11-27T07:57:05Z")
I want to read db with jdbc input. Logstash wants to connect with windows authentication but i dont want this. Because windows authentication is not permitted. But logstash do this and crashes. How can i solve this pro… --- ## [Io error the network adapter could not establish the connection in logstash](https://discuss.elastic.co/t/io-error-the-network-adapter-could-not-establish-the-connection-in-logstash/209236)
**Author:** [@saravana\_hariharan](https://discuss.elastic.co/u/saravana_hariharan)\ **Replies:** 1\ **Last updated:** [November 27, 2019, 6:28am UTC](https://discuss.elastic.co/t/io-error-the-network-adapter-could-not-establish-the-connection-in-logstash/209236 "2019-11-27T06:28:52Z")
unable to connect to database.tried 1times{:error\_message =\>"java::javasql::sqlrecoverableexception: io error the network adapter could not establish the connection in logstash"} im connecting oracle to kibana while run… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=438) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=440)