# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=333 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 334 --- ## [Using other than comma as separator in csv filter](https://discuss.elastic.co/t/using-other-than-comma-as-separator-in-csv-filter/240067)
**Author:** [@Shipper01](https://discuss.elastic.co/u/Shipper01)\ **Replies:** 4\ **Last updated:** [July 7, 2020, 12:53am UTC](https://discuss.elastic.co/t/using-other-than-comma-as-separator-in-csv-filter/240067 "2020-07-07T00:53:01Z")
I have files that has either a semi-colon or pipe as separator. The values within the fields can have commas, so I can't use a commas as separator. I have tried using the csv filter and altering the separator. My conf… --- ## [How to change logstash locale](https://discuss.elastic.co/t/how-to-change-logstash-locale/239729)
**Author:** [@jang](https://discuss.elastic.co/u/jang)\ **Replies:** 1\ **Last updated:** [July 6, 2020, 9:47pm UTC](https://discuss.elastic.co/t/how-to-change-logstash-locale/239729 "2020-07-06T21:47:45Z")
hi. I'm using elastic stack. i want to logstash locale. The default location is 9 hours slower than the area I live in. my logstash.conf index =\> "%{\[@metadata\]\[beat\]}-%{+YYYY.MM.dd}" run it with the above settings… --- ## [Log Ingestion from Splunk](https://discuss.elastic.co/t/log-ingestion-from-splunk/240059)
**Author:** [@mgotechlock](https://discuss.elastic.co/u/mgotechlock)\ **Replies:** 0\ **Last updated:** [July 6, 2020, 8:31pm UTC](https://discuss.elastic.co/t/log-ingestion-from-splunk/240059 "2020-07-06T20:31:57Z")
I am trying to get logs from Splunk to Logstash. I first tried straight up syslog output from Splunk heavy forwarders to logstash, but that wrote each Windows Event Viewer line to a separate syslog event. I don't want to… --- ## [Logstash : jdbc - Unable to connect to database](https://discuss.elastic.co/t/logstash-jdbc-unable-to-connect-to-database/239898)
**Author:** [@Amine\_Aloui](https://discuss.elastic.co/u/Amine_Aloui)\ **Replies:** 2\ **Last updated:** [July 6, 2020, 6:18pm UTC](https://discuss.elastic.co/t/logstash-jdbc-unable-to-connect-to-database/239898 "2020-07-06T18:18:23Z")
this is my file : contact.conf input { jdbc { jdbc\_driver\_library =\> "/tmp/postgresql-42.2.12.jar" jdbc\_driver\_class =\> "org.postgresql.Driver" jdbc\_connection\_st… --- ## [Logstash out put csv with header](https://discuss.elastic.co/t/logstash-out-put-csv-with-header/239968)
**Author:** [@Yungyoung\_Ok](https://discuss.elastic.co/u/Yungyoung_Ok)\ **Replies:** 1\ **Last updated:** [July 6, 2020, 4:54pm UTC](https://discuss.elastic.co/t/logstash-out-put-csv-with-header/239968 "2020-07-06T16:54:27Z")
I am trying to create a csv file by using the output csv in logstash. How can I insert the header only once at the very beginning? my data: es\_field1 es\_field2 es\_field3 1 2 3 4 … --- ## [Logstash pipeline to fill sparse data](https://discuss.elastic.co/t/logstash-pipeline-to-fill-sparse-data/239965)
**Author:** [@henriot-gricad](https://discuss.elastic.co/u/henriot-gricad)\ **Replies:** 0\ **Last updated:** [July 6, 2020, 6:45am UTC](https://discuss.elastic.co/t/logstash-pipeline-to-fill-sparse-data/239965 "2020-07-06T06:45:02Z")
Hi, I have a series of machines whose status, selected from a set of possible values, is inserted into a database upon change with the associated timestamp. There is no new record until the next state change for the mac… --- ## [How to get filename from filebeat json object in logstash filter?](https://discuss.elastic.co/t/how-to-get-filename-from-filebeat-json-object-in-logstash-filter/239844)
**Author:** [@Rashid\_Ali](https://discuss.elastic.co/u/Rashid_Ali)\ **Replies:** 4\ **Last updated:** [July 6, 2020, 6:41am UTC](https://discuss.elastic.co/t/how-to-get-filename-from-filebeat-json-object-in-logstash-filter/239844 "2020-07-06T06:41:48Z")
I want to parse the file from the JSON object sent by filebeat to logstash file. Can anyone please help me here? --- ## [MongoDB Input threw an exception, restarting {:exception=\>org.logstash.MissingConverterException: Missing Converter handling for full class name=org.jruby.gen.RubyObject3, simple name=RubyObject3}](https://discuss.elastic.co/t/mongodb-input-threw-an-exception-restarting-exception-org-logstash-missingconverterexception-missing-converter-handling-for-full-class-name-org-jruby-gen-rubyobject3-simple-name-rubyobject3/239881)
**Author:** [@balamelangi](https://discuss.elastic.co/u/balamelangi)\ **Replies:** 1\ **Last updated:** [July 6, 2020, 6:11am UTC](https://discuss.elastic.co/t/mongodb-input-threw-an-exception-restarting-exception-org-logstash-missingconverterexception-missing-converter-handling-for-full-class-name-org-jruby-gen-rubyobject3-simple-name-rubyobject3/239881 "2020-07-06T06:11:38Z")
Hi @here , I'm getting below error while loading data From mongodb. MongoDB Input threw an exception, restarting {:exception=\>org.logstash.MissingConverterException: Missing Converter handling for full class name=org.j… --- ## [Logstash can't read stored procedure statement as input](https://discuss.elastic.co/t/logstash-cant-read-stored-procedure-statement-as-input/239928)
**Author:** [@Muhammad\_Badawy](https://discuss.elastic.co/u/Muhammad_Badawy)\ **Replies:** 0\ **Last updated:** [July 5, 2020, 3:52pm UTC](https://discuss.elastic.co/t/logstash-cant-read-stored-procedure-statement-as-input/239928 "2020-07-05T15:52:48Z")
I'm trying to let logstash read from MySQL DB through stored procedure. Here is my config input { jdbc { jdbc\_driver\_library =\> "/home/devops/mysql-connector-java-5.1.49/mysql-connector-java-5.1.49-bin.jar" jdbc\_driver… --- ## [Only one record is loaded to Elastic using Logstash conf file](https://discuss.elastic.co/t/only-one-record-is-loaded-to-elastic-using-logstash-conf-file/239684)
**Author:** [@Purnima\_Dhulipudi](https://discuss.elastic.co/u/Purnima_Dhulipudi)\ **Replies:** 5\ **Last updated:** [July 3, 2020, 6:10pm UTC](https://discuss.elastic.co/t/only-one-record-is-loaded-to-elastic-using-logstash-conf-file/239684 "2020-07-03T18:10:12Z")
Hi Below is my logstash conf file , i am trying to load data from oracle table which has 320 records to Elasticsearch but only one record got loaded to ES. I have enabled dead letter queue also but it created file wit… --- ## [CSV : how to ignore new line?](https://discuss.elastic.co/t/csv-how-to-ignore-new-line/239839)
**Author:** [@sauravjha221](https://discuss.elastic.co/u/sauravjha221)\ **Replies:** 0\ **Last updated:** [July 3, 2020, 5:11pm UTC](https://discuss.elastic.co/t/csv-how-to-ignore-new-line/239839 "2020-07-03T17:11:57Z")
"Door Opening: Small: 6.5""W x 7.25""H Medium: 9""W x 12""H Large: 11.75""W x 16""H Extra Large: 16""W x 23.75""H" this is one field but it's being read as multiple lines. How can i fix this. In excel it's working f… --- ## [Creating grok filter to extract each field from Dao logs](https://discuss.elastic.co/t/creating-grok-filter-to-extract-each-field-from-dao-logs/239833)
**Author:** [@kumarvivek633](https://discuss.elastic.co/u/kumarvivek633)\ **Replies:** 0\ **Last updated:** [July 3, 2020, 4:17pm UTC](https://discuss.elastic.co/t/creating-grok-filter-to-extract-each-field-from-dao-logs/239833 "2020-07-03T16:17:22Z")
I want to create a grok filter for my dao logs but as its my first time i stuck wen its going to next line. 2020-07-02 01:13:34.279 \[https-jsse-nio-9143-exec-1\] \[fc7ac7ef-47d6-48f0-b8b4-44f0889a944d\] DEBUG QueryLogger -… --- ## [How to create fields from nested fields for json data in logstash](https://discuss.elastic.co/t/how-to-create-fields-from-nested-fields-for-json-data-in-logstash/239369)
**Author:** [@Ramalakshmi](https://discuss.elastic.co/u/Ramalakshmi)\ **Replies:** 8\ **Last updated:** [July 3, 2020, 4:12pm UTC](https://discuss.elastic.co/t/how-to-create-fields-from-nested-fields-for-json-data-in-logstash/239369 "2020-07-03T16:12:33Z")
Hi, I am having json data that is parsed into kibana using logstash, in that data I am getting nested fields but I want individual fields.Please tell me how to get those fields. service.action.portProbeAction.portProbe… --- ## [Question about nested data](https://discuss.elastic.co/t/question-about-nested-data/239829)
**Author:** [@Mirakuru](https://discuss.elastic.co/u/Mirakuru)\ **Replies:** 5\ **Last updated:** [July 3, 2020, 4:00pm UTC](https://discuss.elastic.co/t/question-about-nested-data/239829 "2020-07-03T16:00:17Z")
Im trying to read a 100k line file into elastic search with logstash. I have a .txt file lines are like this userid;action. I want to use my userids as uniq and import every matching action to it, but there are duplica… --- ## [File input not pulling in data](https://discuss.elastic.co/t/file-input-not-pulling-in-data/239822)
**Author:** [@karlm](https://discuss.elastic.co/u/karlm)\ **Replies:** 1\ **Last updated:** [July 3, 2020, 3:43pm UTC](https://discuss.elastic.co/t/file-input-not-pulling-in-data/239822 "2020-07-03T15:43:05Z")
So i Have a daily log file the this with 3 sections time / priority / message like this below 03-Jul-2020 10:24:00.773 +01:00 \[INF\] User profile is available. this C:/ as key repository and Windows API to encrypt keys a… --- ## [Logstash is not running on Windows 10](https://discuss.elastic.co/t/logstash-is-not-running-on-windows-10/239827)
**Author:** [@senthil1](https://discuss.elastic.co/u/senthil1)\ **Replies:** 0\ **Last updated:** [July 3, 2020, 3:10pm UTC](https://discuss.elastic.co/t/logstash-is-not-running-on-windows-10/239827 "2020-07-03T15:10:45Z")
Whenever I am running the logstash bat file, I am again and again getting the same error "ignoring JAVA\_OPTS=-Xmx1024m -Xms1024m; pass JVM parameters via LS\_JAVA\_OPTS" and Logstash is not runnning. logstash-6.5.1 log… --- ## [Oracle Cloud Infrastructure - Logstash Plugin](https://discuss.elastic.co/t/oracle-cloud-infrastructure-logstash-plugin/239813)
**Author:** [@Kevin\_f](https://discuss.elastic.co/u/Kevin_f)\ **Replies:** 1\ **Last updated:** [July 3, 2020, 2:17pm UTC](https://discuss.elastic.co/t/oracle-cloud-infrastructure-logstash-plugin/239813 "2020-07-03T14:17:34Z")
Hi All, Hoping anyone has ingested logs from OCI Object Storage into Logstash using a specific input plugin similiar to that of logstash-input-s3? The scenario is shown in the diagram below: All the resources in a … --- ## [Logstash - Elasticsearch filter plugin with basic authentication](https://discuss.elastic.co/t/logstash-elasticsearch-filter-plugin-with-basic-authentication/203473)
**Author:** [@AlessandroKP](https://discuss.elastic.co/u/AlessandroKP)\ **Replies:** 2\ **Last updated:** [July 3, 2020, 1:13pm UTC](https://discuss.elastic.co/t/logstash-elasticsearch-filter-plugin-with-basic-authentication/203473 "2020-07-03T13:13:29Z")
Hi all, I have a problem with the Logstash elasticsearch filter plugin when using basic authentication. I have created a specific user called "logstash\_internal\_reader" which has the built-in logstash\_admin role and a … --- ## [Number of rows in an Excel file](https://discuss.elastic.co/t/number-of-rows-in-an-excel-file/238406)
**Author:** [@souma](https://discuss.elastic.co/u/souma)\ **Replies:** 5\ **Last updated:** [July 3, 2020, 1:01pm UTC](https://discuss.elastic.co/t/number-of-rows-in-an-excel-file/238406 "2020-07-03T13:01:46Z")
Hey guys, i have a stack of Excel files, and i use Logstash to send it to Es. All i want is that each time i send a file to Es, it(Logstash config file) count the number of rows and save it in a variable. Is there an… --- ## [How to do Logstash JDBC Output with SSH](https://discuss.elastic.co/t/how-to-do-logstash-jdbc-output-with-ssh/239760)
**Author:** [@JF-GIAMMARI](https://discuss.elastic.co/u/JF-GIAMMARI)\ **Replies:** 0\ **Last updated:** [July 3, 2020, 9:05am UTC](https://discuss.elastic.co/t/how-to-do-logstash-jdbc-output-with-ssh/239760 "2020-07-03T09:05:49Z")
Hi, I'm using JDBC as output to store logs in a mySQL database, but the mySQL server is not on the same server, so I need to launch an SSH connection before making my command, otherwise the connection fails. I've been … --- ## [Get ip source TCP plugin](https://discuss.elastic.co/t/get-ip-source-tcp-plugin/239743)
**Author:** [@Thomas74](https://discuss.elastic.co/u/Thomas74)\ **Replies:** 1\ **Last updated:** [July 3, 2020, 8:05am UTC](https://discuss.elastic.co/t/get-ip-source-tcp-plugin/239743 "2020-07-03T08:05:39Z")
Hi, I wonder if it's possible to get the ip source of a sender when using tcp input in logstash. Is there a way to get this ip even if I don't use http requests ? Best regards, Thomas R. --- ## [Unable to ship data to logstash 6.8 via Python](https://discuss.elastic.co/t/unable-to-ship-data-to-logstash-6-8-via-python/239677)
**Author:** [@Swetha\_B](https://discuss.elastic.co/u/Swetha_B)\ **Replies:** 10\ **Last updated:** [July 3, 2020, 4:32am UTC](https://discuss.elastic.co/t/unable-to-ship-data-to-logstash-6-8-via-python/239677 "2020-07-03T04:32:34Z")
We have a Python script to collect data and ship to Logstash. Please find the python script and logstash config below. import socket import json import logging from datetime import datetime import sys print("starting t… --- ## [Keep index template and ILM when using logstash](https://discuss.elastic.co/t/keep-index-template-and-ilm-when-using-logstash/239736)
**Author:** [@sref](https://discuss.elastic.co/u/sref)\ **Replies:** 0\ **Last updated:** [July 3, 2020, 6:31am UTC](https://discuss.elastic.co/t/keep-index-template-and-ilm-when-using-logstash/239736 "2020-07-03T06:31:51Z")
Im looking for guidance on how to keep ILM working, when sending logs via Logstash to a custom index. I do the classic setup with Winlogbeat-\>Logstash-\>ES. Logstash output config: output { if "CUSTOMTAG" in \[tags\] {… --- ## [Help with converting String to Integer in xml parsing](https://discuss.elastic.co/t/help-with-converting-string-to-integer-in-xml-parsing/239672)
**Author:** [@jyotsna.ganji](https://discuss.elastic.co/u/jyotsna.ganji)\ **Replies:** 1\ **Last updated:** [July 3, 2020, 5:56am UTC](https://discuss.elastic.co/t/help-with-converting-string-to-integer-in-xml-parsing/239672 "2020-07-03T05:56:46Z")
I am parsing a nested xml using the below conf file and it parses individual tag elements perfectly. But when I try to convert string elements to Integer using XPath, it forms arrays of values. input{ http\_poller{ … --- ## [Dead Letter - cannot write event to DLQ: reached maxQueueSize of 2147483648](https://discuss.elastic.co/t/dead-letter-cannot-write-event-to-dlq-reached-maxqueuesize-of-2147483648/239100)
**Author:** [@Ganesh2303](https://discuss.elastic.co/u/Ganesh2303)\ **Replies:** 6\ **Last updated:** [July 3, 2020, 4:24am UTC](https://discuss.elastic.co/t/dead-letter-cannot-write-event-to-dlq-reached-maxqueuesize-of-2147483648/239100 "2020-07-03T04:24:55Z")
Hi Team, I facing syncing issue in Elasticsearch and when i check my logstash log and i observer the below error "cannot write event to DLQ: reached maxQueueSize of 2147483648". whether its causing the syncing issue … --- ## [Optimize logstash configuration](https://discuss.elastic.co/t/optimize-logstash-configuration/239574)
**Author:** [@ohoroyoi](https://discuss.elastic.co/u/ohoroyoi)\ **Replies:** 5\ **Last updated:** [July 3, 2020, 12:32am UTC](https://discuss.elastic.co/t/optimize-logstash-configuration/239574 "2020-07-03T00:32:00Z")
Hello. I'm syncing 4 tables of geo data from postgres to elasticsearch using logstash. I am updating the data to be changed using the doc\_as\_upsert option. One table has more than 700,000 rows. Not many, but sometimes … --- ## [Dissect json array of strings](https://discuss.elastic.co/t/dissect-json-array-of-strings/239711)
**Author:** [@theflakes](https://discuss.elastic.co/u/theflakes)\ **Replies:** 1\ **Last updated:** [July 2, 2020, 9:54pm UTC](https://discuss.elastic.co/t/dissect-json-array-of-strings/239711 "2020-07-02T21:54:52Z")
I have a Json array of strings. I want to find the one entry that starts with “xx” and create a new field with it as the value. The text after “xx” can vary. Not sure how to use the dissect directive on an array of stri… --- ## [Logstash Runner Error](https://discuss.elastic.co/t/logstash-runner-error/239704)
**Author:** [@Kevin\_f](https://discuss.elastic.co/u/Kevin_f)\ **Replies:** 1\ **Last updated:** [July 2, 2020, 9:45pm UTC](https://discuss.elastic.co/t/logstash-runner-error/239704 "2020-07-02T21:45:18Z")
Hi All, I am seeing this error in my logstash logs and not sure how to resolve. Some people say to install xpack plugin. My logstash is on version 6.8 and I believe it already comes with xpack by default with this vers… --- ## [Capturing Start and Stop Times in An Aggregated Data Record?](https://discuss.elastic.co/t/capturing-start-and-stop-times-in-an-aggregated-data-record/239668)
**Author:** [@redapplesonly](https://discuss.elastic.co/u/redapplesonly)\ **Replies:** 4\ **Last updated:** [July 2, 2020, 7:37pm UTC](https://discuss.elastic.co/t/capturing-start-and-stop-times-in-an-aggregated-data-record/239668 "2020-07-02T19:37:54Z")
Hi Logstash Superheroes, Not long ago, this forum helped me with the syntax for an Aggregate Filter Plugin. I got it working, but now I need to append Start and Stop timestamps to the aggregated records. To explain… … --- ## [Beat - logstash - logserver](https://discuss.elastic.co/t/beat-logstash-logserver/239692)
**Author:** [@slawek](https://discuss.elastic.co/u/slawek)\ **Replies:** 4\ **Last updated:** [July 2, 2020, 7:26pm UTC](https://discuss.elastic.co/t/beat-logstash-logserver/239692 "2020-07-02T19:26:47Z")
Hello, Can I configure logstash as a receiver from winlogbeat and send these logs in CEF format to another log server? If yes. How should I do that? Regards [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=332) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=334)