# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=305 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 306 --- ## [LogStash not showing output on console](https://discuss.elastic.co/t/logstash-not-showing-output-on-console/248163)
**Author:** [@Vishal\_Donderia](https://discuss.elastic.co/u/Vishal_Donderia)\ **Replies:** 1\ **Last updated:** [September 10, 2020, 2:57pm UTC](https://discuss.elastic.co/t/logstash-not-showing-output-on-console/248163 "2020-09-10T14:57:55Z")
I was trying to use filebeat to push data into logstash filebeat.yml type: log # Change to true to enable this input configuration. enabled: false #Paths that should be crawled and fetched. Glob based paths. … --- ## [Logstash HA](https://discuss.elastic.co/t/logstash-ha/247956)
**Author:** [@aksshm](https://discuss.elastic.co/u/aksshm)\ **Replies:** 8\ **Last updated:** [September 10, 2020, 2:07pm UTC](https://discuss.elastic.co/t/logstash-ha/247956 "2020-09-10T14:07:21Z")
Hi, We're trying to install/achieve logstash HA. Is logstash support HA? or Is there any way to achieve the same? --- ## [Logstash Error while replicating Data from Oracle to Elasticsearch](https://discuss.elastic.co/t/logstash-error-while-replicating-data-from-oracle-to-elasticsearch/248140)
**Author:** [@tusharnemade](https://discuss.elastic.co/u/tusharnemade)\ **Replies:** 2\ **Last updated:** [September 10, 2020, 9:58am UTC](https://discuss.elastic.co/t/logstash-error-while-replicating-data-from-oracle-to-elasticsearch/248140 "2020-09-10T09:58:17Z")
Hi I am using Elasticsearch version 7.8.0 , Logtash version 7.8.0 RDBMS : Oracle I have configured Logstash to move data from Oracle to Elasticsearch of particular table on continuity basis. Logstash.conf file -- \> … --- ## [Auto\_flush\_interval not working inside my exec plugin](https://discuss.elastic.co/t/auto-flush-interval-not-working-inside-my-exec-plugin/247990)
**Author:** [@sharma\_yash](https://discuss.elastic.co/u/sharma_yash)\ **Replies:** 1\ **Last updated:** [September 10, 2020, 1:09am UTC](https://discuss.elastic.co/t/auto-flush-interval-not-working-inside-my-exec-plugin/247990 "2020-09-10T01:09:47Z")
Hi, I am executing a command using exec plugin. I am getting the response OK. I wanted to keep all the response lines in a single event. exec{ command =\> "my command" interval =\> 300 # To tackle this, I use multiline … --- ## [Maximum length of Grok pattern?](https://discuss.elastic.co/t/maximum-length-of-grok-pattern/248062)
**Author:** [@anda](https://discuss.elastic.co/u/anda)\ **Replies:** 0\ **Last updated:** [September 9, 2020, 7:55pm UTC](https://discuss.elastic.co/t/maximum-length-of-grok-pattern/248062 "2020-09-09T19:55:30Z")
Hello, I am trying to parse a very long log line that contains 294 fields and 11,500 characters. I've had some difficulties testing the grok pattern because it's so long. When trying to use the grok debugger in Kibana,… --- ## [Logstash cannot connect to Oracle cloud using wallet key](https://discuss.elastic.co/t/logstash-cannot-connect-to-oracle-cloud-using-wallet-key/248059)
**Author:** [@Otomar\_Jupiter](https://discuss.elastic.co/u/Otomar_Jupiter)\ **Replies:** 1\ **Last updated:** [September 9, 2020, 7:55pm UTC](https://discuss.elastic.co/t/logstash-cannot-connect-to-oracle-cloud-using-wallet-key/248059 "2020-09-09T19:55:28Z")
Hello community, I'm having hard time to establish Logstash connection towards Oracle cloud autonomous db. I have suspicion that it is connected to wallet key. I was able to connect to Oracle cloud using Ubuntu/sql dev… --- ## [Logstash missing XML records](https://discuss.elastic.co/t/logstash-missing-xml-records/248044)
**Author:** [@stevedbarnes](https://discuss.elastic.co/u/stevedbarnes)\ **Replies:** 0\ **Last updated:** [September 9, 2020, 4:49pm UTC](https://discuss.elastic.co/t/logstash-missing-xml-records/248044 "2020-09-09T16:49:50Z")
Hi, I am consuming large (~600Mb) XML files with one line of data in the file. I am able to (mostly) consume the data using Logstash using xpath to extract fields from the XML and I am stuck on an issue I can't get arou… --- ## [Writing raw mapping\_error events from the Logstash DLQ to Elasticsearch](https://discuss.elastic.co/t/writing-raw-mapping-error-events-from-the-logstash-dlq-to-elasticsearch/248024)
**Author:** [@Yor\_On](https://discuss.elastic.co/u/Yor_On)\ **Replies:** 0\ **Last updated:** [September 9, 2020, 1:59pm UTC](https://discuss.elastic.co/t/writing-raw-mapping-error-events-from-the-logstash-dlq-to-elasticsearch/248024 "2020-09-09T13:59:49Z")
Hi, I'm trying to find a way to get the raw data of messages that fail on mapping errors from my Logstash's Dead Letter Queues to Elasticsearch so I can look into the reasons why the messages ended up in the DLQ in the … --- ## [How to config logstash filter to remove new line in the field csv file](https://discuss.elastic.co/t/how-to-config-logstash-filter-to-remove-new-line-in-the-field-csv-file/247985)
**Author:** [@111207](https://discuss.elastic.co/u/111207)\ **Replies:** 0\ **Last updated:** [September 9, 2020, 9:29am UTC](https://discuss.elastic.co/t/how-to-config-logstash-filter-to-remove-new-line-in-the-field-csv-file/247985 "2020-09-09T09:29:23Z")
Hi Everyone I need to parse csv file with logstash and ingest data to elasticsearch But the some column in csv file have new line This is my input --- ## [Logstash error: cannot parse empty date](https://discuss.elastic.co/t/logstash-error-cannot-parse-empty-date/247983)
**Author:** [@hiba](https://discuss.elastic.co/u/hiba)\ **Replies:** 0\ **Last updated:** [September 9, 2020, 9:21am UTC](https://discuss.elastic.co/t/logstash-error-cannot-parse-empty-date/247983 "2020-09-09T09:21:33Z")
Hi all, i want to load data from an api to elasticserach via logstash. i have a field "depDate" with date type ....in some documents, this field can be empty . what can i do to load this documents in my index ? \[20… --- ## [Logstash s3-sqs input plugin to differentiate 2 types of log](https://discuss.elastic.co/t/logstash-s3-sqs-input-plugin-to-differentiate-2-types-of-log/247971)
**Author:** [@vinodjv\_3](https://discuss.elastic.co/u/vinodjv_3)\ **Replies:** 0\ **Last updated:** [September 9, 2020, 7:49am UTC](https://discuss.elastic.co/t/logstash-s3-sqs-input-plugin-to-differentiate-2-types-of-log/247971 "2020-09-09T07:49:43Z")
Hi all, I have an use case where application logs are pushed to s3 bucket, log types are json and plain. Am using \[logstash-input-s3-sns-sqs\] plugin to read logs from S3 bucket. Both log types are pushed to different p… --- ## [Logstash start failed because of Could not load FFI Provider: (NotImplementedError) FFI not available](https://discuss.elastic.co/t/logstash-start-failed-because-of-could-not-load-ffi-provider-notimplementederror-ffi-not-available/247970)
**Author:** [@qiuwenhui1](https://discuss.elastic.co/u/qiuwenhui1)\ **Replies:** 0\ **Last updated:** [September 9, 2020, 7:47am UTC](https://discuss.elastic.co/t/logstash-start-failed-because-of-could-not-load-ffi-provider-notimplementederror-ffi-not-available/247970 "2020-09-09T07:47:31Z")
java.lang.IllegalStateException: Logstash stopped processing because of an error: (LoadError) Could not load FFI Provider: (NotImplementedError) FFI not available: java.lang.ClassCastException: java.nio.HeapByteBuffer ca… --- ## [How to choose between Logstash and Fluentd in Elastic stack?](https://discuss.elastic.co/t/how-to-choose-between-logstash-and-fluentd-in-elastic-stack/247963)
**Author:** [@Ashish1101](https://discuss.elastic.co/u/Ashish1101)\ **Replies:** 0\ **Last updated:** [September 9, 2020, 7:12am UTC](https://discuss.elastic.co/t/how-to-choose-between-logstash-and-fluentd-in-elastic-stack/247963 "2020-09-09T07:12:59Z")
How do I decide whether to Logstash or Fluentd in elastic stack? what are the criteria? any reference links will be helpful. --- ## [How to parse xml.log via logstash](https://discuss.elastic.co/t/how-to-parse-xml-log-via-logstash/247940)
**Author:** [@gavin-orange](https://discuss.elastic.co/u/gavin-orange)\ **Replies:** 3\ **Last updated:** [September 9, 2020, 6:10am UTC](https://discuss.elastic.co/t/how-to-parse-xml-log-via-logstash/247940 "2020-09-09T06:10:11Z")
Hello Guys , I am totally new into ES and now I want to use logstash to parse the xml.log , hope could get any help . Here is part of xml.log which I have : \ **Author:** [@ankitachow](https://discuss.elastic.co/u/ankitachow)\ **Replies:** 3\ **Last updated:** [September 8, 2020, 11:03pm UTC](https://discuss.elastic.co/t/dateparsefailure-on-time-field/247927 "2020-09-08T23:03:23Z") I have date and time as separate fields in my input file. Date as 2019-08-28 ("yyyy-MM-dd"). Time as 09:33:05.579476547 (HH:mm:ss.SSSSSSSSZ). I want to merge the date and time for every record and create a timestamp fiel… --- ## [Log file parsing and ingesting into ES](https://discuss.elastic.co/t/log-file-parsing-and-ingesting-into-es/247806)
**Author:** [@ankitachow](https://discuss.elastic.co/u/ankitachow)\ **Replies:** 14\ **Last updated:** [September 8, 2020, 6:58pm UTC](https://discuss.elastic.co/t/log-file-parsing-and-ingesting-into-es/247806 "2020-09-08T18:58:36Z")
I have a log file with 1 root of xml as below. I have created a logstash parser and tested it out in manual mode and it seems to work fine. But while ingesting into ES and get the error. Input file: \ **Author:** [@slash665](https://discuss.elastic.co/u/slash665)\ **Replies:** 15\ **Last updated:** [September 8, 2020, 3:22pm UTC](https://discuss.elastic.co/t/filebeat-to-logstash-to-es/247767 "2020-09-08T15:22:15Z") Hello, I am trying to set up a pipeline from Filebeat to Logstash to ES, following the guide from Elastic with the tutorial logs they provide, and for the love of me I just can't get it to work. This is what I've got i… --- ## [How to replace field value when it equals "-"](https://discuss.elastic.co/t/how-to-replace-field-value-when-it-equals/247899)
**Author:** [@gbasic](https://discuss.elastic.co/u/gbasic)\ **Replies:** 0\ **Last updated:** [September 8, 2020, 1:29pm UTC](https://discuss.elastic.co/t/how-to-replace-field-value-when-it-equals/247899 "2020-09-08T13:29:51Z")
How to check if fields in "Data" are equal to "-" and if so change them to something else. This is my data (There are multiple cities in this JSON file with the same fields): { "@version" =\> "1", "@timestamp" … --- ## [Logstash is restarting automatucally with high CPU usage](https://discuss.elastic.co/t/logstash-is-restarting-automatucally-with-high-cpu-usage/246617)
**Author:** [@Tirtharaj\_Karbotkar](https://discuss.elastic.co/u/Tirtharaj_Karbotkar)\ **Replies:** 5\ **Last updated:** [September 8, 2020, 10:39am UTC](https://discuss.elastic.co/t/logstash-is-restarting-automatucally-with-high-cpu-usage/246617 "2020-09-08T10:39:50Z")
Hi, i am trying to run logstash using the below command. sudo systemctl start logstash.service From below logs we can see that logstash service is started than it is stopped and restarted again. \[2020-08-27T11:56:24… --- ## [Build logstash deb-package manually from source](https://discuss.elastic.co/t/build-logstash-deb-package-manually-from-source/247867)
**Author:** [@Maxim\_Lavrovichev](https://discuss.elastic.co/u/Maxim_Lavrovichev)\ **Replies:** 0\ **Last updated:** [September 8, 2020, 10:03am UTC](https://discuss.elastic.co/t/build-logstash-deb-package-manually-from-source/247867 "2020-09-08T10:03:07Z")
Hello, I need to build a deb-package from source code of logstash (https://artifacts.elastic.co/downloads/logstash/logstash-oss-6.7.2.tar.gz). Have you any official instructions for packaging and actual debian scripts(d… --- ## [Keep order of xml in Logstash](https://discuss.elastic.co/t/keep-order-of-xml-in-logstash/247831)
**Author:** [@\_Louw](https://discuss.elastic.co/u/_Louw)\ **Replies:** 0\ **Last updated:** [September 8, 2020, 7:25am UTC](https://discuss.elastic.co/t/keep-order-of-xml-in-logstash/247831 "2020-09-08T07:25:03Z")
I am parsing XML-input in Logstash, and I need to keep the sorting of it, as it is relevant. Example xml: \ \ \ \ **Author:** [@KOTOXJle6](https://discuss.elastic.co/u/KOTOXJle6)\ **Replies:** 2\ **Last updated:** [September 8, 2020, 5:12am UTC](https://discuss.elastic.co/t/logs-duplication/247795 "2020-09-08T05:12:42Z") Hello everyone! Suddenly, the elastic began to double logs. Each log falls into its own index and the duplicate falls into the unkown\_messages index. It seems that this started after adding new indexes, but rolling back… --- ## [Limit of total fields \[1000\] in index exceeded and nested json](https://discuss.elastic.co/t/limit-of-total-fields-1000-in-index-exceeded-and-nested-json/247716)
**Author:** [@Luca\_P](https://discuss.elastic.co/u/Luca_P)\ **Replies:** 2\ **Last updated:** [September 8, 2020, 4:58am UTC](https://discuss.elastic.co/t/limit-of-total-fields-1000-in-index-exceeded-and-nested-json/247716 "2020-09-08T04:58:38Z")
Hi All, I need some help to avoid the well known 'Limit of total fields \[1000\] in index exceeded' problem. I don't consider to raise the default in the settings above 1000. The culprit seems to be a field with a json v… --- ## [How to read multiple inputs in a single logstash instance](https://discuss.elastic.co/t/how-to-read-multiple-inputs-in-a-single-logstash-instance/247776)
**Author:** [@aparna\_shukla](https://discuss.elastic.co/u/aparna_shukla)\ **Replies:** 1\ **Last updated:** [September 8, 2020, 12:24am UTC](https://discuss.elastic.co/t/how-to-read-multiple-inputs-in-a-single-logstash-instance/247776 "2020-09-08T00:24:14Z")
I have metricbeat running on server 1 and filebeat running on server 2 and logstash on server 3. Can we read both metric beat and filebeat logs using one logstash ? both logs(metric and filebeat) are different and we … --- ## [Logstash duplicating records while reading from elasticsearch and writing to bigquery](https://discuss.elastic.co/t/logstash-duplicating-records-while-reading-from-elasticsearch-and-writing-to-bigquery/247702)
**Author:** [@ThiagoSantos](https://discuss.elastic.co/u/ThiagoSantos)\ **Replies:** 3\ **Last updated:** [September 7, 2020, 8:53pm UTC](https://discuss.elastic.co/t/logstash-duplicating-records-while-reading-from-elasticsearch-and-writing-to-bigquery/247702 "2020-09-07T20:53:38Z")
Hi, everyone. Everytime logstash runs, it writes all records from my elasticsearch index to my bigquery database. The problem is that the records get duplicated at every run. My goal is to get only those records that … --- ## [Forward logs from Kiwi to Elasticsearch](https://discuss.elastic.co/t/forward-logs-from-kiwi-to-elasticsearch/247682)
**Author:** [@markb](https://discuss.elastic.co/u/markb)\ **Replies:** 5\ **Last updated:** [September 7, 2020, 1:43pm UTC](https://discuss.elastic.co/t/forward-logs-from-kiwi-to-elasticsearch/247682 "2020-09-07T13:43:15Z")
I would like to forward logs from Kiwi Syslog Server to Elasticsearch by Logstash. After configuring both Kiwi log action and Logstash pipeline I see no log in ES side. This is the Kiwi remote host forward rule: Thi… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=304) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=306)