# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=203 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 204 --- ## [Failed to parse field](https://discuss.elastic.co/t/failed-to-parse-field/281112)
**Author:** [@Senthil\_ak](https://discuss.elastic.co/u/Senthil_ak)\ **Replies:** 4\ **Last updated:** [August 12, 2021, 1:25pm UTC](https://discuss.elastic.co/t/failed-to-parse-field/281112 "2021-08-12T13:25:01Z")
Hi admin, I am getting some strange exception in the logstash all of a sudden. I try to debug but unable to do so. Need your kind help. \[2021-08-11T15:37:35,189\]\[WARN \]\[logstash.outputs.elasticsearch\] Could not index e… --- ## [Unclear how logstash fingerprint treats empty/non-existing fields](https://discuss.elastic.co/t/unclear-how-logstash-fingerprint-treats-empty-non-existing-fields/281157)
**Author:** [@AdmireDirac](https://discuss.elastic.co/u/AdmireDirac)\ **Replies:** 1\ **Last updated:** [August 12, 2021, 12:00pm UTC](https://discuss.elastic.co/t/unclear-how-logstash-fingerprint-treats-empty-non-existing-fields/281157 "2021-08-12T12:00:19Z")
Hello everyone, I have a hard time to understand the behaviour fo the logstash fingerprint filter if a field is empty or non-existent. My scenario is the following: I'm reading a json-file with logstash and create a d… --- ## [Setting up logstash with mongodb not starting](https://discuss.elastic.co/t/setting-up-logstash-with-mongodb-not-starting/280998)
**Author:** [@Bugsbee](https://discuss.elastic.co/u/Bugsbee)\ **Replies:** 5\ **Last updated:** [August 12, 2021, 10:12am UTC](https://discuss.elastic.co/t/setting-up-logstash-with-mongodb-not-starting/280998 "2021-08-12T10:12:05Z")
hi community, need your expertise / advise in this issue im facing when starting logstash which im trying to connect wth mongodb. input { uri =\> 'mongodb://userhere:passwordhere@1234.234.123.12:8081' p… --- ## [Pipeline worker error, the pipeline will be stopped {:pipeline\_id=\>"main", :error=\>"(TypeError) no implicit conversion of nil into Integer](https://discuss.elastic.co/t/pipeline-worker-error-the-pipeline-will-be-stopped-pipeline-id-main-error-typeerror-no-implicit-conversion-of-nil-into-integer/281167)
**Author:** [@ankitdevnalkar](https://discuss.elastic.co/u/ankitdevnalkar)\ **Replies:** 0\ **Last updated:** [August 12, 2021, 8:16am UTC](https://discuss.elastic.co/t/pipeline-worker-error-the-pipeline-will-be-stopped-pipeline-id-main-error-typeerror-no-implicit-conversion-of-nil-into-integer/281167 "2021-08-12T08:16:12Z")
I have configured SQS as input, not sure why I am getting this error. I am not performing any conversation from my Logstash config file. Can anyone help to identify the issue? The following error message comes right af… --- ## [Grok filter by values](https://discuss.elastic.co/t/grok-filter-by-values/280988)
**Author:** [@IsAa](https://discuss.elastic.co/u/IsAa)\ **Replies:** 4\ **Last updated:** [August 12, 2021, 5:19am UTC](https://discuss.elastic.co/t/grok-filter-by-values/280988 "2021-08-12T05:19:42Z")
Hi Everyone, I have a text in my input as follows, The following text contains {"Food":"Fruit","Type":"Apple"} I am trying to structure my output data to also contain fields Food: Fruit Type: Apple I was looking int… --- ## [Dump elasticsearch documents into avro format](https://discuss.elastic.co/t/dump-elasticsearch-documents-into-avro-format/281146)
**Author:** [@Sagar\_kadu](https://discuss.elastic.co/u/Sagar_kadu)\ **Replies:** 0\ **Last updated:** [August 12, 2021, 4:43am UTC](https://discuss.elastic.co/t/dump-elasticsearch-documents-into-avro-format/281146 "2021-08-12T04:43:10Z")
I want to store elasticsearch query results into avro files using logstash..As far as I know there file output logstash plugin which by default stores JSON documents. after going through logstash documentation,I co… --- ## [Parse lvmeng WAF log](https://discuss.elastic.co/t/parse-lvmeng-waf-log/281073)
**Author:** [@star\_cool](https://discuss.elastic.co/u/star_cool)\ **Replies:** 2\ **Last updated:** [August 11, 2021, 11:55pm UTC](https://discuss.elastic.co/t/parse-lvmeng-waf-log/281073 "2021-08-11T23:55:16Z")
hi \<11\>Aug 11 19:14:55 localhost waf: tag:waf\_log\_websec site\_id:1521041933 protect\_id:2623109843 dst\_ip:x.x.x.x dst\_port:80 src\_ip:183.209.173.69 src\_port:17506 method:GET domain:www.domain.com uri:/%5fupload/t… --- ## [Does Http\_poller use something simillar to "sincedb" as File input does?](https://discuss.elastic.co/t/does-http-poller-use-something-simillar-to-sincedb-as-file-input-does/281044)
**Author:** [@panagiss](https://discuss.elastic.co/u/panagiss)\ **Replies:** 2\ **Last updated:** [August 11, 2021, 8:00pm UTC](https://discuss.elastic.co/t/does-http-poller-use-something-simillar-to-sincedb-as-file-input-does/281044 "2021-08-11T20:00:29Z")
I know that File input plugin uses sincedb to "remember" the state of the file and the index position. But what about other input plugins and specifically http\_poller? Because i have noticed that if i make the same req… --- ## [Aggregate error](https://discuss.elastic.co/t/aggregate-error/281080)
**Author:** [@Divya\_Bansal](https://discuss.elastic.co/u/Divya_Bansal)\ **Replies:** 1\ **Last updated:** [August 11, 2021, 4:31pm UTC](https://discuss.elastic.co/t/aggregate-error/281080 "2021-08-11T16:31:34Z")
Hi, I am using the below filter to parse my log file. if \[mesg\] == "ERROR" { #This is prod tomcat log format grok { match =\> { "mesg" =\> \[ "^\\s?\[%{DATA:loglevel}\] %{TIMESTAMP\_ISO8601:logts} \[%{DATA:threadname}\] %{DA… --- ## [Communications link failure consistently after one hour with MySQL](https://discuss.elastic.co/t/communications-link-failure-consistently-after-one-hour-with-mysql/281031)
**Author:** [@Justin\_Morgan](https://discuss.elastic.co/u/Justin_Morgan)\ **Replies:** 1\ **Last updated:** [August 11, 2021, 4:25pm UTC](https://discuss.elastic.co/t/communications-link-failure-consistently-after-one-hour-with-mysql/281031 "2021-08-11T16:25:56Z")
I am using logstash with JDBC to import data from MySQL into Elasticsearch. This has worked before but now it is consistently failing after 1 hour with this exception: \`Exception when executing JDBC query {:exception=\>"… --- ## [Logstash is getting reset after 40 sec ,low uptime](https://discuss.elastic.co/t/logstash-is-getting-reset-after-40-sec-low-uptime/281029)
**Author:** [@Divyank\_Mahalle](https://discuss.elastic.co/u/Divyank_Mahalle)\ **Replies:** 2\ **Last updated:** [August 11, 2021, 4:15pm UTC](https://discuss.elastic.co/t/logstash-is-getting-reset-after-40-sec-low-uptime/281029 "2021-08-11T16:15:21Z")
Hi, I checked the logstash status, after every 40-sec logstash is getting started, Getting error message-How to solve this? Aug 11 09:38:38 learning-1 logstash\[4057\]: ERROR: Failed to parse YAML file "/etc/logstash/lo… --- ## [SOLVED - Logstash http output to Kafka REST Proxy](https://discuss.elastic.co/t/solved-logstash-http-output-to-kafka-rest-proxy/281098)
**Author:** [@stevetu21](https://discuss.elastic.co/u/stevetu21)\ **Replies:** 0\ **Last updated:** [August 11, 2021, 4:02pm UTC](https://discuss.elastic.co/t/solved-logstash-http-output-to-kafka-rest-proxy/281098 "2021-08-11T16:02:44Z")
Wanted to share a solution that I found to Logstash http output to Kafka-REST-Proxy and Logstash to Confluent kafka rest proxy. Was running into the same issue trying to use the http output to write a confluent Kafka RES… --- ## [Logstash SSL configuration invalid SecretKeyFactory not available](https://discuss.elastic.co/t/logstash-ssl-configuration-invalid-secretkeyfactory-not-available/280939)
**Author:** [@Kosodrom](https://discuss.elastic.co/u/Kosodrom)\ **Replies:** 2\ **Last updated:** [August 11, 2021, 11:02am UTC](https://discuss.elastic.co/t/logstash-ssl-configuration-invalid-secretkeyfactory-not-available/280939 "2021-08-11T11:02:46Z")
Hi folks, I am facing following error and I am out of options to fix it: \[2021-08-10T14:43:44,743\]\[ERROR\]\[logstash.inputs.beats \]\[test\_pipeline\] SSL configuration invalid {:exception=\>Java::JavaLang::IllegalArgument… --- ## [Logstash Ruby Script - Access Fields While Avoiding Concurrency Issues](https://discuss.elastic.co/t/logstash-ruby-script-access-fields-while-avoiding-concurrency-issues/280956)
**Author:** [@panagiss](https://discuss.elastic.co/u/panagiss)\ **Replies:** 4\ **Last updated:** [August 11, 2021, 9:43am UTC](https://discuss.elastic.co/t/logstash-ruby-script-access-fields-while-avoiding-concurrency-issues/280956 "2021-08-11T09:43:27Z")
I have a CSV file and what im trying to do is after filtering a bit the data, to output them in a another csv file but i have to check that im not writing SAME info. So i have a Ruby script to check for if a KEY is alrea… --- ## [Logstash jdbc plugin for windows authentication using kerberos in linux](https://discuss.elastic.co/t/logstash-jdbc-plugin-for-windows-authentication-using-kerberos-in-linux/280991)
**Author:** [@akshaybhuradia](https://discuss.elastic.co/u/akshaybhuradia)\ **Replies:** 0\ **Last updated:** [August 11, 2021, 6:23am UTC](https://discuss.elastic.co/t/logstash-jdbc-plugin-for-windows-authentication-using-kerberos-in-linux/280991 "2021-08-11T06:23:55Z")
For kerberos authentication username and password is not required. But plugin make username as required parameter. I have tried diiferent ways but not able to connect to MS Sql Server OS Version : RHEL 8.3 with x64 L… --- ## [Monitoring the logstash postfix queue](https://discuss.elastic.co/t/monitoring-the-logstash-postfix-queue/280986)
**Author:** [@KaPBaJIOJI](https://discuss.elastic.co/u/KaPBaJIOJI)\ **Replies:** 0\ **Last updated:** [August 11, 2021, 5:16am UTC](https://discuss.elastic.co/t/monitoring-the-logstash-postfix-queue/280986 "2021-08-11T05:16:30Z")
Hi all, mb its stupid idei, but i want monitoring postfix queue . I write in file "data", "hostname", "count", how can I transfer this data to elastic? at the moment, i am parse postfix logs, thanks to GitHub - whyscre… --- ## [How to specify pipelines.yml file path](https://discuss.elastic.co/t/how-to-specify-pipelines-yml-file-path/280923)
**Author:** [@GitSpree23](https://discuss.elastic.co/u/GitSpree23)\ **Replies:** 7\ **Last updated:** [August 11, 2021, 3:20am UTC](https://discuss.elastic.co/t/how-to-specify-pipelines-yml-file-path/280923 "2021-08-11T03:20:18Z")
I'm working on Logstash setup in a legacy system with no owners in the organisation. Using ps, i found the process executing the logstash service: /usr/bin/java -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -Djava.awt.headl… --- ## [How to import CSV files, where couple of fields have multiline content](https://discuss.elastic.co/t/how-to-import-csv-files-where-couple-of-fields-have-multiline-content/280453)
**Author:** [@itokai](https://discuss.elastic.co/u/itokai)\ **Replies:** 5\ **Last updated:** [August 11, 2021, 2:00am UTC](https://discuss.elastic.co/t/how-to-import-csv-files-where-couple-of-fields-have-multiline-content/280453 "2021-08-11T02:00:43Z")
Hello, how to accurately import CSV where lines contain fields with multiline content? Default separator is comma, but multiline content is surrounded with double-quotes. For example: Summary,Issue key,Issue id,Parent … --- ## [Logstash http poller - PKIX path building failed](https://discuss.elastic.co/t/logstash-http-poller-pkix-path-building-failed/280966)
**Author:** [@gutierrezfj](https://discuss.elastic.co/u/gutierrezfj)\ **Replies:** 2\ **Last updated:** [August 11, 2021, 1:38am UTC](https://discuss.elastic.co/t/logstash-http-poller-pkix-path-building-failed/280966 "2021-08-11T01:38:09Z")
Hello community. I need your help again! I'm using http poller plugin requesting an url https but have the error: "pkix path building failed sun.security.provider.certpath.suncertpathbuilderexception: unable to find va… --- ## [Logstash filter: aggregate nested arrays](https://discuss.elastic.co/t/logstash-filter-aggregate-nested-arrays/280965)
**Author:** [@me.mohammed](https://discuss.elastic.co/u/me.mohammed)\ **Replies:** 7\ **Last updated:** [August 10, 2021, 10:47pm UTC](https://discuss.elastic.co/t/logstash-filter-aggregate-nested-arrays/280965 "2021-08-10T22:47:50Z")
I'm trying to fetch data from MySQL and push it to ElasticSearch using LogStash, although I'm having trouble creating a config file for LogStash that suits my need I'm trying to achieve this result { "products":\[ … --- ## [DLQ not enabled for logstasha](https://discuss.elastic.co/t/dlq-not-enabled-for-logstasha/280940)
**Author:** [@Krrish\_Raj1](https://discuss.elastic.co/u/Krrish_Raj1)\ **Replies:** 9\ **Last updated:** [August 10, 2021, 9:33pm UTC](https://discuss.elastic.co/t/dlq-not-enabled-for-logstasha/280940 "2021-08-10T21:33:28Z")
I have two events: test\_start and test\_end When I receive test\_start event, I clone one more event from it test\_attempt and index it in elasticsearch. Then when test\_end event arrives, I extract document id and index … --- ## [Warnings on logstash's config execution - "A gauge metric of an unknown type (org.jruby.RubySymbol) has been created for key: status"](https://discuss.elastic.co/t/warnings-on-logstashs-config-execution-a-gauge-metric-of-an-unknown-type-org-jruby-rubysymbol-has-been-created-for-key-status/280954)
**Author:** [@Shreesh\_Narayanan](https://discuss.elastic.co/u/Shreesh_Narayanan)\ **Replies:** 0\ **Last updated:** [August 10, 2021, 5:31pm UTC](https://discuss.elastic.co/t/warnings-on-logstashs-config-execution-a-gauge-metric-of-an-unknown-type-org-jruby-rubysymbol-has-been-created-for-key-status/280954 "2021-08-10T17:31:33Z")
Hello, I see the below warnings on logstash , only when a particular config is used. The warning is as below \[2021-08-10T17:26:22,581\]\[WARN \]\[org.logstash.instrument.metrics.gauge.LazyDelegatingGauge\]\[main\] A gauge m… --- ## [Logstash execute enrich processor](https://discuss.elastic.co/t/logstash-execute-enrich-processor/280949)
**Author:** [@Bigbad](https://discuss.elastic.co/u/Bigbad)\ **Replies:** 0\ **Last updated:** [August 10, 2021, 4:14pm UTC](https://discuss.elastic.co/t/logstash-execute-enrich-processor/280949 "2021-08-10T16:14:25Z")
Hello, I have a logstash pipeline which use ingest pipeline and enrich policy from elasticsearch. But the main problem is to execute \_enrich index everyday. It's possible to figure out kind of scenario? --- ## [Logstash aggregate not working as expected](https://discuss.elastic.co/t/logstash-aggregate-not-working-as-expected/280943)
**Author:** [@emad101](https://discuss.elastic.co/u/emad101)\ **Replies:** 0\ **Last updated:** [August 10, 2021, 3:31pm UTC](https://discuss.elastic.co/t/logstash-aggregate-not-working-as-expected/280943 "2021-08-10T15:31:58Z")
Hello, I am trying to send an entire txt file line in one single field to elastic. I am trying to do this using logstash aggregate, but it seems that it is no longer working, each line is being sent as a separate field … --- ## [Need help on creating Logstash\_FILTER](https://discuss.elastic.co/t/need-help-on-creating-logstash-filter/277928)
**Author:** [@sumit\_n](https://discuss.elastic.co/u/sumit_n)\ **Replies:** 10\ **Last updated:** [August 10, 2021, 12:52pm UTC](https://discuss.elastic.co/t/need-help-on-creating-logstash-filter/277928 "2021-08-10T12:52:28Z")
log:{​​​​​​​​"time\_stamp":"2021-06-11T04:58:32.22154414Z","labels":{​​​​​​​​"vm\_index":0,"session\_docker\_id":"12d333","tenant\_id":"21a64d4","user\_id":"bob","session\_uuid":"2a30"}​​​​​​​​,"event":{​​​​​​​​"kind":"Metric",… --- ## [Unable to start logstash service on Ubuntu server](https://discuss.elastic.co/t/unable-to-start-logstash-service-on-ubuntu-server/280883)
**Author:** [@hiruni.insyncit.net](https://discuss.elastic.co/u/hiruni.insyncit.net)\ **Replies:** 1\ **Last updated:** [August 10, 2021, 12:15pm UTC](https://discuss.elastic.co/t/unable-to-start-logstash-service-on-ubuntu-server/280883 "2021-08-10T12:15:23Z")
I'm using Elasticseach and logstash 7.10.1 on ubuntu 18.04 Logstash service was working properly, few days ago. Suddenly it didn't work. I tried to restart logstash service. But unable to start the service. What can … --- ## [Logstash converts singleton array to string](https://discuss.elastic.co/t/logstash-converts-singleton-array-to-string/280900)
**Author:** [@GitSpree23](https://discuss.elastic.co/u/GitSpree23)\ **Replies:** 1\ **Last updated:** [August 10, 2021, 11:37am UTC](https://discuss.elastic.co/t/logstash-converts-singleton-array-to-string/280900 "2021-08-10T11:37:46Z")
I'm using mutate { add\_field =\> { "new\_field" =\> \["X"\] } } to create an array of integers which can be appended to downstream. But logstash output reads as "new\_field" =\> "X". --- ## [Logstash-codec-nmap undefined Method'\[\]'](https://discuss.elastic.co/t/logstash-codec-nmap-undefined-method/280517)
**Author:** [@111401](https://discuss.elastic.co/u/111401)\ **Replies:** 1\ **Last updated:** [August 10, 2021, 9:33am UTC](https://discuss.elastic.co/t/logstash-codec-nmap-undefined-method/280517 "2021-08-10T09:33:20Z")
Hi, I saved my nmap scan output as a xml file as well use a file input and logstash-codec-nmap to decode, also i use the filter and direct output into elasticsearch just like the following configuration logstash-codec-… --- ## [How to setup multiple Logstash conf reading the same input](https://discuss.elastic.co/t/how-to-setup-multiple-logstash-conf-reading-the-same-input/280878)
**Author:** [@GitSpree23](https://discuss.elastic.co/u/GitSpree23)\ **Replies:** 3\ **Last updated:** [August 10, 2021, 6:50am UTC](https://discuss.elastic.co/t/how-to-setup-multiple-logstash-conf-reading-the-same-input/280878 "2021-08-10T06:50:34Z")
I want to push 2 different transformations of the same data to 2 different outputs. conf-v1: source/.txt -\> filter A -\> output 1 conf-v2: source/.txt -\> filter B -\> output 2 If I use 2 different pipelines, they will s… --- ## [Logstash grok](https://discuss.elastic.co/t/logstash-grok/280779)
**Author:** [@KaPBaJIOJI](https://discuss.elastic.co/u/KaPBaJIOJI)\ **Replies:** 4\ **Last updated:** [August 10, 2021, 3:28am UTC](https://discuss.elastic.co/t/logstash-grok/280779 "2021-08-10T03:28:49Z")
Good afternoon to everyone, my question is quite simple, but I'm still quite new to the elk stack. it is necessary to digest the expression through the grok filter Mon Aug 9 15:18:25 +07 2021 mail 0 the date passes no… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=202) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=204)