# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=157 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 158 --- ## [Logtstash updating fron 7.9 to 7.16.2](https://discuss.elastic.co/t/logtstash-updating-fron-7-9-to-7-16-2/298453)
**Author:** [@Faisal\_Malik](https://discuss.elastic.co/u/Faisal_Malik)\ **Replies:** 3\ **Last updated:** [February 28, 2022, 11:54pm UTC](https://discuss.elastic.co/t/logtstash-updating-fron-7-9-to-7-16-2/298453 "2022-02-28T23:54:10Z")
Hi i am trying to update logstash from 7.9.0 to 7.16.2 (Using Helm chart ) Getting error. Pipeline error {:pipeline\_id=\>"main", :exception=\>#\ **Author:** [@djschny](https://discuss.elastic.co/u/djschny)\ **Replies:** 2\ **Last updated:** [February 28, 2022, 11:48pm UTC](https://discuss.elastic.co/t/logstash-in-docker-fails-after-plugin-installed/298446 "2022-02-28T23:48:19Z") When using the official image for logstash (8.0.0) if a plugin is installed, after that Logstash will not load. If I use the image without the plugin installed then it works. For example: docker run --rm --name my-logst… --- ## [Logstash-plugin install logstash-output-influxdb error proxy](https://discuss.elastic.co/t/logstash-plugin-install-logstash-output-influxdb-error-proxy/298451)
**Author:** [@bob\_monty](https://discuss.elastic.co/u/bob_monty)\ **Replies:** 0\ **Last updated:** [February 28, 2022, 9:38pm UTC](https://discuss.elastic.co/t/logstash-plugin-install-logstash-output-influxdb-error-proxy/298451 "2022-02-28T21:38:52Z")
Greetings im trying to logstash-plugin install logstash-output-influxdb but im behind a firewall and we have a proxy. i set the proxy at the command line with HTTP\_PROXY, and also tried setting it in logstash-7.16.1/vend… --- ## [Logstash installed on GCP compute engine having issues connecting to Mongo Atlas](https://discuss.elastic.co/t/logstash-installed-on-gcp-compute-engine-having-issues-connecting-to-mongo-atlas/296461)
**Author:** [@justjaidev](https://discuss.elastic.co/u/justjaidev)\ **Replies:** 1\ **Last updated:** [February 28, 2022, 8:33pm UTC](https://discuss.elastic.co/t/logstash-installed-on-gcp-compute-engine-having-issues-connecting-to-mongo-atlas/296461 "2022-02-28T20:33:33Z")
Logstash installed on GCP compute engine is unable to connect Mongo Atlas.. Tried all the below 3 versions of mongo output plugin. 3.1.5 3.1.6 3.1.7 Below error observed with 3.1.5 version of mongo output plugin \[WA… --- ## [Support of mongodb+srv to connect with Atlas Mongo](https://discuss.elastic.co/t/support-of-mongodb-srv-to-connect-with-atlas-mongo/298445)
**Author:** [@zvazquez](https://discuss.elastic.co/u/zvazquez)\ **Replies:** 0\ **Last updated:** [February 28, 2022, 8:26pm UTC](https://discuss.elastic.co/t/support-of-mongodb-srv-to-connect-with-atlas-mongo/298445 "2022-02-28T20:26:55Z")
Hi, I am trying to push data coming into Logstash to MongoDB on Atlas. My output configuration is as following: mongodb { id =\> "logs" collection =\> "logentries" database =\> "logs" uri =\>… --- ## [\[ERROR\] Logstash JDBC plugin - PKIX path building failed](https://discuss.elastic.co/t/error-logstash-jdbc-plugin-pkix-path-building-failed/298439)
**Author:** [@Rossana](https://discuss.elastic.co/u/Rossana)\ **Replies:** 0\ **Last updated:** [February 28, 2022, 6:25pm UTC](https://discuss.elastic.co/t/error-logstash-jdbc-plugin-pkix-path-building-failed/298439 "2022-02-28T18:25:29Z")
Hi, I have an issue with logstash jdbc plugin configuration for MSSQL. My config file: input { jdbc { jdbc\_driver\_library =\> "/opt/elastic/drivers/mssql-jdbc-10.2.0.jre11.jar" jdbc\_driver\_class =\> "com… --- ## [Logstash Even Hub input. error when running as a service, but not when I run single pipeline](https://discuss.elastic.co/t/logstash-even-hub-input-error-when-running-as-a-service-but-not-when-i-run-single-pipeline/298423)
**Author:** [@ElasticLiver](https://discuss.elastic.co/u/ElasticLiver)\ **Replies:** 0\ **Last updated:** [February 28, 2022, 3:43pm UTC](https://discuss.elastic.co/t/logstash-even-hub-input-error-when-running-as-a-service-but-not-when-i-run-single-pipeline/298423 "2022-02-28T15:43:10Z")
Hi Im trying to consume data from Azure Event Hub, I have 15 different logstash pipelines pointing to different storages, and Even Hubs, they all work well,and I can get data when I run the single pipelines alone, but wh… --- ## [Jdbc\_driver\_library default folder in old versions](https://discuss.elastic.co/t/jdbc-driver-library-default-folder-in-old-versions/298416)
**Author:** [@serfisar](https://discuss.elastic.co/u/serfisar)\ **Replies:** 0\ **Last updated:** [February 28, 2022, 2:56pm UTC](https://discuss.elastic.co/t/jdbc-driver-library-default-folder-in-old-versions/298416 "2022-02-28T14:56:49Z")
Hello! Could someone suggest where is default folder for jars in logstash version 2.4.1? I know that beginning from 6 version it is - /usr/share/logstash/logstash-core/lib/jars/ but previous versions had another pa… --- ## [How to use parameters in a stored script that's executed using logstash output plugin](https://discuss.elastic.co/t/how-to-use-parameters-in-a-stored-script-thats-executed-using-logstash-output-plugin/298336)
**Author:** [@Rui\_Goncalves](https://discuss.elastic.co/u/Rui_Goncalves)\ **Replies:** 2\ **Last updated:** [February 28, 2022, 2:34pm UTC](https://discuss.elastic.co/t/how-to-use-parameters-in-a-stored-script-thats-executed-using-logstash-output-plugin/298336 "2022-02-28T14:34:08Z")
Hi, I've a stored script : POST \_scripts/1\_jd\_job\_domain { "script": { "lang": "painless", "params": { "generic\_domain": "generic domain" \[...\] }, "source" : """if ( ctx.\_source.car?.current\_assignmen… --- ## [Parsing message for haproxy logs no matched](https://discuss.elastic.co/t/parsing-message-for-haproxy-logs-no-matched/298395)
**Author:** [@Romain.Depreux](https://discuss.elastic.co/u/Romain.Depreux)\ **Replies:** 0\ **Last updated:** [February 28, 2022, 10:31am UTC](https://discuss.elastic.co/t/parsing-message-for-haproxy-logs-no-matched/298395 "2022-02-28T10:31:43Z")
Hello, can anyone kindly help me to solve this problem? I'm trying to parse the fields of a haproxy log then dissect that message to map the different message values into fields for logstash. But I have the impression … --- ## [Logstash and MongoDB input](https://discuss.elastic.co/t/logstash-and-mongodb-input/297535)
**Author:** [@serfisar](https://discuss.elastic.co/u/serfisar)\ **Replies:** 4\ **Last updated:** [February 28, 2022, 10:20am UTC](https://discuss.elastic.co/t/logstash-and-mongodb-input/297535 "2022-02-28T10:20:15Z")
Hi! Logstash version: 7.17.0 Could you help me with logstash-mongodb issue, in view of input plugin for Logstash. I work on some task, where I need to get logs from postgre and mongo. Regarding to postgre - everythi… --- ## [I have a problem when I want to send logs of PFSense (2.5.2 amd64) to EK version 7.14.2](https://discuss.elastic.co/t/i-have-a-problem-when-i-want-to-send-logs-of-pfsense-2-5-2-amd64-to-ek-version-7-14-2/297603)
**Author:** [@khouloud1](https://discuss.elastic.co/u/khouloud1)\ **Replies:** 18\ **Last updated:** [February 28, 2022, 9:19am UTC](https://discuss.elastic.co/t/i-have-a-problem-when-i-want-to-send-logs-of-pfsense-2-5-2-amd64-to-ek-version-7-14-2/297603 "2022-02-28T09:19:20Z")
I have a problem when I want to send logs from PFSense (2.5.2 amd64) to EK version 7.14.2 I did configure PFSense to send logs to EK but I did not find the best procedure to configure Elasticsearch and Kibana (7.14.2) --- ## [Logstash input: Twitter](https://discuss.elastic.co/t/logstash-input-twitter/298351)
**Author:** [@Rysiu](https://discuss.elastic.co/u/Rysiu)\ **Replies:** 1\ **Last updated:** [February 27, 2022, 2:42pm UTC](https://discuss.elastic.co/t/logstash-input-twitter/298351 "2022-02-27T14:42:32Z")
Hello, I have a simple pipeline setup in Logstash (keys changed of course): input { twitter { consumer\_key =\> "rcLfXIhprrPKLgVI467Slbux0" consumer\_secret =\>"uYk68znfoDQq70lfWXS3I… --- ## [Timestamp does not match data](https://discuss.elastic.co/t/timestamp-does-not-match-data/298344)
**Author:** [@tjunge](https://discuss.elastic.co/u/tjunge)\ **Replies:** 2\ **Last updated:** [February 27, 2022, 2:21pm UTC](https://discuss.elastic.co/t/timestamp-does-not-match-data/298344 "2022-02-27T14:21:44Z")
Hi, I am new to Elastic Stack and trying to filter my fail2ban log. In Grok Debugger I tried to simulate my Grok pattern on my data but have problems with the timestamp. I always get the simulate error: Provided Grok p… --- ## [Parsing Json logs separtly](https://discuss.elastic.co/t/parsing-json-logs-separtly/298329)
**Author:** [@frank\_rib](https://discuss.elastic.co/u/frank_rib)\ **Replies:** 0\ **Last updated:** [February 26, 2022, 11:08am UTC](https://discuss.elastic.co/t/parsing-json-logs-separtly/298329 "2022-02-26T11:08:22Z")
Hello every body, I want to send the results of this json file to ELK through Logstash, \` {“logsourcetypename\_deviceType": "Stonesoft Management Center", "High Level Category": "Access"}, {"logsourcetypename\_deviceTy… --- ## [\[0\] "\_jsonparsefailure" how do you remove this?](https://discuss.elastic.co/t/0-jsonparsefailure-how-do-you-remove-this/298225)
**Author:** [@itschobot](https://discuss.elastic.co/u/itschobot)\ **Replies:** 3\ **Last updated:** [February 26, 2022, 3:41am UTC](https://discuss.elastic.co/t/0-jsonparsefailure-how-do-you-remove-this/298225 "2022-02-26T03:41:51Z")
how do you get rid of the \[0\] "\_jsonparsefailure? { "origin" =\> "some\_platform", "ls-source" =\> "some\_platform", "port" =\> 1234, "event" =\> { … --- ## [CSV filter not working properly](https://discuss.elastic.co/t/csv-filter-not-working-properly/298274)
**Author:** [@elasticity2](https://discuss.elastic.co/u/elasticity2)\ **Replies:** 2\ **Last updated:** [February 25, 2022, 11:33pm UTC](https://discuss.elastic.co/t/csv-filter-not-working-properly/298274 "2022-02-25T23:33:37Z")
Hi all, I have a folder which contains two different types of CSV files. My logstash config is as below. When I run logstash, it just ingests the csvs as is without the columns getting generated. If I explicitly mention … --- ## [How to merge two data sets into one with Logstash?](https://discuss.elastic.co/t/how-to-merge-two-data-sets-into-one-with-logstash/298218)
**Author:** [@mohsin106](https://discuss.elastic.co/u/mohsin106)\ **Replies:** 7\ **Last updated:** [February 25, 2022, 5:24pm UTC](https://discuss.elastic.co/t/how-to-merge-two-data-sets-into-one-with-logstash/298218 "2022-02-25T17:24:56Z")
Hi, I'm trying to merge two data sets into one document that will get stored in ES. My first data set looks like this: "\_source": { "protocol-name": "BGP", "name-tag": "default", "@version": "1", "host… --- ## [Help with Aggregate filter](https://discuss.elastic.co/t/help-with-aggregate-filter/298167)
**Author:** [@BhawanaSharma](https://discuss.elastic.co/u/BhawanaSharma)\ **Replies:** 3\ **Last updated:** [February 25, 2022, 5:23pm UTC](https://discuss.elastic.co/t/help-with-aggregate-filter/298167 "2022-02-25T17:23:32Z")
aggregate { timeout\_tags =\> \['\_aggregatetimeout'\] task\_id =\> "%{\[log\]\[file\]\[path\]}" code =\> 'map\["build\_name"\] = event.get("\[build\_name\_long\]") ; map\["build\_link"\] = event.get("\[build\_name\_start\]") ; event.set("bui… --- ## [Updating logstahs 7.9 to 7.16.2](https://discuss.elastic.co/t/updating-logstahs-7-9-to-7-16-2/298299)
**Author:** [@Faisal\_Malik](https://discuss.elastic.co/u/Faisal_Malik)\ **Replies:** 1\ **Last updated:** [February 25, 2022, 5:06pm UTC](https://discuss.elastic.co/t/updating-logstahs-7-9-to-7-16-2/298299 "2022-02-25T17:06:05Z")
updating logstash from 7.9.0 to 7.16.2 using helm chat. but getting error. :39:59,607\]\[ERROR\]\[logstash.agent \] Failed to execute action {:action=\>LogStash::PipelineAction::Create/pipeline\_id:main, :exception=\>… --- ## [Logstash output loki tcp error](https://discuss.elastic.co/t/logstash-output-loki-tcp-error/298298)
**Author:** [@bianca6](https://discuss.elastic.co/u/bianca6)\ **Replies:** 0\ **Last updated:** [February 25, 2022, 3:44pm UTC](https://discuss.elastic.co/t/logstash-output-loki-tcp-error/298298 "2022-02-25T15:44:41Z")
Hi everyone! I have a first pipeline running well Fileabeat -\> Logstash -\> Elasticsearch. Now I'm trying to run Fileabeat -\> Logstash -\> Loki but there is a connection error, the message come from Filebeat: {"log.leve… --- ## [Splitting message log](https://discuss.elastic.co/t/splitting-message-log/296805)
**Author:** [@wizard](https://discuss.elastic.co/u/wizard)\ **Replies:** 3\ **Last updated:** [February 25, 2022, 3:04pm UTC](https://discuss.elastic.co/t/splitting-message-log/296805 "2022-02-25T15:04:17Z")
Hello, When I am collecting logs from Syslog and transferring those logs to kibana using filebeat, I am getting a tab named by message in row format as below mention: I want this to be in a separate field like below… --- ## [Filter JSON Array Using Logstash](https://discuss.elastic.co/t/filter-json-array-using-logstash/298253)
**Author:** [@AJEE123](https://discuss.elastic.co/u/AJEE123)\ **Replies:** 1\ **Last updated:** [February 25, 2022, 2:59pm UTC](https://discuss.elastic.co/t/filter-json-array-using-logstash/298253 "2022-02-25T14:59:16Z")
Hi guys, can anybody help me? I have JSON Array as an input like this: "reported\_devices": \[ { "id": "636e6930-6361-3a65-393a-36393a31643a", "ips": \[ … --- ## [How to write log into ElasticSearch (provide by k8s) from logstash?](https://discuss.elastic.co/t/how-to-write-log-into-elasticsearch-provide-by-k8s-from-logstash/298240)
**Author:** [@dalei2019](https://discuss.elastic.co/u/dalei2019)\ **Replies:** 0\ **Last updated:** [February 25, 2022, 4:45am UTC](https://discuss.elastic.co/t/how-to-write-log-into-elasticsearch-provide-by-k8s-from-logstash/298240 "2022-02-25T04:45:18Z")
Hi Team: The current architecture is like this, Logstash writes to the remote (ignore the intranet ip in the error log below) Elasticsearch cluster, but the default Elasticsearch Output Plugin does not take effect. The… --- ## [JSON parse error, original data now in message field](https://discuss.elastic.co/t/json-parse-error-original-data-now-in-message-field/298065)
**Author:** [@d6036de2b54af16665f4](https://discuss.elastic.co/u/d6036de2b54af16665f4)\ **Replies:** 2\ **Last updated:** [February 24, 2022, 8:54pm UTC](https://discuss.elastic.co/t/json-parse-error-original-data-now-in-message-field/298065 "2022-02-24T20:54:13Z")
HI, Error I am receiving in logstash logs is : \[2022-02-22T15:19:27,369\]\[ERROR\]\[logstash.codecs.json \] JSON parse error, original data now in message field {:error=\>#\ **Author:** [@martiros\_martiros](https://discuss.elastic.co/u/martiros_martiros)\ **Replies:** 1\ **Last updated:** [February 24, 2022, 8:49pm UTC](https://discuss.elastic.co/t/cant-send-logs-to-data-view/298176 "2022-02-24T20:49:47Z") I got this in my spring boot this is my conf. file input { file { type =\> "java" path =\> "/UUUU\*\*\*\*\*\*\*\*/IdeaProjects/elk-stack-logging-example/elk-example.log" codec =\> multiline { … --- ## [Log stream with logstash](https://discuss.elastic.co/t/log-stream-with-logstash/298195)
**Author:** [@laolao](https://discuss.elastic.co/u/laolao)\ **Replies:** 1\ **Last updated:** [February 24, 2022, 8:45pm UTC](https://discuss.elastic.co/t/log-stream-with-logstash/298195 "2022-02-24T20:45:11Z")
I have a basic configuration of logstash sending syslogs to Elasticsearch. I created my index pattern and can make visualizations, etc. However, I want to view the logs in real time on my dashboard. My dashboard has 4 d… --- ## [How to automate reading logs?](https://discuss.elastic.co/t/how-to-automate-reading-logs/297873)
**Author:** [@Mariem](https://discuss.elastic.co/u/Mariem)\ **Replies:** 4\ **Last updated:** [February 24, 2022, 1:38pm UTC](https://discuss.elastic.co/t/how-to-automate-reading-logs/297873 "2022-02-24T13:38:14Z")
Hi, i'm using elk on docker and i want to automate reading logs from different server can someone help me ?? --- ## [Logstash 7.17 on Windows - Pipeline running but logs are not present in Elasticsearch](https://discuss.elastic.co/t/logstash-7-17-on-windows-pipeline-running-but-logs-are-not-present-in-elasticsearch/297519)
**Author:** [@bianca6](https://discuss.elastic.co/u/bianca6)\ **Replies:** 5\ **Last updated:** [February 24, 2022, 10:56am UTC](https://discuss.elastic.co/t/logstash-7-17-on-windows-pipeline-running-but-logs-are-not-present-in-elasticsearch/297519 "2022-02-24T10:56:03Z")
Hi, I'm on Windows 10. I'm building a pipeline with Filebeat / Logstash / Elasticsearch / Kibana. Each one of them is at the latest version (downloaded thie previous week). I'm trying to ship a simple log file (ndjson… --- ## [Pipeline number of arguments error](https://discuss.elastic.co/t/pipeline-number-of-arguments-error/297367)
**Author:** [@Michael\_Bailey](https://discuss.elastic.co/u/Michael_Bailey)\ **Replies:** 4\ **Last updated:** [February 24, 2022, 10:13am UTC](https://discuss.elastic.co/t/pipeline-number-of-arguments-error/297367 "2022-02-24T10:13:14Z")
Hi, For some reason I have started to get the following fatal error with logstash 8.0.0. \[LogStash::Runner\] runner - An unexpected error occurred! {:error=\>#\