# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=58 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 59 --- ## [Logstash cannot connect to my postgresql database, they are both running in docker containers on the same compose network](https://discuss.elastic.co/t/logstash-cannot-connect-to-my-postgresql-database-they-are-both-running-in-docker-containers-on-the-same-compose-network/343668)
**Author:** [@descholar-ceo](https://discuss.elastic.co/u/descholar-ceo)\ **Replies:** 0\ **Last updated:** [September 23, 2023, 9:33am UTC](https://discuss.elastic.co/t/logstash-cannot-connect-to-my-postgresql-database-they-are-both-running-in-docker-containers-on-the-same-compose-network/343668 "2023-09-23T09:33:39Z")
I am looking for a help, I am struggling with connecting Logstash to my postgres db, they are both running on the same docker compose network and the connection string I passed works from my application which is running … --- ## [Logstash Condiational Filtering Issue with Geo Location](https://discuss.elastic.co/t/logstash-condiational-filtering-issue-with-geo-location/343596)
**Author:** [@M\_Hatam](https://discuss.elastic.co/u/M_Hatam)\ **Replies:** 1\ **Last updated:** [September 22, 2023, 5:57pm UTC](https://discuss.elastic.co/t/logstash-condiational-filtering-issue-with-geo-location/343596 "2023-09-22T17:57:09Z")
Hi Everyone, Sorry if this is answered somewhere else and I would appreciate if you can help. I'm sending logs from FortiGate to Logstash and I want to set geoip location to be sent to Elasticsearch. Since some traffic… --- ## [ECS Field Name (Fortigate Dataset)](https://discuss.elastic.co/t/ecs-field-name-fortigate-dataset/343492)
**Author:** [@wwalker](https://discuss.elastic.co/u/wwalker)\ **Replies:** 3\ **Last updated:** [September 22, 2023, 5:19pm UTC](https://discuss.elastic.co/t/ecs-field-name-fortigate-dataset/343492 "2023-09-22T17:19:13Z")
I am collecting Fortigate logs with the Elastic Agent and the Fortigate integration. These are then being shipped to Logstash for some custom enrichment before being pushed to Elastic Cloud. One of the custom enrichmen… --- ## [{:exception=\>"Java::OrgLogstash::MissingConverterException: Missing Converter handling for full class name=org.bson.types.ObjectId, simple name=ObjectId"}](https://discuss.elastic.co/t/exception-java-missing-converter-handling-for-full-class-name-org-bson-types-objectid-simple-name-objectid/343636)
**Author:** [@EL\_MALKI\_MOHAMED](https://discuss.elastic.co/u/EL_MALKI_MOHAMED)\ **Replies:** 0\ **Last updated:** [September 22, 2023, 3:26pm UTC](https://discuss.elastic.co/t/exception-java-missing-converter-handling-for-full-class-name-org-bson-types-objectid-simple-name-objectid/343636 "2023-09-22T15:26:13Z")
It not works. Please help me.the problem still persists this is my config : input { jdbc { jdbc\_driver\_library =\> "/etc/logstash/jdbc/mongojdbc3.1.jar" jdbc\_driver\_class =\> "com.dbschema… --- ## [Logstash jdk vulnerability](https://discuss.elastic.co/t/logstash-jdk-vulnerability/343578)
**Author:** [@eh2021-elastic](https://discuss.elastic.co/u/eh2021-elastic)\ **Replies:** 1\ **Last updated:** [September 22, 2023, 11:12am UTC](https://discuss.elastic.co/t/logstash-jdk-vulnerability/343578 "2023-09-22T11:12:48Z")
We are currently running logstash 7.16.3 but are getting flagged for the version of JDK 11.0.13 that it is using and are being told we need to upgrade the JDK version to something higher that 11.0.13. How do one upgrade… --- ## [Failed to parse field \[request.body\] of type \[text\] in document with id](https://discuss.elastic.co/t/failed-to-parse-field-request-body-of-type-text-in-document-with-id/343543)
**Author:** [@kaldaray](https://discuss.elastic.co/u/kaldaray)\ **Replies:** 1\ **Last updated:** [September 21, 2023, 1:06pm UTC](https://discuss.elastic.co/t/failed-to-parse-field-request-body-of-type-text-in-document-with-id/343543 "2023-09-21T13:06:52Z")
Hi all, i have the following log {"@timestamp": "2023-09-21T15:04:43.583+03:00","@version": "1","message": "Request log","thread\_name": "http-nio-8080-exec-9","level": "INFO","level\_value": 20000,"X-REQUEST-ID": "ca17be… --- ## [How to make multiple lines of json file to single line json file](https://discuss.elastic.co/t/how-to-make-multiple-lines-of-json-file-to-single-line-json-file/343424)
**Author:** [@Narayan\_Rao](https://discuss.elastic.co/u/Narayan_Rao)\ **Replies:** 2\ **Last updated:** [September 21, 2023, 11:29am UTC](https://discuss.elastic.co/t/how-to-make-multiple-lines-of-json-file-to-single-line-json-file/343424 "2023-09-21T11:29:08Z")
I am having issue with multiple lines json file. With single line json file able to process the file with below configuration. logstash.conf input { beats{ port =\> "5044" codec =\> json } } filter { json… --- ## [How to parse multiple nested arrays](https://discuss.elastic.co/t/how-to-parse-multiple-nested-arrays/343409)
**Author:** [@Ankita\_Pachauri](https://discuss.elastic.co/u/Ankita_Pachauri)\ **Replies:** 2\ **Last updated:** [September 21, 2023, 8:17am UTC](https://discuss.elastic.co/t/how-to-parse-multiple-nested-arrays/343409 "2023-09-21T08:17:16Z")
Hello everyone, I am trying to parse a json document using logstash version 8.3.3. The json document has multiple nested arrays, to flatten the document split is being used inside the filter. The issue is that the split… --- ## [Logstash not create index in Elasticsearch](https://discuss.elastic.co/t/logstash-not-create-index-in-elasticsearch/343429)
**Author:** [@Raffy\_Revanza](https://discuss.elastic.co/u/Raffy_Revanza)\ **Replies:** 3\ **Last updated:** [September 21, 2023, 7:15am UTC](https://discuss.elastic.co/t/logstash-not-create-index-in-elasticsearch/343429 "2023-09-21T07:15:15Z")
So, i want to send my CSV file in my laptop to elasticsearch to build a dashboard. I have configured the conf files and it success on my logstash, but the index not readable by elastic ? why ? here's the logs "response:… --- ## [How to get or extract Count of fields using logstash](https://discuss.elastic.co/t/how-to-get-or-extract-count-of-fields-using-logstash/343501)
**Author:** [@Ayushi\_bhardwaj](https://discuss.elastic.co/u/Ayushi_bhardwaj)\ **Replies:** 0\ **Last updated:** [September 21, 2023, 4:42am UTC](https://discuss.elastic.co/t/how-to-get-or-extract-count-of-fields-using-logstash/343501 "2023-09-21T04:42:50Z")
Hello, I want to extract or get count of fields using logstash. Below is the query i have written but when i put this query in logstash it does not work this query does not give me the count. pls help GET /abc-int-apl… --- ## [Aggregate function help](https://discuss.elastic.co/t/aggregate-function-help/343432)
**Author:** [@Ameeruddin\_Mohammed](https://discuss.elastic.co/u/Ameeruddin_Mohammed)\ **Replies:** 3\ **Last updated:** [September 20, 2023, 11:55pm UTC](https://discuss.elastic.co/t/aggregate-function-help/343432 "2023-09-20T23:55:05Z")
hi, i have logs in this format and i want to start the aggregation when start comes in the line and end the aggregation when end occurs. the aggregation is based on "username". i was able to use aggregate function but… --- ## [Logstash with email output plugin "no such file to load -- net/smtp"](https://discuss.elastic.co/t/logstash-with-email-output-plugin-no-such-file-to-load-net-smtp/343490)
**Author:** [@lee.clemens](https://discuss.elastic.co/u/lee.clemens)\ **Replies:** 2\ **Last updated:** [September 20, 2023, 9:21pm UTC](https://discuss.elastic.co/t/logstash-with-email-output-plugin-no-such-file-to-load-net-smtp/343490 "2023-09-20T21:21:47Z")
Hello, I recently upgraded RHEL 9 from logstash-8.8.2-1.x86\_64 to logstash-8.10.1-1.x86\_64 and now logstash fails to start. We are using the email output plugin and see this error in the logs: \[2023-09-20T13:48:49,401… --- ## [How can I remove the duplicate in the logs and prevent to create new docs](https://discuss.elastic.co/t/how-can-i-remove-the-duplicate-in-the-logs-and-prevent-to-create-new-docs/343417)
**Author:** [@Cruz](https://discuss.elastic.co/u/Cruz)\ **Replies:** 7\ **Last updated:** [September 20, 2023, 5:34pm UTC](https://discuss.elastic.co/t/how-can-i-remove-the-duplicate-in-the-logs-and-prevent-to-create-new-docs/343417 "2023-09-20T17:34:44Z")
Hello everyone! I have this logs { "\_index": ".ds-my-neoada-stream-2023.09.14-000005", "\_id": "T8v5sIoB0eBbzdbCLRnW", "\_version": 1, "\_score": 0, "\_source": { "from\_plant": "N/A", "tick\_current": "IT-… --- ## [Invalid cron expression for schedule field of elasticsearch input plugin](https://discuss.elastic.co/t/invalid-cron-expression-for-schedule-field-of-elasticsearch-input-plugin/343478)
**Author:** [@mikec1](https://discuss.elastic.co/u/mikec1)\ **Replies:** 5\ **Last updated:** [September 20, 2023, 5:08pm UTC](https://discuss.elastic.co/t/invalid-cron-expression-for-schedule-field-of-elasticsearch-input-plugin/343478 "2023-09-20T17:08:07Z")
I have a pipeline whereby the input section looks like this: input { elasticsearch { hosts =\> \["elasticsearch.my.host.here:port"\] index =\> 'my\_index\_name\_pattern' query =\> '{ "query": { "mat… --- ## [Use variable with logstash](https://discuss.elastic.co/t/use-variable-with-logstash/343462)
**Author:** [@sam1975](https://discuss.elastic.co/u/sam1975)\ **Replies:** 1\ **Last updated:** [September 20, 2023, 1:48pm UTC](https://discuss.elastic.co/t/use-variable-with-logstash/343462 "2023-09-20T13:48:41Z")
Hello I have a long list like this if \[monitoring\_data\_name\] == "componentFault" { pipeline { send\_to =\> "componentFault" } } else if \[monitoring\_data\_name\] == "localAccountPasswordModification" { pipeline { send\_t… --- ## [Logstash json input file only required fields to output](https://discuss.elastic.co/t/logstash-json-input-file-only-required-fields-to-output/342400)
**Author:** [@Narayan\_Rao](https://discuss.elastic.co/u/Narayan_Rao)\ **Replies:** 11\ **Last updated:** [September 20, 2023, 12:53pm UTC](https://discuss.elastic.co/t/logstash-json-input-file-only-required-fields-to-output/342400 "2023-09-20T12:53:37Z")
I'm new in ELK & I have logs in JSON format. Below is the json sample log file. I want only item level array, others fields not required. Sample logs { "source": "mdm/pim", "topic": "pim-record-globalfields", "subj… --- ## [Logstash JDBC plugin](https://discuss.elastic.co/t/logstash-jdbc-plugin/343456)
**Author:** [@Akulainelastic](https://discuss.elastic.co/u/Akulainelastic)\ **Replies:** 0\ **Last updated:** [September 20, 2023, 12:04pm UTC](https://discuss.elastic.co/t/logstash-jdbc-plugin/343456 "2023-09-20T12:04:15Z")
Hello All , so I have a requirement where I need to use JDBC plugin to fetch the database data and reflect it in kibana , although I have successfully ingested data and pipelined it in logstash , the pipelines are runnin… --- ## [What is the max throughput of the stdout?](https://discuss.elastic.co/t/what-is-the-max-throughput-of-the-stdout/343416)
**Author:** [@Daniel9](https://discuss.elastic.co/u/Daniel9)\ **Replies:** 0\ **Last updated:** [September 20, 2023, 5:00am UTC](https://discuss.elastic.co/t/what-is-the-max-throughput-of-the-stdout/343416 "2023-09-20T05:00:14Z")
Here is my out output configration below: output { stdout { codec =\> json\_lines } } Here is my verification result: |Logs/s(In)|Logs/s (out)|Bytes/log (out)|Queue increase size (m)| |1500|200|580|700m| |780|200|5… --- ## [Pfelk logstash data parsing](https://discuss.elastic.co/t/pfelk-logstash-data-parsing/343284)
**Author:** [@kozistan](https://discuss.elastic.co/u/kozistan)\ **Replies:** 8\ **Last updated:** [September 19, 2023, 7:54pm UTC](https://discuss.elastic.co/t/pfelk-logstash-data-parsing/343284 "2023-09-19T19:54:03Z")
Hello would appreciate help with logstash parsing data into elastisearch. Please check my log output. Using opnsense syslog to logstash's pfelk addon and can not figure out whe right mutate filter to get this done. Thank… --- ## [Logstash 7.x Log4j CVE remediation on Windows server](https://discuss.elastic.co/t/logstash-7-x-log4j-cve-remediation-on-windows-server/343374)
**Author:** [@newschapmj1](https://discuss.elastic.co/u/newschapmj1)\ **Replies:** 0\ **Last updated:** [September 19, 2023, 2:22pm UTC](https://discuss.elastic.co/t/logstash-7-x-log4j-cve-remediation-on-windows-server/343374 "2023-09-19T14:22:04Z")
Logstash 7.x Log4j Windows script Contains Linux and MacOs Installations and Docker. Has anyone got the same script/steps for Windows server? --- ## [Logstash - How to Dynamic Parse Log's value](https://discuss.elastic.co/t/logstash-how-to-dynamic-parse-logs-value/343125)
**Author:** [@Huy\_Hoang\_Le](https://discuss.elastic.co/u/Huy_Hoang_Le)\ **Replies:** 8\ **Last updated:** [September 19, 2023, 3:36am UTC](https://discuss.elastic.co/t/logstash-how-to-dynamic-parse-logs-value/343125 "2023-09-19T03:36:25Z")
Hi I have this sample Document \[Thread-13\]\[2023-09-15 09:32:35\]\[INFO\]:{'\[Sub\]0-BaseTransformer\]': '0.0004', '\[Sub\]1-NGINX Feature Extractor Service\]': '0.0135', '\[Dataloader\]\[#0.-PutToQueue\]': '0.0005', '\[Sub\]\[#1.EMA\_FP… --- ## [Logstash vulnerabilities around ruby-maven-libs](https://discuss.elastic.co/t/logstash-vulnerabilities-around-ruby-maven-libs/343278)
**Author:** [@balakr](https://discuss.elastic.co/u/balakr)\ **Replies:** 2\ **Last updated:** [September 18, 2023, 4:23pm UTC](https://discuss.elastic.co/t/logstash-vulnerabilities-around-ruby-maven-libs/343278 "2023-09-18T16:23:12Z")
my company is pushing me for fixing vulnerablities in logstash, at this point i am in learning mode. when i looking at the below vulnerablity, does this need ruby-maven-libs upgrade or just guava upgrade Required\_Versi… --- ## [Error installing gems (\> invalid source release: 11)](https://discuss.elastic.co/t/error-installing-gems-invalid-source-release-11/343201)
**Author:** [@balakr](https://discuss.elastic.co/u/balakr)\ **Replies:** 2\ **Last updated:** [September 18, 2023, 3:31pm UTC](https://discuss.elastic.co/t/error-installing-gems-invalid-source-release-11/343201 "2023-09-18T15:31:23Z")
Task :downloadPreviousJRuby UP-TO-DATE Task :downloadJRuby UP-TO-DATE Download jruby-dist-9.3.10.0-bin.tar.gz Task :benchmark-cli:compileJava FAILED FAILURE: Build failed with an exception. What went wrong: … --- ## [Send logs Citrix to logstash/elasticsearch](https://discuss.elastic.co/t/send-logs-citrix-to-logstash-elasticsearch/343263)
**Author:** [@mulbzh](https://discuss.elastic.co/u/mulbzh)\ **Replies:** 0\ **Last updated:** [September 18, 2023, 1:01pm UTC](https://discuss.elastic.co/t/send-logs-citrix-to-logstash-elasticsearch/343263 "2023-09-18T13:01:52Z")
Hello and sorry for my bad english :slight\_smile: , I am new in logstash/elasticsearch. I have a server installed by older technician. So, i understand globally how it works but i have one trouble. I send logs from my… --- ## [Grok not parsing](https://discuss.elastic.co/t/grok-not-parsing/343098)
**Author:** [@pshas](https://discuss.elastic.co/u/pshas)\ **Replies:** 1\ **Last updated:** [September 18, 2023, 9:24am UTC](https://discuss.elastic.co/t/grok-not-parsing/343098 "2023-09-18T09:24:33Z")
logstash.conf # Sample Logstash configuration for creating a simple # Beats -\> Logstash -\> Elasticsearch pipeline. input { beats { port =\> 5044 type = "test" } } filter { if \[type\] == "log" { grok { … --- ## [Logstash stop working due to FFI not available: null](https://discuss.elastic.co/t/logstash-stop-working-due-to-ffi-not-available-null/343174)
**Author:** [@lalchand\_rajak](https://discuss.elastic.co/u/lalchand_rajak)\ **Replies:** 2\ **Last updated:** [September 18, 2023, 5:37am UTC](https://discuss.elastic.co/t/logstash-stop-working-due-to-ffi-not-available-null/343174 "2023-09-18T05:37:38Z")
Logstash stopped working due to FFI not available: null . I have already provided the tmp path in Jvm.options # set the I/O temp directory #-Djava.io.tmpdir=$HOME -Djava.io.tmpdir=/home/apmuser/tmp drwxrwxr-x. 2 logsta… --- ## [Fixing vulnerablities in logstash code](https://discuss.elastic.co/t/fixing-vulnerablities-in-logstash-code/343168)
**Author:** [@balakr](https://discuss.elastic.co/u/balakr)\ **Replies:** 1\ **Last updated:** [September 16, 2023, 5:05am UTC](https://discuss.elastic.co/t/fixing-vulnerablities-in-logstash-code/343168 "2023-09-16T05:05:34Z")
my company check for vulnerablities and i see bunch of vulnerablities in logstash. an example is below to fix this vulnerablity, should i upgrade guava or does jruby needs to be upgraded. if jruby needs to be upgraded … --- ## [Logstash container not receiving log files from Filebeats running on host](https://discuss.elastic.co/t/logstash-container-not-receiving-log-files-from-filebeats-running-on-host/343162)
**Author:** [@David\_Locarno](https://discuss.elastic.co/u/David_Locarno)\ **Replies:** 1\ **Last updated:** [September 15, 2023, 10:48pm UTC](https://discuss.elastic.co/t/logstash-container-not-receiving-log-files-from-filebeats-running-on-host/343162 "2023-09-15T22:48:20Z")
I am running a RHEL VM with Filebeats installed and three Podman containers running Kibana, Elasticsearch, and Logstash. Almost everything works, except for sending files from Filebeats to my Logstash container's pipelin… --- ## [Cannot parse logs - problem with multiline parse failures](https://discuss.elastic.co/t/cannot-parse-logs-problem-with-multiline-parse-failures/343146)
**Author:** [@danmed](https://discuss.elastic.co/u/danmed)\ **Replies:** 4\ **Last updated:** [September 15, 2023, 10:27pm UTC](https://discuss.elastic.co/t/cannot-parse-logs-problem-with-multiline-parse-failures/343146 "2023-09-15T22:27:04Z")
Hi All, I am having a lot of problems parsing logs especially with different dates and logs having multiline tags. For example: A head (very first 10 lines) of one of my log files, specifically, catalina.out, could be… --- ## [Drop filter in Logstash filter not working for the below event](https://discuss.elastic.co/t/drop-filter-in-logstash-filter-not-working-for-the-below-event/343120)
**Author:** [@Subrato1](https://discuss.elastic.co/u/Subrato1)\ **Replies:** 1\ **Last updated:** [September 15, 2023, 6:31pm UTC](https://discuss.elastic.co/t/drop-filter-in-logstash-filter-not-working-for-the-below-event/343120 "2023-09-15T18:31:11Z")
Event in Logstash : { "timestamp" =\> "2023-09-13T05:10:52.527038098Z", "user" =\> "admin", "type" =\> "icd\_postgresql", "status" =\> "INSERT INTO t1 SELECT i/100, i/500 FROM generate\_series(1,1… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=57) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=59)