# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=172 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 173 --- ## [Calculate time difference between two log lines - Logstash](https://discuss.elastic.co/t/calculate-time-difference-between-two-log-lines-logstash/292284)
**Author:** [@giancringo](https://discuss.elastic.co/u/giancringo)\ **Replies:** 4\ **Last updated:** [December 22, 2021, 2:56pm UTC](https://discuss.elastic.co/t/calculate-time-difference-between-two-log-lines-logstash/292284 "2021-12-22T14:56:34Z")
Hello, I'm looking for a solution to calculate the difference between two time on two different log lines as below: 2020-06-11 15:31:22,817 \[http-8080-0\] INFO it.example.sign.sign.SignTool - \[TYPE\] trackingID: x12e65… --- ## [Delete and recreate index in elasticsearch](https://discuss.elastic.co/t/delete-and-recreate-index-in-elasticsearch/292651)
**Author:** [@Vikrant1](https://discuss.elastic.co/u/Vikrant1)\ **Replies:** 1\ **Last updated:** [December 22, 2021, 2:17pm UTC](https://discuss.elastic.co/t/delete-and-recreate-index-in-elasticsearch/292651 "2021-12-22T14:17:51Z")
Hi All, Thanks for taking your time to read this query - I have to delete the existing index and the need to create same Index in Elasticsearch using the logstash config file before data loading start (reading from or… --- ## [Logstash S3 input plugin - filter based on time modified](https://discuss.elastic.co/t/logstash-s3-input-plugin-filter-based-on-time-modified/292581)
**Author:** [@kmualem](https://discuss.elastic.co/u/kmualem)\ **Replies:** 9\ **Last updated:** [December 22, 2021, 2:03pm UTC](https://discuss.elastic.co/t/logstash-s3-input-plugin-filter-based-on-time-modified/292581 "2021-12-22T14:03:27Z")
I have a Logstash container that is configured to read objects from S3. The requirement is to filter old objects, let's say objects before 3 months should be dropped. I noticed that I can expose the s3 metadata, so I h… --- ## [How to parse nested json events](https://discuss.elastic.co/t/how-to-parse-nested-json-events/292039)
**Author:** [@Anusha\_Kusanghi](https://discuss.elastic.co/u/Anusha_Kusanghi)\ **Replies:** 3\ **Last updated:** [December 22, 2021, 2:02pm UTC](https://discuss.elastic.co/t/how-to-parse-nested-json-events/292039 "2021-12-22T14:02:10Z")
Hello! I have nested json objects like this : { "\_index": "impe\_logs-2021.11.19-1", "\_type": "\_doc", "\_id": "tyu", "\_version": 1, "\_score": null, "\_source": { "port": 54640, "impe": { "abp": {… --- ## [Logstash JDBC input metadata is overriding](https://discuss.elastic.co/t/logstash-jdbc-input-metadata-is-overriding/292691)
**Author:** [@Vinayak\_G](https://discuss.elastic.co/u/Vinayak_G)\ **Replies:** 0\ **Last updated:** [December 22, 2021, 1:47pm UTC](https://discuss.elastic.co/t/logstash-jdbc-input-metadata-is-overriding/292691 "2021-12-22T13:47:41Z")
Hey Everybody I am new to ELK I was running logstash 6.8.3 on my machine, I wanted to upgrade to 6.8.21, so I created a new machine and installed logstash 6.8.21, then I pushed my pipeline's and metadata files so that … --- ## [ERROR logstash.inputs.jdbc](https://discuss.elastic.co/t/error-logstash-inputs-jdbc/292610)
**Author:** [@Gabriel\_Munoz](https://discuss.elastic.co/u/Gabriel_Munoz)\ **Replies:** 1\ **Last updated:** [December 22, 2021, 12:21pm UTC](https://discuss.elastic.co/t/error-logstash-inputs-jdbc/292610 "2021-12-22T12:21:58Z")
El controlador no pudo establecer una conexión segura con SQL Server con el cifrado de Capa de sockets seguros (SSL). Error: "No appropriate protocol (protocol is disabled or cipher suites are inappropriate) \[2021-12-21… --- ## [Logstash S3 output plugin, how do we get logging?](https://discuss.elastic.co/t/logstash-s3-output-plugin-how-do-we-get-logging/292615)
**Author:** [@Colin\_K](https://discuss.elastic.co/u/Colin_K)\ **Replies:** 6\ **Last updated:** [December 22, 2021, 11:07am UTC](https://discuss.elastic.co/t/logstash-s3-output-plugin-how-do-we-get-logging/292615 "2021-12-22T11:07:16Z")
When I use the file output plugin I see INFO log lines that we are opening and closing a file to write to. But when using the s3 output plugin I get writes but no logging. log4j2.properties status = debug name = Logst… --- ## [Urgent - Incomplete fix for Apache Log4j vulnerability v2.15.0](https://discuss.elastic.co/t/urgent-incomplete-fix-for-apache-log4j-vulnerability-v2-15-0/291893)
**Author:** [@ppafford](https://discuss.elastic.co/u/ppafford)\ **Replies:** 2\ **Last updated:** [December 22, 2021, 4:58am UTC](https://discuss.elastic.co/t/urgent-incomplete-fix-for-apache-log4j-vulnerability-v2-15-0/291893 "2021-12-22T04:58:48Z")
Looks like there is another issue and another fix And the release of Logstash 7.16.1 Release Notes | Logstash Reference \[7.16\] | Elastic has the incomplete fix. When can this be addressed? --- ## [Logstash Ruby plugin code block not raising Exception](https://discuss.elastic.co/t/logstash-ruby-plugin-code-block-not-raising-exception/291376)
**Author:** [@Colin\_K](https://discuss.elastic.co/u/Colin_K)\ **Replies:** 4\ **Last updated:** [December 21, 2021, 6:57pm UTC](https://discuss.elastic.co/t/logstash-ruby-plugin-code-block-not-raising-exception/291376 "2021-12-21T18:57:35Z")
I have a ruby block. The "happy" path works where I see the expected map output. When I provide an invalid scheme for an Array with a log line and an element that does not contain an object with field, I do not throw the… --- ## [Logstash stopped processing because of an error: (SystemExit) exit org.jruby.exceptions.SystemExit: (SystemExit) exit](https://discuss.elastic.co/t/logstash-stopped-processing-because-of-an-error-systemexit-exit-org-jruby-exceptions-systemexit-systemexit-exit/292432)
**Author:** [@tobias.greis](https://discuss.elastic.co/u/tobias.greis)\ **Replies:** 4\ **Last updated:** [December 21, 2021, 5:20pm UTC](https://discuss.elastic.co/t/logstash-stopped-processing-because-of-an-error-systemexit-exit-org-jruby-exceptions-systemexit-systemexit-exit/292432 "2021-12-21T17:20:08Z")
Hi, I have a problem with my Logstash Component. When starting I get the following failure and can't find helpful information online. I use a SLES 15 with Logstash 7.16.1 and OpenJDK11. \[2021-12-20T07:56:23,219\]\[INFO \]… --- ## [Logstash update doc fields if exits](https://discuss.elastic.co/t/logstash-update-doc-fields-if-exits/292206)
**Author:** [@patilanna](https://discuss.elastic.co/u/patilanna)\ **Replies:** 3\ **Last updated:** [December 21, 2021, 5:13pm UTC](https://discuss.elastic.co/t/logstash-update-doc-fields-if-exits/292206 "2021-12-21T17:13:26Z")
Blockquote logstash to update doc fields if exits or insert new to ELK stack, sorry if any thing not as per forum I am sending kafka stream of events logs, there was two types of logs, open-event log and close-event l… --- ## [Logstash to logstash not indexing logs when is enabled](https://discuss.elastic.co/t/logstash-to-logstash-not-indexing-logs-when-is-enabled/292546)
**Author:** [@jhaos](https://discuss.elastic.co/u/jhaos)\ **Replies:** 1\ **Last updated:** [December 21, 2021, 5:12pm UTC](https://discuss.elastic.co/t/logstash-to-logstash-not-indexing-logs-when-is-enabled/292546 "2021-12-21T17:12:27Z")
Hi, we are facing an issue using Logstash to Logstash configuration Logstash-to-Logstash Communication | Logstash Reference \[7.16\] | Elastic. Our architecture is the following using two different inputs in filebeat: Fil… --- ## [Logstash Rspec Test Library, how does it fundamentally work when creating pipelines?](https://discuss.elastic.co/t/logstash-rspec-test-library-how-does-it-fundamentally-work-when-creating-pipelines/292537)
**Author:** [@UXabre](https://discuss.elastic.co/u/UXabre)\ **Replies:** 0\ **Last updated:** [December 21, 2021, 8:54am UTC](https://discuss.elastic.co/t/logstash-rspec-test-library-how-does-it-fundamentally-work-when-creating-pipelines/292537 "2021-12-21T08:54:33Z")
So, I've been using rspec for a while now while testing my "simple" pipeline configs. However, as of recent, I'd like to use "complex" filters like "aggregate" and "memcached" but it is hard, if not impossible to find t… --- ## [Logstash pipelines failing after JNDI lookup class removed from version 7.4.2](https://discuss.elastic.co/t/logstash-pipelines-failing-after-jndi-lookup-class-removed-from-version-7-4-2/292539)
**Author:** [@Rahul\_Dey](https://discuss.elastic.co/u/Rahul_Dey)\ **Replies:** 1\ **Last updated:** [December 21, 2021, 9:07am UTC](https://discuss.elastic.co/t/logstash-pipelines-failing-after-jndi-lookup-class-removed-from-version-7-4-2/292539 "2021-12-21T09:07:11Z")
For threat mitigation of log4j vulnerability in Logstash, I have removed that JNDI look up class from Logstash docker image by adding the below lines in Dockerfile as advised by Elastic team. RUN bin/ruby -rzip -e \\ '… --- ## [CVE-2021-44228 - Logstahs - ERROR StatusLogger Log4j2 could not find a logging implementation. Please add log4j-core to the classpath](https://discuss.elastic.co/t/cve-2021-44228-logstahs-error-statuslogger-log4j2-could-not-find-a-logging-implementation-please-add-log4j-core-to-the-classpath/291798)
**Author:** [@prajyodh](https://discuss.elastic.co/u/prajyodh)\ **Replies:** 3\ **Last updated:** [December 21, 2021, 6:41am UTC](https://discuss.elastic.co/t/cve-2021-44228-logstahs-error-statuslogger-log4j2-could-not-find-a-logging-implementation-please-add-log4j-core-to-the-classpath/291798 "2021-12-21T06:41:37Z")
Hi I was checking on impact of CVE-2021-44228 on ELK stack and came across the link When i tried the solution mentioned for logstash to remove the JNDI Class zip -q -d \/logstash-core/lib/jars/log4j-c… --- ## [Logstash unable to connect to Elasticsearch over https](https://discuss.elastic.co/t/logstash-unable-to-connect-to-elasticsearch-over-https/292362)
**Author:** [@Bastian\_Jager](https://discuss.elastic.co/u/Bastian_Jager)\ **Replies:** 6\ **Last updated:** [December 20, 2021, 8:38pm UTC](https://discuss.elastic.co/t/logstash-unable-to-connect-to-elasticsearch-over-https/292362 "2021-12-20T20:38:59Z")
Hello, I am facing connection issues from logstash to my elastic stack. The very strange this is, that it worked until yesterday. Today (without any modification from my side) I get: Dec 17 22:00:55 ubuntu logstash-app… --- ## [What exactly does sincedb\_path do?](https://discuss.elastic.co/t/what-exactly-does-sincedb-path-do/292504)
**Author:** [@kibanauser4](https://discuss.elastic.co/u/kibanauser4)\ **Replies:** 1\ **Last updated:** [December 20, 2021, 8:09pm UTC](https://discuss.elastic.co/t/what-exactly-does-sincedb-path-do/292504 "2021-12-20T20:09:50Z")
I'm having little trouble understanding Logstash a bit. I've already imported a file using Logstash and I've tried to update the index but it updates it only if I set sincedb\_path to NUL. Can someone explain? Thanks. --- ## [OSS Logstash 7.16.1 incompatible with Elasticsearch 7.10.2](https://discuss.elastic.co/t/oss-logstash-7-16-1-incompatible-with-elasticsearch-7-10-2/292283)
**Author:** [@AlanMark](https://discuss.elastic.co/u/AlanMark)\ **Replies:** 2\ **Last updated:** [December 20, 2021, 3:20pm UTC](https://discuss.elastic.co/t/oss-logstash-7-16-1-incompatible-with-elasticsearch-7-10-2/292283 "2021-12-20T15:20:09Z")
I am running an upgrade from some older versions of opendistro and OSS logstash to the latest versions, in order to mitigate some of the latest vulnerabilities. According to Elastic, Logstash OSS 7.16.x should be compat… --- ## [Logstash error](https://discuss.elastic.co/t/logstash-error/291738)
**Author:** [@Jasmina\_Desai](https://discuss.elastic.co/u/Jasmina_Desai)\ **Replies:** 1\ **Last updated:** [December 20, 2021, 2:40pm UTC](https://discuss.elastic.co/t/logstash-error/291738 "2021-12-20T14:40:23Z")
We are currently using logstash OSS version 7.12.0 downloaded from Past Releases of Elastic Stack Software | Elastic. However, the version 7.16.1 that has the log4j2 vulnerability fix throws a 404 error. How do I get t… --- ## [Logstash CSV Timestamp extract](https://discuss.elastic.co/t/logstash-csv-timestamp-extract/292188)
**Author:** [@Robsen\_Inc](https://discuss.elastic.co/u/Robsen_Inc)\ **Replies:** 5\ **Last updated:** [December 20, 2021, 9:38am UTC](https://discuss.elastic.co/t/logstash-csv-timestamp-extract/292188 "2021-12-20T09:38:42Z")
Hello everyone, I have a problem with my log data. After hours of try-and-error I am now looking for help from you. An exemplary line looks like this: 09/04/2021 11:30:53 0 0 0 0 0 0 0 0 false false false false false … --- ## [Need help to evaluate Logstash for syncing Oracle data into Elasticsearch (combining multiple 25+ table records into single nested document)](https://discuss.elastic.co/t/need-help-to-evaluate-logstash-for-syncing-oracle-data-into-elasticsearch-combining-multiple-25-table-records-into-single-nested-document/292430)
**Author:** [@Tejas\_Damle](https://discuss.elastic.co/u/Tejas_Damle)\ **Replies:** 1\ **Last updated:** [December 20, 2021, 9:38am UTC](https://discuss.elastic.co/t/need-help-to-evaluate-logstash-for-syncing-oracle-data-into-elasticsearch-combining-multiple-25-table-records-into-single-nested-document/292430 "2021-12-20T09:38:25Z")
Requirement: We are planning to use Logstash to sync data from Oracle to Elasticsearch. There are around 25+ tables in Oracle. Plan is to to have single index in Elasticsearch with nested/flattened documents which will… --- ## [CVE-2021-45046 : Incomplete fix for Apache Log4j vulnerability](https://discuss.elastic.co/t/cve-2021-45046-incomplete-fix-for-apache-log4j-vulnerability/291925)
**Author:** [@Ravi\_GH](https://discuss.elastic.co/u/Ravi_GH)\ **Replies:** 1\ **Last updated:** [December 15, 2021, 9:13pm UTC](https://discuss.elastic.co/t/cve-2021-45046-incomplete-fix-for-apache-log4j-vulnerability/291925 "2021-12-15T21:13:46Z")
With respect to "Incomplete fix for Apache Log4j vulnerability" @ Incomplete fix for Apache Log4j vulnerability · CVE-2021-45046 · GitHub Advisory Database · GitHub I have two questions: Q1) is JndiLookup class remova… --- ## [Logstash update log4j version](https://discuss.elastic.co/t/logstash-update-log4j-version/291946)
**Author:** [@mangeshs](https://discuss.elastic.co/u/mangeshs)\ **Replies:** 1\ **Last updated:** [December 15, 2021, 9:43pm UTC](https://discuss.elastic.co/t/logstash-update-log4j-version/291946 "2021-12-15T21:43:02Z")
I have to change log4j 2.11 to 2.16 so I have replaced all jars. now its saying Problems loading a plugin with {:type=\>"input", :name=\>"beats", :path=\>"logstash/inputs/beats", :error\_message=\>"\\n\\n\\tyou might need to r… --- ## [Grok to multiline ingestion does not handle rows](https://discuss.elastic.co/t/grok-to-multiline-ingestion-does-not-handle-rows/292412)
**Author:** [@tjswe](https://discuss.elastic.co/u/tjswe)\ **Replies:** 2\ **Last updated:** [December 19, 2021, 8:17pm UTC](https://discuss.elastic.co/t/grok-to-multiline-ingestion-does-not-handle-rows/292412 "2021-12-19T20:17:56Z")
Hi! Im ingesting .txt-files containing navtex-messages (see example below). Each message comes in a separate .txt-file appearing continuously over 24h landning in a sub-directory with the current date. input { file… --- ## [Logstash filter CSV inside email content](https://discuss.elastic.co/t/logstash-filter-csv-inside-email-content/292244)
**Author:** [@pcpdoc02](https://discuss.elastic.co/u/pcpdoc02)\ **Replies:** 2\ **Last updated:** [December 19, 2021, 8:08am UTC](https://discuss.elastic.co/t/logstash-filter-csv-inside-email-content/292244 "2021-12-19T08:08:13Z")
I would like to use Logstash to stream CSV data in email to Elasticsearch. IMAP and CSV filter seems suitable to extract email content and parse the CSV contents respectively. How can I use both filter together to output… --- ## [Issector mapping, pattern not found error in Logstash](https://discuss.elastic.co/t/issector-mapping-pattern-not-found-error-in-logstash/292226)
**Author:** [@arunpmohan](https://discuss.elastic.co/u/arunpmohan)\ **Replies:** 2\ **Last updated:** [December 18, 2021, 1:45am UTC](https://discuss.elastic.co/t/issector-mapping-pattern-not-found-error-in-logstash/292226 "2021-12-18T01:45:07Z")
I have an log message which I split using grok and then use an if conditional to parse a field using dissect filter. But it is giving me Dissector mapping, pattern not found error. I tried the dissect pattern with http… --- ## [Logstash using multiline for big log file](https://discuss.elastic.co/t/logstash-using-multiline-for-big-log-file/292150)
**Author:** [@pan1](https://discuss.elastic.co/u/pan1)\ **Replies:** 3\ **Last updated:** [December 17, 2021, 5:18pm UTC](https://discuss.elastic.co/t/logstash-using-multiline-for-big-log-file/292150 "2021-12-17T17:18:50Z")
Hello everyone, I have to handle big log-files (arround 50k lines) using ELK and want to extract some information out of it. A long story short, I want to use filter for searching for specific informations, store thos… --- ## [Update by query in Logstash](https://discuss.elastic.co/t/update-by-query-in-logstash/292333)
**Author:** [@Pablo\_Albertengo](https://discuss.elastic.co/u/Pablo_Albertengo)\ **Replies:** 0\ **Last updated:** [December 17, 2021, 4:09pm UTC](https://discuss.elastic.co/t/update-by-query-in-logstash/292333 "2021-12-17T16:09:30Z")
Hi! I'm trying to solve the following problem. I have some information in the database that I want to send to an Elasticsearch index through Logstash. It is something that I have already done on some occasion but the … --- ## [Logstash failing to build with custom java Plugin after log4j issue fix in 7.16.1 build](https://discuss.elastic.co/t/logstash-failing-to-build-with-custom-java-plugin-after-log4j-issue-fix-in-7-16-1-build/292316)
**Author:** [@Rahulkumar\_Pawar](https://discuss.elastic.co/u/Rahulkumar_Pawar)\ **Replies:** 0\ **Last updated:** [December 17, 2021, 1:50pm UTC](https://discuss.elastic.co/t/logstash-failing-to-build-with-custom-java-plugin-after-log4j-issue-fix-in-7-16-1-build/292316 "2021-12-17T13:50:03Z")
I upgraded logstash source with 7.16.1 to have log4j vulnerability fix. Followed below steps Downloaded source code -- GitHub - elastic/logstash at v7.16.1 for source repo, ran ./gradlew assemble followed by rake … --- ## [FATAL error after updating to 7.16.1](https://discuss.elastic.co/t/fatal-error-after-updating-to-7-16-1/291959)
**Author:** [@maltewhiite](https://discuss.elastic.co/u/maltewhiite)\ **Replies:** 14\ **Last updated:** [December 17, 2021, 2:11pm UTC](https://discuss.elastic.co/t/fatal-error-after-updating-to-7-16-1/291959 "2021-12-17T14:11:19Z")
\[FATAL\] 2021-12-15 11:35:53.567 \[main\] Logstash - Logstash was unable to start due to an unexpected Gemfile change. What is the Gemfile change? How do I debug this? Anyone else experienced this? I can't find anything… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=171) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=173)