# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=92 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 93 --- ## [Logstash jdbc input mantain sql row order](https://discuss.elastic.co/t/logstash-jdbc-input-mantain-sql-row-order/324470)
**Author:** [@Ricardo\_Canuto](https://discuss.elastic.co/u/Ricardo_Canuto)\ **Replies:** 8\ **Last updated:** [February 3, 2023, 2:51pm UTC](https://discuss.elastic.co/t/logstash-jdbc-input-mantain-sql-row-order/324470 "2023-02-03T14:51:27Z")
Hi, I'm new to logstash and I'm trying to check if an sql job (and its steps) runs successfully. The query gets the step 0 (job output) and all the steps that have errors or warnings. I want to gather all the steps… --- ## [Logstash - Dateformat yyyyMMdd HHmmss](https://discuss.elastic.co/t/logstash-dateformat-yyyymmdd-hhmmss/324311)
**Author:** [@A.Klos](https://discuss.elastic.co/u/A.Klos)\ **Replies:** 3\ **Last updated:** [February 3, 2023, 2:08pm UTC](https://discuss.elastic.co/t/logstash-dateformat-yyyymmdd-hhmmss/324311 "2023-02-03T14:08:04Z")
Hi, I have still problem to get right timestamp in elastic. One Row of log looks like: 20210611 111146 SOME Date Field ... I tried: grok { match =\> \[ "message" , "%{DATA:timestamp}" \] } date { … --- ## [Index CSV Timestamp](https://discuss.elastic.co/t/index-csv-timestamp/324579)
**Author:** [@gabrile\_jaime\_gomez](https://discuss.elastic.co/u/gabrile_jaime_gomez)\ **Replies:** 6\ **Last updated:** [February 3, 2023, 1:06pm UTC](https://discuss.elastic.co/t/index-csv-timestamp/324579 "2023-02-03T13:06:11Z")
hello I want to index some csv files. I want to be indexed with the modification date, not with the date entered in Elastic. Example My file has a modification date of 01/23/2022, that is the date that I would like to… --- ## [Logstash cann not read encrypted key](https://discuss.elastic.co/t/logstash-cann-not-read-encrypted-key/324629)
**Author:** [@AfeefGhannam](https://discuss.elastic.co/u/AfeefGhannam)\ **Replies:** 0\ **Last updated:** [February 3, 2023, 9:23am UTC](https://discuss.elastic.co/t/logstash-cann-not-read-encrypted-key/324629 "2023-02-03T09:23:26Z")
Hi, when we create an encrypted and compatible Logstash key, Logstash can not read the key and give the following error in log: message=\>"File does not contain valid private key: /etc/logstash/certs/logstash-pkcs8.key"… --- ## [I am also facing the same issue, did someone found the solution or workaround for this](https://discuss.elastic.co/t/i-am-also-facing-the-same-issue-did-someone-found-the-solution-or-workaround-for-this/324342)
**Author:** [@mohit\_bairagi](https://discuss.elastic.co/u/mohit_bairagi)\ **Replies:** 1\ **Last updated:** [February 3, 2023, 3:50am UTC](https://discuss.elastic.co/t/i-am-also-facing-the-same-issue-did-someone-found-the-solution-or-workaround-for-this/324342 "2023-02-03T03:50:27Z")
Continuing the discussion from S3 Input Plugin Following Errors Execution Expired and Net::OpenTimeout: --- ## [Logstash ruby filter](https://discuss.elastic.co/t/logstash-ruby-filter/324590)
**Author:** [@sheetal3](https://discuss.elastic.co/u/sheetal3)\ **Replies:** 0\ **Last updated:** [February 3, 2023, 2:20am UTC](https://discuss.elastic.co/t/logstash-ruby-filter/324590 "2023-02-03T02:20:15Z")
Getting Error: \[2023-02-02T19:25:48,535\]\[ERROR\]\[logstash.filters.ruby \]\[main\]\[b7126651d97050c2a450765cb1ad946624dc0b4a1ea72baecc762eb17b892bdd\] Ruby exception occurred: undefined method each' for #\ … --- ## [How to read logs with permanent \[NULL\]-characters at the end of file?](https://discuss.elastic.co/t/how-to-read-logs-with-permanent-null-characters-at-the-end-of-file/324408)
**Author:** [@Evgenii\_X](https://discuss.elastic.co/u/Evgenii_X)\ **Replies:** 1\ **Last updated:** [February 2, 2023, 6:46am UTC](https://discuss.elastic.co/t/how-to-read-logs-with-permanent-null-characters-at-the-end-of-file/324408 "2023-02-02T06:46:37Z")
We need to collect MT4 logs (MetaTrader 4 trading platform). Log-saving "Feature" in MT4 is implemented according to the following algorithm: When creating a log file (or adding new logs to the current one), the platfo… --- ## [Syntax error](https://discuss.elastic.co/t/syntax-error/324471)
**Author:** [@moep](https://discuss.elastic.co/u/moep)\ **Replies:** 3\ **Last updated:** [February 2, 2023, 12:32am UTC](https://discuss.elastic.co/t/syntax-error/324471 "2023-02-02T00:32:40Z")
Hey there, I'm working playing with Logstash and wrote alot of grok patterns. But right now, I have a syntax error in this snippet: if \[message\] =~ "SMTP error from remote mail server after RCPT TO" { grok { … --- ## [Java heap oom on logstash](https://discuss.elastic.co/t/java-heap-oom-on-logstash/324426)
**Author:** [@INS](https://discuss.elastic.co/u/INS)\ **Replies:** 2\ **Last updated:** [February 1, 2023, 1:21pm UTC](https://discuss.elastic.co/t/java-heap-oom-on-logstash/324426 "2023-02-01T13:21:03Z")
Hi I'm using the latest one version of logstash 8.6.0 I meet the same processing error every day for one of pipeline in the meantime I decreased of count of workers and event and increased heap but still is not enough … --- ## [Logtash copy one field to another in a different log](https://discuss.elastic.co/t/logtash-copy-one-field-to-another-in-a-different-log/324423)
**Author:** [@tegerei](https://discuss.elastic.co/u/tegerei)\ **Replies:** 0\ **Last updated:** [February 1, 2023, 12:39pm UTC](https://discuss.elastic.co/t/logtash-copy-one-field-to-another-in-a-different-log/324423 "2023-02-01T12:39:45Z")
Hello, I have the following sample log. Feb 1 15:30:49 sudo: pam\_unix(sudo-i:auth): authentication failure; logname= uid=10050 euid=0 tty=/dev/pts/2 user=test Feb 1 15:30:50 sudo: pam\_sss(sudo-i:auth): authent… --- ## [Read Oracle Database Table Data vi Logstash to insert into ElasticSearch](https://discuss.elastic.co/t/read-oracle-database-table-data-vi-logstash-to-insert-into-elasticsearch/323717)
**Author:** [@SP003](https://discuss.elastic.co/u/SP003)\ **Replies:** 3\ **Last updated:** [February 1, 2023, 11:32am UTC](https://discuss.elastic.co/t/read-oracle-database-table-data-vi-logstash-to-insert-into-elasticsearch/323717 "2023-02-01T11:32:17Z")
Dear all, I am trying to connect with table from Oracle Database, and read records through logstash, and then to insert into Elastic search. And the same I want to do in a Linux system. So far I understood that I have … --- ## [Logstash occupies superior folder of log folder](https://discuss.elastic.co/t/logstash-occupies-superior-folder-of-log-folder/324379)
**Author:** [@Steven29](https://discuss.elastic.co/u/Steven29)\ **Replies:** 0\ **Last updated:** [February 1, 2023, 2:27am UTC](https://discuss.elastic.co/t/logstash-occupies-superior-folder-of-log-folder/324379 "2023-02-01T02:27:12Z")
I assigned the path ' /AllLog/Logstash'. But sometimes when I start the logstash, log(log of logstash) is not stacked because the AllLog folder is full(capacity). But when I stop the logstash, the capacity of AllLog f… --- ## [Multiline Logstash that handles timestamp on each line](https://discuss.elastic.co/t/multiline-logstash-that-handles-timestamp-on-each-line/322556)
**Author:** [@Scotsie](https://discuss.elastic.co/u/Scotsie)\ **Replies:** 5\ **Last updated:** [January 31, 2023, 9:58pm UTC](https://discuss.elastic.co/t/multiline-logstash-that-handles-timestamp-on-each-line/322556 "2023-01-31T21:58:46Z")
I'm currently ingesting logs from multiple devices successfully, one document per row. One particular brand, Polycom, is sending a multiline entry that includes the timestamp for each row. Sample Logging (with normal an… --- ## [Convert NanoSecond Unix timestamp](https://discuss.elastic.co/t/convert-nanosecond-unix-timestamp/324368)
**Author:** [@maskrider1111](https://discuss.elastic.co/u/maskrider1111)\ **Replies:** 8\ **Last updated:** [January 31, 2023, 9:57pm UTC](https://discuss.elastic.co/t/convert-nanosecond-unix-timestamp/324368 "2023-01-31T21:57:29Z")
Hi Folks, Any idea how to convert the nanosecond unix timestamp in logstash filter? date { match =\> \[ "eventtime","UNIX\_MS", "ISO8601" \] target =\> "Epoch" timezone =\> "UT… --- ## [Line break in grok pattern](https://discuss.elastic.co/t/line-break-in-grok-pattern/324369)
**Author:** [@mariana17](https://discuss.elastic.co/u/mariana17)\ **Replies:** 1\ **Last updated:** [January 31, 2023, 8:52pm UTC](https://discuss.elastic.co/t/line-break-in-grok-pattern/324369 "2023-01-31T20:52:38Z")
I am trying to get the data from a log, however in the middle of the log there is a line break which prevents the Grok filter from reading it correctly. If I adjust it to a single line it works, however, it would require… --- ## [I have a problema with send data from filebeat to Logstash](https://discuss.elastic.co/t/i-have-a-problema-with-send-data-from-filebeat-to-logstash/324366)
**Author:** [@odelacruzc](https://discuss.elastic.co/u/odelacruzc)\ **Replies:** 3\ **Last updated:** [January 31, 2023, 8:40pm UTC](https://discuss.elastic.co/t/i-have-a-problema-with-send-data-from-filebeat-to-logstash/324366 "2023-01-31T20:40:01Z")
Hello, can you help me pleae, I have a filebeat in my local computer and logstash in a VM Ubuntu in VirtualBox, so I check conectivity from my PC to remote server ubuntu with telnet 192.168.1.12 5044 and was successfull … --- ## [Twitter Input - Logstash filter mutate remove\_field - Elasticsearch](https://discuss.elastic.co/t/twitter-input-logstash-filter-mutate-remove-field-elasticsearch/324290)
**Author:** [@xalmer](https://discuss.elastic.co/u/xalmer)\ **Replies:** 10\ **Last updated:** [January 31, 2023, 8:38pm UTC](https://discuss.elastic.co/t/twitter-input-logstash-filter-mutate-remove-field-elasticsearch/324290 "2023-01-31T20:38:45Z")
Hello eveybody, Im trying to discover the fantastic world of possibilities of ELK. But i stop in a problem, and maybe someone can solve this "equation". Im using the Twitter Input Plugin to receive Twitter data, but i … --- ## [Logstash will not start with /tmp mounted noexec](https://discuss.elastic.co/t/logstash-will-not-start-with-tmp-mounted-noexec/324125)
**Author:** [@chuck1](https://discuss.elastic.co/u/chuck1)\ **Replies:** 6\ **Last updated:** [January 31, 2023, 7:39pm UTC](https://discuss.elastic.co/t/logstash-will-not-start-with-tmp-mounted-noexec/324125 "2023-01-31T19:39:53Z")
Logstash will not start with the /tmp directory mounted as noexec on RHEL 8.6. We fixed this with Elasticsearch. However, do not have the proper variables, guidance on how to fix this with Logstash. Thank You For Your … --- ## [TCP input and answer to client](https://discuss.elastic.co/t/tcp-input-and-answer-to-client/324338)
**Author:** [@M\_K1](https://discuss.elastic.co/u/M_K1)\ **Replies:** 4\ **Last updated:** [January 31, 2023, 6:01pm UTC](https://discuss.elastic.co/t/tcp-input-and-answer-to-client/324338 "2023-01-31T18:01:22Z")
Hello! is it possible to do in logstash? i need to receive strings throuth tcp input and after succssesful recievment i need to answer the client with string for example "ok" or "bad format" --- ## [Logstash filter split array of json into individual objects](https://discuss.elastic.co/t/logstash-filter-split-array-of-json-into-individual-objects/324245)
**Author:** [@sajjad\_akram](https://discuss.elastic.co/u/sajjad_akram)\ **Replies:** 4\ **Last updated:** [January 31, 2023, 1:43pm UTC](https://discuss.elastic.co/t/logstash-filter-split-array-of-json-into-individual-objects/324245 "2023-01-31T13:43:34Z")
Hi , iam trying to ingest each json object of array as a new entry/event in dynatrace using logstash. This is my json array {"RequestEventList":\[{"Instant":"2023-01-27T09:00:01.16141Z","RequestKey":"3fcbef69-9-10a608… --- ## [Not able to skip reading oids for down host in logstash Snmp file](https://discuss.elastic.co/t/not-able-to-skip-reading-oids-for-down-host-in-logstash-snmp-file/324335)
**Author:** [@himanshu\_rajput2](https://discuss.elastic.co/u/himanshu_rajput2)\ **Replies:** 0\ **Last updated:** [January 31, 2023, 1:20pm UTC](https://discuss.elastic.co/t/not-able-to-skip-reading-oids-for-down-host-in-logstash-snmp-file/324335 "2023-01-31T13:20:11Z")
I am using 7.6.2 version of logstash. We are fetching snmp data using walk in logstash conf file. I have used feature of multiple hosts. It is working fine but if any one host is down then it reads all oid for that host … --- ## [Dynamical way of getting name and path of Logstash config file](https://discuss.elastic.co/t/dynamical-way-of-getting-name-and-path-of-logstash-config-file/324324)
**Author:** [@sigbo](https://discuss.elastic.co/u/sigbo)\ **Replies:** 0\ **Last updated:** [January 31, 2023, 12:22pm UTC](https://discuss.elastic.co/t/dynamical-way-of-getting-name-and-path-of-logstash-config-file/324324 "2023-01-31T12:22:23Z")
We have a lot of Logstash configuration files and some handle almost the same data. Along with Logstash we have several other scripts ingesting data into Elasticsearch. Because of the sheer amount, we'd like to be able … --- ## [Error on Running Logstash](https://discuss.elastic.co/t/error-on-running-logstash/324299)
**Author:** [@Meghana1](https://discuss.elastic.co/u/Meghana1)\ **Replies:** 2\ **Last updated:** [January 31, 2023, 11:38am UTC](https://discuss.elastic.co/t/error-on-running-logstash/324299 "2023-01-31T11:38:12Z")
When I run the following command bin/logstash -f /etc/logstash/logstash-sample.conf . I get the following error runner - An unexpected error occurred! {:error=\>java.nio.file.AccessDeniedException: /usr/share/logstash/… --- ## [Logstash csv export =\> export of unwanted documents multiple times](https://discuss.elastic.co/t/logstash-csv-export-export-of-unwanted-documents-multiple-times/324315)
**Author:** [@SKiD](https://discuss.elastic.co/u/SKiD)\ **Replies:** 0\ **Last updated:** [January 31, 2023, 10:26am UTC](https://discuss.elastic.co/t/logstash-csv-export-export-of-unwanted-documents-multiple-times/324315 "2023-01-31T10:26:44Z")
Hello, I'm currently experiencing weird behavior of my logstash pipeline. Maybe someone has an idea what I'm currently doing wrong. What I'm trying to do I use a logstash pipeline to extract data from elasticsearch an… --- ## [Logstash unable to process more no of documents](https://discuss.elastic.co/t/logstash-unable-to-process-more-no-of-documents/324213)
**Author:** [@PRASHANT\_MEHTA](https://discuss.elastic.co/u/PRASHANT_MEHTA)\ **Replies:** 2\ **Last updated:** [January 31, 2023, 9:44am UTC](https://discuss.elastic.co/t/logstash-unable-to-process-more-no-of-documents/324213 "2023-01-31T09:44:01Z")
Hello All, I'm tyring to process 40000 documents by running perl script.The issue faced is that in stack management I can see the index being get cereated but the documents are not inserting the index. For a seperate u… --- ## [Parse json and non-json logs](https://discuss.elastic.co/t/parse-json-and-non-json-logs/324282)
**Author:** [@true64gurus](https://discuss.elastic.co/u/true64gurus)\ **Replies:** 1\ **Last updated:** [January 31, 2023, 2:52am UTC](https://discuss.elastic.co/t/parse-json-and-non-json-logs/324282 "2023-01-31T02:52:29Z")
I have multiple Kubernetes clusters sending logs to S3 , then I use logstash to read logs off S3 files. Some logs messages are in JSON , others in structured format. How to dynamically detect and parse JSON , and save n… --- ## [Proper way to escape escape characters](https://discuss.elastic.co/t/proper-way-to-escape-escape-characters/324143)
**Author:** [@hexoffender](https://discuss.elastic.co/u/hexoffender)\ **Replies:** 3\ **Last updated:** [January 30, 2023, 5:31pm UTC](https://discuss.elastic.co/t/proper-way-to-escape-escape-characters/324143 "2023-01-30T17:31:18Z")
Hello, I have a weird problem. I'm trying to replace \\x5c with \\ in a mutate. However, the logstash config fails to parse when I do this: input { beats { # The port to listen on for filebeat connections. … --- ## [Logstash Array of JSON](https://discuss.elastic.co/t/logstash-array-of-json/324178)
**Author:** [@creative\_sha](https://discuss.elastic.co/u/creative_sha)\ **Replies:** 4\ **Last updated:** [January 30, 2023, 5:53am UTC](https://discuss.elastic.co/t/logstash-array-of-json/324178 "2023-01-30T05:53:48Z")
How can I Split Array of JSON into different JSON so that per request we will get 1 row? --- ## [Kafka\_consumer\_lag in Logstash](https://discuss.elastic.co/t/kafka-consumer-lag-in-logstash/324108)
**Author:** [@Ondrej\_S](https://discuss.elastic.co/u/Ondrej_S)\ **Replies:** 2\ **Last updated:** [January 30, 2023, 2:28pm UTC](https://discuss.elastic.co/t/kafka-consumer-lag-in-logstash/324108 "2023-01-30T14:28:23Z")
Hello, Is it possible to get the value or verify value of “kafka\_consumer\_lag” directly in Logstash 7.17? The idea is to verify: if \[kafka\_consumer\_lag\]\[lag\] == 0 In Logstash Output and if yes run http plugin with u… --- ## [Trouble replacing leading/trailing whitespace in nested json](https://discuss.elastic.co/t/trouble-replacing-leading-trailing-whitespace-in-nested-json/324129)
**Author:** [@mark54g](https://discuss.elastic.co/u/mark54g)\ **Replies:** 4\ **Last updated:** [January 30, 2023, 9:33am UTC](https://discuss.elastic.co/t/trouble-replacing-leading-trailing-whitespace-in-nested-json/324129 "2023-01-30T09:33:59Z")
Hey, folks Trying to figure out a clean way to solve this problem I have nested json coming in from an SQS queue, and I've been playing with mocking it up with a static file example and filebeat, which I know is not pe… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=91) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=93)