# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=310 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 311 --- ## [Does the jdbc\_static loader run once upon logstash start or restart no matter the scheduler?](https://discuss.elastic.co/t/does-the-jdbc-static-loader-run-once-upon-logstash-start-or-restart-no-matter-the-scheduler/246681)
**Author:** [@chapmantrain](https://discuss.elastic.co/u/chapmantrain)\ **Replies:** 2\ **Last updated:** [August 27, 2020, 8:34pm UTC](https://discuss.elastic.co/t/does-the-jdbc-static-loader-run-once-upon-logstash-start-or-restart-no-matter-the-scheduler/246681 "2020-08-27T20:34:59Z")
When you start up or restart logstash with a jdbc\_static filter, does the loader automatically run once and then pick up the scheduler? In other words if I have the scheduler set to run once a day at midnight, and I sta… --- ## [Logstash and PKCS#12 format](https://discuss.elastic.co/t/logstash-and-pkcs-12-format/246499)
**Author:** [@ManuelF](https://discuss.elastic.co/u/ManuelF)\ **Replies:** 2\ **Last updated:** [August 27, 2020, 8:05pm UTC](https://discuss.elastic.co/t/logstash-and-pkcs-12-format/246499 "2020-08-27T20:05:51Z")
Hi, \*Running ELK 7.9 I am in the process of securing my ELK nodes and I have been struggling with the security settings for the last few days. After spending some time on this, I finally have Elasticsearch and Kibana c… --- ## [Syslog input config for logstash](https://discuss.elastic.co/t/syslog-input-config-for-logstash/245714)
**Author:** [@Cihan\_Tunali](https://discuss.elastic.co/u/Cihan_Tunali)\ **Replies:** 7\ **Last updated:** [August 27, 2020, 6:19pm UTC](https://discuss.elastic.co/t/syslog-input-config-for-logstash/245714 "2020-08-27T18:19:38Z")
Hello all, Im using ES 7.9.0. My goal is to get syslogs from my firewall to logstash. I tried every config that i found on net. But could not solve it. I configured rsyslog to forward incoming syslogs to my logstash whi… --- ## [Translate filter is not working when the input is from filebeat module](https://discuss.elastic.co/t/translate-filter-is-not-working-when-the-input-is-from-filebeat-module/246671)
**Author:** [@Jeeth](https://discuss.elastic.co/u/Jeeth)\ **Replies:** 1\ **Last updated:** [August 27, 2020, 6:04pm UTC](https://discuss.elastic.co/t/translate-filter-is-not-working-when-the-input-is-from-filebeat-module/246671 "2020-08-27T18:04:09Z")
Hi all, We have two different logstash pipelines for two different location data. Paloalto --\> syslog --\> filebeat --\> Logstash --\> Elasticsearch (Parsing with filebeat panw module) Paloalto --\> syslog --\> Logstash --\>… --- ## [How to parse a TimeSpan value which displays the difference between to timestamps](https://discuss.elastic.co/t/how-to-parse-a-timespan-value-which-displays-the-difference-between-to-timestamps/246502)
**Author:** [@pheathers](https://discuss.elastic.co/u/pheathers)\ **Replies:** 7\ **Last updated:** [August 27, 2020, 5:29pm UTC](https://discuss.elastic.co/t/how-to-parse-a-timespan-value-which-displays-the-difference-between-to-timestamps/246502 "2020-08-27T17:29:45Z")
I'm trying to parse through a timespan value (hh:mm:ss:SSSSSS). Here is the message ... \`2020-08-05 13:28:53.128 +03:00 \[DBG\] {"Timestamp":"2020-08-05T13:28:53.1286882+03:00","Profile":"db","ProcessID":26688,"SessionId… --- ## [Problem with Cloudwatch Input Login for ECS Namespace](https://discuss.elastic.co/t/problem-with-cloudwatch-input-login-for-ecs-namespace/246661)
**Author:** [@nareshbitla](https://discuss.elastic.co/u/nareshbitla)\ **Replies:** 0\ **Last updated:** [August 27, 2020, 4:27pm UTC](https://discuss.elastic.co/t/problem-with-cloudwatch-input-login-for-ecs-namespace/246661 "2020-08-27T16:27:29Z")
Hi, I am trying send AWS/ECS metrics from cloudwatch to kibana using logstash. Below is my config input { cloudwatch { "access\_key\_id" =\> "XXXXXXXXXXXX" "secret\_access\_key" =\> "XXXXXXXXX" #role\_arn =\> "XXXXXXXXXXX… --- ## [Logstash 7.x service fails to start on Ubuntu 18.04 LTS with OpenJDK 11 JDK as a service but starts otherwise](https://discuss.elastic.co/t/logstash-7-x-service-fails-to-start-on-ubuntu-18-04-lts-with-openjdk-11-jdk-as-a-service-but-starts-otherwise/246506)
**Author:** [@Henri.C.Lefevre](https://discuss.elastic.co/u/Henri.C.Lefevre)\ **Replies:** 4\ **Last updated:** [August 27, 2020, 3:58pm UTC](https://discuss.elastic.co/t/logstash-7-x-service-fails-to-start-on-ubuntu-18-04-lts-with-openjdk-11-jdk-as-a-service-but-starts-otherwise/246506 "2020-08-27T15:58:15Z")
I am trying to use logstash a SystemV service on Ubuntu 18.04 LTS. Whenever I try to use the service command, the service startup fails \> service logstash start && journalctl -u logstash logstash.service: Failed to ex… --- ## [Logstash Dynamic Template and ILM Creation](https://discuss.elastic.co/t/logstash-dynamic-template-and-ilm-creation/246653)
**Author:** [@Ramalakshmi](https://discuss.elastic.co/u/Ramalakshmi)\ **Replies:** 4\ **Last updated:** [August 27, 2020, 3:50pm UTC](https://discuss.elastic.co/t/logstash-dynamic-template-and-ilm-creation/246653 "2020-08-27T15:50:35Z")
Hi I am able to create the multiple index patterns dynamically using below configuration. output { stdout { codec =\> rubydebug } elasticsearch { hosts =\> \[ es\_host:9200" \] index =\> "lstas… --- ## [Thread leaks in logstash-output-s3 when wrong access keys are passed](https://discuss.elastic.co/t/thread-leaks-in-logstash-output-s3-when-wrong-access-keys-are-passed/246657)
**Author:** [@vinaykambam](https://discuss.elastic.co/u/vinaykambam)\ **Replies:** 0\ **Last updated:** [August 27, 2020, 3:39pm UTC](https://discuss.elastic.co/t/thread-leaks-in-logstash-output-s3-when-wrong-access-keys-are-passed/246657 "2020-08-27T15:39:40Z")
Hi There, I've observed thread leaks (S3, Stale factory sweeper) from logstash-output-s3 plugin when wrong access keys are passed in s3 output configuration. I tried same with different versions of logstash but it is th… --- ## [New field base on the sum of other 3 fields not working](https://discuss.elastic.co/t/new-field-base-on-the-sum-of-other-3-fields-not-working/246612)
**Author:** [@nino](https://discuss.elastic.co/u/nino)\ **Replies:** 3\ **Last updated:** [August 27, 2020, 3:17pm UTC](https://discuss.elastic.co/t/new-field-base-on-the-sum-of-other-3-fields-not-working/246612 "2020-08-27T15:17:46Z")
Hello all, i need some help, i can't get this working. I need to create a new file base on the sum of other three fields. field1, field2, field3 are defined as float fields ruby { code =\> "event\['new\_field'\] =''" } mu… --- ## [Create a new field (customdate) like yyyy-mm(2020-08) format from eventstamp(1597739955692) type Epoch](https://discuss.elastic.co/t/create-a-new-field-customdate-like-yyyy-mm-2020-08-format-from-eventstamp-1597739955692-type-epoch/246516)
**Author:** [@wishnu2016](https://discuss.elastic.co/u/wishnu2016)\ **Replies:** 2\ **Last updated:** [August 27, 2020, 3:17pm UTC](https://discuss.elastic.co/t/create-a-new-field-customdate-like-yyyy-mm-2020-08-format-from-eventstamp-1597739955692-type-epoch/246516 "2020-08-27T15:17:23Z")
I want to create a new field with a year-month format (yyyy-mm, 2020-08) from the eventstamp field which I am receiving in Epoch format. I am new to logstash, I was able to add a new field but I don't know how to covert … --- ## [TLS 1.2 enable on elasticsearch output plugin](https://discuss.elastic.co/t/tls-1-2-enable-on-elasticsearch-output-plugin/246553)
**Author:** [@Wasantha\_Herath](https://discuss.elastic.co/u/Wasantha_Herath)\ **Replies:** 1\ **Last updated:** [August 27, 2020, 3:14pm UTC](https://discuss.elastic.co/t/tls-1-2-enable-on-elasticsearch-output-plugin/246553 "2020-08-27T15:14:56Z")
Hi Team, Currently we are using a ES cloud cluster and separate logstash (v6.3) instance which communicate with ES cloud cluster using HTTPS host name. Like below output { elasticsearch { hosts =\> \[ "https… --- ## [Troubles moving from Elasticsearch ingest pipeline to Logstash](https://discuss.elastic.co/t/troubles-moving-from-elasticsearch-ingest-pipeline-to-logstash/246490)
**Author:** [@Daniele\_Stefanucci](https://discuss.elastic.co/u/Daniele_Stefanucci)\ **Replies:** 3\ **Last updated:** [August 27, 2020, 3:14pm UTC](https://discuss.elastic.co/t/troubles-moving-from-elasticsearch-ingest-pipeline-to-logstash/246490 "2020-08-27T15:14:12Z")
Hi! I'm facing an issue while moving from elasticsearch ingest pipeline to logstash pipeline, I've correctly set up filebeat for IIS logs and configured the elasticsearch pipeline as per documentation (with logstash call… --- ## [Ruby + replace from array lines](https://discuss.elastic.co/t/ruby-replace-from-array-lines/246647)
**Author:** [@thaideval](https://discuss.elastic.co/u/thaideval)\ **Replies:** 0\ **Last updated:** [August 27, 2020, 2:53pm UTC](https://discuss.elastic.co/t/ruby-replace-from-array-lines/246647 "2020-08-27T14:53:00Z")
Hello there. First of all i am not even sure if its possible, but figured - should still ask. so.. i have two fields. String with text: {field1} = "Some text from blah X.KEY1 Some more text X.KEY2 and text here... " … --- ## [Modifying a CSV file - Logstash](https://discuss.elastic.co/t/modifying-a-csv-file-logstash/246333)
**Author:** [@Aniket\_Valsangkar](https://discuss.elastic.co/u/Aniket_Valsangkar)\ **Replies:** 3\ **Last updated:** [August 27, 2020, 11:48am UTC](https://discuss.elastic.co/t/modifying-a-csv-file-logstash/246333 "2020-08-27T11:48:26Z")
Hi Team, I have a CSV file which gets updated (delete, add, change in the data in fields) regularly. The columns stay constant only the data changes. I want to write a code in logstash, which should be able to capture t… --- ## [Logstash - Unable to start the pipeline(Cipher Issue)](https://discuss.elastic.co/t/logstash-unable-to-start-the-pipeline-cipher-issue/246598)
**Author:** [@anandd4](https://discuss.elastic.co/u/anandd4)\ **Replies:** 0\ **Last updated:** [August 27, 2020, 10:31am UTC](https://discuss.elastic.co/t/logstash-unable-to-start-the-pipeline-cipher-issue/246598 "2020-08-27T10:31:07Z")
Logstash throws following error on turning on SSL - Failed to execute action {:id=\>:main, :action\_type=\>LogStash::ConvergeResult::FailedAction, :message=\>"Could not execute action: PipelineAction::Create\, action\_r… --- ## [Logstash with filebeat and batching](https://discuss.elastic.co/t/logstash-with-filebeat-and-batching/246597)
**Author:** [@Carlos\_Magalhaes](https://discuss.elastic.co/u/Carlos_Magalhaes)\ **Replies:** 0\ **Last updated:** [August 27, 2020, 10:30am UTC](https://discuss.elastic.co/t/logstash-with-filebeat-and-batching/246597 "2020-08-27T10:30:04Z")
Hi all, I know Logstash isn't made to be a batch processing tool i.e. only process logs at a certain period or time? Scenario - I have large amounts of logs coming from a firewall via filebeat to a logstash server. It … --- ## [Logstash parsing timestamp incorrectly](https://discuss.elastic.co/t/logstash-parsing-timestamp-incorrectly/244773)
**Author:** [@anandd4](https://discuss.elastic.co/u/anandd4)\ **Replies:** 2\ **Last updated:** [August 27, 2020, 8:47am UTC](https://discuss.elastic.co/t/logstash-parsing-timestamp-incorrectly/244773 "2020-08-27T08:47:05Z")
Problem Statement - Filebeat streaming logs via logstash into elasticsearch (7.5) Logstash applies the filter on epoch time & update @timestamp field during data ingestion Logstash is incorrectly ingesting the timesta… --- ## [Logstash-keystore failed with error "ArgumentError: invalid byte sequence in CP949"](https://discuss.elastic.co/t/logstash-keystore-failed-with-error-argumenterror-invalid-byte-sequence-in-cp949/246582)
**Author:** [@jaykim](https://discuss.elastic.co/u/jaykim)\ **Replies:** 0\ **Last updated:** [August 27, 2020, 8:33am UTC](https://discuss.elastic.co/t/logstash-keystore-failed-with-error-argumenterror-invalid-byte-sequence-in-cp949/246582 "2020-08-27T08:33:32Z")
Hello team, I'm working to secure configuration on ELK stack. While creating keystore in logstash, there was an error : 'ArgumentError: invalid byte sequence in CP949' It also occured in v7.8.0, don't know what happens… --- ## [Error with elasticsearch-logstash](https://discuss.elastic.co/t/error-with-elasticsearch-logstash/246556)
**Author:** [@Aparna\_Vodapalli](https://discuss.elastic.co/u/Aparna_Vodapalli)\ **Replies:** 0\ **Last updated:** [August 27, 2020, 6:34am UTC](https://discuss.elastic.co/t/error-with-elasticsearch-logstash/246556 "2020-08-27T06:34:15Z")
I get below error in my ELK setup. This error is not there in logstash version 6.3.2 and beacuse of this error i am not able to upgrade my logstash as this error is still not fixed in latest logstash 7.8.0 \[ERROR\]\[filew… --- ## [Logstash JDBC input for oracle SSL connections](https://discuss.elastic.co/t/logstash-jdbc-input-for-oracle-ssl-connections/246528)
**Author:** [@erickgvsat](https://discuss.elastic.co/u/erickgvsat)\ **Replies:** 0\ **Last updated:** [August 27, 2020, 1:10am UTC](https://discuss.elastic.co/t/logstash-jdbc-input-for-oracle-ssl-connections/246528 "2020-08-27T01:10:04Z")
I am trying to connect to Oracle database with JDBC input plugin using SSL (TCPS) connection. I am able to connect with regular connection (non SSL). However when thin URL with SSL TCPS, it doesn't work. Here is examp… --- ## [Logstash KV Filter Help](https://discuss.elastic.co/t/logstash-kv-filter-help/246199)
**Author:** [@kopacko](https://discuss.elastic.co/u/kopacko)\ **Replies:** 1\ **Last updated:** [August 27, 2020, 1:07am UTC](https://discuss.elastic.co/t/logstash-kv-filter-help/246199 "2020-08-27T01:07:42Z")
I am working on a project and I have setup a chatbot that connects to Twitch chat channels. I am trying to parse chat messages and having some issues related to a KV filter. An example log looks like this: @badge-info… --- ## [Index not created using log stash](https://discuss.elastic.co/t/index-not-created-using-log-stash/246443)
**Author:** [@Adarsh\_S](https://discuss.elastic.co/u/Adarsh_S)\ **Replies:** 2\ **Last updated:** [August 27, 2020, 12:31am UTC](https://discuss.elastic.co/t/index-not-created-using-log-stash/246443 "2020-08-27T00:31:36Z")
I am using elasticsearch with logstash \[Version : 7.9.0\] to visualize a data set with kibana. I am doing this task on Windows10. Logstash config file - input { jdbc { jdbc\_driver\_library =\> "$PATH\\mysql-connect… --- ## [How can format json file when using logstash](https://discuss.elastic.co/t/how-can-format-json-file-when-using-logstash/246520)
**Author:** [@bertrand1](https://discuss.elastic.co/u/bertrand1)\ **Replies:** 0\ **Last updated:** [August 26, 2020, 10:29pm UTC](https://discuss.elastic.co/t/how-can-format-json-file-when-using-logstash/246520 "2020-08-26T22:29:20Z")
Hello, I am newby on logstash, i am trying to send a log with the following in ES/kibana ===\> {"Brique":"Echange", "Environnement":"prd", "UUID":"7cc9584f-3e89-4a1c-b62f-bf753a3cf693", "Script":"script\_envoi.ksh How ca… --- ## [Logstash dynamically create field name](https://discuss.elastic.co/t/logstash-dynamically-create-field-name/246326)
**Author:** [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Replies:** 8\ **Last updated:** [August 26, 2020, 10:00pm UTC](https://discuss.elastic.co/t/logstash-dynamically-create-field-name/246326 "2020-08-26T22:00:42Z")
this is my input {"dataflow":\[{"gate": "v204.06", "attrmath":"v206.07"}\]} I want output like dataflow.m1: gate dataflow.m1\_version: v204.06 dataflow.m2: attrmath dataflow.m2\_version: v206.07 I want this m1, m2, m3… --- ## [How to use \\n as a delimeter in dissect filter](https://discuss.elastic.co/t/how-to-use-n-as-a-delimeter-in-dissect-filter/246485)
**Author:** [@shani](https://discuss.elastic.co/u/shani)\ **Replies:** 9\ **Last updated:** [August 26, 2020, 7:55pm UTC](https://discuss.elastic.co/t/how-to-use-n-as-a-delimeter-in-dissect-filter/246485 "2020-08-26T19:55:49Z")
Hi All, dissect filter doesn't work when I use \\n as a delimiter, it return error as Dissector mapping, pattern not found {"field"=\>"message", "pattern"=\>"%{test}\\n%{test1}" It works on any other delimeter but \\n does… --- ## [\[help\]how to write config for multiplication](https://discuss.elastic.co/t/help-how-to-write-config-for-multiplication/246448)
**Author:** [@manojb1-in.ibm.com](https://discuss.elastic.co/u/manojb1-in.ibm.com)\ **Replies:** 3\ **Last updated:** [August 26, 2020, 6:05pm UTC](https://discuss.elastic.co/t/help-how-to-write-config-for-multiplication/246448 "2020-08-26T18:05:44Z")
I am writing logstash config for iostat output of aix\_server. sample lines is given Below Disks: xfers read write … --- ## [Grok for IP then geoip and ip2proxy plugin?](https://discuss.elastic.co/t/grok-for-ip-then-geoip-and-ip2proxy-plugin/246362)
**Author:** [@mortimerj](https://discuss.elastic.co/u/mortimerj)\ **Replies:** 3\ **Last updated:** [August 26, 2020, 5:25pm UTC](https://discuss.elastic.co/t/grok-for-ip-then-geoip-and-ip2proxy-plugin/246362 "2020-08-26T17:25:20Z")
Hello! I'm trying to grok just the IP address from the field ClientIP, which can present as \[1.2.3.4:4444\] or 1.2.3.4:4444, then create a new field (clientipfixed), and apply the geoip and ip2proxy plugins to enrich it.… --- ## [Set threshold or condition when to Email in logstash script](https://discuss.elastic.co/t/set-threshold-or-condition-when-to-email-in-logstash-script/246373)
**Author:** [@David03](https://discuss.elastic.co/u/David03)\ **Replies:** 1\ **Last updated:** [August 26, 2020, 4:33pm UTC](https://discuss.elastic.co/t/set-threshold-or-condition-when-to-email-in-logstash-script/246373 "2020-08-26T16:33:38Z")
Is it possible to set a threshold or condition when to email to avoid a continuous firing of email? This is my current script. if \[message\] =~ "Caused by:" { grok{ match =\>\["message", "Caused by:"\] add\_tag =\>\["shoul… --- ## [Parse Generic Nestaed json without splitting](https://discuss.elastic.co/t/parse-generic-nestaed-json-without-splitting/246435)
**Author:** [@Bhargav\_Patel1](https://discuss.elastic.co/u/Bhargav_Patel1)\ **Replies:** 1\ **Last updated:** [August 26, 2020, 4:28pm UTC](https://discuss.elastic.co/t/parse-generic-nestaed-json-without-splitting/246435 "2020-08-26T16:28:22Z")
Hello, I have valid json string which i want to parse as json and want to stored in elastic { "id": "101834545", "date": "06-Apr-2016", "Gender": "Male", "ActiveStatus": "Active", "NameDetails": { "Name": \[{ … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=309) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=311)