# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=366 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 367 --- ## [Date parse failure - jdbc](https://discuss.elastic.co/t/date-parse-failure-jdbc/230090)
**Author:** [@mimimike](https://discuss.elastic.co/u/mimimike)\ **Replies:** 4\ **Last updated:** [April 29, 2020, 7:20am UTC](https://discuss.elastic.co/t/date-parse-failure-jdbc/230090 "2020-04-29T07:20:42Z")
Hi, I've tried to index some docs with logstash using jdbc. But it always fail because it is not able to index date with format yyyyMMdd input { jdbc { jdbc\_connection\_string =\> "jdbc:sqlserver://localhost;databaseNa… --- ## [Extract substring from filename](https://discuss.elastic.co/t/extract-substring-from-filename/230322)
**Author:** [@mohammad\_hasan\_dabba](https://discuss.elastic.co/u/mohammad_hasan_dabba)\ **Replies:** 0\ **Last updated:** [April 29, 2020, 7:18am UTC](https://discuss.elastic.co/t/extract-substring-from-filename/230322 "2020-04-29T07:18:22Z")
hello, I have a typical issue but couldn't find the answer I want to extract file name and date-time from a file path this is how my file path looks like: /home/p188-mobiletradingpro-stresstest/stresstest/logs/999-241-… --- ## [Copy data from ms-sql to elasticsearch using logstash for Nested type](https://discuss.elastic.co/t/copy-data-from-ms-sql-to-elasticsearch-using-logstash-for-nested-type/230314)
**Author:** [@ssupale](https://discuss.elastic.co/u/ssupale)\ **Replies:** 0\ **Last updated:** [April 29, 2020, 6:35am UTC](https://discuss.elastic.co/t/copy-data-from-ms-sql-to-elasticsearch-using-logstash-for-nested-type/230314 "2020-04-29T06:35:19Z")
I have a following index in Elasticsearch (Create Index Syntax is as below), This index is having a nested type for field jsonarrayversion. We also have the similar table in mssql with columns as (Id int, Name nvarchar(… --- ## [Reloading the config file, do I need it?](https://discuss.elastic.co/t/reloading-the-config-file-do-i-need-it/230307)
**Author:** [@juuuhuuu](https://discuss.elastic.co/u/juuuhuuu)\ **Replies:** 0\ **Last updated:** [April 29, 2020, 6:16am UTC](https://discuss.elastic.co/t/reloading-the-config-file-do-i-need-it/230307 "2020-04-29T06:16:43Z")
Hi all of you, I would like to know if I need reloading of the config file for elka stack -\> filebeat on one server and kibana, elasticsearch and logstash on onther server. https://www.elastic.co/guide/en/logstash/cu… --- ## [Logstash not removing non running pipeline](https://discuss.elastic.co/t/logstash-not-removing-non-running-pipeline/230273)
**Author:** [@Pink11](https://discuss.elastic.co/u/Pink11)\ **Replies:** 0\ **Last updated:** [April 28, 2020, 11:48pm UTC](https://discuss.elastic.co/t/logstash-not-removing-non-running-pipeline/230273 "2020-04-28T23:48:28Z")
I have 2 pipeline s running in one logstash via pipelines.yml file If I remove one pipeline from pipelines.yml that pipeline stays as a non running pipeline and If I add that removed pipeline again in pipelines.… --- ## [Logstash loads the log file at initial run. But when the logfile is updated it does not show in Kibana](https://discuss.elastic.co/t/logstash-loads-the-log-file-at-initial-run-but-when-the-logfile-is-updated-it-does-not-show-in-kibana/230272)
**Author:** [@rajagopalss](https://discuss.elastic.co/u/rajagopalss)\ **Replies:** 0\ **Last updated:** [April 28, 2020, 11:43pm UTC](https://discuss.elastic.co/t/logstash-loads-the-log-file-at-initial-run-but-when-the-logfile-is-updated-it-does-not-show-in-kibana/230272 "2020-04-28T23:43:48Z")
Hello. I'm new to logstash Logstash, ElasticSearch and Kibana are running in docker container on windows machine on same network. Springboot microservice logs to a log file - which is configured for logstash When i st… --- ## [Does client verification require host based certificates?](https://discuss.elastic.co/t/does-client-verification-require-host-based-certificates/230243)
**Author:** [@sjne](https://discuss.elastic.co/u/sjne)\ **Replies:** 2\ **Last updated:** [April 28, 2020, 10:47pm UTC](https://discuss.elastic.co/t/does-client-verification-require-host-based-certificates/230243 "2020-04-28T22:47:53Z")
When using ssl\_verify\_mode: force\_peer for client verification with Filebeat, does the client certificate need to be issued to the specific host name/IP? I.e the CN is the host name of the node. For example can I use (i… --- ## [Performance split for logstash and s3](https://discuss.elastic.co/t/performance-split-for-logstash-and-s3/230260)
**Author:** [@sc5283](https://discuss.elastic.co/u/sc5283)\ **Replies:** 2\ **Last updated:** [April 28, 2020, 10:25pm UTC](https://discuss.elastic.co/t/performance-split-for-logstash-and-s3/230260 "2020-04-28T22:25:14Z")
Is there a way to split the load on multiple logstash instances to grab data from S3? example: I would like for logstash instance 1 to grab only files ending in 1 and 2 logstash instance 2 to grab files ending in 3 an… --- ## [Float 1.0 does not get converted to boolean true](https://discuss.elastic.co/t/float-1-0-does-not-get-converted-to-boolean-true/230231)
**Author:** [@sc5283](https://discuss.elastic.co/u/sc5283)\ **Replies:** 5\ **Last updated:** [April 28, 2020, 10:19pm UTC](https://discuss.elastic.co/t/float-1-0-does-not-get-converted-to-boolean-true/230231 "2020-04-28T22:19:24Z")
logstash 7.6.2 From mutate filter specifies float 1.0 is converted to boolean true but this is not the case, all others work (0, 0.0, "0", etc). I have tried also the translate filter with no success input : ---\> … --- ## [Error: Received fatal alert: handshake\_failure Exception: Faraday::SSLError](https://discuss.elastic.co/t/error-received-fatal-alert-handshake-failure-exception-faraday-sslerror/230253)
**Author:** [@edster](https://discuss.elastic.co/u/edster)\ **Replies:** 0\ **Last updated:** [April 28, 2020, 9:14pm UTC](https://discuss.elastic.co/t/error-received-fatal-alert-handshake-failure-exception-faraday-sslerror/230253 "2020-04-28T21:14:15Z")
Hello, I am running into an issue using the input to connect to our elastic nodes to retrieve an index, however, im running into the following error Error: Received fatal alert: handshake\_failure Exception: Faraday::S… --- ## [org.logstash.beats.InvalidFrameProtocolException](https://discuss.elastic.co/t/org-logstash-beats-invalidframeprotocolexception/229862)
**Author:** [@Louis\_Zhang](https://discuss.elastic.co/u/Louis_Zhang)\ **Replies:** 1\ **Last updated:** [April 28, 2020, 8:51pm UTC](https://discuss.elastic.co/t/org-logstash-beats-invalidframeprotocolexception/229862 "2020-04-28T20:51:34Z")
Looks like logstash is receving incorrect event type, that's due to beats on remote machines mis-configuration, we have hundreds of remote machines, is there any possible way to extrac the remote: 172.16.234.66 inofmrat… --- ## [XML parsing error nil:NilClass \[2020-04-22T16:13:54,199\]\[ERROR\]\[logstash.filters.ruby \]\[main\] Ruby exception occurred: undefined method](https://discuss.elastic.co/t/xml-parsing-error-nil-nilclass-2020-04-22t1654-199-error-logstash-filters-ruby-main-ruby-exception-occurred-undefined-method/230201)
**Author:** [@yagoza](https://discuss.elastic.co/u/yagoza)\ **Replies:** 6\ **Last updated:** [April 28, 2020, 7:54pm UTC](https://discuss.elastic.co/t/xml-parsing-error-nil-nilclass-2020-04-22t1654-199-error-logstash-filters-ruby-main-ruby-exception-occurred-undefined-method/230201 "2020-04-28T19:54:19Z")
Hi I am getting this error while parsing XML data using logstash \[2020-04-22T16:13:54,092\]\[ERROR\]\[logstash.filters.ruby \]\[main\] Ruby exception occurred: undefined method \[\]' for nil:NilClass \[2020-04-22T16:13:54,199\]\[E… --- ## [Elasticsearch not showing the new index](https://discuss.elastic.co/t/elasticsearch-not-showing-the-new-index/230228)
**Author:** [@thekradev](https://discuss.elastic.co/u/thekradev)\ **Replies:** 0\ **Last updated:** [April 28, 2020, 5:55pm UTC](https://discuss.elastic.co/t/elasticsearch-not-showing-the-new-index/230228 "2020-04-28T17:55:20Z")
Hi everyone, I'm new to Elastic stack. I'm using Elasticsearch, Kibana, Logstash, and filebeat to parse server logs. I follow this link After I run this command curl -XGET 'localhost:9200/logstash-2020.04.28/\_search?p… --- ## [Join information between 2 Data inputs](https://discuss.elastic.co/t/join-information-between-2-data-inputs/230219)
**Author:** [@pmo](https://discuss.elastic.co/u/pmo)\ **Replies:** 1\ **Last updated:** [April 28, 2020, 5:12pm UTC](https://discuss.elastic.co/t/join-information-between-2-data-inputs/230219 "2020-04-28T17:12:17Z")
Hi experts, i would like to know if it's possible to join two input file in order to create a new file and send the result to elasticsearch ? Ex: I've input data from SQL and another input data from JSON file. I would… --- ## [Create a field with information from another field](https://discuss.elastic.co/t/create-a-field-with-information-from-another-field/230216)
**Author:** [@pmo](https://discuss.elastic.co/u/pmo)\ **Replies:** 1\ **Last updated:** [April 28, 2020, 5:11pm UTC](https://discuss.elastic.co/t/create-a-field-with-information-from-another-field/230216 "2020-04-28T17:11:51Z")
Hi expert, I would like to know if it's possible to create a new field with information that are taken from another field. I'm using sql connection. I've a field named : Code\_postal (that has 3 or 5 digits - I only… --- ## [GROK not match](https://discuss.elastic.co/t/grok-not-match/230122)
**Author:** [@Hoang\_Duy](https://discuss.elastic.co/u/Hoang_Duy)\ **Replies:** 1\ **Last updated:** [April 28, 2020, 4:31pm UTC](https://discuss.elastic.co/t/grok-not-match/230122 "2020-04-28T16:31:59Z")
Pls help.. I don't know why my logstash filter config doesn't match system message "Failed password for invalid user bbbbbbbbbbbbbb from 10.10.64.74 port 62561 ssh2" My filter config: filter { if \[fileset\]\[module\] =… --- ## [Multiple input filebeat to logstash](https://discuss.elastic.co/t/multiple-input-filebeat-to-logstash/230194)
**Author:** [@GedeoN](https://discuss.elastic.co/u/GedeoN)\ **Replies:** 1\ **Last updated:** [April 28, 2020, 3:28pm UTC](https://discuss.elastic.co/t/multiple-input-filebeat-to-logstash/230194 "2020-04-28T15:28:46Z")
Hi all experts, I'm trying inject many logs from many files. Actually, i receive all logs but in just one index. I tried to tags, but it's not working actually, and i don't find my error. filebeat configuration: fil… --- ## [Run config file from a web app (flask)](https://discuss.elastic.co/t/run-config-file-from-a-web-app-flask/230197)
**Author:** [@inchirah](https://discuss.elastic.co/u/inchirah)\ **Replies:** 0\ **Last updated:** [April 28, 2020, 2:36pm UTC](https://discuss.elastic.co/t/run-config-file-from-a-web-app-flask/230197 "2020-04-28T14:36:18Z")
Hello, I build a web app with flask. Through a button the user gonna choose a csv file from his desktop and when he clicks to submit a LOGSTASH config file gonna run to import that file (the user previously chosen) into… --- ## [Calculating count and percentage](https://discuss.elastic.co/t/calculating-count-and-percentage/229029)
**Author:** [@rcganesh](https://discuss.elastic.co/u/rcganesh)\ **Replies:** 1\ **Last updated:** [April 28, 2020, 2:03pm UTC](https://discuss.elastic.co/t/calculating-count-and-percentage/229029 "2020-04-28T14:03:36Z")
Hello Experts, I am couple of weeks old to Logstash and I need some help in trasforming data through logstash and send it to either some file or Elasticsearch. Below is the data-flow we are implementing. Input data in… --- ## [Azureblob input plugin - error](https://discuss.elastic.co/t/azureblob-input-plugin-error/229088)
**Author:** [@Kevin\_f](https://discuss.elastic.co/u/Kevin_f)\ **Replies:** 5\ **Last updated:** [April 28, 2020, 1:35pm UTC](https://discuss.elastic.co/t/azureblob-input-plugin-error/229088 "2020-04-28T13:35:15Z")
Hello, Hoping that someone within the community has come across this issue we are facing and may have managed to resolve it or have any idea? Basically we have installed the azureblob plugin (0.9.13) to ingest logs fro… --- ## [Http\_poller input](https://discuss.elastic.co/t/http-poller-input/229764)
**Author:** [@hiba](https://discuss.elastic.co/u/hiba)\ **Replies:** 2\ **Last updated:** [April 28, 2020, 1:22pm UTC](https://discuss.elastic.co/t/http-poller-input/229764 "2020-04-28T13:22:25Z")
Hi , i want to receive data through HTTP requests using logstash. my file.conf look like this : input { http\_poller { urls =\> { url =\> "http://api.v4.boosterbc.net/suggest/cities?term=hamma&lang=fr " } request\_t… --- ## [Logstash is not receiving logs using UDP protocol (tiny limit on the message size)](https://discuss.elastic.co/t/logstash-is-not-receiving-logs-using-udp-protocol-tiny-limit-on-the-message-size/230176)
**Author:** [@Zol](https://discuss.elastic.co/u/Zol)\ **Replies:** 0\ **Last updated:** [April 28, 2020, 1:09pm UTC](https://discuss.elastic.co/t/logstash-is-not-receiving-logs-using-udp-protocol-tiny-limit-on-the-message-size/230176 "2020-04-28T13:09:26Z")
We have a NodeJs application (Expressjs framework) in two environments staging and production. We are using Winston library for logging to Logstash. We have two opened ports on Logstash, one for production and another f… --- ## [Logstash disable Stdout](https://discuss.elastic.co/t/logstash-disable-stdout/229479)
**Author:** [@RAM\_NATHAN](https://discuss.elastic.co/u/RAM_NATHAN)\ **Replies:** 4\ **Last updated:** [April 28, 2020, 11:51am UTC](https://discuss.elastic.co/t/logstash-disable-stdout/229479 "2020-04-28T11:51:58Z")
Hi I'm having below code in output section { hosts =\> \["localhost:9200"\] document\_id =\> "%{authsid}" index =\> "dashboard\_write" action =\> "delete" } … --- ## [Future of Logstash within Elastic Stack](https://discuss.elastic.co/t/future-of-logstash-within-elastic-stack/230144)
**Author:** [@kelk](https://discuss.elastic.co/u/kelk)\ **Replies:** 0\ **Last updated:** [April 28, 2020, 11:07am UTC](https://discuss.elastic.co/t/future-of-logstash-within-elastic-stack/230144 "2020-04-28T11:07:51Z")
hi All, We were looking into data aggregation platforms and analytics, and wanted to design Elastic Cluster. I wanted to get your views on "future" of logstash in the Elastic Stack. With the advent of Beats and more pow… --- ## [Batch http lookup filter](https://discuss.elastic.co/t/batch-http-lookup-filter/228789)
**Author:** [@AmirBahrami](https://discuss.elastic.co/u/AmirBahrami)\ **Replies:** 1\ **Last updated:** [April 28, 2020, 5:52am UTC](https://discuss.elastic.co/t/batch-http-lookup-filter/228789 "2020-04-28T05:52:04Z")
Hey everybody I'm looking for a way to do batch lookups by http filter on some event field an then merge those parts together. I thought it'd be possible with something temporary like '@metadata' field but as I found m… --- ## [Logstash to transfer data from Elastic search source to Elastic Search Destination Realtime](https://discuss.elastic.co/t/logstash-to-transfer-data-from-elastic-search-source-to-elastic-search-destination-realtime/230055)
**Author:** [@vikrant\_baraiya1](https://discuss.elastic.co/u/vikrant_baraiya1)\ **Replies:** 0\ **Last updated:** [April 28, 2020, 12:49am UTC](https://discuss.elastic.co/t/logstash-to-transfer-data-from-elastic-search-source-to-elastic-search-destination-realtime/230055 "2020-04-28T00:49:41Z")
Hello ES/Logstash Gurus, As I am new with Elastic Stack, I need your help. I am working on one of the Use Case which requires to transfer the data from one Elastic Search Cluster to another Elastic Search cluster in re… --- ## [Run Logstash config file from a web app](https://discuss.elastic.co/t/run-logstash-config-file-from-a-web-app/230052)
**Author:** [@inchirah](https://discuss.elastic.co/u/inchirah)\ **Replies:** 0\ **Last updated:** [April 28, 2020, 12:25am UTC](https://discuss.elastic.co/t/run-logstash-config-file-from-a-web-app/230052 "2020-04-28T00:25:08Z")
Hello, I build a web app with flask. Through a button the user gonna choose a csv file from his desktop and when he clicks to submit a LOGSTASH config file gonna run to import that file (the user previously chosen) into… --- ## [GROK match multiple pattern drop the log](https://discuss.elastic.co/t/grok-match-multiple-pattern-drop-the-log/229986)
**Author:** [@tharu85](https://discuss.elastic.co/u/tharu85)\ **Replies:** 1\ **Last updated:** [April 27, 2020, 9:20pm UTC](https://discuss.elastic.co/t/grok-match-multiple-pattern-drop-the-log/229986 "2020-04-27T21:20:46Z")
I used following grok pattern to extract IIS log. The match message containing multiple grok pattern due to IIS log configuration on several server. Unfortunately one I have execute this filter, no messages were parsers… --- ## [Dotted fieldnames, how to create?](https://discuss.elastic.co/t/dotted-fieldnames-how-to-create/230018)
**Author:** [@WimDH](https://discuss.elastic.co/u/WimDH)\ **Replies:** 2\ **Last updated:** [April 27, 2020, 9:16pm UTC](https://discuss.elastic.co/t/dotted-fieldnames-how-to-create/230018 "2020-04-27T21:16:29Z")
Hi all, Not sure if this is the right place to ask... I'm new to ElasticStack, and I'm trying to find documentation on how fieldnames are built. When I look at Kibana Discover, I see fieldnames like host.name and syst… --- ## [In jvm.options file pass kill -9 for killing logstash on outofmemory](https://discuss.elastic.co/t/in-jvm-options-file-pass-kill-9-for-killing-logstash-on-outofmemory/227414)
**Author:** [@Sunil\_Vp](https://discuss.elastic.co/u/Sunil_Vp)\ **Replies:** 2\ **Last updated:** [April 27, 2020, 7:58pm UTC](https://discuss.elastic.co/t/in-jvm-options-file-pass-kill-9-for-killing-logstash-on-outofmemory/227414 "2020-04-27T19:58:12Z")
Hi, I wanted to pass -XX:OnOutOfMemoryError=kill -9 %p option to the JVM arguments of logstash. I added the same in jvm.options file, But once i add it i get: Error: Could not create the Java Virtual Machine. Error: A… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=365) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=367)