# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=368 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 369 --- ## [Logstash with ruby command](https://discuss.elastic.co/t/logstash-with-ruby-command/229576)
**Author:** [@111228](https://discuss.elastic.co/u/111228)\ **Replies:** 1\ **Last updated:** [April 24, 2020, 4:53pm UTC](https://discuss.elastic.co/t/logstash-with-ruby-command/229576 "2020-04-24T16:53:42Z")
Hello everyone I use ruby in my logstash to increase the value of the field by 1000 and I do it but in logstash logs I constantly see a message with error \[ERROR\]\[logstash.filters.ruby \]\[main\] Ruby exception occurred… --- ## [Unknown type in Discover even with Dynamic Mapping](https://discuss.elastic.co/t/unknown-type-in-discover-even-with-dynamic-mapping/229423)
**Author:** [@Bibas](https://discuss.elastic.co/u/Bibas)\ **Replies:** 3\ **Last updated:** [April 24, 2020, 4:52pm UTC](https://discuss.elastic.co/t/unknown-type-in-discover-even-with-dynamic-mapping/229423 "2020-04-24T16:52:31Z")
Hello ! I'm new to the ELK stack so this question might be a bit dumb and I apologize. I have logs going through Logstash, then Elastic Search and arriving to Kibana without problems. My logs contains severals custom f… --- ## [Remove nested fields from parsed xml](https://discuss.elastic.co/t/remove-nested-fields-from-parsed-xml/229665)
**Author:** [@toms130](https://discuss.elastic.co/u/toms130)\ **Replies:** 0\ **Last updated:** [April 24, 2020, 1:36pm UTC](https://discuss.elastic.co/t/remove-nested-fields-from-parsed-xml/229665 "2020-04-24T13:36:51Z")
Hi all, I'm trying to remove some fields from an xml parsed document. I got a lot of fields (~150), and I would like to delete some of them (~60), so I can't list them individually (and I don't know them in advance). … --- ## [Logstash not outputing data to any output plugin](https://discuss.elastic.co/t/logstash-not-outputing-data-to-any-output-plugin/229626)
**Author:** [@tbhaxor](https://discuss.elastic.co/u/tbhaxor)\ **Replies:** 1\ **Last updated:** [April 24, 2020, 12:51pm UTC](https://discuss.elastic.co/t/logstash-not-outputing-data-to-any-output-plugin/229626 "2020-04-24T12:51:48Z")
I am using logstash in the SSL environment, in such a way that beats are connecting to logstash but logstash itself not sending data to elasticsearch. So I added the file output plugin, it seems the problem is not resolv… --- ## [Wrong document\_id when end with 0](https://discuss.elastic.co/t/wrong-document-id-when-end-with-0/229646)
**Author:** [@dgonzalezp](https://discuss.elastic.co/u/dgonzalezp)\ **Replies:** 0\ **Last updated:** [April 24, 2020, 12:12pm UTC](https://discuss.elastic.co/t/wrong-document-id-when-end-with-0/229646 "2020-04-24T12:12:42Z")
Hi everyone, I have the following ouput configuration file: output { if "\_jdbcstreamingdefaultsused" in \[tags\] { elasticsearch { hosts =\> \["esdatahot01.siacloud.es:9200","esda… --- ## [Remove Double quotes from field JSON file](https://discuss.elastic.co/t/remove-double-quotes-from-field-json-file/229641)
**Author:** [@alfonso.viso](https://discuss.elastic.co/u/alfonso.viso)\ **Replies:** 0\ **Last updated:** [April 24, 2020, 11:57am UTC](https://discuss.elastic.co/t/remove-double-quotes-from-field-json-file/229641 "2020-04-24T11:57:09Z")
Hi Community!!, I need some help with this issue. Our source is a json file and one of these fields has this value: "IF\_NON\_MATCH":""99914b932bd37a50b983c5e7c90ae93b"" I try remove the double quotes with gsub using th… --- ## [Installation Error](https://discuss.elastic.co/t/installation-error/229544)
**Author:** [@InfiniteLoops](https://discuss.elastic.co/u/InfiniteLoops)\ **Replies:** 2\ **Last updated:** [April 24, 2020, 11:10am UTC](https://discuss.elastic.co/t/installation-error/229544 "2020-04-24T11:10:57Z")
I am trying to setup a DLQ. I was getting Errors related to the plugin. I tried uninstalling the plugin and reinstalling it but the same error was happening that tells me it didn't exist. I decided to reinstall Logstash.… --- ## [Logstash - Grok - Parse Digit](https://discuss.elastic.co/t/logstash-grok-parse-digit/229596)
**Author:** [@pjo](https://discuss.elastic.co/u/pjo)\ **Replies:** 1\ **Last updated:** [April 24, 2020, 8:42am UTC](https://discuss.elastic.co/t/logstash-grok-parse-digit/229596 "2020-04-24T08:42:38Z")
Hello, I have this simple chain that represent a date (with 2digit year format) 200424 I'm looking for a way to parse it to obtain this output : year : 20 month : 04 montday : 24 Is it possible with grok to extra… --- ## [New pipelines are not loaded correctly](https://discuss.elastic.co/t/new-pipelines-are-not-loaded-correctly/229305)
**Author:** [@lsambolino](https://discuss.elastic.co/u/lsambolino)\ **Replies:** 2\ **Last updated:** [April 24, 2020, 8:02am UTC](https://discuss.elastic.co/t/new-pipelines-are-not-loaded-correctly/229305 "2020-04-24T08:02:12Z")
Hi, In our ELK 7.6.2 setup, we configured 4 new pipelines, so having a total of 8 pipelines: elkserver01# ls -la total 76 drwxrwxr-x 10 logstash logstash 4096 Apr 22 14:18 . drwxr-xr-x 4 logstash logs… --- ## [Centralized pipeline management](https://discuss.elastic.co/t/centralized-pipeline-management/229586)
**Author:** [@probson](https://discuss.elastic.co/u/probson)\ **Replies:** 0\ **Last updated:** [April 24, 2020, 7:06am UTC](https://discuss.elastic.co/t/centralized-pipeline-management/229586 "2020-04-24T07:06:46Z")
Hi, I was considering using the centralized pipeline management, I have tried to find some upto date videos/guides, I found an old blog with aspirations for central management but no further info other than the elastic … --- ## [Ruby Code in Logstash to extract specific line](https://discuss.elastic.co/t/ruby-code-in-logstash-to-extract-specific-line/229385)
**Author:** [@vikramaddagulla](https://discuss.elastic.co/u/vikramaddagulla)\ **Replies:** 4\ **Last updated:** [April 24, 2020, 5:47am UTC](https://discuss.elastic.co/t/ruby-code-in-logstash-to-extract-specific-line/229385 "2020-04-24T05:47:03Z")
Hello, I have a java stack error message and I would like to extract "Caused by" line from the full error message. I had followed the instructions stated in the below post : I was able to achieve the results but with… --- ## ["Timeout waiting for connection from pool" from http\_poller plugin](https://discuss.elastic.co/t/timeout-waiting-for-connection-from-pool-from-http-poller-plugin/229014)
**Author:** [@fr3eze](https://discuss.elastic.co/u/fr3eze)\ **Replies:** 4\ **Last updated:** [April 24, 2020, 1:50am UTC](https://discuss.elastic.co/t/timeout-waiting-for-connection-from-pool-from-http-poller-plugin/229014 "2020-04-24T01:50:57Z")
I have a single logstash configuration with single http\_poller plugin, and a lot of URL stacked inside: input { http\_poller { urls =\> { A =\> { method =\> get url =\> "https://xxx.xxx" h… --- ## [Json date is not parsing to @timestamp](https://discuss.elastic.co/t/json-date-is-not-parsing-to-timestamp/229539)
**Author:** [@Bhupal\_Bobbadi](https://discuss.elastic.co/u/Bhupal_Bobbadi)\ **Replies:** 4\ **Last updated:** [April 23, 2020, 10:42pm UTC](https://discuss.elastic.co/t/json-date-is-not-parsing-to-timestamp/229539 "2020-04-23T22:42:13Z")
I'm new to logstash and I'm trying to index json data into Elastic search with logstash and my data has datetime field, and I'm trying to use that field as @timestamp but I could not parse it to the field. Any suggestio… --- ## [General Help](https://discuss.elastic.co/t/general-help/229532)
**Author:** [@loatswil](https://discuss.elastic.co/u/loatswil)\ **Replies:** 2\ **Last updated:** [April 23, 2020, 10:10pm UTC](https://discuss.elastic.co/t/general-help/229532 "2020-04-23T22:10:10Z")
I have set up Elastic, Logstash and Kibana on a spare machine to get my feet wet and I am having trouble just getting data into the system and visible in Kibana. I have various systems sending rsyslog data to a director… --- ## [Logstash stuck](https://discuss.elastic.co/t/logstash-stuck/229494)
**Author:** [@Burga](https://discuss.elastic.co/u/Burga)\ **Replies:** 5\ **Last updated:** [April 23, 2020, 8:13pm UTC](https://discuss.elastic.co/t/logstash-stuck/229494 "2020-04-23T20:13:17Z")
Guys need your help I'm desperate, I'm trying to configure pipelines on logstash server , but it doesn't work , please take a look I'll be grateful to get some advice how to make it work when I start logstash in the l… --- ## [Unable to install logstash on Ubuntu 18.0.4 LTS](https://discuss.elastic.co/t/unable-to-install-logstash-on-ubuntu-18-0-4-lts/229515)
**Author:** [@tbhaxor](https://discuss.elastic.co/u/tbhaxor)\ **Replies:** 0\ **Last updated:** [April 23, 2020, 4:35pm UTC](https://discuss.elastic.co/t/unable-to-install-logstash-on-ubuntu-18-0-4-lts/229515 "2020-04-23T16:35:56Z")
I was installing logstash 7.6.1 for opendistro compatibility. After downling the package from elastic artifacts, I am getting this error Errno::EISDIR: Is a directory - /etc/default/logstash write at org/jruby… --- ## [Condition statements output Logstash](https://discuss.elastic.co/t/condition-statements-output-logstash/229508)
**Author:** [@GedeoN](https://discuss.elastic.co/u/GedeoN)\ **Replies:** 2\ **Last updated:** [April 23, 2020, 4:32pm UTC](https://discuss.elastic.co/t/condition-statements-output-logstash/229508 "2020-04-23T16:32:12Z")
Hello, I'm working on possibilities to have many output in logstash. I actually don't understand what is wrong. I have 7.x versions. If i just use this, it's working. Blockquote output { if "ERRORLOG" in \[tags\] { … --- ## [HELP WITH LOGSTASH PLUGIN](https://discuss.elastic.co/t/help-with-logstash-plugin/229438)
**Author:** [@J-JENNY](https://discuss.elastic.co/u/J-JENNY)\ **Replies:** 12\ **Last updated:** [April 23, 2020, 4:22pm UTC](https://discuss.elastic.co/t/help-with-logstash-plugin/229438 "2020-04-23T16:22:49Z")
I am trying to analyze whatsapp chats on elastic, but my logstash config file seem to be the problem this is my config file input { beats { type =\> "whatsapp-alerts" port =\> 5012 } } filter { if \[type\]=="whatsa… --- ## [Write your own Java filter plugin - graceful error handling](https://discuss.elastic.co/t/write-your-own-java-filter-plugin-graceful-error-handling/229511)
**Author:** [@mlefebvre](https://discuss.elastic.co/u/mlefebvre)\ **Replies:** 0\ **Last updated:** [April 23, 2020, 4:08pm UTC](https://discuss.elastic.co/t/write-your-own-java-filter-plugin-graceful-error-handling/229511 "2020-04-23T16:08:29Z")
Write your own Java filter plugin - graceful error handling Hello everyone, I hope you are all doing great ! I've been writing my own filter logstash plugin in Java the past couple days and it was quite a learning expe… --- ## [Cannot see Logstash index](https://discuss.elastic.co/t/cannot-see-logstash-index/229488)
**Author:** [@BlessingEzeama](https://discuss.elastic.co/u/BlessingEzeama)\ **Replies:** 0\ **Last updated:** [April 23, 2020, 2:27pm UTC](https://discuss.elastic.co/t/cannot-see-logstash-index/229488 "2020-04-23T14:27:33Z")
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size red open filebeat-7.6.2-2020.04.23 1\_mZNRLHRD6hAJ7YcFildw 1 1 yellow o… --- ## [Logstash template for different IIS versions](https://discuss.elastic.co/t/logstash-template-for-different-iis-versions/229005)
**Author:** [@wonderlust](https://discuss.elastic.co/u/wonderlust)\ **Replies:** 3\ **Last updated:** [April 23, 2020, 1:44pm UTC](https://discuss.elastic.co/t/logstash-template-for-different-iis-versions/229005 "2020-04-23T13:44:33Z")
Hello, I would like to create a logstash template that uses different grok filters for different IIS log versions. So far, Im using an if statement to check the message if contains IIS log version 10.0 or else for older… --- ## [\[types removal\] Specifying types in bulk requests is deprecated](https://discuss.elastic.co/t/types-removal-specifying-types-in-bulk-requests-is-deprecated/229467)
**Author:** [@PerseiX1](https://discuss.elastic.co/u/PerseiX1)\ **Replies:** 0\ **Last updated:** [April 23, 2020, 12:46pm UTC](https://discuss.elastic.co/t/types-removal-specifying-types-in-bulk-requests-is-deprecated/229467 "2020-04-23T12:46:13Z")
Hello, I try to use ELK from docker all with version 7.6.0 but in Elasticsearch I have an error elasticsearch\_1 | {"type": "deprecation", "timestamp": "2020-04-23T12:14:46,194Z", "level": "WARN", "component": "o.e.d.a… --- ## [Debug issue](https://discuss.elastic.co/t/debug-issue/229429)
**Author:** [@Ido\_Cohen](https://discuss.elastic.co/u/Ido_Cohen)\ **Replies:** 0\ **Last updated:** [April 23, 2020, 9:45am UTC](https://discuss.elastic.co/t/debug-issue/229429 "2020-04-23T09:45:51Z")
Hi all, I am running a docker containter of logstash - I am using logstash to send csv reports to elastic. Part of the lines in the csv are not parsing well. When I reading logstash log I can find the error but cannot… --- ## [Flattening array and dynamically create the field name](https://discuss.elastic.co/t/flattening-array-and-dynamically-create-the-field-name/229268)
**Author:** [@manto](https://discuss.elastic.co/u/manto)\ **Replies:** 2\ **Last updated:** [April 23, 2020, 9:42am UTC](https://discuss.elastic.co/t/flattening-array-and-dynamically-create-the-field-name/229268 "2020-04-23T09:42:17Z")
Hello, I have an JSON array input below: "coverageStats": \[ { "label": "Branches", "position": 6, "total": 30, "covered": … --- ## [Digital signature on logs](https://discuss.elastic.co/t/digital-signature-on-logs/229409)
**Author:** [@chiara](https://discuss.elastic.co/u/chiara)\ **Replies:** 1\ **Last updated:** [April 23, 2020, 9:32am UTC](https://discuss.elastic.co/t/digital-signature-on-logs/229409 "2020-04-23T09:32:35Z")
For Compliance rules I need to digitally sign my logs and make them unchangeable. What is the best way to do it? --- ## [Input jdbc plugin while using with sql\_last\_value gives error](https://discuss.elastic.co/t/input-jdbc-plugin-while-using-with-sql-last-value-gives-error/229419)
**Author:** [@rahul24](https://discuss.elastic.co/u/rahul24)\ **Replies:** 0\ **Last updated:** [April 23, 2020, 9:17am UTC](https://discuss.elastic.co/t/input-jdbc-plugin-while-using-with-sql-last-value-gives-error/229419 "2020-04-23T09:17:59Z")
Hello, I am trying to poke my sql db to get only the updated values from the table using column ChangedAtas the tracking column with below configuration: \`input { jdbc { type =\> "actlock" jdbc\_c… --- ## [HTTP POST with CSV](https://discuss.elastic.co/t/http-post-with-csv/228052)
**Author:** [@karnamonkster](https://discuss.elastic.co/u/karnamonkster)\ **Replies:** 4\ **Last updated:** [April 23, 2020, 7:23am UTC](https://discuss.elastic.co/t/http-post-with-csv/228052 "2020-04-23T07:23:37Z")
Hi , This is in connection with one of the issues here I have a typical scenario where i am not able to find this thing working. I could see the message at the other end (tested with localhost running : nc -l -k port)… --- ## [Please remove](https://discuss.elastic.co/t/please-remove/229319)
**Author:** [@sera](https://discuss.elastic.co/u/sera)\ **Replies:** 0\ **Last updated:** [April 22, 2020, 4:49pm UTC](https://discuss.elastic.co/t/please-remove/229319 "2020-04-22T16:49:37Z")
Please remove this post. --- ## [Best Practices in logstash](https://discuss.elastic.co/t/best-practices-in-logstash/229379)
**Author:** [@Vignesh2912](https://discuss.elastic.co/u/Vignesh2912)\ **Replies:** 0\ **Last updated:** [April 23, 2020, 5:16am UTC](https://discuss.elastic.co/t/best-practices-in-logstash/229379 "2020-04-23T05:16:06Z")
Hi All, My current setup is as below, Amazon S3 -\> Logstash -\> Elasticsearch -\> Kibana In logstash, we are trying to poll files from 4 s3 buckets ie each for each environment. so we are expecting data load per day of … --- ## [Parsing multiple formats in the same log file](https://discuss.elastic.co/t/parsing-multiple-formats-in-the-same-log-file/229377)
**Author:** [@vikramaddagulla](https://discuss.elastic.co/u/vikramaddagulla)\ **Replies:** 0\ **Last updated:** [April 23, 2020, 4:49am UTC](https://discuss.elastic.co/t/parsing-multiple-formats-in-the-same-log-file/229377 "2020-04-23T04:49:08Z")
Hello, Is it possible to parse multiple formats in a single log file ? For ex : My log file can write different type of data belonging to different sub-system within the same application which is writing the logs. The… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=367) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=369)