# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=709 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 710 --- ## [Logstash JDBC input : Encrypted username and password](https://discuss.elastic.co/t/logstash-jdbc-input-encrypted-username-and-password/142731)
**Author:** [@Goyal.vikas87](https://discuss.elastic.co/u/Goyal.vikas87)\ **Replies:** 1\ **Last updated:** [August 2, 2018, 9:43am UTC](https://discuss.elastic.co/t/logstash-jdbc-input-encrypted-username-and-password/142731 "2018-08-02T09:43:09Z")
Please help how I can use encrypted username and password in logstash config files rather than hard coding in input section. Is there any way that I can read the parameters from an external file. --- ## [Running Logstash from a Rails application](https://discuss.elastic.co/t/running-logstash-from-a-rails-application/142730)
**Author:** [@AmiSazar](https://discuss.elastic.co/u/AmiSazar)\ **Replies:** 0\ **Last updated:** [August 2, 2018, 9:40am UTC](https://discuss.elastic.co/t/running-logstash-from-a-rails-application/142730 "2018-08-02T09:40:03Z")
Sorry if this is a newbie question or misplaced here - I have found no information on this by searching. I have a Rails app with its own Ruby environment. From this app, I am trying to start logstash via Ruby's system()… --- ## [Input / Filter problems with 6.3.2](https://discuss.elastic.co/t/input-filter-problems-with-6-3-2/142721)
**Author:** [@DanielE](https://discuss.elastic.co/u/DanielE)\ **Replies:** 2\ **Last updated:** [August 2, 2018, 9:23am UTC](https://discuss.elastic.co/t/input-filter-problems-with-6-3-2/142721 "2018-08-02T09:23:55Z")
Hi, i´m having a problem with my new 6.3.2 stack. I am using three different conf files for Input, filter and ouput. At the Moment i can´t add a field in my Input to start using my filter. Also my experiements to use ta… --- ## [How file input plugin work?](https://discuss.elastic.co/t/how-file-input-plugin-work/142703)
**Author:** [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)\ **Replies:** 2\ **Last updated:** [August 2, 2018, 8:25am UTC](https://discuss.elastic.co/t/how-file-input-plugin-work/142703 "2018-08-02T08:25:49Z")
Hello All, Currently i am using filebeat agent to forward the logs from file to logstash server. But i want to know how file input plugin work in logstash? I am confused how it will fetch the logs from remote server? If… --- ## [Get value out of key from document](https://discuss.elastic.co/t/get-value-out-of-key-from-document/142701)
**Author:** [@JPelastic](https://discuss.elastic.co/u/JPelastic)\ **Replies:** 1\ **Last updated:** [August 2, 2018, 7:45am UTC](https://discuss.elastic.co/t/get-value-out-of-key-from-document/142701 "2018-08-02T07:45:53Z")
Hi I don't understand the filter possibilities yet: I have output documents { "xml" =\> "some xml here", "@timestamp" =\> some date here, "@version" =\> 1 } This is my filter - the input is not important. What I w… --- ## [Encountered a retryable error. Will Retry with exponential backoff {:code=\>400](https://discuss.elastic.co/t/encountered-a-retryable-error-will-retry-with-exponential-backoff-code-400/142702)
**Author:** [@myName](https://discuss.elastic.co/u/myName)\ **Replies:** 0\ **Last updated:** [August 2, 2018, 7:44am UTC](https://discuss.elastic.co/t/encountered-a-retryable-error-will-retry-with-exponential-backoff-code-400/142702 "2018-08-02T07:44:14Z")
Hello We are running logstash 5.6.3, and sometimes it stops inserting data into ES After restarting the logstash, the data appears in the kibana but the logstash log is filled with messages like this: \[2018-08-02T07:… --- ## [Accessing hash fields with the JDBC\_STATIC filter](https://discuss.elastic.co/t/accessing-hash-fields-with-the-jdbc-static-filter/142689)
**Author:** [@victor.nilsson](https://discuss.elastic.co/u/victor.nilsson)\ **Replies:** 3\ **Last updated:** [August 2, 2018, 7:20am UTC](https://discuss.elastic.co/t/accessing-hash-fields-with-the-jdbc-static-filter/142689 "2018-08-02T07:20:40Z")
Hi, I am currently receiving logs from windows servers that are shipped with winlogbeat and adding a field from the metadata that winlogbeat is sending, like this: 1001-input-beats.conf: input { beats { port =\> … --- ## [Jdbc\_static does not write any staging file](https://discuss.elastic.co/t/jdbc-static-does-not-write-any-staging-file/142695)
**Author:** [@alesnav](https://discuss.elastic.co/u/alesnav)\ **Replies:** 0\ **Last updated:** [August 2, 2018, 7:08am UTC](https://discuss.elastic.co/t/jdbc-static-does-not-write-any-staging-file/142695 "2018-08-02T07:08:38Z")
Hi there! I am facing a problem with jdbc\_static, because it does not write any staging file. I have scheduled each load \*/8 hours, but it is querying the database many more times (near 1 per minute) because it does no… --- ## [How to display error messages or syslogs message date wise](https://discuss.elastic.co/t/how-to-display-error-messages-or-syslogs-message-date-wise/142489)
**Author:** [@geetika-06](https://discuss.elastic.co/u/geetika-06)\ **Replies:** 1\ **Last updated:** [August 2, 2018, 6:32am UTC](https://discuss.elastic.co/t/how-to-display-error-messages-or-syslogs-message-date-wise/142489 "2018-08-02T06:32:49Z")
Hi as i want to display my syslogs or error messages in kibana "discover page" as group wise and it should be grouped on the basis of date but in kibana it shows logs of eg. apache as date hours mins and sec eg August 1… --- ## [Read input from http request](https://discuss.elastic.co/t/read-input-from-http-request/142273)
**Author:** [@jibinoosae](https://discuss.elastic.co/u/jibinoosae)\ **Replies:** 1\ **Last updated:** [August 2, 2018, 6:11am UTC](https://discuss.elastic.co/t/read-input-from-http-request/142273 "2018-08-02T06:11:19Z")
Hi , I want to read input from a http request which give response as json can anyone help me to do witha simple example --- ## [Hello World for ELK Stack with Filebeats in windows 10, Please Help](https://discuss.elastic.co/t/hello-world-for-elk-stack-with-filebeats-in-windows-10-please-help/142677)
**Author:** [@Ab\_A](https://discuss.elastic.co/u/Ab_A)\ **Replies:** 0\ **Last updated:** [August 2, 2018, 5:04am UTC](https://discuss.elastic.co/t/hello-world-for-elk-stack-with-filebeats-in-windows-10-please-help/142677 "2018-08-02T05:04:54Z")
I am new to ELK Stack and unable to get basic working. Please help. Steps Done: Started elasticsearch.bat (Started Fine) Open http://localhost:9200 in Chrome and got: { "name" : "SrfDQVN", "cluster\_name… --- ## [ERR Failed to publish events caused by: read tcp X.XXXX:55860-\>XXXXXXX:5044: i/o timeout](https://discuss.elastic.co/t/err-failed-to-publish-events-caused-by-read-tcp-x-xxxx-55860-xxxxxxx-i-o-timeout/142000)
**Author:** [@rajkumar.m](https://discuss.elastic.co/u/rajkumar.m)\ **Replies:** 7\ **Last updated:** [August 2, 2018, 3:58am UTC](https://discuss.elastic.co/t/err-failed-to-publish-events-caused-by-read-tcp-x-xxxx-55860-xxxxxxx-i-o-timeout/142000 "2018-08-02T03:58:55Z")
file beat is not able to send data and getting timeout error with logstash, what could be the reason --- ## [Event dependent configuration broken with multiple pipelines?](https://discuss.elastic.co/t/event-dependent-configuration-broken-with-multiple-pipelines/141869)
**Author:** [@Steven\_Ensslen](https://discuss.elastic.co/u/Steven_Ensslen)\ **Replies:** 4\ **Last updated:** [August 2, 2018, 3:32am UTC](https://discuss.elastic.co/t/event-dependent-configuration-broken-with-multiple-pipelines/141869 "2018-08-02T03:32:28Z")
Hi, If I set pipeline.workers \> 1 then event dependent configuration, "%{}", appears to be disabled. Here is my test case. #!/bin/bash # This works export TIX\_VERSION=1 gunzip --stdout tmp/cart.gz | logstash --pipelin… --- ## [Remove starting and ending forward slash](https://discuss.elastic.co/t/remove-starting-and-ending-forward-slash/142659)
**Author:** [@abhishek007](https://discuss.elastic.co/u/abhishek007)\ **Replies:** 3\ **Last updated:** [August 2, 2018, 2:34am UTC](https://discuss.elastic.co/t/remove-starting-and-ending-forward-slash/142659 "2018-08-02T02:34:28Z")
Hi, I have to get rid of the starting forward slash in a field value. Field value is "/abc/par/xyz/" I want this ti be muted as abc\_par\_xyz I can replace '/' with '\_' using gsub, but it gives me "abc\_par\_xyz" How ca… --- ## [Logstash rds filtering?](https://discuss.elastic.co/t/logstash-rds-filtering/142655)
**Author:** [@jspriesterbach](https://discuss.elastic.co/u/jspriesterbach)\ **Replies:** 2\ **Last updated:** [August 1, 2018, 11:09pm UTC](https://discuss.elastic.co/t/logstash-rds-filtering/142655 "2018-08-01T23:09:06Z")
Hello, I currently have a script that will download all the error & slow logs from RDS for the day and put them in a folder. I am now trying to get logstash to read those files and put them into our ELK stack for use wi… --- ## [How to re-ingest logs after upgrade to Elastic Stack 6.3.0](https://discuss.elastic.co/t/how-to-re-ingest-logs-after-upgrade-to-elastic-stack-6-3-0/140743)
**Author:** [@Reena\_Deberry](https://discuss.elastic.co/u/Reena_Deberry)\ **Replies:** 22\ **Last updated:** [August 1, 2018, 9:17pm UTC](https://discuss.elastic.co/t/how-to-re-ingest-logs-after-upgrade-to-elastic-stack-6-3-0/140743 "2018-08-01T21:17:50Z")
I've re-installed the ELK stack from scratch on a server that previously had a lower version. I though i had it all cleaned up prior to installing the latest version of ELK. When i go to the Kibana console i am not seein… --- ## [Exit logstash and jdbc input completes](https://discuss.elastic.co/t/exit-logstash-and-jdbc-input-completes/142446)
**Author:** [@daq](https://discuss.elastic.co/u/daq)\ **Replies:** 2\ **Last updated:** [August 1, 2018, 9:10pm UTC](https://discuss.elastic.co/t/exit-logstash-and-jdbc-input-completes/142446 "2018-08-01T21:10:02Z")
Is there any way to exit logstash after (an unscheduled) jdbc input completes? --- ## [How To Integrate an Logstash in androidapplication?](https://discuss.elastic.co/t/how-to-integrate-an-logstash-in-androidapplication/142285)
**Author:** [@rajeshwaran](https://discuss.elastic.co/u/rajeshwaran)\ **Replies:** 1\ **Last updated:** [August 1, 2018, 8:44pm UTC](https://discuss.elastic.co/t/how-to-integrate-an-logstash-in-androidapplication/142285 "2018-08-01T20:44:17Z")
please any one help me to implement logstash in androia applications --- ## [How is -%{+YYYY.MM.dd} generated on Logstash output](https://discuss.elastic.co/t/how-is-yyyy-mm-dd-generated-on-logstash-output/142189)
**Author:** [@A\_B](https://discuss.elastic.co/u/A_B)\ **Replies:** 3\ **Last updated:** [August 1, 2018, 8:35pm UTC](https://discuss.elastic.co/t/how-is-yyyy-mm-dd-generated-on-logstash-output/142189 "2018-08-01T20:35:52Z")
Hello, I use daily ES indices that are created by Logstash using output { elasticsearch { ... index =\> "%{\[@metadata\]\[foo\]}-%{+YYYY.MM.dd}" } } I always assumed that Logstash looks up the date from the sy… --- ## [Not able to install plugin http poller](https://discuss.elastic.co/t/not-able-to-install-plugin-http-poller/142176)
**Author:** [@jibinoosae](https://discuss.elastic.co/u/jibinoosae)\ **Replies:** 1\ **Last updated:** [August 1, 2018, 8:32pm UTC](https://discuss.elastic.co/t/not-able-to-install-plugin-http-poller/142176 "2018-08-01T20:32:42Z")
please help to install the http pollen plugin root@elastic:/usr/share/logstash# ./bin/logstash-plugin install logstash-input-http\_poller-3.3.4.gem Validating logstash-input-http\_poller-3.3.4.gem Plugin logstash-input… --- ## [Remove \\n and \\t from message](https://discuss.elastic.co/t/remove-n-and-t-from-message/142620)
**Author:** [@ArunANayagam](https://discuss.elastic.co/u/ArunANayagam)\ **Replies:** 5\ **Last updated:** [August 1, 2018, 6:21pm UTC](https://discuss.elastic.co/t/remove-n-and-t-from-message/142620 "2018-08-01T18:21:47Z")
Hi, I am trying to ingest Java exceptions into Elasticsearch via logstash. In Kibana, they show up like this, \\n\\tat org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)\\n\\tat org.apache.coyote… --- ## [Using global variables or reading values from a file](https://discuss.elastic.co/t/using-global-variables-or-reading-values-from-a-file/142618)
**Author:** [@lucas.alvarez](https://discuss.elastic.co/u/lucas.alvarez)\ **Replies:** 1\ **Last updated:** [August 1, 2018, 5:56pm UTC](https://discuss.elastic.co/t/using-global-variables-or-reading-values-from-a-file/142618 "2018-08-01T17:56:29Z")
Hello everybody!. I was wondering if there's a way to either define global variables that I can use from the output.conf file or read them from a given file. My scenario is the following, in 201-output.conf file I have … --- ## [XML on Elasticsearch](https://discuss.elastic.co/t/xml-on-elasticsearch/142601)
**Author:** [@Fram\_Souza](https://discuss.elastic.co/u/Fram_Souza)\ **Replies:** 9\ **Last updated:** [August 1, 2018, 5:50pm UTC](https://discuss.elastic.co/t/xml-on-elasticsearch/142601 "2018-08-01T17:50:48Z")
Hi Guys! I need index file XML on cluster Elasticsearch, following a flux: S3 -\> Logstash -\> Elasticsearch I read about xpath on xml filter, BUT my xml is very large then I don't get map all xml I need that each fiel… --- ## [SQL to Elastic](https://discuss.elastic.co/t/sql-to-elastic/142408)
**Author:** [@Gian\_Carlo\_Menotti](https://discuss.elastic.co/u/Gian_Carlo_Menotti)\ **Replies:** 3\ **Last updated:** [August 1, 2018, 5:32pm UTC](https://discuss.elastic.co/t/sql-to-elastic/142408 "2018-08-01T17:32:54Z")
Hi, i would like to know, how to add 40 tables in elastic using logstash. the best way is make 40 conf files? --- ## [How to exclude duplicates in Aggregate filter?](https://discuss.elastic.co/t/how-to-exclude-duplicates-in-aggregate-filter/140939)
**Author:** [@Jevgenij](https://discuss.elastic.co/u/Jevgenij)\ **Replies:** 6\ **Last updated:** [August 1, 2018, 4:53pm UTC](https://discuss.elastic.co/t/how-to-exclude-duplicates-in-aggregate-filter/140939 "2018-08-01T16:53:00Z")
Hello! I'm fetching some data from db with multiple associated tables. My query: SELECT \`item\`.\`id\`, \`item\`.\`name\`, \`colour\`.\`name\` AS \`colour\`, \`material\`.\`name\` AS \`material\` FROM \`item\` LEFT JOIN \`item\_has\_colour\` … --- ## [Logstash JSON parsing of @timestamp](https://discuss.elastic.co/t/logstash-json-parsing-of-timestamp/142476)
**Author:** [@mykael](https://discuss.elastic.co/u/mykael)\ **Replies:** 4\ **Last updated:** [August 1, 2018, 4:03pm UTC](https://discuss.elastic.co/t/logstash-json-parsing-of-timestamp/142476 "2018-08-01T16:03:14Z")
The docs for the logstrash JSON plugin say: If the parsed data contains a @timestamp field, we will try to use it for the event’s @timestamp, if the parsing fails, the field will be renamed to \_@timestamp and the event… --- ## [Geoip lookup fails](https://discuss.elastic.co/t/geoip-lookup-fails/142612)
**Author:** [@jmoffitt](https://discuss.elastic.co/u/jmoffitt)\ **Replies:** 1\ **Last updated:** [August 1, 2018, 3:51pm UTC](https://discuss.elastic.co/t/geoip-lookup-fails/142612 "2018-08-01T15:51:29Z")
As far as I can tell I've followed the directions, but all of my geoip lookups fail. What did I miss? input { udp { port =\> 11001 codec =\> netflow { versions =\> \[5, 9\] } t… --- ## [Upgrade Logstash Elasticsearch output plugin without updating Logstash core](https://discuss.elastic.co/t/upgrade-logstash-elasticsearch-output-plugin-without-updating-logstash-core/142535)
**Author:** [@Denis\_Filatov](https://discuss.elastic.co/u/Denis_Filatov)\ **Replies:** 6\ **Last updated:** [August 1, 2018, 2:16pm UTC](https://discuss.elastic.co/t/upgrade-logstash-elasticsearch-output-plugin-without-updating-logstash-core/142535 "2018-08-01T14:16:49Z")
Hello! We using 5.1.2 version of Elastic stack (Logstash, Kibana, Elastic). And now we need to dead letter queue feature. The best way i see - upgrade only elasticsearch output plugin from Version: 5.4.0 to version Ve… --- ## [Mssql](https://discuss.elastic.co/t/mssql/141805)
**Author:** [@Gian\_Carlo\_Menotti](https://discuss.elastic.co/u/Gian_Carlo_Menotti)\ **Replies:** 2\ **Last updated:** [August 1, 2018, 1:47pm UTC](https://discuss.elastic.co/t/mssql/141805 "2018-08-01T13:47:08Z")
how can i connect to MS SQL to Elastic? i search a lot but i cant find a tutorial that works.. im using ubuntu 18 --- ## [Output jdbc](https://discuss.elastic.co/t/output-jdbc/142564)
**Author:** [@hamza90](https://discuss.elastic.co/u/hamza90)\ **Replies:** 2\ **Last updated:** [August 1, 2018, 1:36pm UTC](https://discuss.elastic.co/t/output-jdbc/142564 "2018-08-01T13:36:06Z")
im collecting csv data with logstash and i want to store with mysql .what is Logstash output plugins available for mysql? thanks [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=708) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=710)