# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=874 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 875 --- ## [Ruby code in logstash](https://discuss.elastic.co/t/ruby-code-in-logstash/107735)
**Author:** [@Shekhar\_Pandey](https://discuss.elastic.co/u/Shekhar_Pandey)\ **Replies:** 12\ **Last updated:** [November 16, 2017, 8:41am UTC](https://discuss.elastic.co/t/ruby-code-in-logstash/107735 "2017-11-16T08:41:54Z")
I want to fetch the column name in ruby... for example mrp = Rs.23 brand = Apple Above is my mysql database.... So I am using event.get(mrp) and getting value. But I also want mrp as a name. So how can I get. And … --- ## [How to get a record from ElasticSearch in the logstash filter?](https://discuss.elastic.co/t/how-to-get-a-record-from-elasticsearch-in-the-logstash-filter/107297)
**Author:** [@Harim\_Tong](https://discuss.elastic.co/u/Harim_Tong)\ **Replies:** 2\ **Last updated:** [November 16, 2017, 6:48am UTC](https://discuss.elastic.co/t/how-to-get-a-record-from-elasticsearch-in-the-logstash-filter/107297 "2017-11-16T06:48:29Z")
How to get a record by \_id (or other fields) from ElasticSearch in a logstash filter stage? I know the Elasticsearch provides the Get API like: curl -XHEAD -i 'http://localhost:9200/twitter/tweet/1' to get a record from… --- ## [Logstash pipeline is not executing changes when I run it for second time](https://discuss.elastic.co/t/logstash-pipeline-is-not-executing-changes-when-i-run-it-for-second-time/107846)
**Author:** [@Seyf\_Souissi](https://discuss.elastic.co/u/Seyf_Souissi)\ **Replies:** 1\ **Last updated:** [November 16, 2017, 6:37am UTC](https://discuss.elastic.co/t/logstash-pipeline-is-not-executing-changes-when-i-run-it-for-second-time/107846 "2017-11-16T06:37:09Z")
Hi, When I run logstash config file, I can see the index created in elastic search. But when I run it again with few changes( like changing index name) I can see logstash pipeline has started but the index is not create… --- ## [Remove { and " from field name?](https://discuss.elastic.co/t/remove-and-from-field-name/106977)
**Author:** [@Sjaak01](https://discuss.elastic.co/u/Sjaak01)\ **Replies:** 10\ **Last updated:** [November 16, 2017, 6:29am UTC](https://discuss.elastic.co/t/remove-and-from-field-name/106977 "2017-11-16T06:29:59Z")
Hi, I'm outputting some log data to disk where and logstash is marking the first field name with { and the last field value with }. For whatever reason I also have a field name called "date so that field isn't parsed by… --- ## [Logstash6 recommended way of filtering for the future?](https://discuss.elastic.co/t/logstash6-recommended-way-of-filtering-for-the-future/107748)
**Author:** [@MarcusCaepio](https://discuss.elastic.co/u/MarcusCaepio)\ **Replies:** 4\ **Last updated:** [November 16, 2017, 6:19am UTC](https://discuss.elastic.co/t/logstash6-recommended-way-of-filtering-for-the-future/107748 "2017-11-16T06:19:31Z")
Hi all, GZ for Elastic Stack 6. I am still a noobie in ELK and after reading most of the changelog I have a question: In https://www.elastic.co/blog/elasticsearch-6-0-0-released it is said that elastic is already goin… --- ## [Creating a temporary variable in logstash](https://discuss.elastic.co/t/creating-a-temporary-variable-in-logstash/107820)
**Author:** [@joconner](https://discuss.elastic.co/u/joconner)\ **Replies:** 3\ **Last updated:** [November 15, 2017, 8:27pm UTC](https://discuss.elastic.co/t/creating-a-temporary-variable-in-logstash/107820 "2017-11-15T20:27:32Z")
As I experiment with logstash, I have tried to create temporary variables to hold data that I hope to later use to use in an add\_field statement. However, I often see configuration errors when I try to do this. How does… --- ## [ELK web APIs](https://discuss.elastic.co/t/elk-web-apis/107794)
**Author:** [@kundavaram](https://discuss.elastic.co/u/kundavaram)\ **Replies:** 4\ **Last updated:** [November 15, 2017, 8:16pm UTC](https://discuss.elastic.co/t/elk-web-apis/107794 "2017-11-15T20:16:13Z")
Hello, I'm new to ELK Stack. We would like to feed logs that are generated from our middleware application (Biztalk). I'm considering couple of options: My app generated an XML logs and whats the best way to feed th… --- ## [Logstash 5.6.3 isn't recognizing X-pack](https://discuss.elastic.co/t/logstash-5-6-3-isnt-recognizing-x-pack/107801)
**Author:** [@nkj](https://discuss.elastic.co/u/nkj)\ **Replies:** 1\ **Last updated:** [November 15, 2017, 8:08pm UTC](https://discuss.elastic.co/t/logstash-5-6-3-isnt-recognizing-x-pack/107801 "2017-11-15T20:08:47Z")
Logstash has X-pack installed but won't recognize it. I'm getting the following logs as output: \[2017-11-15T11:03:25,399\]\[FATAL\]\[logstash.runner \] An unexpected error occurred! {:error=\>#\ **Author:** [@mbvelo](https://discuss.elastic.co/u/mbvelo)\ **Replies:** 29\ **Last updated:** [November 15, 2017, 6:48pm UTC](https://discuss.elastic.co/t/grok-failure/107270 "2017-11-15T18:48:25Z") please help , i have tried almost everything here is my log and filebeat and logstash configs OUT:999.777.666.172\\tErrorCode=00000\\tOpStatus=00000\\tExceptionReason=null\\tJSON={\\"op\_gsn\\":\\"ghhhh.intra.test.com\\",\\"op\_s… --- ## [Input Json file](https://discuss.elastic.co/t/input-json-file/107781)
**Author:** [@mussa572](https://discuss.elastic.co/u/mussa572)\ **Replies:** 5\ **Last updated:** [November 15, 2017, 6:21pm UTC](https://discuss.elastic.co/t/input-json-file/107781 "2017-11-15T18:21:50Z")
Hi I am trying to ingest some of inventory data from JSON files using the following logstash config. input { file { type =\> "json" path =\> "/opt/uploaddata/\*.json" start\_position =\> "beginning" } } filter { jso… --- ## [Polling with logstash using http\_poller](https://discuss.elastic.co/t/polling-with-logstash-using-http-poller/106709)
**Author:** [@citanionarrei](https://discuss.elastic.co/u/citanionarrei)\ **Replies:** 4\ **Last updated:** [November 15, 2017, 4:30pm UTC](https://discuss.elastic.co/t/polling-with-logstash-using-http-poller/106709 "2017-11-15T16:30:05Z")
hi friends i'm trying to make the polling of json log from this site http://stream.meetup.com/2/rsvps input { http\_poller { urls =\> { myurl =\> "http://stream.meetup.com/2/rsvps" } interval =\> 5 } } but the… --- ## [Logstash 6.0 http input not accepting utf-8](https://discuss.elastic.co/t/logstash-6-0-http-input-not-accepting-utf-8/107731)
**Author:** [@Conall\_Prendergast](https://discuss.elastic.co/u/Conall_Prendergast)\ **Replies:** 3\ **Last updated:** [November 15, 2017, 3:36pm UTC](https://discuss.elastic.co/t/logstash-6-0-http-input-not-accepting-utf-8/107731 "2017-11-15T15:36:26Z")
Hi, I have a logstash pipeline with a http input plugin. I want to use a json codec but this fails when the input contains the character "é". This fails with the error: \[ERROR\] 2017-11-15 11:32:24.663 \[Ruby-0-Thread-2… --- ## [Impact logstash-receiver arrêté](https://discuss.elastic.co/t/impact-logstash-receiver-arrete/107762)
**Author:** [@crapoulou](https://discuss.elastic.co/u/crapoulou)\ **Replies:** 3\ **Last updated:** [November 15, 2017, 3:24pm UTC](https://discuss.elastic.co/t/impact-logstash-receiver-arrete/107762 "2017-11-15T15:24:27Z")
Bonjour, Je tente de comprendre 2 choses : Comment fonctionne logstash et les shippers réagissent lorsque le service Logstash-receiver (ou indexer d'ailleurs) est arrêté : perte de données ? Comment sont horodatés les… --- ## [How to save next line with ruby filter](https://discuss.elastic.co/t/how-to-save-next-line-with-ruby-filter/107773)
**Author:** [@cy8el1us](https://discuss.elastic.co/u/cy8el1us)\ **Replies:** 2\ **Last updated:** [November 15, 2017, 3:23pm UTC](https://discuss.elastic.co/t/how-to-save-next-line-with-ruby-filter/107773 "2017-11-15T15:23:07Z")
I wanted to know if it's possible to write an advanced ruby code into a logstash filter in order to keep specific value. I have something like: ruby { code =\> " \_isInCyclic = false \_isJump = false \_loggerName … --- ## [Help Needed in improving the data ingestion time](https://discuss.elastic.co/t/help-needed-in-improving-the-data-ingestion-time/107767)
**Author:** [@arun\_s](https://discuss.elastic.co/u/arun_s)\ **Replies:** 4\ **Last updated:** [November 15, 2017, 2:50pm UTC](https://discuss.elastic.co/t/help-needed-in-improving-the-data-ingestion-time/107767 "2017-11-15T14:50:58Z")
Hi I am having csv file of size 1.6 GB and i used default configuration of logstash and elasticsearch to upload the data via logstash , it took more than 15hrs and when i tried with LS\_HEAP\_SIZE=2gb the data uploaded in… --- ## [SQLException: ORA-02396: exceeded maximum idle time](https://discuss.elastic.co/t/sqlexception-ora-02396-exceeded-maximum-idle-time/107764)
**Author:** [@kanchan](https://discuss.elastic.co/u/kanchan)\ **Replies:** 2\ **Last updated:** [November 15, 2017, 2:47pm UTC](https://discuss.elastic.co/t/sqlexception-ora-02396-exceeded-maximum-idle-time/107764 "2017-11-15T14:47:59Z")
Hi Team, I tried to push bulk data using logstash configuration file.I have given all database connection related parameters and query but after sometime connection closed and getting below exception: Exception when ex… --- ## [Create geopoint data](https://discuss.elastic.co/t/create-geopoint-data/107580)
**Author:** [@Rym\_Guerbi\_Michaut](https://discuss.elastic.co/u/Rym_Guerbi_Michaut)\ **Replies:** 4\ **Last updated:** [November 15, 2017, 12:54pm UTC](https://discuss.elastic.co/t/create-geopoint-data/107580 "2017-11-15T12:54:41Z")
Hi there, I want to create a beautiful visualisation of some geopoints on a map. I have a small view of 5 points in a relational database and I'm trying to put a correct code in Logstash filter to import the right dat… --- ## [To extract fields from a query string](https://discuss.elastic.co/t/to-extract-fields-from-a-query-string/107680)
**Author:** [@neal1991](https://discuss.elastic.co/u/neal1991)\ **Replies:** 4\ **Last updated:** [November 15, 2017, 12:45pm UTC](https://discuss.elastic.co/t/to-extract-fields-from-a-query-string/107680 "2017-11-15T12:45:23Z")
Assume that there is a field like: Cmd=FolderSync&User=ABC&DeviceId=androidc737881212&DeviceType=Android&Log=V141 I can use mutate to split the field into an array. it will just like \['Cmd=FolderSync....\] But I want … --- ## [Logstash 5.6.2 Open file limit Error](https://discuss.elastic.co/t/logstash-5-6-2-open-file-limit-error/106964)
**Author:** [@shashwat](https://discuss.elastic.co/u/shashwat)\ **Replies:** 4\ **Last updated:** [November 15, 2017, 11:10am UTC](https://discuss.elastic.co/t/logstash-5-6-2-open-file-limit-error/106964 "2017-11-15T11:10:45Z")
Hi, I am facing issue while running logstash. I am getting below warning in logstash logs : "\[WARN \]\[logstash.inputs.file \] Reached open files limit: 4095, set by the 'max\_open\_files' option or default, files yet t… --- ## [Ruby exception occurred: -1](https://discuss.elastic.co/t/ruby-exception-occurred-1/107148)
**Author:** [@Mojster](https://discuss.elastic.co/u/Mojster)\ **Replies:** 9\ **Last updated:** [November 15, 2017, 10:57am UTC](https://discuss.elastic.co/t/ruby-exception-occurred-1/107148 "2017-11-15T10:57:07Z")
I'm running Elastic stack 5.6.3. Since this version or 5.6.2. I'm getting random LS error: \[2017-11-09T07:50:24,829\]\[ERROR\]\[logstash.filters.ruby \] Ruby exception occurred: -1 I've tried to recreate it, but as it looks… --- ## [Aggregation task\_id](https://discuss.elastic.co/t/aggregation-task-id/107720)
**Author:** [@ssasporta](https://discuss.elastic.co/u/ssasporta)\ **Replies:** 0\ **Last updated:** [November 15, 2017, 10:26am UTC](https://discuss.elastic.co/t/aggregation-task-id/107720 "2017-11-15T10:26:07Z")
I am trying to project the task\_id to be part of my index name or to have it as fields in the aggregated events. I try to have it as %{\[task\_id\]} or some other forms, but without success. Any idea how to be able to exp… --- ## [How to manage output side errors](https://discuss.elastic.co/t/how-to-manage-output-side-errors/107708)
**Author:** [@sbek1](https://discuss.elastic.co/u/sbek1)\ **Replies:** 0\ **Last updated:** [November 15, 2017, 9:44am UTC](https://discuss.elastic.co/t/how-to-manage-output-side-errors/107708 "2017-11-15T09:44:53Z")
Hi, I use the jdbc output plugin. I would like to know if it is possible to catch the eventual exception of the plugin (Sql exception for example). And in general, is it possible to handle errors on the output side? --- ## [ERROR "PASS JVM parameters via LS\_JAVA\_OPTS"](https://discuss.elastic.co/t/error-pass-jvm-parameters-via-ls-java-opts/107707)
**Author:** [@Gobinda\_Nandi](https://discuss.elastic.co/u/Gobinda_Nandi)\ **Replies:** 0\ **Last updated:** [November 15, 2017, 9:43am UTC](https://discuss.elastic.co/t/error-pass-jvm-parameters-via-ls-java-opts/107707 "2017-11-15T09:43:00Z")
WHICH FILE AND WHAT PARAMETERS --- ## [X-Pack logstash management - credentials should not be mandatory](https://discuss.elastic.co/t/x-pack-logstash-management-credentials-should-not-be-mandatory/107698)
**Author:** [@Charles.w](https://discuss.elastic.co/u/Charles.w)\ **Replies:** 0\ **Last updated:** [November 15, 2017, 9:16am UTC](https://discuss.elastic.co/t/x-pack-logstash-management-credentials-should-not-be-mandatory/107698 "2017-11-15T09:16:29Z")
Hi, As it is possible to use X-Pack logstash management features with all security features being disabled, the following settings should not be mandatory as they are not always required. xpack.management.elasticsearc… --- ## [Corporate Web gateway blocking logstash zip download](https://discuss.elastic.co/t/corporate-web-gateway-blocking-logstash-zip-download/107560)
**Author:** [@makibroshett](https://discuss.elastic.co/u/makibroshett)\ **Replies:** 2\ **Last updated:** [November 15, 2017, 7:51am UTC](https://discuss.elastic.co/t/corporate-web-gateway-blocking-logstash-zip-download/107560 "2017-11-15T07:51:17Z")
Hi, I'm attempting to install logstash 5.6.4 from https://artifacts.elastic.co/downloads/logstash/logstash-5.6.4.zip. The corporate gateway is complaining about an encrypted zip file inside the package: logstash-5.6.4… --- ## [Grok for Scala](https://discuss.elastic.co/t/grok-for-scala/107343)
**Author:** [@sneha\_krishnamurthy](https://discuss.elastic.co/u/sneha_krishnamurthy)\ **Replies:** 4\ **Last updated:** [November 15, 2017, 7:18am UTC](https://discuss.elastic.co/t/grok-for-scala/107343 "2017-11-15T07:18:48Z")
How do i use the grok plugin in my Scala code to parse data? --- ## [Not working for redis sorted\_set datatype in logstash 5.6.4](https://discuss.elastic.co/t/not-working-for-redis-sorted-set-datatype-in-logstash-5-6-4/107663)
**Author:** [@Samy\_Nathan](https://discuss.elastic.co/u/Samy_Nathan)\ **Replies:** 1\ **Last updated:** [November 15, 2017, 7:09am UTC](https://discuss.elastic.co/t/not-working-for-redis-sorted-set-datatype-in-logstash-5-6-4/107663 "2017-11-15T07:09:28Z")
created logstashredis.conf file input { redis { host =\> "localhost" type =\> "redis-input" data\_type =\> "sorted\_set" key =\> "logstash" # We use json\_event here since the sender is a logstash agent codec =\> json } … --- ## [How to create Grok Filter Pattern for Tomcat logs](https://discuss.elastic.co/t/how-to-create-grok-filter-pattern-for-tomcat-logs/107143)
**Author:** [@shrikantgulia](https://discuss.elastic.co/u/shrikantgulia)\ **Replies:** 4\ **Last updated:** [November 15, 2017, 6:57am UTC](https://discuss.elastic.co/t/how-to-create-grok-filter-pattern-for-tomcat-logs/107143 "2017-11-15T06:57:12Z")
Jun 29, 2008 11:16:20 AM org.apache.catalina.core.ApplicationContext log INFO: ContextListener: contextInitialized() Jun 29, 2008 11:16:20 AM org.apache.catalina.core.ApplicationContext log INFO: SessionListener: cont… --- ## [Logstash windows installation JVM error](https://discuss.elastic.co/t/logstash-windows-installation-jvm-error/107667)
**Author:** [@Gobinda\_Nandi](https://discuss.elastic.co/u/Gobinda_Nandi)\ **Replies:** 3\ **Last updated:** [November 15, 2017, 6:19am UTC](https://discuss.elastic.co/t/logstash-windows-installation-jvm-error/107667 "2017-11-15T06:19:31Z")
Logstash 6.0.0 aborts with “warning: ignoring JAVA\_OPTS=-Xms256m -Xmx512m; pass JVM parameters via LS\_JAVA\_OPTS” --- ## [S3 input plugin not working](https://discuss.elastic.co/t/s3-input-plugin-not-working/107500)
**Author:** [@shrvnkmr](https://discuss.elastic.co/u/shrvnkmr)\ **Replies:** 0\ **Last updated:** [November 14, 2017, 7:28am UTC](https://discuss.elastic.co/t/s3-input-plugin-not-working/107500 "2017-11-14T07:28:24Z")
Hi, I am trying to read json files under sub directory(non-prod/abc/) in a s3 bucket(abc-bucket) using s3 input plugin with IAM roles And Below is my configuration input { s3 { bucket =\> "abc-buc… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=873) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=875)