# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1000 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1001 --- ## [Merge logs](https://discuss.elastic.co/t/merge-logs/82339)
**Author:** [@amakarenkov](https://discuss.elastic.co/u/amakarenkov)\ **Replies:** 0\ **Last updated:** [April 13, 2017, 6:30pm UTC](https://discuss.elastic.co/t/merge-logs/82339 "2017-04-13T18:30:42Z")
Hi there! I have 2 different logs. One of them has this 2 keys: SrcIP and DstIP. Second has this 2 keys: SrcIP and Username. I want to merge this logs using SrcIP unique key so I will have SrcIP, DstIP, Username keys. … --- ## [Help to GROK parse JBoss EAP 6 access log](https://discuss.elastic.co/t/help-to-grok-parse-jboss-eap-6-access-log/82306)
**Author:** [@bsl.lacerda](https://discuss.elastic.co/u/bsl.lacerda)\ **Replies:** 1\ **Last updated:** [April 13, 2017, 3:06pm UTC](https://discuss.elastic.co/t/help-to-grok-parse-jboss-eap-6-access-log/82306 "2017-04-13T15:06:34Z")
Greetings!!! I'm trying to parse JBoss Access Log Message without success. In the grok debugger, the regex works fine, but in the LOGSTASH 5.2.0 Grok Filter I'm getting the \_grokparsefailure. This is the LOGPATTERN: … --- ## [Logstash jdbc plugin input for mutliple table joins of one to many relationships](https://discuss.elastic.co/t/logstash-jdbc-plugin-input-for-mutliple-table-joins-of-one-to-many-relationships/59933)
**Author:** [@Syed\_Ather](https://discuss.elastic.co/u/Syed_Ather)\ **Replies:** 4\ **Last updated:** [April 13, 2017, 2:42pm UTC](https://discuss.elastic.co/t/logstash-jdbc-plugin-input-for-mutliple-table-joins-of-one-to-many-relationships/59933 "2017-04-13T14:42:28Z")
HI All, I have two tables linked in a one to many relationship , i.e for each row in first table there will be multiple rows in second table . One employee can have many orders assigned to them . ex : how can i a… --- ## [With multi level splitting(3 levels) output flow is just inserting the last split event values in elastic search](https://discuss.elastic.co/t/with-multi-level-splitting-3-levels-output-flow-is-just-inserting-the-last-split-event-values-in-elastic-search/62837)
**Author:** [@sukumar1987](https://discuss.elastic.co/u/sukumar1987)\ **Replies:** 2\ **Last updated:** [April 13, 2017, 2:33pm UTC](https://discuss.elastic.co/t/with-multi-level-splitting-3-levels-output-flow-is-just-inserting-the-last-split-event-values-in-elastic-search/62837 "2017-04-13T14:33:21Z")
we have telemetry output coming to logstash via kafka which we are splitting to three levels to get the required data out of it. Here, I see that filter block output is fine but when the output flow is sending data to el… --- ## [Translate Config works from STDIN but not Beats input](https://discuss.elastic.co/t/translate-config-works-from-stdin-but-not-beats-input/81653)
**Author:** [@dshepard](https://discuss.elastic.co/u/dshepard)\ **Replies:** 2\ **Last updated:** [April 13, 2017, 2:11pm UTC](https://discuss.elastic.co/t/translate-config-works-from-stdin-but-not-beats-input/81653 "2017-04-13T14:11:24Z")
The following code works if I paste the data into STDIN, but when the data comes from BEATS it fails to create the geoip fields. I just get a translation field with the result in it. This is from the logstash config … --- ## [Logstash csv configuration to access nested json field](https://discuss.elastic.co/t/logstash-csv-configuration-to-access-nested-json-field/82307)
**Author:** [@tejamenneni](https://discuss.elastic.co/u/tejamenneni)\ **Replies:** 0\ **Last updated:** [April 13, 2017, 2:03pm UTC](https://discuss.elastic.co/t/logstash-csv-configuration-to-access-nested-json-field/82307 "2017-04-13T14:03:15Z")
Following is my logstash configuration to load es data and convert to csv format input { elasticsearch { hosts =\> "localhost:9200" index =\> "chats" query =\> '{ "query": { "range" : { … --- ## [Dynamic splitting of an event into multiple events](https://discuss.elastic.co/t/dynamic-splitting-of-an-event-into-multiple-events/82302)
**Author:** [@riddhijit\_roy](https://discuss.elastic.co/u/riddhijit_roy)\ **Replies:** 0\ **Last updated:** [April 13, 2017, 1:46pm UTC](https://discuss.elastic.co/t/dynamic-splitting-of-an-event-into-multiple-events/82302 "2017-04-13T13:46:50Z")
I have been trying to parse a complex log pattern for a while . Using multiline pattern matching in filebeat I have been able to group it as per my needs . The sample log format is as follows: { \[Timestamp\] \[ClassA\] \[… --- ## [How to use a dynamic value (from shell?) in input { } configuration?](https://discuss.elastic.co/t/how-to-use-a-dynamic-value-from-shell-in-input-configuration/82208)
**Author:** [@endersonmaia](https://discuss.elastic.co/u/endersonmaia)\ **Replies:** 3\ **Last updated:** [April 13, 2017, 12:41pm UTC](https://discuss.elastic.co/t/how-to-use-a-dynamic-value-from-shell-in-input-configuration/82208 "2017-04-13T12:41:43Z")
I'm testing logstash for the first time, and I'm using it to get data from a database. I've already managed to configure jdbc-input pluging, and I'm using a query that uses a date as a parameter. ... parameters =\> { … --- ## [Logstash-plugin install produces \*thousands\* of ENOENTs](https://discuss.elastic.co/t/logstash-plugin-install-produces-thousands-of-enoents/82277)
**Author:** [@bh9](https://discuss.elastic.co/u/bh9)\ **Replies:** 0\ **Last updated:** [April 13, 2017, 11:02am UTC](https://discuss.elastic.co/t/logstash-plugin-install-produces-thousands-of-enoents/82277 "2017-04-13T11:02:39Z")
Version 5.2.2, OS Centos 7, default config file logstash-plugin install is really slow, so I ran strace -o install-test -ff /usr/share/logstash/bin/logstash-plugin install logstash-filter-elasticsearch logstash-filter-tr… --- ## [Trying to figure out geo\_point mapping](https://discuss.elastic.co/t/trying-to-figure-out-geo-point-mapping/81820)
**Author:** [@mw\_jko](https://discuss.elastic.co/u/mw_jko)\ **Replies:** 5\ **Last updated:** [April 13, 2017, 11:24am UTC](https://discuss.elastic.co/t/trying-to-figure-out-geo-point-mapping/81820 "2017-04-13T11:24:25Z")
Hi there, I'm using the latest ELK stack (official 5.3.0 docker images) and trying to figure out a setup for geoip mapping. Without much success and I don't really know why: logstash.conf input { file { … --- ## [How to improve Logstash performance imrpovement](https://discuss.elastic.co/t/how-to-improve-logstash-performance-imrpovement/80289)
**Author:** [@udaykirankona](https://discuss.elastic.co/u/udaykirankona)\ **Replies:** 8\ **Last updated:** [April 13, 2017, 9:08am UTC](https://discuss.elastic.co/t/how-to-improve-logstash-performance-imrpovement/80289 "2017-04-13T09:08:12Z")
I have 5 node cluster as follows 4 nodes with each 16 GB RAM 4 Cores - ES 1 node with 32 GB RAM 8 cores - Logstash I am able to process 550000 records per one minute, I want improve it to more. So what changes I ca… --- ## [Json array splitting in logstash](https://discuss.elastic.co/t/json-array-splitting-in-logstash/82139)
**Author:** [@Shweta](https://discuss.elastic.co/u/Shweta)\ **Replies:** 10\ **Last updated:** [April 13, 2017, 8:18am UTC](https://discuss.elastic.co/t/json-array-splitting-in-logstash/82139 "2017-04-13T08:18:28Z")
Hi, We have following kind of logs in json files. {"info":{"tstmp":1.2,"from":"avshdd","hostid":"jahgcjha","log":{"version":"jhasgc","id":"jsadh","jobname":"xzcj","lognm":{"class":"jscjks","msg":\[{"users":"kjdfk","c… --- ## [Grok QS includes quotes, how can I remove them?](https://discuss.elastic.co/t/grok-qs-includes-quotes-how-can-i-remove-them/82195)
**Author:** [@mikemi](https://discuss.elastic.co/u/mikemi)\ **Replies:** 2\ **Last updated:** [April 13, 2017, 7:04am UTC](https://discuss.elastic.co/t/grok-qs-includes-quotes-how-can-i-remove-them/82195 "2017-04-13T07:04:31Z")
I'm using the QS grok pattern. It includes the encapsulating quotes in the values. For example, I'm using it to get the referrer in a custom apache log. How can I remove the quotes before shipping over to elasticsearch… --- ## [How do I confirm DNS round robin is working correctly?](https://discuss.elastic.co/t/how-do-i-confirm-dns-round-robin-is-working-correctly/82068)
**Author:** [@red888](https://discuss.elastic.co/u/red888)\ **Replies:** 7\ **Last updated:** [April 13, 2017, 3:38am UTC](https://discuss.elastic.co/t/how-do-i-confirm-dns-round-robin-is-working-correctly/82068 "2017-04-13T03:38:52Z")
I have logstash pointing to a round robin DNS record: elasticsearch { hosts =\> my.cluster.com index =\> "myindex" ....etc How can I confirm logstash is load balancing across all those servers? Or more importantly… --- ## [Logstash-jdbc-input Timezone issue](https://discuss.elastic.co/t/logstash-jdbc-input-timezone-issue/58783)
**Author:** [@slion](https://discuss.elastic.co/u/slion)\ **Replies:** 7\ **Last updated:** [April 12, 2017, 7:10pm UTC](https://discuss.elastic.co/t/logstash-jdbc-input-timezone-issue/58783 "2017-04-12T19:10:26Z")
When I use logstash-jdbc-input plugin to import data from Sql Server, the imported time is utc-based which is not same with my origin data. How to solve this? Second, the sql\_last\_value always logged in UTC timezone ei… --- ## [Grok filter is not working (custom pattern)](https://discuss.elastic.co/t/grok-filter-is-not-working-custom-pattern/82017)
**Author:** [@ibrahimsharaf](https://discuss.elastic.co/u/ibrahimsharaf)\ **Replies:** 5\ **Last updated:** [April 12, 2017, 6:21pm UTC](https://discuss.elastic.co/t/grok-filter-is-not-working-custom-pattern/82017 "2017-04-12T18:21:29Z")
I've a log file (http://codepad.org/vAMFhhR2), and I want to extract a specific number out of it (line 18) I wrote a custom pattern grok filter, tested it on http://grokdebug.herokuapp.com/, it works fine and extracts my… --- ## [Logstash install plugin extremely slow?](https://discuss.elastic.co/t/logstash-install-plugin-extremely-slow/50642)
**Author:** [@xdzhou](https://discuss.elastic.co/u/xdzhou)\ **Replies:** 4\ **Last updated:** [April 12, 2017, 6:07pm UTC](https://discuss.elastic.co/t/logstash-install-plugin-extremely-slow/50642 "2017-04-12T18:07:30Z")
Did anyone experienced the similar issue. When using logstash-plugin to install and uninstall plugins, either online or offline, it becomes extremely slow, mean 10-30 minutes, but somehow it finished successfully. for … --- ## [My multiline input codec is broken](https://discuss.elastic.co/t/my-multiline-input-codec-is-broken/82203)
**Author:** [@ibrahimsharaf](https://discuss.elastic.co/u/ibrahimsharaf)\ **Replies:** 1\ **Last updated:** [April 12, 2017, 5:27pm UTC](https://discuss.elastic.co/t/my-multiline-input-codec-is-broken/82203 "2017-04-12T17:27:12Z")
Hello I am using a multiline input codec to merge all the lines starting with either a whitespace or {' with the previous one, here's a snippet of my log file (http://codepad.org/vAMFhhR2), I want this snippet to be pars… --- ## [Aggregate filter only works sometimes, randomly doesn't work](https://discuss.elastic.co/t/aggregate-filter-only-works-sometimes-randomly-doesnt-work/82201)
**Author:** [@brandono](https://discuss.elastic.co/u/brandono)\ **Replies:** 0\ **Last updated:** [April 12, 2017, 4:59pm UTC](https://discuss.elastic.co/t/aggregate-filter-only-works-sometimes-randomly-doesnt-work/82201 "2017-04-12T16:59:50Z")
Hi, I'm using the aggregate function to sum together groups of lines coming from a CSV. I am using push\_previous\_map\_as\_event =\> true to handle the rows. All rows are grouped by a unique name, and are in the correct f… --- ## [Having issue with Logstash and filebeat](https://discuss.elastic.co/t/having-issue-with-logstash-and-filebeat/82196)
**Author:** [@RAKESH\_JOSHI](https://discuss.elastic.co/u/RAKESH_JOSHI)\ **Replies:** 0\ **Last updated:** [April 12, 2017, 4:18pm UTC](https://discuss.elastic.co/t/having-issue-with-logstash-and-filebeat/82196 "2017-04-12T16:18:34Z")
Hi, I'm using ELK docker images install on Ubuntu 14.04 and i'm using Filebeat to shipping the logs which is installed on windows 10. Now the problem is i'm not able to send the logs to Logstash through filebeat, its … --- ## [Error with basic geo\_point setup in logstash](https://discuss.elastic.co/t/error-with-basic-geo-point-setup-in-logstash/82175)
**Author:** [@jclose](https://discuss.elastic.co/u/jclose)\ **Replies:** 3\ **Last updated:** [April 12, 2017, 4:10pm UTC](https://discuss.elastic.co/t/error-with-basic-geo-point-setup-in-logstash/82175 "2017-04-12T16:10:24Z")
I am trying use a basic geo\_point/geoip setup in logstash. My template for ES has the following field: "source\_geoip" : { "type" : "geo\_point" } Then, in my logstash config, I am trying the following: ge… --- ## [Use ruby 2 for jruby](https://discuss.elastic.co/t/use-ruby-2-for-jruby/82129)
**Author:** [@codenamelxl](https://discuss.elastic.co/u/codenamelxl)\ **Replies:** 2\ **Last updated:** [April 12, 2017, 4:04pm UTC](https://discuss.elastic.co/t/use-ruby-2-for-jruby/82129 "2017-04-12T16:04:40Z")
I'm trying to write a plugin for logstash. My plugin uses a library that require ruby 2+ to run. After searching around, i found that JRuby 1.7 can support Ruby 2.x. How can i force logstash jruby to use ruby 2.0. Thanks… --- ## [Bug logstash](https://discuss.elastic.co/t/bug-logstash/82186)
**Author:** [@zak](https://discuss.elastic.co/u/zak)\ **Replies:** 1\ **Last updated:** [April 12, 2017, 3:46pm UTC](https://discuss.elastic.co/t/bug-logstash/82186 "2017-04-12T15:46:05Z")
hi, when I launch logstash configuration. input { file { path =\> "/elasticsearch/data/elastic\_stack/logstash-2.4.0/projects.csv" start\_position =\> "beginning"}} filter { xml {source =\> "message"}} outpu… --- ## [Parse logfile date into Kibana's timestamp](https://discuss.elastic.co/t/parse-logfile-date-into-kibanas-timestamp/82163)
**Author:** [@ibrahimsharaf](https://discuss.elastic.co/u/ibrahimsharaf)\ **Replies:** 17\ **Last updated:** [April 12, 2017, 3:05pm UTC](https://discuss.elastic.co/t/parse-logfile-date-into-kibanas-timestamp/82163 "2017-04-12T15:05:44Z")
Hello, I have a date in log file like this one 2017-01-01 07:57:22 , I want to extract only the month and the day 01-01, using a logstash filter, then use it as Kibana's timestamp. here's how my logstash filter looks l… --- ## [Http\_poller: dynamic URL parameters](https://discuss.elastic.co/t/http-poller-dynamic-url-parameters/81986)
**Author:** [@volkera](https://discuss.elastic.co/u/volkera)\ **Replies:** 3\ **Last updated:** [April 12, 2017, 3:04pm UTC](https://discuss.elastic.co/t/http-poller-dynamic-url-parameters/81986 "2017-04-12T15:04:26Z")
Is it possible to use a variables instead of hardcoded date in the url? http\_poller { type =\> app\_android urls =\> { app\_android =\> { method =\> get url =\> "Asbury Park Press NJ | Jersey Shore & New Jersey News" he… --- ## [Error while performing resurretion after restarting elasticsearch](https://discuss.elastic.co/t/error-while-performing-resurretion-after-restarting-elasticsearch/63980)
**Author:** [@nick.e](https://discuss.elastic.co/u/nick.e)\ **Replies:** 6\ **Last updated:** [April 12, 2017, 2:47pm UTC](https://discuss.elastic.co/t/error-while-performing-resurretion-after-restarting-elasticsearch/63980 "2017-04-12T14:47:58Z")
Hello together, I installed a single-node-cluster with the 5.0.0rc1 version of the ELK-Stack for some testing. When restarting the elasticsearch-service logstash tries to reconnect: \[ERROR\]\[logstash.outputs.elastics… --- ## [Java process does not stop](https://discuss.elastic.co/t/java-process-does-not-stop/79391)
**Author:** [@laurent1](https://discuss.elastic.co/u/laurent1)\ **Replies:** 0\ **Last updated:** [March 21, 2017, 11:00am UTC](https://discuss.elastic.co/t/java-process-does-not-stop/79391 "2017-03-21T11:00:43Z")
Hello, we create a logstash custom with java process, we make a script for a rebooting of the service in question all the day below the script in question REM ############### REM # Variables # REM ############### se… --- ## [Sending logs to logstash using http](https://discuss.elastic.co/t/sending-logs-to-logstash-using-http/81361)
**Author:** [@Leela\_Kodali](https://discuss.elastic.co/u/Leela_Kodali)\ **Replies:** 9\ **Last updated:** [April 12, 2017, 2:21pm UTC](https://discuss.elastic.co/t/sending-logs-to-logstash-using-http/81361 "2017-04-12T14:21:41Z")
Hi, My logs are exposed on http. , how to push logs to logstash using http, earlier i use file beat it is working fine, but if i use http how is my logstash.json file will be: input { http { host =\> "127.0.0.1" #… --- ## [Loading Logstash pipeline from a central source (perhaps via HTTP)?](https://discuss.elastic.co/t/loading-logstash-pipeline-from-a-central-source-perhaps-via-http/82174)
**Author:** [@rcowart](https://discuss.elastic.co/u/rcowart)\ **Replies:** 1\ **Last updated:** [April 12, 2017, 2:15pm UTC](https://discuss.elastic.co/t/loading-logstash-pipeline-from-a-central-source-perhaps-via-http/82174 "2017-04-12T14:15:41Z")
I am looking for ideas on how to easily spin up multiple Logstash instances from a common pipeline config source. Ideally I would be able to specify a URL (e.g. S3 bucket), instead of a local file or directory, from whic… --- ## [Logstash 5.0 A plugin had an unrecoverable error](https://discuss.elastic.co/t/logstash-5-0-a-plugin-had-an-unrecoverable-error/64403)
**Author:** [@bopa](https://discuss.elastic.co/u/bopa)\ **Replies:** 6\ **Last updated:** [April 12, 2017, 2:05pm UTC](https://discuss.elastic.co/t/logstash-5-0-a-plugin-had-an-unrecoverable-error/64403 "2017-04-12T14:05:52Z")
I have upgraded my ELK stack to 5.0. All seems running well but I'm not receiving any events to my elastic search from logstash. Following error occurs in logstash-plain.log file, \[2016-10-31T10:59:30,638\]\[ERROR\]\[logst… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=999) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1001)