# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1004 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1005 --- ## [eWAS grok and multiline parsing](https://discuss.elastic.co/t/ewas-grok-and-multiline-parsing/81540)
**Author:** [@sam281](https://discuss.elastic.co/u/sam281)\ **Replies:** 1\ **Last updated:** [April 7, 2017, 5:26am UTC](https://discuss.elastic.co/t/ewas-grok-and-multiline-parsing/81540 "2017-04-07T05:26:57Z")
Can somebody help me with multi line parsing of an eWAS (embedded Websphere Application Server) application log messages which looks something like below. And the log has single line events as well when its not java stac… --- ## [Applying ruby code to several fields in an event?](https://discuss.elastic.co/t/applying-ruby-code-to-several-fields-in-an-event/81541)
**Author:** [@Grigory\_Shamov](https://discuss.elastic.co/u/Grigory_Shamov)\ **Replies:** 1\ **Last updated:** [April 7, 2017, 5:04am UTC](https://discuss.elastic.co/t/applying-ruby-code-to-several-fields-in-an-event/81541 "2017-04-07T05:04:38Z")
Hi, What would be the best way to applyc same ruby code to several fields? To grok something, then do (the same) math on the fields from the grok. Or to convert date/time to/from epoch. It is possible just to repeat th… --- ## [Iteration in Logstash](https://discuss.elastic.co/t/iteration-in-logstash/26123)
**Author:** [@jasonct05](https://discuss.elastic.co/u/jasonct05)\ **Replies:** 4\ **Last updated:** [April 7, 2017, 5:02am UTC](https://discuss.elastic.co/t/iteration-in-logstash/26123 "2017-04-07T05:02:07Z")
I have a string data in Logstash that has been parsed as that looks like "id": "0,3,10" "value": "100,200,300" I would like to match the individual "id" to the corresponding "value" with respect to their order in the … --- ## [Duplicated data into field from xpath](https://discuss.elastic.co/t/duplicated-data-into-field-from-xpath/81537)
**Author:** [@daian.martinho](https://discuss.elastic.co/u/daian.martinho)\ **Replies:** 0\ **Last updated:** [April 6, 2017, 9:04pm UTC](https://discuss.elastic.co/t/duplicated-data-into-field-from-xpath/81537 "2017-04-06T21:04:01Z")
i'm getting duplicated data at only one field. All others are ok. xml is like: \ **Author:** [@ZillaG](https://discuss.elastic.co/u/ZillaG)\ **Replies:** 0\ **Last updated:** [April 6, 2017, 7:42pm UTC](https://discuss.elastic.co/t/which-elasticsearch-node-should-logstash-use-in-output-filter/81528 "2017-04-06T19:42:51Z") I have an ELK infrastructure that has - 1 Kibana/Logstash node - 1 Elasticsearch client node - 3 Elasticsearch master nodes - 2 Elasticsearch data nodes On my Logstash's output filter, which node should I point my Logs… --- ## [LogStash Startup error](https://discuss.elastic.co/t/logstash-startup-error/81196)
**Author:** [@anshul\_pandey](https://discuss.elastic.co/u/anshul_pandey)\ **Replies:** 2\ **Last updated:** [April 6, 2017, 6:57pm UTC](https://discuss.elastic.co/t/logstash-startup-error/81196 "2017-04-06T18:57:06Z")
hi all, during the logstash startup i am getting following message. I don't think i have installed monitoring tools, for either elastic or logstash... Not sure, what's causing this error and how to fix this. {:c… --- ## [Rabbitmq input with cipher](https://discuss.elastic.co/t/rabbitmq-input-with-cipher/80733)
**Author:** [@piyush](https://discuss.elastic.co/u/piyush)\ **Replies:** 2\ **Last updated:** [April 6, 2017, 6:44pm UTC](https://discuss.elastic.co/t/rabbitmq-input-with-cipher/80733 "2017-04-06T18:44:58Z")
Hi Team, We setup Rabbitmq and configured logstash to read messages from queue. The configurations was working fine and now we updated Rabbitmq to accept only TLSv1.2,TLSv1.1 with specific ciphers. After this logstash ca… --- ## [How to reference fields outside an XML event?](https://discuss.elastic.co/t/how-to-reference-fields-outside-an-xml-event/81522)
**Author:** [@chg](https://discuss.elastic.co/u/chg)\ **Replies:** 0\ **Last updated:** [April 6, 2017, 6:01pm UTC](https://discuss.elastic.co/t/how-to-reference-fields-outside-an-xml-event/81522 "2017-04-06T18:01:45Z")
Example XML: \ \ \ \ \1\ \ \ www.google.com \ \ www.elastic.co \ \ ... \ \ … --- ## [How to pattern input codec multiline](https://discuss.elastic.co/t/how-to-pattern-input-codec-multiline/81520)
**Author:** [@kongupradeep](https://discuss.elastic.co/u/kongupradeep)\ **Replies:** 0\ **Last updated:** [April 6, 2017, 5:44pm UTC](https://discuss.elastic.co/t/how-to-pattern-input-codec-multiline/81520 "2017-04-06T17:44:48Z")
please help me that my xml contain \ \ Empire Burlesque Bob Dylan USA Columbia 10.90 1985 … --- ## [Jms\_reply\_to field and logstash-input-jms plugin - Value not showing?](https://discuss.elastic.co/t/jms-reply-to-field-and-logstash-input-jms-plugin-value-not-showing/81515)
**Author:** [@jeff-4ed6](https://discuss.elastic.co/u/jeff-4ed6)\ **Replies:** 0\ **Last updated:** [April 6, 2017, 5:01pm UTC](https://discuss.elastic.co/t/jms-reply-to-field-and-logstash-input-jms-plugin-value-not-showing/81515 "2017-04-06T17:01:18Z")
Hello, everyone. I have installed logstash-input-jms 3.0.1 plugin to Logstash 5.3.0, configured it, and it is consuming messages off of a Qpid queue using the Qpid Client 6.1.2 libraries. I have Logstash output set to … --- ## [Support hebrew characters in logs](https://discuss.elastic.co/t/support-hebrew-characters-in-logs/81168)
**Author:** [@orlevy08](https://discuss.elastic.co/u/orlevy08)\ **Replies:** 4\ **Last updated:** [April 6, 2017, 3:29pm UTC](https://discuss.elastic.co/t/support-hebrew-characters-in-logs/81168 "2017-04-06T15:29:58Z")
Hi All, I have an issue with logstash parsing using grok: When some hebrew characters are placed in the log the result i see is ?????? instead of the user name (for example: if the user name is בלהבלה i get ??????) Is … --- ## [Logstash doesn't read file from beginning](https://discuss.elastic.co/t/logstash-doesnt-read-file-from-beginning/81487)
**Author:** [@luciferdude](https://discuss.elastic.co/u/luciferdude)\ **Replies:** 1\ **Last updated:** [April 6, 2017, 2:48pm UTC](https://discuss.elastic.co/t/logstash-doesnt-read-file-from-beginning/81487 "2017-04-06T14:48:16Z")
I'm installed logstash on one of our linux server and i'm trying to forward application log to ES and visualize in Kibana. Any New Line being written to the log are being visualized but the file from beginning is not … --- ## [Generic advice to a newb. Best practises](https://discuss.elastic.co/t/generic-advice-to-a-newb-best-practises/81469)
**Author:** [@bhu73](https://discuss.elastic.co/u/bhu73)\ **Replies:** 3\ **Last updated:** [April 6, 2017, 1:26pm UTC](https://discuss.elastic.co/t/generic-advice-to-a-newb-best-practises/81469 "2017-04-06T13:26:39Z")
I have just installed ELK, got my first data into to it, done a bit of grokking and "kv"..ing. I am however an absolute noob and the problem is more complex than can be solved with this. So what I am looking for is gen… --- ## [Logstash with elasticsearch input loops results and always gives the error " Failed to parse request body "](https://discuss.elastic.co/t/logstash-with-elasticsearch-input-loops-results-and-always-gives-the-error-failed-to-parse-request-body/80947)
**Author:** [@hacker\_21](https://discuss.elastic.co/u/hacker_21)\ **Replies:** 1\ **Last updated:** [April 6, 2017, 1:15pm UTC](https://discuss.elastic.co/t/logstash-with-elasticsearch-input-loops-results-and-always-gives-the-error-failed-to-parse-request-body/80947 "2017-04-06T13:15:38Z")
I'm getting this error no matter how I tweak my very basic logstash configuration: Error: \[400\] {"error":{"root\_cause":\[{"type":"illegal\_argument\_exception","reason":"Failed to parse request body"}\],"type":"illegal\_argum… --- ## [Multiline codec and xml](https://discuss.elastic.co/t/multiline-codec-and-xml/81446)
**Author:** [@flalar](https://discuss.elastic.co/u/flalar)\ **Replies:** 0\ **Last updated:** [April 6, 2017, 10:39am UTC](https://discuss.elastic.co/t/multiline-codec-and-xml/81446 "2017-04-06T10:39:49Z")
We're experienceing difficulties pulling events from xml input The xml input looks like this \ \ **Author:** [@Raja1](https://discuss.elastic.co/u/Raja1)\ **Replies:** 1\ **Last updated:** [April 6, 2017, 8:25am UTC](https://discuss.elastic.co/t/kafka-not-reading-the-logstash-logs/81419 "2017-04-06T08:25:17Z") Hi, I have done the below configuration. I am able start the logstash but kafka not reading the logs. Please help me to resolve this issue. input { file{ path=\> \["/etc/logstash/\*.log"\] } } output{ kafka { b… --- ## [Elasticsearch Host unreachable](https://discuss.elastic.co/t/elasticsearch-host-unreachable/81261)
**Author:** [@Huda\_Aqilah](https://discuss.elastic.co/u/Huda_Aqilah)\ **Replies:** 3\ **Last updated:** [April 6, 2017, 6:17am UTC](https://discuss.elastic.co/t/elasticsearch-host-unreachable/81261 "2017-04-06T06:17:15Z")
I tried to migrate data from MySQL into Elasticsearch using logstash. unfortunately , i got an error that my elasticsearch is unreachable. this is the error . Marking url as de ad.Last error: \[LogStash::Outputs::E… --- ## ["interval" should not be required in logstash-input-exec plugin](https://discuss.elastic.co/t/interval-should-not-be-required-in-logstash-input-exec-plugin/81321)
**Author:** [@shin0higuchi](https://discuss.elastic.co/u/shin0higuchi)\ **Replies:** 1\ **Last updated:** [April 6, 2017, 5:30am UTC](https://discuss.elastic.co/t/interval-should-not-be-required-in-logstash-input-exec-plugin/81321 "2017-04-06T05:30:20Z")
Logstash Version : 5.3.0 "interval" setting is now required in logstash-input-exec plugin. When I want to execute some command only once, I have to set interval. Interval should be optional and the default should be… --- ## [Logstash performance issue with filter and influxdb](https://discuss.elastic.co/t/logstash-performance-issue-with-filter-and-influxdb/81408)
**Author:** [@k1ng87](https://discuss.elastic.co/u/k1ng87)\ **Replies:** 0\ **Last updated:** [April 6, 2017, 5:27am UTC](https://discuss.elastic.co/t/logstash-performance-issue-with-filter-and-influxdb/81408 "2017-04-06T05:27:25Z")
I currenlty have a logstash config that is written as such and I've noticed something interesting. The throughput of this filter is about 400 messages/sec which is awfully slow. input { kafka { … --- ## [How to filter/remove fields that have certain naming pattern](https://discuss.elastic.co/t/how-to-filter-remove-fields-that-have-certain-naming-pattern/80562)
**Author:** [@dixitnikhil2004](https://discuss.elastic.co/u/dixitnikhil2004)\ **Replies:** 3\ **Last updated:** [April 6, 2017, 5:15am UTC](https://discuss.elastic.co/t/how-to-filter-remove-fields-that-have-certain-naming-pattern/80562 "2017-04-06T05:15:09Z")
with JSON input, i have field like Name:Value build.artifacts.hyper-mingw64-x86\_64-debug-master.0.0.1677.rd3e0b27a.tgz.archive: "SOME VALUE1" build.artifacts.hyper-mingw64-x86\_64-debug-master.0.0.1677.rd3e0b27a.zip.archi… --- ## [Logstash output to RabbitMQ - Connection Close](https://discuss.elastic.co/t/logstash-output-to-rabbitmq-connection-close/81202)
**Author:** [@brickpattern](https://discuss.elastic.co/u/brickpattern)\ **Replies:** 2\ **Last updated:** [April 6, 2017, 5:03am UTC](https://discuss.elastic.co/t/logstash-output-to-rabbitmq-connection-close/81202 "2017-04-06T05:03:21Z")
Logstash version 5.0.0 attempt to send information to Rabbit MQ in output results in below error \[ERROR\]\[logstash.agent \] Pipeline aborted due to error {:exception=\>java.util.concurrent.TimeoutException, … --- ## [Finding most accessed URI-paths / Transactions using Logstash](https://discuss.elastic.co/t/finding-most-accessed-uri-paths-transactions-using-logstash/81263)
**Author:** [@anisen](https://discuss.elastic.co/u/anisen)\ **Replies:** 1\ **Last updated:** [April 6, 2017, 4:36am UTC](https://discuss.elastic.co/t/finding-most-accessed-uri-paths-transactions-using-logstash/81263 "2017-04-06T04:36:38Z")
Hi, My log file looks like this: 2015-06-12:00:08:54 100.220.144.1 GET /site/path1.aspx 200 2015-06-12:00:08:55 100.220.144.1 GET /site/path2.aspx 200 2015-06-12:00:08:56 100.220.144.1 GET /site/path3.aspx 200 2015… --- ## [error=\>#\](https://discuss.elastic.co/t/error-argumenterror-setting-hasnt-been-registered/78666)
**Author:** [@Ragnar](https://discuss.elastic.co/u/Ragnar)\ **Replies:** 15\ **Last updated:** [April 6, 2017, 1:46am UTC](https://discuss.elastic.co/t/error-argumenterror-setting-hasnt-been-registered/78666 "2017-04-06T01:46:24Z")
Hi there, I want to send a +8Go csv file to my ES server form my machine. I use Logstash to send the file with this conf : input { file { path =\> "/Users/karnag/Downloads/siren201703.csv" start\_position … --- ## [Logstash input speed slow, around 8,000events / sec writing to local file](https://discuss.elastic.co/t/logstash-input-speed-slow-around-8-000events-sec-writing-to-local-file/2591)
**Author:** [@nathaniel](https://discuss.elastic.co/u/nathaniel)\ **Replies:** 9\ **Last updated:** [April 5, 2017, 11:22pm UTC](https://discuss.elastic.co/t/logstash-input-speed-slow-around-8-000events-sec-writing-to-local-file/2591 "2017-04-05T23:22:36Z")
Hello guys, thanks for noticing this question. I am trying to build up a centralized logging system using ELK framework, but the speed is very slow. I checked and the problem is on logstash. I used a generator to generat… --- ## [Update document on multiple indices](https://discuss.elastic.co/t/update-document-on-multiple-indices/81316)
**Author:** [@shaharmor](https://discuss.elastic.co/u/shaharmor)\ **Replies:** 1\ **Last updated:** [April 5, 2017, 11:24pm UTC](https://discuss.elastic.co/t/update-document-on-multiple-indices/81316 "2017-04-05T23:24:58Z")
Hi, I'm tracking user sessions as docs in ES. Each session is a single doc that gets updated when ever the session state changes. I'm using daily based indices, and the ID of the doc is the session ID (UUID). If a se… --- ## [Run Logstash with local ruby2.4](https://discuss.elastic.co/t/run-logstash-with-local-ruby2-4/81380)
**Author:** [@shabirmean](https://discuss.elastic.co/u/shabirmean)\ **Replies:** 2\ **Last updated:** [April 5, 2017, 11:03pm UTC](https://discuss.elastic.co/t/run-logstash-with-local-ruby2-4/81380 "2017-04-05T23:03:28Z")
Hi, I am a newbie in-terms of Ruby. I am trying to extend the ruby-filewatch library used by the logstash-input-file plugin. For the extension I'm using the external lib - rest-client. This external lib requires … --- ## [Read log file that is feed from top not from bottom](https://discuss.elastic.co/t/read-log-file-that-is-feed-from-top-not-from-bottom/81362)
**Author:** [@exorcista](https://discuss.elastic.co/u/exorcista)\ **Replies:** 1\ **Last updated:** [April 5, 2017, 10:46pm UTC](https://discuss.elastic.co/t/read-log-file-that-is-feed-from-top-not-from-bottom/81362 "2017-04-05T22:46:38Z")
Hi folks. I have a log file that is feed from top (new entries are added from top), Is there a way to let know logstash read this file from top? Thanks in advance. --- ## [Enriching logs with data from Elasticsearch (logstash-filter-elasticsearch)](https://discuss.elastic.co/t/enriching-logs-with-data-from-elasticsearch-logstash-filter-elasticsearch/81325)
**Author:** [@matejzero](https://discuss.elastic.co/u/matejzero)\ **Replies:** 3\ **Last updated:** [April 5, 2017, 8:07pm UTC](https://discuss.elastic.co/t/enriching-logs-with-data-from-elasticsearch-logstash-filter-elasticsearch/81325 "2017-04-05T20:07:49Z")
Hello, we are trying to enrich out firewall/router logs with custom geoip data (such as coordinates, client info, internal client ID,...), which we have saved in a dedicated index on our ES cluster. I'm currently tr… --- ## [Can't use type in logstash conf file](https://discuss.elastic.co/t/cant-use-type-in-logstash-conf-file/81282)
**Author:** [@vikrams](https://discuss.elastic.co/u/vikrams)\ **Replies:** 2\ **Last updated:** [April 5, 2017, 6:27pm UTC](https://discuss.elastic.co/t/cant-use-type-in-logstash-conf-file/81282 "2017-04-05T18:27:50Z")
I am trying to use "type" for conditional filtering from two different inputs , Below is a simplified configuration I dont get any data in my index using this file. But if i simply remove "type" and conditions using "typ… --- ## [Splitting greedydata](https://discuss.elastic.co/t/splitting-greedydata/78408)
**Author:** [@Alex\_Mendez](https://discuss.elastic.co/u/Alex_Mendez)\ **Replies:** 6\ **Last updated:** [April 5, 2017, 6:18pm UTC](https://discuss.elastic.co/t/splitting-greedydata/78408 "2017-04-05T18:18:28Z")
Greetings, I'd like to know if its possible to split the message in GREEDYDATA further into searchable data. Example line in log stash grok { match =\> \[ "message", "%{SYSLOGTIMESTAMP:syslog\_timestamp} %{HO… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1003) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1005)