# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1027 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1028 --- ## [XML filter plugin is not working!](https://discuss.elastic.co/t/xml-filter-plugin-is-not-working/76723)
**Author:** [@Ritesh\_Sharma](https://discuss.elastic.co/u/Ritesh_Sharma)\ **Replies:** 3\ **Last updated:** [March 1, 2017, 1:35pm UTC](https://discuss.elastic.co/t/xml-filter-plugin-is-not-working/76723 "2017-03-01T13:35:56Z")
Hi People , I am pulling some stream of XML files from Kafka topic and trying to fetch one of the element of XML file through Logstash using XML filter plugin but unable to do so ; mentioning details below ; XML bl… --- ## [How to fetch syslog date from body in logstash filter](https://discuss.elastic.co/t/how-to-fetch-syslog-date-from-body-in-logstash-filter/76961)
**Author:** [@sundar1](https://discuss.elastic.co/u/sundar1)\ **Replies:** 1\ **Last updated:** [March 1, 2017, 1:14pm UTC](https://discuss.elastic.co/t/how-to-fetch-syslog-date-from-body-in-logstash-filter/76961 "2017-03-01T13:14:55Z")
The below one is my json and fetch the date from the body and convert to logstash timestamp. "Mar 1 04:57:10" date should come as a @timestamp. { "host" =\> "vc2cm.com", "@version" =\> "1", "location… --- ## [Condition on value in array?](https://discuss.elastic.co/t/condition-on-value-in-array/76967)
**Author:** [@asp](https://discuss.elastic.co/u/asp)\ **Replies:** 0\ **Last updated:** [March 1, 2017, 11:00am UTC](https://discuss.elastic.co/t/condition-on-value-in-array/76967 "2017-03-01T11:00:51Z")
Hi, I have following fields in my event: ... "stage": \[ "shared", "QA", "dev\_trunk" \], ... I want to set a new meta field to to staging, if the stage filed contains, QA, dev\_trunk or PreProd. … --- ## [Logstash and geopoint error, several types for the same field](https://discuss.elastic.co/t/logstash-and-geopoint-error-several-types-for-the-same-field/76959)
**Author:** [@DrBatoon](https://discuss.elastic.co/u/DrBatoon)\ **Replies:** 1\ **Last updated:** [March 1, 2017, 10:39am UTC](https://discuss.elastic.co/t/logstash-and-geopoint-error-several-types-for-the-same-field/76959 "2017-03-01T10:39:22Z")
Hi! Im trying to load tweets directly into elasticsearch 5.2.1 via logstash. If I'm just running the logstash file before setting up my mapping, all fields except geoloca with the type geo\_point are fine. Therefore I am … --- ## [Logstash 2.1 UDP Listener Died](https://discuss.elastic.co/t/logstash-2-1-udp-listener-died/76526)
**Author:** [@alexkru](https://discuss.elastic.co/u/alexkru)\ **Replies:** 1\ **Last updated:** [March 1, 2017, 10:18am UTC](https://discuss.elastic.co/t/logstash-2-1-udp-listener-died/76526 "2017-03-01T10:18:33Z")
Hi, We are using a single logstash instance which runs on c4.xlarge instance in AWS. The input plugin is as following: udp { port =\> 7192 workers =\> 4 queue\_size =\> 40000 codec =\> json { charset =\> "UTF-8" } … --- ## [Handling persistent ES output failures](https://discuss.elastic.co/t/handling-persistent-es-output-failures/76387)
**Author:** [@Dan\_Markhasin](https://discuss.elastic.co/u/Dan_Markhasin)\ **Replies:** 8\ **Last updated:** [March 1, 2017, 10:16am UTC](https://discuss.elastic.co/t/handling-persistent-es-output-failures/76387 "2017-03-01T10:16:24Z")
Hi, I am trying to understand if there is a method in logstash to handle failures that occur when writing to ElasticSearch. I know that logstash has a retry policy for certain types of exceptions, but the failures I am… --- ## [PR, add support for redis ZSET input and output](https://discuss.elastic.co/t/pr-add-support-for-redis-zset-input-and-output/76958)
**Author:** [@amaury-d](https://discuss.elastic.co/u/amaury-d)\ **Replies:** 0\ **Last updated:** [March 1, 2017, 9:48am UTC](https://discuss.elastic.co/t/pr-add-support-for-redis-zset-input-and-output/76958 "2017-03-01T09:48:16Z")
Hello to all, Just wanted to ping the forum about two pull requests i've made a few weeks ago to add support for redis zset to logstash aka priority queues. Is everything all right ? Is there anything else I can … --- ## [How to manually insert NONE value to elasticsearch index field using Logstash?](https://discuss.elastic.co/t/how-to-manually-insert-none-value-to-elasticsearch-index-field-using-logstash/76954)
**Author:** [@elasticheart](https://discuss.elastic.co/u/elasticheart)\ **Replies:** 0\ **Last updated:** [March 1, 2017, 9:23am UTC](https://discuss.elastic.co/t/how-to-manually-insert-none-value-to-elasticsearch-index-field-using-logstash/76954 "2017-03-01T09:23:47Z")
Hi, I have logfiles which contain numeric data, and I have manually created the map accordingly, specifying the number fields. But the problem is that in case if there is no data for field, log files will be generated … --- ## [Logstash upload csv not working on the cloud](https://discuss.elastic.co/t/logstash-upload-csv-not-working-on-the-cloud/76943)
**Author:** [@davidker](https://discuss.elastic.co/u/davidker)\ **Replies:** 2\ **Last updated:** [March 1, 2017, 8:53am UTC](https://discuss.elastic.co/t/logstash-upload-csv-not-working-on-the-cloud/76943 "2017-03-01T08:53:50Z")
I have a trial set up and wanted to inport a CSV file I have a local copy of logstash running and want it to load data into the hosted elasticsearch cluster. I got the following error : Attempted to resurrect connectio… --- ## [Run service with multiple configuration files](https://discuss.elastic.co/t/run-service-with-multiple-configuration-files/76895)
**Author:** [@ggp](https://discuss.elastic.co/u/ggp)\ **Replies:** 1\ **Last updated:** [March 1, 2017, 7:13am UTC](https://discuss.elastic.co/t/run-service-with-multiple-configuration-files/76895 "2017-03-01T07:13:12Z")
how to run logstash service on debian or rpm if we have multiple configuration files ? --- ## [Need some help for parsing Log](https://discuss.elastic.co/t/need-some-help-for-parsing-log/76836)
**Author:** [@wawa](https://discuss.elastic.co/u/wawa)\ **Replies:** 1\ **Last updated:** [March 1, 2017, 6:57am UTC](https://discuss.elastic.co/t/need-some-help-for-parsing-log/76836 "2017-03-01T06:57:50Z")
Hi logstash community, I'm trying to get some sort of grok pattern to work with the following logging format : 2017-02-21 23:45:41.7734 | MyLogger | INFO | PathInfo \[/json/api/GetData\] Duration \[821,1381 ms\] IpAddress… --- ## [Logstash 5 ruby filter](https://discuss.elastic.co/t/logstash-5-ruby-filter/76851)
**Author:** [@pjanzen](https://discuss.elastic.co/u/pjanzen)\ **Replies:** 3\ **Last updated:** [March 1, 2017, 6:55am UTC](https://discuss.elastic.co/t/logstash-5-ruby-filter/76851 "2017-03-01T06:55:20Z")
Hi All, I have the following code in my logstash config and the ruby part causes logstash to continuously restart, if I remove the ruby part it works just fine. There are no errors in the log what so ever, shall I repor… --- ## [Can I change total\_fields.limit from logstash's output?](https://discuss.elastic.co/t/can-i-change-total-fields-limit-from-logstashs-output/76904)
**Author:** [@camkidman](https://discuss.elastic.co/u/camkidman)\ **Replies:** 0\ **Last updated:** [March 1, 2017, 3:14am UTC](https://discuss.elastic.co/t/can-i-change-total-fields-limit-from-logstashs-output/76904 "2017-03-01T03:14:46Z")
Hi there, I've got the following in my elasticsearch configuration file: output { elasticsearch { hosts =\> \["https://...."\] index =\> "production-%{+YYYY.MM.dd}" } stdout { codec =\> rubydebug } } I'm runni… --- ## [Jdbc add field](https://discuss.elastic.co/t/jdbc-add-field/76899)
**Author:** [@Greentea](https://discuss.elastic.co/u/Greentea)\ **Replies:** 1\ **Last updated:** [March 1, 2017, 2:52am UTC](https://discuss.elastic.co/t/jdbc-add-field/76899 "2017-03-01T02:52:19Z")
Hi! I am trying to add field in to a type by using if statement. But logstash report error \[LogStash::Runner\] ERROR logstash.agent - fetched an invalid config Here is my filter part filter { if \[type\]=product{ mutat… --- ## [Filter to drop all packets that are null not working](https://discuss.elastic.co/t/filter-to-drop-all-packets-that-are-null-not-working/76838)
**Author:** [@Ben\_Hoffman](https://discuss.elastic.co/u/Ben_Hoffman)\ **Replies:** 3\ **Last updated:** [March 1, 2017, 12:08am UTC](https://discuss.elastic.co/t/filter-to-drop-all-packets-that-are-null-not-working/76838 "2017-03-01T00:08:45Z")
So I wrote a filter to drop any event that has a certain field with a value of null: filter { if\[type\] == "flow" and \[packet\_source\]\[ip\] == "" { drop { } } } However, this does not work. Does anyo… --- ## [Unable to create index after x-pack](https://discuss.elastic.co/t/unable-to-create-index-after-x-pack/76559)
**Author:** [@ankitc7](https://discuss.elastic.co/u/ankitc7)\ **Replies:** 5\ **Last updated:** [February 28, 2017, 11:09pm UTC](https://discuss.elastic.co/t/unable-to-create-index-after-x-pack/76559 "2017-02-28T23:09:30Z")
Hello, Filebeat, Logstash, Elasticsearch were working sucessfully until I installed x-pack. Now when I try to index data, I get this: \[2017-02-26T16:10:45,438\]\[WARN \]\[logstash.outputs.elasticsearch\] Failed action. {:sta… --- ## [How to setup logstash-input-jms plugin?](https://discuss.elastic.co/t/how-to-setup-logstash-input-jms-plugin/76884)
**Author:** [@Ranieri\_Mazili\_de\_Ol](https://discuss.elastic.co/u/Ranieri_Mazili_de_Ol)\ **Replies:** 0\ **Last updated:** [February 28, 2017, 10:56pm UTC](https://discuss.elastic.co/t/how-to-setup-logstash-input-jms-plugin/76884 "2017-02-28T22:56:57Z")
Hello, I just installed Logstash and then installed logstash-input-jms plugin with the following command # logstash-plugin install logstash-input-jms Validating logstash-input-jms Installing logstash-input-jms Installa… --- ## [Broken download links for current and past releases](https://discuss.elastic.co/t/broken-download-links-for-current-and-past-releases/76870)
**Author:** [@anhlqn](https://discuss.elastic.co/u/anhlqn)\ **Replies:** 1\ **Last updated:** [February 28, 2017, 8:37pm UTC](https://discuss.elastic.co/t/broken-download-links-for-current-and-past-releases/76870 "2017-02-28T20:37:46Z")
Links to dowload both ES and LS are brokens. I'm not sure if it is just me or the download site https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.2.2.zip https://download.elastic.co/elasticsearch/r… --- ## [Sporadic failures with X-Pack/Shield](https://discuss.elastic.co/t/sporadic-failures-with-x-pack-shield/76259)
**Author:** [@Greek0](https://discuss.elastic.co/u/Greek0)\ **Replies:** 6\ **Last updated:** [February 28, 2017, 7:51pm UTC](https://discuss.elastic.co/t/sporadic-failures-with-x-pack-shield/76259 "2017-02-28T19:51:53Z")
We started using xpack/shield with ElasticSearch 5.2. Unfortunately, we get random auth errors and can't even get logstash running because of it (it was running fine before). When requesting http://kibana:9200/\_templat… --- ## [Error expecting # line](https://discuss.elastic.co/t/error-expecting-line/76708)
**Author:** [@zbay](https://discuss.elastic.co/u/zbay)\ **Replies:** 10\ **Last updated:** [February 28, 2017, 7:46pm UTC](https://discuss.elastic.co/t/error-expecting-line/76708 "2017-02-28T19:46:38Z")
Good evening everyone, I've been trying to track down some errors and get my first job running and collecting data from twitter, I found some examples on github and slightly modified them to fit my needs. At this poin… --- ## [Logstash requires setting a file](https://discuss.elastic.co/t/logstash-requires-setting-a-file/64658)
**Author:** [@w0lverine](https://discuss.elastic.co/u/w0lverine)\ **Replies:** 10\ **Last updated:** [February 28, 2017, 6:28pm UTC](https://discuss.elastic.co/t/logstash-requires-setting-a-file/64658 "2017-02-28T18:28:39Z")
Logstash Info error: A setting file is missing I installed the deb file of logstash 5.0 and I starting logstash for the first time. I get an INFO error: INFO: Logstash requires a setting file which is typically locate… --- ## [Logstash CVS filter is too slow](https://discuss.elastic.co/t/logstash-cvs-filter-is-too-slow/76671)
**Author:** [@lyres](https://discuss.elastic.co/u/lyres)\ **Replies:** 2\ **Last updated:** [February 28, 2017, 6:20pm UTC](https://discuss.elastic.co/t/logstash-cvs-filter-is-too-slow/76671 "2017-02-28T18:20:28Z")
Hi, I'm trying to parse my logs by CVS filter, but this plugin is too slow. My restriction is to use only 1 worker, because we cannot use more than 1 CPU for parsing. With 1 worker CVS filter parses about 600 lines per … --- ## [Logstash forwarding AIX Help](https://discuss.elastic.co/t/logstash-forwarding-aix-help/76835)
**Author:** [@stevenmarrocco](https://discuss.elastic.co/u/stevenmarrocco)\ **Replies:** 0\ **Last updated:** [February 28, 2017, 5:10pm UTC](https://discuss.elastic.co/t/logstash-forwarding-aix-help/76835 "2017-02-28T17:10:54Z")
Hello, I'm currently attempting to setup logstash-forwarder-java and am not having much luck. I've gone through countless documentation and guides online but they seem to be dated. I do know that AIX does not suppo… --- ## [Help with \_jsonparsefailure?](https://discuss.elastic.co/t/help-with--jsonparsefailure/76446)
**Author:** [@ZillaG](https://discuss.elastic.co/u/ZillaG)\ **Replies:** 4\ **Last updated:** [February 28, 2017, 3:37pm UTC](https://discuss.elastic.co/t/help-with--jsonparsefailure/76446 "2017-02-28T15:37:29Z")
I have a multi-line java exception log that I'm correctly sending via rsyslog from a remote server to my ELK stack, like the one below (sorry for the verbosity). It looks, to me, like a valid json file. { "type":"pe-lo… --- ## [How to make something like an IP address be stored in an array of integers instead of a string](https://discuss.elastic.co/t/how-to-make-something-like-an-ip-address-be-stored-in-an-array-of-integers-instead-of-a-string/76444)
**Author:** [@Ben\_Hoffman](https://discuss.elastic.co/u/Ben_Hoffman)\ **Replies:** 2\ **Last updated:** [February 28, 2017, 3:32pm UTC](https://discuss.elastic.co/t/how-to-make-something-like-an-ip-address-be-stored-in-an-array-of-integers-instead-of-a-string/76444 "2017-02-28T15:32:51Z")
So I want to basically do a string split at the " . " in one of my outputs from a log file (It is an IP address). Is there any way to do this with a Logstash filter? It would make the requests from my application to my … --- ## [Only read new lines](https://discuss.elastic.co/t/only-read-new-lines/76804)
**Author:** [@bertvervaele](https://discuss.elastic.co/u/bertvervaele)\ **Replies:** 2\ **Last updated:** [February 28, 2017, 3:11pm UTC](https://discuss.elastic.co/t/only-read-new-lines/76804 "2017-02-28T15:11:19Z")
Hello I just started with elastic stack so maybe it's a stupid question. I want to read log files live. So after every chance I only want to read the new lines not the whole file again. Is this possible? Greetings Bert … --- ## [Logstash Output File Rotate](https://discuss.elastic.co/t/logstash-output-file-rotate/76782)
**Author:** [@sdeshpande](https://discuss.elastic.co/u/sdeshpande)\ **Replies:** 9\ **Last updated:** [February 28, 2017, 3:11pm UTC](https://discuss.elastic.co/t/logstash-output-file-rotate/76782 "2017-02-28T15:11:09Z")
Hi Guys, Can someone please tell me if it is possible to rotate the Output File generated by Logstash output plugin ? In Logstash config I have following lines and was wondering if I can specify option to configure rot… --- ## [Multi pattern grok failing to load](https://discuss.elastic.co/t/multi-pattern-grok-failing-to-load/76422)
**Author:** [@mhooper](https://discuss.elastic.co/u/mhooper)\ **Replies:** 4\ **Last updated:** [February 28, 2017, 3:08pm UTC](https://discuss.elastic.co/t/multi-pattern-grok-failing-to-load/76422 "2017-02-28T15:08:50Z")
After I added the following grok pattern, my config fails to load. grok { match =\> { "tempMessage" =\> "WebAPI - Request Uri: %{DATA:action} http://webservice.bigcompany.com/%{WORD:endpoint}/%{INT:number}/%{WORD:cancel… --- ## [Trouble with Greedydata being a bit too greedy](https://discuss.elastic.co/t/trouble-with-greedydata-being-a-bit-too-greedy/76799)
**Author:** [@Mark\_Dolan](https://discuss.elastic.co/u/Mark_Dolan)\ **Replies:** 0\ **Last updated:** [February 28, 2017, 2:48pm UTC](https://discuss.elastic.co/t/trouble-with-greedydata-being-a-bit-too-greedy/76799 "2017-02-28T14:48:25Z")
I have some log lines that sometimes have a description appended, in the "Details: bla bla bla" string. The first and third lines have it here: Naam: St Nationaal Epilepsie Fon ds/ de Macht Vh Kleine Machtig… --- ## [Best Practice for different log files with multi line Grok patterns](https://discuss.elastic.co/t/best-practice-for-different-log-files-with-multi-line-grok-patterns/76623)
**Author:** [@Mohsen\_Sabbaghi](https://discuss.elastic.co/u/Mohsen_Sabbaghi)\ **Replies:** 1\ **Last updated:** [February 28, 2017, 2:29pm UTC](https://discuss.elastic.co/t/best-practice-for-different-log-files-with-multi-line-grok-patterns/76623 "2017-02-28T14:29:38Z")
Hi and Respect to all the ELK Gigs ! scenario : i have several log files (my app logs, application server logs and...) that each of them contains several different of log patterns in their lines. i'm new in ES and i ju… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1026) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1028)