# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=798 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 799 --- ## [Grok not working with PING multiline](https://discuss.elastic.co/t/grok-not-working-with-ping-multiline/124706)
**Author:** [@chuski92](https://discuss.elastic.co/u/chuski92)\ **Replies:** 3\ **Last updated:** [March 20, 2018, 11:43am UTC](https://discuss.elastic.co/t/grok-not-working-with-ping-multiline/124706 "2018-03-20T11:43:07Z")
Hello. I'm having troubles with parsing the response of making PING to our servers, my failure is with the PING error packets. With a good PING its ok, but parsing the next line doesn't work: --- 192.168.1.1 ping stat… --- ## [Logstash does not auto recover after disconnection with destination](https://discuss.elastic.co/t/logstash-does-not-auto-recover-after-disconnection-with-destination/124742)
**Author:** [@ashurawat](https://discuss.elastic.co/u/ashurawat)\ **Replies:** 0\ **Last updated:** [March 20, 2018, 11:22am UTC](https://discuss.elastic.co/t/logstash-does-not-auto-recover-after-disconnection-with-destination/124742 "2018-03-20T11:22:32Z")
Hello dear, I am very new with Logstash. and facing this issue for so long. Tried many things but not able to resolve it. Please find below cases which I am using right now. Filebeat is running on VM1 and Logstash is… --- ## [Sending filebeat to logstash loses system fields](https://discuss.elastic.co/t/sending-filebeat-to-logstash-loses-system-fields/124736)
**Author:** [@KevSex](https://discuss.elastic.co/u/KevSex)\ **Replies:** 0\ **Last updated:** [March 20, 2018, 11:07am UTC](https://discuss.elastic.co/t/sending-filebeat-to-logstash-loses-system-fields/124736 "2018-03-20T11:07:16Z")
Hi all, Have been setting up ELK stack and hitting some issues that's been bugging me for a while. When I set up FileBeat (RHEL server) to output to ES, everthing seems to work correctly in that I'm able to filter by s… --- ## [\[simple question\] import JSON into elasticsearch](https://discuss.elastic.co/t/simple-question-import-json-into-elasticsearch/122123)
**Author:** [@Kevin\_Csuka](https://discuss.elastic.co/u/Kevin_Csuka)\ **Replies:** 13\ **Last updated:** [March 20, 2018, 10:24am UTC](https://discuss.elastic.co/t/simple-question-import-json-into-elasticsearch/122123 "2018-03-20T10:24:29Z")
I am new to Elasticsearch and I just need to use it once. Just setup a working fresh Elasticsearch 6.2.2 and Kibana is working in my browser. I've got no indexes or other stuff, it just booted. I've got one simple json… --- ## [License problem foe xpack for logstash-6.2.2](https://discuss.elastic.co/t/license-problem-foe-xpack-for-logstash-6-2-2/124544)
**Author:** [@kanchan\_pingale](https://discuss.elastic.co/u/kanchan_pingale)\ **Replies:** 2\ **Last updated:** [March 20, 2018, 9:17am UTC](https://discuss.elastic.co/t/license-problem-foe-xpack-for-logstash-6-2-2/124544 "2018-03-20T09:17:23Z")
Once I start logstash, the error is as follows: Elasticsearch at URL 'http://localhost:9200/\_xpack' \[2018-03-19T04:33:00,946\]\[ERROR\]\[logstash.licensechecker.licensemanager\] Unable to retrieve license information from l… --- ## [Logstash not creating index after few days](https://discuss.elastic.co/t/logstash-not-creating-index-after-few-days/124705)
**Author:** [@hmmxp](https://discuss.elastic.co/u/hmmxp)\ **Replies:** 0\ **Last updated:** [March 20, 2018, 8:38am UTC](https://discuss.elastic.co/t/logstash-not-creating-index-after-few-days/124705 "2018-03-20T08:38:21Z")
Dear All, Facing issue on logstash 5.6.6 netflow module where index is not created after a few days, and the only way to resolve it is to stop logstash and starting logstash again. During stopping logstash, it will tak… --- ## [Log contains normal text and JSON](https://discuss.elastic.co/t/log-contains-normal-text-and-json/124293)
**Author:** [@jof300](https://discuss.elastic.co/u/jof300)\ **Replies:** 4\ **Last updated:** [March 20, 2018, 8:09am UTC](https://discuss.elastic.co/t/log-contains-normal-text-and-json/124293 "2018-03-20T08:09:46Z")
Hello, We have some application logs which contains one part in normal text and sometimes (when ERROR) one part in JSON format. Here is an exemple : 2018-03-15 17:48:21.087 ERROR com.example.heraclesservice.account.Acc… --- ## [Kv plugin value is empty](https://discuss.elastic.co/t/kv-plugin-value-is-empty/124698)
**Author:** [@tyb1222](https://discuss.elastic.co/u/tyb1222)\ **Replies:** 1\ **Last updated:** [March 20, 2018, 7:52am UTC](https://discuss.elastic.co/t/kv-plugin-value-is-empty/124698 "2018-03-20T07:52:53Z")
I had a problem when the key-value pair like this ip= error\_code=404.the result would be ip=error\_code=404。 --- ## [How to extract root cause of error pattern occurrence](https://discuss.elastic.co/t/how-to-extract-root-cause-of-error-pattern-occurrence/124568)
**Author:** [@prasad\_elk](https://discuss.elastic.co/u/prasad_elk)\ **Replies:** 1\ **Last updated:** [March 20, 2018, 7:43am UTC](https://discuss.elastic.co/t/how-to-extract-root-cause-of-error-pattern-occurrence/124568 "2018-03-20T07:43:50Z")
please help me to extract the set of root cause details caused to occur particular error log format: \[time\] \[applicaitonsession\] \[log type\] \[request\] \[request id\] \[time\] \[applicaitonsession\] \[log type\] \[response\] \[re… --- ## [Logstash 6.2.2 JDBC-DB2 input problem](https://discuss.elastic.co/t/logstash-6-2-2-jdbc-db2-input-problem/123975)
**Author:** [@lsi](https://discuss.elastic.co/u/lsi)\ **Replies:** 2\ **Last updated:** [March 20, 2018, 7:33am UTC](https://discuss.elastic.co/t/logstash-6-2-2-jdbc-db2-input-problem/123975 "2018-03-20T07:33:24Z")
Hi All, I was using an earlier version of Logstash (5.\*) and reading data from a DB2 instance with no issues using the following input params input { jdbc { jdbc\_connection\_string =\> "jdbc:db2://...URL" jdbc\_user =\> … --- ## [Automatic XML parsing, how to avoid array creation](https://discuss.elastic.co/t/automatic-xml-parsing-how-to-avoid-array-creation/124572)
**Author:** [@l3m0ntr33](https://discuss.elastic.co/u/l3m0ntr33)\ **Replies:** 3\ **Last updated:** [March 20, 2018, 7:27am UTC](https://discuss.elastic.co/t/automatic-xml-parsing-how-to-avoid-array-creation/124572 "2018-03-20T07:27:34Z")
Hello, I want to parse Windows Event XML Log with logstash. I use MULTILINE input plugin and XML filter but i did not manage to have the output i want. This is my input file: \ **Author:** [@Pinno\_Lin](https://discuss.elastic.co/u/Pinno_Lin)\ **Replies:** 7\ **Last updated:** [March 20, 2018, 6:52am UTC](https://discuss.elastic.co/t/how-to-parse-mail-log-from-syslog/124287 "2018-03-20T06:52:08Z") Hi, I have a .TXT file abut mail log from syslog, i try to parse log with grok, but failed. the source row: 2018-03-14 13:56:17 local0.info ms1.zootek.com.tw CEF:0|Cellopoint|CelloOS|4.1.5|FILTER\_LOG|Filter by mail pro… --- ## [Bug or not?](https://discuss.elastic.co/t/bug-or-not/124621)
**Author:** [@pjanzen](https://discuss.elastic.co/u/pjanzen)\ **Replies:** 1\ **Last updated:** [March 20, 2018, 6:23am UTC](https://discuss.elastic.co/t/bug-or-not/124621 "2018-03-20T06:23:44Z")
Hi All, I am getting the below error on my dissect filter. Is this a bug and should I file a report for this? I do not seem to be able the find the logline that this is causing it. full stacktrace Here (github.com) E… --- ## [Dissect vs. GROK + other tweak questions](https://discuss.elastic.co/t/dissect-vs-grok-other-tweak-questions/124677)
**Author:** [@kabilan](https://discuss.elastic.co/u/kabilan)\ **Replies:** 1\ **Last updated:** [March 20, 2018, 5:57am UTC](https://discuss.elastic.co/t/dissect-vs-grok-other-tweak-questions/124677 "2018-03-20T05:57:38Z")
I am looking to migrate many of parsers away from grokking and towards dissect when feasible. i had a question pertaining to performance %{header}-%{timestamp}-%{payload} -Dissect logs into 3 segments -Timestamp will … --- ## [Ingesting different formats from one json file](https://discuss.elastic.co/t/ingesting-different-formats-from-one-json-file/124658)
**Author:** [@Pororo](https://discuss.elastic.co/u/Pororo)\ **Replies:** 3\ **Last updated:** [March 20, 2018, 5:37am UTC](https://discuss.elastic.co/t/ingesting-different-formats-from-one-json-file/124658 "2018-03-20T05:37:59Z")
Hi, My json file has two different formats of data. For simple visualization, \[37\]\[47\] I have read the document before and heard there is "conditional" operation in logstash filter. I am wondering how do I actually m… --- ## [Running logstash properly](https://discuss.elastic.co/t/running-logstash-properly/124020)
**Author:** [@retxedue](https://discuss.elastic.co/u/retxedue)\ **Replies:** 15\ **Last updated:** [March 20, 2018, 3:32am UTC](https://discuss.elastic.co/t/running-logstash-properly/124020 "2018-03-20T03:32:36Z")
I have installed my logstash using rpm and enabled it using systemctl. I have to stash a csv file and create table in grafana daily. Do i have to run bin/logstash -f /etc/logstash/conf.d/logstash.config via cron if i wa… --- ## [Logstash GROK filter for tomcat logs](https://discuss.elastic.co/t/logstash-grok-filter-for-tomcat-logs/124213)
**Author:** [@ashok9177](https://discuss.elastic.co/u/ashok9177)\ **Replies:** 3\ **Last updated:** [March 19, 2018, 9:39pm UTC](https://discuss.elastic.co/t/logstash-grok-filter-for-tomcat-logs/124213 "2018-03-19T21:39:53Z")
Hello elk team , i am new to elk , i need pattern that can find " transactionid following value from tomcat logs and create separate field sample logs are like below ... 2018-03-14 10:58:36,853 INFO so:165 - Female … --- ## [Logstash convert field to local time](https://discuss.elastic.co/t/logstash-convert-field-to-local-time/124645)
**Author:** [@motts](https://discuss.elastic.co/u/motts)\ **Replies:** 1\ **Last updated:** [March 19, 2018, 9:33pm UTC](https://discuss.elastic.co/t/logstash-convert-field-to-local-time/124645 "2018-03-19T21:33:16Z")
Hello there, I am working with some IIS logs, but they are exporting the date as UTC only. Is there a way to logstash that so that it takes UTC time and converts that to the local server time? I know there is an @timest… --- ## [How to add all elements in an array from json objects as fields in logstash](https://discuss.elastic.co/t/how-to-add-all-elements-in-an-array-from-json-objects-as-fields-in-logstash/124549)
**Author:** [@Sunil\_Villuri](https://discuss.elastic.co/u/Sunil_Villuri)\ **Replies:** 6\ **Last updated:** [March 19, 2018, 8:25pm UTC](https://discuss.elastic.co/t/how-to-add-all-elements-in-an-array-from-json-objects-as-fields-in-logstash/124549 "2018-03-19T20:25:57Z")
Sample : "message": \[ "key1" : "value1", "key2" : "value2", "key3" : "value3", "key4" : "value4", "key5" : "value5", "keyn" : "valuen" \] i need to create the Additional Fields from each and every key in the mess… --- ## [Remove all URLs and emails from text](https://discuss.elastic.co/t/remove-all-urls-and-emails-from-text/124258)
**Author:** [@pavuk](https://discuss.elastic.co/u/pavuk)\ **Replies:** 1\ **Last updated:** [March 19, 2018, 7:25pm UTC](https://discuss.elastic.co/t/remove-all-urls-and-emails-from-text/124258 "2018-03-19T19:25:53Z")
Hi, I would like to remove all URLs and email addresses from plain text field. I suppose that gsub should be used like mutate { gsub =\> \["text", "http\\S+", ""\] } Please correct me if I'm wrong. And what about re… --- ## [Filter mutate add\_field : select a value in an inner field](https://discuss.elastic.co/t/filter-mutate-add-field-select-a-value-in-an-inner-field/124626)
**Author:** [@Lemec\_Cinq](https://discuss.elastic.co/u/Lemec_Cinq)\ **Replies:** 1\ **Last updated:** [March 19, 2018, 7:02pm UTC](https://discuss.elastic.co/t/filter-mutate-add-field-select-a-value-in-an-inner-field/124626 "2018-03-19T19:02:29Z")
I have a json input : {"@version":"1","@timestamp":"2018-03-19T18:24:01.919Z","fields":"{\\"numdocksavailable\\":0,\\"is\_installed\\":0,\\"is\_returning\\":0,\\"is\_renting\\":0,\\"last\_reported\\":1519703208,\\"station\_id\\":6650… --- ## [Create several es type from one json input](https://discuss.elastic.co/t/create-several-es-type-from-one-json-input/124473)
**Author:** [@Viven7512](https://discuss.elastic.co/u/Viven7512)\ **Replies:** 1\ **Last updated:** [March 19, 2018, 6:39pm UTC](https://discuss.elastic.co/t/create-several-es-type-from-one-json-input/124473 "2018-03-19T18:39:18Z")
I have a json input : \[ { "fields": { "last\_reported": 1519897717, "is\_returning": 1, "numbikesavailable": 4, "numdocksavailable": 25, "lon": 2.275342586850291, "lat": 48.8296843300915, "is\_installed": 1, "i… --- ## [IF statement block only if number of messages received](https://discuss.elastic.co/t/if-statement-block-only-if-number-of-messages-received/124582)
**Author:** [@Ben19](https://discuss.elastic.co/u/Ben19)\ **Replies:** 2\ **Last updated:** [March 19, 2018, 5:10pm UTC](https://discuss.elastic.co/t/if-statement-block-only-if-number-of-messages-received/124582 "2018-03-19T17:10:10Z")
Hello, so we have a requirement to forward emails using the logstash email plugin when certain conditions exist. We have been using this for the last few years with no issues, we are looking to try implement an if statem… --- ## [Exception in pipelineworker](https://discuss.elastic.co/t/exception-in-pipelineworker/124369)
**Author:** [@franck.lefebure](https://discuss.elastic.co/u/franck.lefebure)\ **Replies:** 2\ **Last updated:** [March 19, 2018, 4:34pm UTC](https://discuss.elastic.co/t/exception-in-pipelineworker/124369 "2018-03-19T16:34:31Z")
Hi, I have several times / day this exception on a oracle jdbc input : \[2018-03-16T03:30:29,368\]\[ERROR\]\[logstash.pipeline \] Exception in pipelineworker, the pipeline stopped processing new events, please check your fil… --- ## [Why \_grokparsefailer doesn't give specific error](https://discuss.elastic.co/t/why-grokparsefailer-doesnt-give-specific-error/124462)
**Author:** [@jason\_smith](https://discuss.elastic.co/u/jason_smith)\ **Replies:** 5\ **Last updated:** [March 19, 2018, 4:07pm UTC](https://discuss.elastic.co/t/why-grokparsefailer-doesnt-give-specific-error/124462 "2018-03-19T16:07:45Z")
Hello Good folks, when we have grokparseerror, how can I understand where the error is generated while parsing? I just tried to grab the first two fields from the file using the following grok filter. It threw error wi… --- ## [Remove key of a json key-value input](https://discuss.elastic.co/t/remove-key-of-a-json-key-value-input/124597)
**Author:** [@Lemec\_Cinq](https://discuss.elastic.co/u/Lemec_Cinq)\ **Replies:** 0\ **Last updated:** [March 19, 2018, 3:33pm UTC](https://discuss.elastic.co/t/remove-key-of-a-json-key-value-input/124597 "2018-03-19T15:33:04Z")
I have a json input which contains a key-value where the value is an array, I want to keep the array value but directly without the key : the input: { "records": \[ { "geometry": { "coordinates": \[… --- ## [Dissect filter warn when log contains Chinese word](https://discuss.elastic.co/t/dissect-filter-warn-when-log-contains-chinese-word/121966)
**Author:** [@lafrinte](https://discuss.elastic.co/u/lafrinte)\ **Replies:** 2\ **Last updated:** [March 19, 2018, 3:07pm UTC](https://discuss.elastic.co/t/dissect-filter-warn-when-log-contains-chinese-word/121966 "2018-03-19T15:07:47Z")
I use dissect filter to parse a log which contains some Chinese word, it will prompt warn 'Dissector mapping, field found in event but it was empty'. What can i do for this warn. This is my filter proxy: filter { dis… --- ## [Filter records that contain a specfic field name](https://discuss.elastic.co/t/filter-records-that-contain-a-specfic-field-name/124476)
**Author:** [@Lemec\_Cinq](https://discuss.elastic.co/u/Lemec_Cinq)\ **Replies:** 4\ **Last updated:** [March 19, 2018, 3:06pm UTC](https://discuss.elastic.co/t/filter-records-that-contain-a-specfic-field-name/124476 "2018-03-19T15:06:18Z")
I have a json input with different fields name and I would like to filter the data in order to keep at the end of the chain only the records that have a specific field For example in the data set below I just want to ke… --- ## [Flatten JSON Array in Logstash Filter](https://discuss.elastic.co/t/flatten-json-array-in-logstash-filter/124562)
**Author:** [@EricPSU](https://discuss.elastic.co/u/EricPSU)\ **Replies:** 2\ **Last updated:** [March 19, 2018, 2:41pm UTC](https://discuss.elastic.co/t/flatten-json-array-in-logstash-filter/124562 "2018-03-19T14:41:17Z")
I have a REST API call that returns the structure below. When I send this structure to Elasticsearch, all of the records within the "data" array are combined into one Elasticsearch document. { "data": \[ { … --- ## [Logstash5 is more then 3 time slower then logstash2](https://discuss.elastic.co/t/logstash5-is-more-then-3-time-slower-then-logstash2/124571)
**Author:** [@vkakhnych](https://discuss.elastic.co/u/vkakhnych)\ **Replies:** 0\ **Last updated:** [March 19, 2018, 2:10pm UTC](https://discuss.elastic.co/t/logstash5-is-more-then-3-time-slower-then-logstash2/124571 "2018-03-19T14:10:26Z")
Upgraded logstash2 with default config to logstash5 with also default config and got horrible performance on the same application config. Is it well known issue? We use it to get messages from AWS SQS and push to ES (EL… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=797) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=799)