# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=552 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 553 --- ## [Want to extract error(no response AudioSetVolum) message from log file using logstash also count of error message occured and it's timing. I am new to this Can anyone help me on this?](https://discuss.elastic.co/t/want-to-extract-error-no-response-audiosetvolum-message-from-log-file-using-logstash-also-count-of-error-message-occured-and-its-timing-i-am-new-to-this-can-anyone-help-me-on-this/180142)
**Author:** [@veera](https://discuss.elastic.co/u/veera)\ **Replies:** 0\ **Last updated:** [May 8, 2019, 9:26am UTC](https://discuss.elastic.co/t/want-to-extract-error-no-response-audiosetvolum-message-from-log-file-using-logstash-also-count-of-error-message-occured-and-its-timing-i-am-new-to-this-can-anyone-help-me-on-this/180142 "2019-05-08T09:26:15Z")
\-------------Sample log file below------------------- 4/10/2019 12:19:17:437 - #ERROR#NAK# for AudioSetGain\_0x0329, slot=0x82, ctrl=0 4/10/2019 12:19:17:437 - VTS\_RANGE\_ERROR 0x0004 4/10/2019 12:19:17:453 - #ERROR#NAK… --- ## [Complex xml/json data insert](https://discuss.elastic.co/t/complex-xml-json-data-insert/179796)
**Author:** [@Niels\_van\_Rijn](https://discuss.elastic.co/u/Niels_van_Rijn)\ **Replies:** 8\ **Last updated:** [May 8, 2019, 7:59am UTC](https://discuss.elastic.co/t/complex-xml-json-data-insert/179796 "2019-05-08T07:59:49Z")
Hi there! I'm new to the ELK stack and I need some advice for inserting my complex xml's into the ELK stack here you can see an example of only one of 10.163 xml's I have used the xml filter in logstash but this creat… --- ## [How to dynamically move nested key value to root level](https://discuss.elastic.co/t/how-to-dynamically-move-nested-key-value-to-root-level/180006)
**Author:** [@arefeh](https://discuss.elastic.co/u/arefeh)\ **Replies:** 2\ **Last updated:** [May 8, 2019, 4:56am UTC](https://discuss.elastic.co/t/how-to-dynamically-move-nested-key-value-to-root-level/180006 "2019-05-08T04:56:31Z")
This is my logstash config file: input { http { id =\> bulkHttpInput port =\> 8088 additional\_codecs =\> {"application/json" =\> "es\_bulk"} codec =\> es\_bulk } } filter { mutate { remove\_field =\> \[… --- ## [Unable to Load Plugin - Packetfu](https://discuss.elastic.co/t/unable-to-load-plugin-packetfu/180093)
**Author:** [@apooniajjn](https://discuss.elastic.co/u/apooniajjn)\ **Replies:** 0\ **Last updated:** [May 8, 2019, 4:26am UTC](https://discuss.elastic.co/t/unable-to-load-plugin-packetfu/180093 "2019-05-08T04:26:36Z")
I am creating a custom output plugin and use it on logstash. I am using packetfu library in my plugin definition when I am running plugin it fails because it couldn't load packetfu. Where do I have to define packetfu so… --- ## [Is there a way to add new field in bulk in logstash filter?](https://discuss.elastic.co/t/is-there-a-way-to-add-new-field-in-bulk-in-logstash-filter/179912)
**Author:** [@david3721](https://discuss.elastic.co/u/david3721)\ **Replies:** 2\ **Last updated:** [May 7, 2019, 12:33pm UTC](https://discuss.elastic.co/t/is-there-a-way-to-add-new-field-in-bulk-in-logstash-filter/179912 "2019-05-07T12:33:05Z")
I am using elastiflow to analysis netflow, now I am trying to add a new field, and fill the corresponding value when matching a specific IP. The filter conf is working like this, but the issue is I have thousands IPs to … --- ## [Out of Memory right after staring logstash](https://discuss.elastic.co/t/out-of-memory-right-after-staring-logstash/179568)
**Author:** [@halolk](https://discuss.elastic.co/u/halolk)\ **Replies:** 6\ **Last updated:** [May 8, 2019, 12:02am UTC](https://discuss.elastic.co/t/out-of-memory-right-after-staring-logstash/179568 "2019-05-08T00:02:22Z")
Hi guys, My company is running a logstash instance to ingest logs from customer by several input conf files, it crashed after about 1 day due to OOM, and then every time I try to start it again, it crash by OOM immediat… --- ## [Logstash ruby filter issues with HTTP queries](https://discuss.elastic.co/t/logstash-ruby-filter-issues-with-http-queries/180017)
**Author:** [@ayesyev](https://discuss.elastic.co/u/ayesyev)\ **Replies:** 4\ **Last updated:** [May 7, 2019, 7:47pm UTC](https://discuss.elastic.co/t/logstash-ruby-filter-issues-with-http-queries/180017 "2019-05-07T19:47:13Z")
Hi there! I'm stuck with the issue, I have with an http client in my ruby filter in Logstash 6.5.4. Here is a snippet of the filter code require 'net/http' require 'uri' require 'json' def register(params) @http = … --- ## [Date field being converted to timestamps](https://discuss.elastic.co/t/date-field-being-converted-to-timestamps/180044)
**Author:** [@Bofus](https://discuss.elastic.co/u/Bofus)\ **Replies:** 3\ **Last updated:** [May 7, 2019, 7:44pm UTC](https://discuss.elastic.co/t/date-field-being-converted-to-timestamps/180044 "2019-05-07T19:44:53Z")
Hello everyone, Im new in this world.... I hope this question is not really dumb, but Ive being searching the net the whole day and couldnt find an answer. The thing is simple, Im trying to import some data from a MySQ… --- ## [Changing eventTime to ISO8601 standard](https://discuss.elastic.co/t/changing-eventtime-to-iso8601-standard/180034)
**Author:** [@robkeik](https://discuss.elastic.co/u/robkeik)\ **Replies:** 1\ **Last updated:** [May 7, 2019, 6:38pm UTC](https://discuss.elastic.co/t/changing-eventtime-to-iso8601-standard/180034 "2019-05-07T18:38:15Z")
I am trying to change the eventTime to the ISO standard. I am looking at the filter date plugin but not sure how to code this in dissect. Any suggestions. thanks dissect { mapping =\> { "message" =\> '%{} %{} %{eventTim… --- ## [Error while using amazon\_es](https://discuss.elastic.co/t/error-while-using-amazon-es/179294)
**Author:** [@Shubham\_Yadav1](https://discuss.elastic.co/u/Shubham_Yadav1)\ **Replies:** 4\ **Last updated:** [May 7, 2019, 6:16pm UTC](https://discuss.elastic.co/t/error-while-using-amazon-es/179294 "2019-05-07T18:16:15Z")
I am running logstash on RHEL with amazon\_es output plugin to send data to aws elastic search but getting this output and i don't see index in kibana. how do i make sure everything is working fine? WARNING: Could not f… --- ## [Logstash Custom Plugin can't add the local dependencies](https://discuss.elastic.co/t/logstash-custom-plugin-cant-add-the-local-dependencies/179830)
**Author:** [@paul1243](https://discuss.elastic.co/u/paul1243)\ **Replies:** 1\ **Last updated:** [May 7, 2019, 6:10pm UTC](https://discuss.elastic.co/t/logstash-custom-plugin-cant-add-the-local-dependencies/179830 "2019-05-07T18:10:56Z")
Hi, We use artifactory as the gem source and we have some locally built gems. I tried to use one of the gem in the customized logstash filter plugin as a runtime dependency and tried to call inside the filter, which ru… --- ## [Illegal unquoted character for google cloud storage input plugin](https://discuss.elastic.co/t/illegal-unquoted-character-for-google-cloud-storage-input-plugin/180021)
**Author:** [@msukmana](https://discuss.elastic.co/u/msukmana)\ **Replies:** 0\ **Last updated:** [May 7, 2019, 3:47pm UTC](https://discuss.elastic.co/t/illegal-unquoted-character-for-google-cloud-storage-input-plugin/180021 "2019-05-07T15:47:38Z")
I am trying to download stackdriver logs from my GCP bucket using logstash with this configuration: google\_cloud\_storage { interval =\> 60 bucket\_id =\> "stackdriverlog-cloudraid" json\_key\_file =\> "/home/boli/Document… --- ## [Index matched patterns and drop unmatched](https://discuss.elastic.co/t/index-matched-patterns-and-drop-unmatched/179871)
**Author:** [@pep\_sdg](https://discuss.elastic.co/u/pep_sdg)\ **Replies:** 1\ **Last updated:** [May 7, 2019, 3:44pm UTC](https://discuss.elastic.co/t/index-matched-patterns-and-drop-unmatched/179871 "2019-05-07T15:44:54Z")
Hi. I'm new to Logstash grok patterns and don't have enough knowledge to right the grok filter to match and drop events. I'm trying to index events where the pattern matches event\_type = tripwire write to tripwire index… --- ## [Parse HTTP requests to the NASA Kennedy Space Center WWW server in Florida](https://discuss.elastic.co/t/parse-http-requests-to-the-nasa-kennedy-space-center-www-server-in-florida/180009)
**Author:** [@jailbreakerSN](https://discuss.elastic.co/u/jailbreakerSN)\ **Replies:** 2\ **Last updated:** [May 7, 2019, 3:42pm UTC](https://discuss.elastic.co/t/parse-http-requests-to-the-nasa-kennedy-space-center-www-server-in-florida/180009 "2019-05-07T15:42:01Z")
Hi, I'm working on Nasa Kennedy Space Center web server data set and I want to monitore them in Kibana. So what I want is to collect them with Filebeat and parse them with Logstash. So is somebody know how to configur… --- ## [If Array element not present](https://discuss.elastic.co/t/if-array-element-not-present/179938)
**Author:** [@ghost3h](https://discuss.elastic.co/u/ghost3h)\ **Replies:** 1\ **Last updated:** [May 7, 2019, 3:20pm UTC](https://discuss.elastic.co/t/if-array-element-not-present/179938 "2019-05-07T15:20:47Z")
Is there a way to only set a field to an array element when its populated? split =\> { "HOSTALIAS" =\> ", " } add\_field =\> { "ci\_alias" =\> "%{\[HOSTALIAS\]\[0\]}" "blueprint-id" =\> "… --- ## [Conditional output from filbeat documents logstash to elasticsearch](https://discuss.elastic.co/t/conditional-output-from-filbeat-documents-logstash-to-elasticsearch/179866)
**Author:** [@rajsolanki](https://discuss.elastic.co/u/rajsolanki)\ **Replies:** 2\ **Last updated:** [May 7, 2019, 2:52pm UTC](https://discuss.elastic.co/t/conditional-output-from-filbeat-documents-logstash-to-elasticsearch/179866 "2019-05-07T14:52:51Z")
I hope its not a repeat Question. I m trying to ingest multiple files from multiple directories via filebeat --\> logstash. in my file beat input config (filebeat.config.inputs: statements) i m adding log\_type field for… --- ## [Logstash avro error can't start](https://discuss.elastic.co/t/logstash-avro-error-cant-start/179914)
**Author:** [@zqc0512](https://discuss.elastic.co/u/zqc0512)\ **Replies:** 3\ **Last updated:** [May 7, 2019, 1:45pm UTC](https://discuss.elastic.co/t/logstash-avro-error-cant-start/179914 "2019-05-07T13:45:31Z")
logstash 7.0.1 \[2019-05-07T15:51:16,703\]\[FATAL\]\[logstash.runner \] An unexpected error occurred! {:error=\>#\, :backtrace=\>\["org/jruby/ext/stringio/StringIO.java:852:in \`r… --- ## [Avoiding Field Reference Grammer in JSON Parsing](https://discuss.elastic.co/t/avoiding-field-reference-grammer-in-json-parsing/177899)
**Author:** [@hal74](https://discuss.elastic.co/u/hal74)\ **Replies:** 3\ **Last updated:** [May 7, 2019, 12:57pm UTC](https://discuss.elastic.co/t/avoiding-field-reference-grammer-in-json-parsing/177899 "2019-05-07T12:57:13Z")
Hey all, I have been processing stringified JSON using the json filter plugin for some time: json { skip\_on\_invalid\_json =\> true source =\> "data" } The JSON being parsed is dynamically generated by another sys… --- ## [Won't accept logs using standard syslog date format](https://discuss.elastic.co/t/wont-accept-logs-using-standard-syslog-date-format/179745)
**Author:** [@elasticSmash](https://discuss.elastic.co/u/elasticSmash)\ **Replies:** 2\ **Last updated:** [May 7, 2019, 11:59am UTC](https://discuss.elastic.co/t/wont-accept-logs-using-standard-syslog-date-format/179745 "2019-05-07T11:59:07Z")
So far I've added hosts to my logstash input that have used ISO8601 time format in their syslog config. Those have imported ok and the date/time is interpreted into the timestamp field. I have the following stanza in m… --- ## [Data format issue](https://discuss.elastic.co/t/data-format-issue/179941)
**Author:** [@Ravindra\_Nath](https://discuss.elastic.co/u/Ravindra_Nath)\ **Replies:** 0\ **Last updated:** [May 7, 2019, 11:04am UTC](https://discuss.elastic.co/t/data-format-issue/179941 "2019-05-07T11:04:37Z")
Is there any possibility for logstash to take @timestamp from the log field and my date format coming from mysql is "Tue Apr 30 17:04:35 IST 2019" which is in VARCHAR and also want to change the format to which elastic s… --- ## [GC (Allocation Failure)](https://discuss.elastic.co/t/gc-allocation-failure/179744)
**Author:** [@hemantkhokhar](https://discuss.elastic.co/u/hemantkhokhar)\ **Replies:** 3\ **Last updated:** [May 7, 2019, 10:35am UTC](https://discuss.elastic.co/t/gc-allocation-failure/179744 "2019-05-07T10:35:14Z")
while i am getting below logs then kiban not showing logs ...need help. 10014.695: \[GC (Allocation Failure) 1531122K-\>1258470K(4160256K), 0.0214068 secs\] 10015.756: \[GC (Allocation Failure) 1531110K-\>1258425K(4160256… --- ## [Cisco Switch logs](https://discuss.elastic.co/t/cisco-switch-logs/179376)
**Author:** [@Labidi\_Ayoub](https://discuss.elastic.co/u/Labidi_Ayoub)\ **Replies:** 2\ **Last updated:** [May 7, 2019, 10:08am UTC](https://discuss.elastic.co/t/cisco-switch-logs/179376 "2019-05-07T10:08:24Z")
Hello everyone, Please i would be grateful if someone help me how i collect cisco Switch logs with ELK, it is my first time with ELK , I installed ELK in centos 7 . --- ## [Syslog are not indexing to logstash-2019-x.x](https://discuss.elastic.co/t/syslog-are-not-indexing-to-logstash-2019-x-x/179694)
**Author:** [@wcpoon](https://discuss.elastic.co/u/wcpoon)\ **Replies:** 3\ **Last updated:** [May 7, 2019, 9:16am UTC](https://discuss.elastic.co/t/syslog-are-not-indexing-to-logstash-2019-x-x/179694 "2019-05-07T09:16:30Z")
Hi guys, after upgrade from 6.7 to 7.0, i can received syslog from my remote devices syslog to /var/log/messages, however, i cannot display all the latest logs in kibana discover tab. all latest data is not indexing to… --- ## [Logstash behavior with cacert field in elasticsearch output (6.4 logstash, 5.6 ES)](https://discuss.elastic.co/t/logstash-behavior-with-cacert-field-in-elasticsearch-output-6-4-logstash-5-6-es/179798)
**Author:** [@preddy75](https://discuss.elastic.co/u/preddy75)\ **Replies:** 1\ **Last updated:** [May 7, 2019, 8:45am UTC](https://discuss.elastic.co/t/logstash-behavior-with-cacert-field-in-elasticsearch-output-6-4-logstash-5-6-es/179798 "2019-05-07T08:45:18Z")
Hi, What exactly is behavior of logstash with this configuration? output { elasticsearch { user =\> "" password =\> "" ssl =\> "true" ssl\_certificate\_verification =\> false hosts =\> \[\] cacert =\> "/rootcafile.pem" m… --- ## [Is there a way to integrate custom decoder in logstash](https://discuss.elastic.co/t/is-there-a-way-to-integrate-custom-decoder-in-logstash/179882)
**Author:** [@Sheenam\_Jindal](https://discuss.elastic.co/u/Sheenam_Jindal)\ **Replies:** 0\ **Last updated:** [May 7, 2019, 4:56am UTC](https://discuss.elastic.co/t/is-there-a-way-to-integrate-custom-decoder-in-logstash/179882 "2019-05-07T04:56:09Z")
I have binary logs which I want to get decoded by my own custom decoder to get the logs in text format. Can anyone help with some pointers? --- ## [How to take input from CORBA in logstash](https://discuss.elastic.co/t/how-to-take-input-from-corba-in-logstash/178886)
**Author:** [@Ku1deep](https://discuss.elastic.co/u/Ku1deep)\ **Replies:** 2\ **Last updated:** [May 7, 2019, 7:13am UTC](https://discuss.elastic.co/t/how-to-take-input-from-corba-in-logstash/178886 "2019-05-07T07:13:12Z")
How can I take input from CORBA in logstash? Any help please --- ## [Getting "lumberjack All host unavailable, sleeping" error when Logstash-to-Logstash communication is configured](https://discuss.elastic.co/t/getting-lumberjack-all-host-unavailable-sleeping-error-when-logstash-to-logstash-communication-is-configured/179903)
**Author:** [@garima](https://discuss.elastic.co/u/garima)\ **Replies:** 0\ **Last updated:** [May 7, 2019, 6:59am UTC](https://discuss.elastic.co/t/getting-lumberjack-all-host-unavailable-sleeping-error-when-logstash-to-logstash-communication-is-configured/179903 "2019-05-07T06:59:38Z")
Logstash Version: 6.4.1 Use case: Logstash-to-Logstash Communication Upstream hostname : node-1 (IP 172.30.64.98) Downstream hostname : node-2 (IP 172.30.64.115) In /etc/hosts both hostname is already mentioned and b… --- ## [Upgrade from 6.7 to 7.0, logstash cannot index event to elasticsearch](https://discuss.elastic.co/t/upgrade-from-6-7-to-7-0-logstash-cannot-index-event-to-elasticsearch/179901)
**Author:** [@wcpoon](https://discuss.elastic.co/u/wcpoon)\ **Replies:** 0\ **Last updated:** [May 7, 2019, 6:55am UTC](https://discuss.elastic.co/t/upgrade-from-6-7-to-7-0-logstash-cannot-index-event-to-elasticsearch/179901 "2019-05-07T06:55:44Z")
Hi guys, my logstash 7.0 cannot index event to Elasticsearch, below are the logs details, \[2019-05-06T10:10:35,215\]\[WARN \]\[logstash.outputs.elasticsearch\] Could not index event to Elasticsearch. {:status=\>400, :action=\>… --- ## [Logstash unable to bind to external IP](https://discuss.elastic.co/t/logstash-unable-to-bind-to-external-ip/179877)
**Author:** [@Abhilash\_B](https://discuss.elastic.co/u/Abhilash_B)\ **Replies:** 4\ **Last updated:** [May 7, 2019, 6:10am UTC](https://discuss.elastic.co/t/logstash-unable-to-bind-to-external-ip/179877 "2019-05-07T06:10:57Z")
I have filebeat and logstash running of different droplets in digitalocean. When I use internal IP of logstash droplet in output.logstash section of filebeat.yml, everything works fine. When I change it to an external IP… --- ## [Indexing for only updated data using logstash](https://discuss.elastic.co/t/indexing-for-only-updated-data-using-logstash/179705)
**Author:** [@pyerunka](https://discuss.elastic.co/u/pyerunka)\ **Replies:** 4\ **Last updated:** [May 7, 2019, 5:28am UTC](https://discuss.elastic.co/t/indexing-for-only-updated-data-using-logstash/179705 "2019-05-07T05:28:50Z")
Hello, Using jdbc input plugin in logstash, I have fetched all the data which I have wanted the most from database tables. Now I want only updated data from all that db tables. Every time I don’t want to index whole da… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=551) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=553)