# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=937 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 938 --- ## [Org.logstash.beats Invalid frame protocol exception](https://discuss.elastic.co/t/org-logstash-beats-invalid-frame-protocol-exception/95010)
**Author:** [@sweety](https://discuss.elastic.co/u/sweety)\ **Replies:** 0\ **Last updated:** [July 28, 2017, 8:32pm UTC](https://discuss.elastic.co/t/org-logstash-beats-invalid-frame-protocol-exception/95010 "2017-07-28T20:32:38Z")
I am trying to send logs to logstash using filebeats On local machine: --filebeats --logstash --kibana On server --elastcisearch I ran filebeat and logstash on two terminals on my local machine: ran logstash - b… --- ## [Logstash and x-pack monitoring](https://discuss.elastic.co/t/logstash-and-x-pack-monitoring/92929)
**Author:** [@jupp](https://discuss.elastic.co/u/jupp)\ **Replies:** 6\ **Last updated:** [July 28, 2017, 6:37pm UTC](https://discuss.elastic.co/t/logstash-and-x-pack-monitoring/92929 "2017-07-28T18:37:49Z")
hi people, why i can not write my logstash x-pack monitoring data directly in a dedicated monitoring cluster? greetz jupp --- ## [Overwriting json @timestamp](https://discuss.elastic.co/t/overwriting-json-timestamp/94631)
**Author:** [@2knarf](https://discuss.elastic.co/u/2knarf)\ **Replies:** 11\ **Last updated:** [July 28, 2017, 4:11pm UTC](https://discuss.elastic.co/t/overwriting-json-timestamp/94631 "2017-07-28T16:11:50Z")
Hello, I am sending json logs to logstash and I want to overwrite the @timestamp field. To do this I do: json.keys\_under\_root: true json.overwrite\_keys: true The log entries goes to logstash, but in Kibana th… --- ## [Multiple site configuration](https://discuss.elastic.co/t/multiple-site-configuration/94882)
**Author:** [@plackey](https://discuss.elastic.co/u/plackey)\ **Replies:** 3\ **Last updated:** [July 28, 2017, 3:33pm UTC](https://discuss.elastic.co/t/multiple-site-configuration/94882 "2017-07-28T15:33:37Z")
I have 10 remote sites that I need to backhaul the logs to a central site for processing. I was going to have logstash at each of the remote sites and use lumberjack to forward the logs to the central site. The question … --- ## [Delete all extracted events before a specific field](https://discuss.elastic.co/t/delete-all-extracted-events-before-a-specific-field/94847)
**Author:** [@ibrahimsharaf](https://discuss.elastic.co/u/ibrahimsharaf)\ **Replies:** 3\ **Last updated:** [July 28, 2017, 3:05pm UTC](https://discuss.elastic.co/t/delete-all-extracted-events-before-a-specific-field/94847 "2017-07-28T15:05:59Z")
Hi there, I am having an important message in my logfile (logfile stats, comes at the end of the logfile). Sometimes the logfile is not finished (before sending it to logstash), so the stats message is not written yet, s… --- ## [Logstash Parsing for tags](https://discuss.elastic.co/t/logstash-parsing-for-tags/94703)
**Author:** [@jayden.kim](https://discuss.elastic.co/u/jayden.kim)\ **Replies:** 4\ **Last updated:** [July 28, 2017, 2:39pm UTC](https://discuss.elastic.co/t/logstash-parsing-for-tags/94703 "2017-07-28T14:39:14Z")
I am using rsyslog to send selected logs to a server with ELK. It is hard to manage the list of logs through rsyslog.d conf, thus I want to send all logs from a certain directory in /var/log. The problem I encounter is t… --- ## [Permissions Driving Me Nuts!](https://discuss.elastic.co/t/permissions-driving-me-nuts/94859)
**Author:** [@GV27](https://discuss.elastic.co/u/GV27)\ **Replies:** 2\ **Last updated:** [July 28, 2017, 1:59pm UTC](https://discuss.elastic.co/t/permissions-driving-me-nuts/94859 "2017-07-28T13:59:48Z")
First time LogStash install and I can't get it to run due to permissions issues that I 'm not understanding. Started by complaining about /usr/share/logstash/data not being writable so I fixed that, then ./data/queue s… --- ## [Ruby filter limitations to hash tables?](https://discuss.elastic.co/t/ruby-filter-limitations-to-hash-tables/94946)
**Author:** [@markchagers](https://discuss.elastic.co/u/markchagers)\ **Replies:** 2\ **Last updated:** [July 28, 2017, 1:16pm UTC](https://discuss.elastic.co/t/ruby-filter-limitations-to-hash-tables/94946 "2017-07-28T13:16:24Z")
Are there any limitations to what you can do with a hash table in the ruby filter? I’m trying things that work well in regular ruby, but which cause logstash --config.test\_and\_exit to throw an error. I have searched the… --- ## [Rotate elastic indices based on size](https://discuss.elastic.co/t/rotate-elastic-indices-based-on-size/94933)
**Author:** [@abhiroyg](https://discuss.elastic.co/u/abhiroyg)\ **Replies:** 2\ **Last updated:** [July 28, 2017, 11:24am UTC](https://discuss.elastic.co/t/rotate-elastic-indices-based-on-size/94933 "2017-07-28T11:24:09Z")
Hi, How to achieve rotation of elastic indices based on its size using elasticsearch-output plugin ? If I use an external program like Rollover API, will I lose data ? Do I have to bring logstash down, do the rollover, … --- ## [Increase pipeline.batch.size](https://discuss.elastic.co/t/increase-pipeline-batch-size/94756)
**Author:** [@mitabrev](https://discuss.elastic.co/u/mitabrev)\ **Replies:** 1\ **Last updated:** [July 28, 2017, 11:21am UTC](https://discuss.elastic.co/t/increase-pipeline-batch-size/94756 "2017-07-28T11:21:46Z")
Hi all, Currently I have default settings in logstash.yml: pipeline.workers: 4 pipeline.batch.size: 125 I tried to increase the batch size to 200 and 500, but the logstash service wouldn't start. Even when I decrease… --- ## [Setting "dead\_letter\_queue.enable" hasn't been registered](https://discuss.elastic.co/t/setting-dead-letter-queue-enable-hasnt-been-registered/94934)
**Author:** [@mitabrev](https://discuss.elastic.co/u/mitabrev)\ **Replies:** 4\ **Last updated:** [July 28, 2017, 11:10am UTC](https://discuss.elastic.co/t/setting-dead-letter-queue-enable-hasnt-been-registered/94934 "2017-07-28T11:10:38Z")
I’m reading a little the docs, want to optimize my indexing and I think the setting dead\_letter\_queue.enable: true would be useful. So I added this line to my logstash.yml, started the service but got the error: \`\[2017… --- ## [Extract substring of a log line](https://discuss.elastic.co/t/extract-substring-of-a-log-line/94788)
**Author:** [@DFrant](https://discuss.elastic.co/u/DFrant)\ **Replies:** 2\ **Last updated:** [July 28, 2017, 10:28am UTC](https://discuss.elastic.co/t/extract-substring-of-a-log-line/94788 "2017-07-28T10:28:24Z")
Hello, I have some logs lines wich are only one string, for example : 01999918000170702135929%WS%00000000000070030819078820913135929050000080RRN0002W900500000000C0000000000000500024464063100100 03081907882 300 R00 I… --- ## [Iterate over array in logstash](https://discuss.elastic.co/t/iterate-over-array-in-logstash/94491)
**Author:** [@vids](https://discuss.elastic.co/u/vids)\ **Replies:** 2\ **Last updated:** [July 28, 2017, 6:53am UTC](https://discuss.elastic.co/t/iterate-over-array-in-logstash/94491 "2017-07-28T06:53:58Z")
can we iterate over array in logstash under xml filter by using for loop for dynamic array? --- ## [Path for custom ruby gems?](https://discuss.elastic.co/t/path-for-custom-ruby-gems/94907)
**Author:** [@german23](https://discuss.elastic.co/u/german23)\ **Replies:** 0\ **Last updated:** [July 28, 2017, 6:32am UTC](https://discuss.elastic.co/t/path-for-custom-ruby-gems/94907 "2017-07-28T06:32:42Z")
Hey folks, currently i am trying to implement a caching solution via memcache to cache our netbios name lookups. /usr/share/logstash/bin/logstash --version logstash 5.5.0 For this task i installed a ruby plugin called… --- ## [No location field Geoip plugin](https://discuss.elastic.co/t/no-location-field-geoip-plugin/94644)
**Author:** [@xiguazhi](https://discuss.elastic.co/u/xiguazhi)\ **Replies:** 4\ **Last updated:** [July 28, 2017, 12:32am UTC](https://discuss.elastic.co/t/no-location-field-geoip-plugin/94644 "2017-07-28T00:32:04Z")
After using Geoip plugin in a filter specifying the ip address field I am getting Location.lat and Location.lon fields but I am not getting a location field defined as geopoint, or anything for that matter I have inst… --- ## [Logstash output to logstash](https://discuss.elastic.co/t/logstash-output-to-logstash/94780)
**Author:** [@ibrahimsharaf](https://discuss.elastic.co/u/ibrahimsharaf)\ **Replies:** 1\ **Last updated:** [July 27, 2017, 9:10pm UTC](https://discuss.elastic.co/t/logstash-output-to-logstash/94780 "2017-07-27T21:10:28Z")
Hello, I am thinking about sending my logfiles to a logstash instance to parse some events, then send the output for another logstash instance to parse another events, based on the extracted events by the first logstash … --- ## [Logstash crashes in dns filter](https://discuss.elastic.co/t/logstash-crashes-in-dns-filter/94810)
**Author:** [@Tom\_Mortimer](https://discuss.elastic.co/u/Tom_Mortimer)\ **Replies:** 3\ **Last updated:** [July 27, 2017, 6:00pm UTC](https://discuss.elastic.co/t/logstash-crashes-in-dns-filter/94810 "2017-07-27T18:00:38Z")
Hi all, Trying the dns filter in Logstash for the first time, and I'm getting this error in the log: \[2017-07-27T14:54:43,886\]\[ERROR\]\[logstash.pipeline \] Exception in pipelineworker, the pipeline stopped proce… --- ## [Multiline defined in filebeat will not work in logstash](https://discuss.elastic.co/t/multiline-defined-in-filebeat-will-not-work-in-logstash/94763)
**Author:** [@Platypus](https://discuss.elastic.co/u/Platypus)\ **Replies:** 1\ **Last updated:** [July 27, 2017, 4:54pm UTC](https://discuss.elastic.co/t/multiline-defined-in-filebeat-will-not-work-in-logstash/94763 "2017-07-27T16:54:41Z")
I have a classic ELK environment. I have configured some single line and some multiline propectors in my filebeat config. I have check the logs of filebeat and filebeat is sending correctly both my multiline and single … --- ## [Logstash using abrevations on another index](https://discuss.elastic.co/t/logstash-using-abrevations-on-another-index/94820)
**Author:** [@arunvijayam](https://discuss.elastic.co/u/arunvijayam)\ **Replies:** 1\ **Last updated:** [July 27, 2017, 4:19pm UTC](https://discuss.elastic.co/t/logstash-using-abrevations-on-another-index/94820 "2017-07-27T16:19:57Z")
Hi All, I am using a different index for field abbreviations and its descriptions. How to use that index on another index to change field value based on abbreviations. Simply I am asking that one index hold US state … --- ## [Hashmap usage in logstash filter plugin](https://discuss.elastic.co/t/hashmap-usage-in-logstash-filter-plugin/94821)
**Author:** [@arunvijayam](https://discuss.elastic.co/u/arunvijayam)\ **Replies:** 1\ **Last updated:** [July 27, 2017, 4:18pm UTC](https://discuss.elastic.co/t/hashmap-usage-in-logstash-filter-plugin/94821 "2017-07-27T16:18:45Z")
Hi All, I have to populate a field in index with its full form from its short form. For this I have to use hashmap or any property holder to convert the field value from its short form to long form(full form). If a f… --- ## [FileBeats to Logstash](https://discuss.elastic.co/t/filebeats-to-logstash/94399)
**Author:** [@FR-TheS](https://discuss.elastic.co/u/FR-TheS)\ **Replies:** 4\ **Last updated:** [July 27, 2017, 2:42pm UTC](https://discuss.elastic.co/t/filebeats-to-logstash/94399 "2017-07-27T14:42:12Z")
Hi everyone, We have facing a weird issue about filebeats to logstash. We have at this moment 5 conf files in logstash with jdbc query. We wants to add a new conf which is for filebeats so listen on port 5044 and outpu… --- ## [Help with grok for logfile](https://discuss.elastic.co/t/help-with-grok-for-logfile/94792)
**Author:** [@dhiraj\_khanna](https://discuss.elastic.co/u/dhiraj_khanna)\ **Replies:** 1\ **Last updated:** [July 27, 2017, 1:14pm UTC](https://discuss.elastic.co/t/help-with-grok-for-logfile/94792 "2017-07-27T13:14:35Z")
I am very new to the ELK stack and have been trying to look at some tutorials to get me started. Would appreciate if someone could point me in the right direction for ingesting the following type of logs. I assume that … --- ## [Date filter parsing for Hours](https://discuss.elastic.co/t/date-filter-parsing-for-hours/94797)
**Author:** [@josephjohney](https://discuss.elastic.co/u/josephjohney)\ **Replies:** 0\ **Last updated:** [July 27, 2017, 12:59pm UTC](https://discuss.elastic.co/t/date-filter-parsing-for-hours/94797 "2017-07-27T12:59:51Z")
Continuing the discussion from Date filter not working for AM/PM: Continuing the discussion from Filebeat: send fail; Error publishing events (retrying): read tcp 10.13.37.73:49990-\>10.13.37.99:5044: i/o timeout: Cont… --- ## [Timestamp seems lost when I stop all the 3 logstash nodes then start them?](https://discuss.elastic.co/t/timestamp-seems-lost-when-i-stop-all-the-3-logstash-nodes-then-start-them/93360)
**Author:** [@KeithTt](https://discuss.elastic.co/u/KeithTt)\ **Replies:** 11\ **Last updated:** [July 27, 2017, 10:57am UTC](https://discuss.elastic.co/t/timestamp-seems-lost-when-i-stop-all-the-3-logstash-nodes-then-start-them/93360 "2017-07-27T10:57:05Z")
logstash version: 5.41 There are 3 logstash nodes in my ELK-STACK, I configure load-balancing in filebeat. I use BELK to collect and analysis nginx access log, and I change the log-format to json. Once, I stop all … --- ## [How to send logs directly to ELK (no intermediate file storage involved)](https://discuss.elastic.co/t/how-to-send-logs-directly-to-elk-no-intermediate-file-storage-involved/94649)
**Author:** [@mla](https://discuss.elastic.co/u/mla)\ **Replies:** 4\ **Last updated:** [July 27, 2017, 8:24am UTC](https://discuss.elastic.co/t/how-to-send-logs-directly-to-elk-no-intermediate-file-storage-involved/94649 "2017-07-27T08:24:08Z")
The applications I'm running in a Docker container, managed by openshift, should not use volumes for log files. So i would like to write directly from the applications, using some module (appender) in log4j/logback/....… --- ## [Error in Logstash log . "Error: Disc quota exceeded"](https://discuss.elastic.co/t/error-in-logstash-log-error-disc-quota-exceeded/94102)
**Author:** [@sreepatch](https://discuss.elastic.co/u/sreepatch)\ **Replies:** 4\ **Last updated:** [July 27, 2017, 6:55am UTC](https://discuss.elastic.co/t/error-in-logstash-log-error-disc-quota-exceeded/94102 "2017-07-27T06:55:37Z")
i am getting Error: Disc quota exceeded", :level=\>:error in the logstash log. why this occurs and how can it be resolved? Logstash is being used to Push the data to ELastic Search. In the OuptFilter of Logstash we hav… --- ## [Plugin installation behind authenticated proxy is not working in Logstash version 5.5.0](https://discuss.elastic.co/t/plugin-installation-behind-authenticated-proxy-is-not-working-in-logstash-version-5-5-0/94632)
**Author:** [@lkpatel](https://discuss.elastic.co/u/lkpatel)\ **Replies:** 1\ **Last updated:** [July 27, 2017, 6:55am UTC](https://discuss.elastic.co/t/plugin-installation-behind-authenticated-proxy-is-not-working-in-logstash-version-5-5-0/94632 "2017-07-27T06:55:36Z")
Hi all, I am trying to install logstash-filter-plugin behind authenticated proxy and failed to do so. So far I have used following methods Setting environment variable HTTP\[S\]\_PROXY does not work with following error… --- ## [Logstash 1.5.0 email output plugin issue-connection failed](https://discuss.elastic.co/t/logstash-1-5-0-email-output-plugin-issue-connection-failed/94742)
**Author:** [@abc1](https://discuss.elastic.co/u/abc1)\ **Replies:** 0\ **Last updated:** [July 27, 2017, 5:26am UTC](https://discuss.elastic.co/t/logstash-1-5-0-email-output-plugin-issue-connection-failed/94742 "2017-07-27T05:26:15Z")
I want to send mail when particular log occurs again and again so I am trying to send email using logstash 1.5.0 when a line matches with he given grok pattern but due to connection failure i am not getting email. (I am … --- ## [Logstash output plugin solr](https://discuss.elastic.co/t/logstash-output-plugin-solr/94749)
**Author:** [@soumya1](https://discuss.elastic.co/u/soumya1)\ **Replies:** 0\ **Last updated:** [July 27, 2017, 6:25am UTC](https://discuss.elastic.co/t/logstash-output-plugin-solr/94749 "2017-07-27T06:25:56Z")
pushing the log data to solr from logstash through solr plugin worked fine.after n no of events solr didnt receive any input from logstash .logs {"BACULA\_LOG\_MAXSTART"=\>"Fatal error: Job canceled because max start delay… --- ## [Logstash to solr](https://discuss.elastic.co/t/logstash-to-solr/94745)
**Author:** [@soumya1](https://discuss.elastic.co/u/soumya1)\ **Replies:** 0\ **Last updated:** [July 27, 2017, 5:54am UTC](https://discuss.elastic.co/t/logstash-to-solr/94745 "2017-07-27T05:54:49Z")
i am pushing my application logs from logstash to solr . one layer logs am sending to one collection & the other one to other collection,i cannot see logs pushed into one layer dynamically. logstash is pushing flush onto… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=936) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=938)