# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1264 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1265 --- ## [Harvet : (offset snapshot:0) error in logstash-forwarder](https://discuss.elastic.co/t/harvet-offset-snapshot-0-error-in-logstash-forwarder/1166)
**Author:** [@sunilmchaudhari](https://discuss.elastic.co/u/sunilmchaudhari)\ **Replies:** 1\ **Last updated:** [May 24, 2015, 2:56am UTC](https://discuss.elastic.co/t/harvet-offset-snapshot-0-error-in-logstash-forwarder/1166 "2015-05-24T02:56:39Z")
Hello friends, I have just configured LSF to read some crm and sales logs under /var/log. For some old files it says Skipping file (older than dead time of 24h0m0s): which is fine. However for remaining files it says … --- ## [Logstash heap size](https://discuss.elastic.co/t/logstash-heap-size/622)
**Author:** [@it2](https://discuss.elastic.co/u/it2)\ **Replies:** 5\ **Last updated:** [May 23, 2015, 10:35am UTC](https://discuss.elastic.co/t/logstash-heap-size/622 "2015-05-23T10:35:26Z")
Hello friends! Just say, I'm not an expert in Logstash or elasticsearch. I have logstash 1.4.2 cluster, one master and two nodes for data. I have a lot of logs sending to my cluster every day. And once, logstash stopped … --- ## [New to logstash: file input and stdout output not working](https://discuss.elastic.co/t/new-to-logstash-file-input-and-stdout-output-not-working/1140)
**Author:** [@kaustabhd](https://discuss.elastic.co/u/kaustabhd)\ **Replies:** 2\ **Last updated:** [May 22, 2015, 10:26pm UTC](https://discuss.elastic.co/t/new-to-logstash-file-input-and-stdout-output-not-working/1140 "2015-05-22T22:26:50Z")
I have a logstash config: input { file { path =\> "/home/user/data/trace\_collector.log" } } filter { grok { match =\> { "message" =\> "%{GREEDYDATA: log}" } } } output { … --- ## [Getting date stamp from file properties](https://discuss.elastic.co/t/getting-date-stamp-from-file-properties/300)
**Author:** [@Andy](https://discuss.elastic.co/u/Andy)\ **Replies:** 9\ **Last updated:** [May 22, 2015, 11:56am UTC](https://discuss.elastic.co/t/getting-date-stamp-from-file-properties/300 "2015-05-22T11:56:43Z")
Hi, I'm wondering if I can get at log data in files that only have a time stamp (no date) for each line in the file. The file name contains the date that the file was created but only the day number. The file is overwr… --- ## [Logstash stops processing files after a while](https://discuss.elastic.co/t/logstash-stops-processing-files-after-a-while/944)
**Author:** [@Centopus](https://discuss.elastic.co/u/Centopus)\ **Replies:** 5\ **Last updated:** [May 22, 2015, 5:22am UTC](https://discuss.elastic.co/t/logstash-stops-processing-files-after-a-while/944 "2015-05-22T05:22:49Z")
Hello I'm having a problem while trying to parse weblogic log files, via file input plugin. After a while (took him around 20 minutes to stop reading anything with debug enabled, last time I've tried it survived 5 minu… --- ## [1.5.0 works from CLI, but not from Puppet elasticsearc/logstash module](https://discuss.elastic.co/t/1-5-0-works-from-cli-but-not-from-puppet-elasticsearc-logstash-module/873)
**Author:** [@Chris\_Neal](https://discuss.elastic.co/u/Chris_Neal)\ **Replies:** 11\ **Last updated:** [May 21, 2015, 9:25pm UTC](https://discuss.elastic.co/t/1-5-0-works-from-cli-but-not-from-puppet-elasticsearc-logstash-module/873 "2015-05-21T21:25:10Z")
Hi all. Super strange problem here. I upgraded my Logstash version that is managed by the Puppet Forge module elasticsearch/logstash from 1.5.0 beta 1 to 1.5.0-1. The uninstall of the beta worked fine. The install of… --- ## [Mixed type lumberjack {} input](https://discuss.elastic.co/t/mixed-type-lumberjack-input/1118)
**Author:** [@Oneiroi](https://discuss.elastic.co/u/Oneiroi)\ **Replies:** 0\ **Last updated:** [May 21, 2015, 4:30pm UTC](https://discuss.elastic.co/t/mixed-type-lumberjack-input/1118 "2015-05-21T16:30:53Z")
Currently have a single input { lumberjack { port =\> 5XXXX ssl\_certificate =\> "/logstash.crt" ssl\_key =\> "/logstash.key" codec =\> json } } Currently however mixed types are bei… --- ## [How could I set index & type at logstash?](https://discuss.elastic.co/t/how-could-i-set-index-type-at-logstash/985)
**Author:** [@Ducheol\_Kim](https://discuss.elastic.co/u/Ducheol_Kim)\ **Replies:** 4\ **Last updated:** [May 21, 2015, 2:29pm UTC](https://discuss.elastic.co/t/how-could-i-set-index-type-at-logstash/985 "2015-05-21T14:29:38Z")
Hi All. I'm switching current river to logstash and want to keep the same mapping that I used at river. I test logstash to set in the logstash config and it work fine.But, how I could set type ? output { elastics… --- ## [Creating a New Event and Adding Tags in a Filter Programmatically](https://discuss.elastic.co/t/creating-a-new-event-and-adding-tags-in-a-filter-programmatically/1026)
**Author:** [@jekeller](https://discuss.elastic.co/u/jekeller)\ **Replies:** 1\ **Last updated:** [May 21, 2015, 2:15pm UTC](https://discuss.elastic.co/t/creating-a-new-event-and-adding-tags-in-a-filter-programmatically/1026 "2015-05-21T14:15:56Z")
I am writing a custom filter plugin based loosely on the Throttle.rb filter. The filter needs to deduplicate some netflow data by filtering all the data related to the same flow and producing a new event that is an aggre… --- ## [Any tips for monitoring Logstash performance](https://discuss.elastic.co/t/any-tips-for-monitoring-logstash-performance/1004)
**Author:** [@madCow](https://discuss.elastic.co/u/madCow)\ **Replies:** 1\ **Last updated:** [May 20, 2015, 11:35pm UTC](https://discuss.elastic.co/t/any-tips-for-monitoring-logstash-performance/1004 "2015-05-20T23:35:13Z")
Hi, I have a simple ELK stack at present which looks to be working fine. I'm running on a single machine and getting around 60 messages every 5 mins and plan to double that soon. Can some more experienced users recomm… --- ## [The Logstash Book has been updated for Logstash v1.5.0!](https://discuss.elastic.co/t/the-logstash-book-has-been-updated-for-logstash-v1-5-0/1035)
**Author:** [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Replies:** 0\ **Last updated:** [May 20, 2015, 9:18pm UTC](https://discuss.elastic.co/t/the-logstash-book-has-been-updated-for-logstash-v1-5-0/1035 "2015-05-20T21:18:49Z")
I've copied this from the Logstash mailing list; I am pleased to announce that The Logstash Book has been updated to reflect the 1.5.0 release of Logstash. There are updates to installation, configuration, Kibana an… --- ## [Logstash 1.5 system metrics plugin](https://discuss.elastic.co/t/logstash-1-5-system-metrics-plugin/988)
**Author:** [@dimitri](https://discuss.elastic.co/u/dimitri)\ **Replies:** 2\ **Last updated:** [May 20, 2015, 6:43pm UTC](https://discuss.elastic.co/t/logstash-1-5-system-metrics-plugin/988 "2015-05-20T18:43:39Z")
Hey all, Before Logstash 1.5 i was using collectd to collect system metrics. Now in the 1.5 version of logstash it seems that this plugin is not available anymore? Is there an alternative that can be used? Or is it s… --- ## [Logstash 1.5.0 Performance Regression](https://discuss.elastic.co/t/logstash-1-5-0-performance-regression/842)
**Author:** [@Drew\_Town](https://discuss.elastic.co/u/Drew_Town)\ **Replies:** 10\ **Last updated:** [May 20, 2015, 3:42pm UTC](https://discuss.elastic.co/t/logstash-1-5-0-performance-regression/842 "2015-05-20T15:42:02Z")
Setup: Redis -\> 6 Logstash instances (json codec, date mutate) -\> elasticsearch We have 6 Logstash instances pulling from Redis with a batch size of 1,000 for very small documents \<1KB. We don't really do any process… --- ## [File output gzip configuration](https://discuss.elastic.co/t/file-output-gzip-configuration/912)
**Author:** [@kebbkh](https://discuss.elastic.co/u/kebbkh)\ **Replies:** 2\ **Last updated:** [May 20, 2015, 8:28am UTC](https://discuss.elastic.co/t/file-output-gzip-configuration/912 "2015-05-20T08:28:24Z")
Hello, I am trying to use the gzip option with the file output, however, I wanted to know how would I go about looking into the contents of the generated gzip file? file { path =\> "/throttled/output.txt" … --- ## [Recommended ELK + PHP "topology" & how to set variable type from within PHP?](https://discuss.elastic.co/t/recommended-elk-php-topology-how-to-set-variable-type-from-within-php/851)
**Author:** [@RVN\_BR](https://discuss.elastic.co/u/RVN_BR)\ **Replies:** 3\ **Last updated:** [May 20, 2015, 8:10am UTC](https://discuss.elastic.co/t/recommended-elk-php-topology-how-to-set-variable-type-from-within-php/851 "2015-05-20T08:10:59Z")
Hi, I have a two-fold question. I will try to keep it as simple as possible. I have searched a lot and found examples in both ways, but still havent found any pros/cons to be decisive? Objective: - Multi-server syste… --- ## [Grok in realtime fails but in Debuuger works happy](https://discuss.elastic.co/t/grok-in-realtime-fails-but-in-debuuger-works-happy/552)
**Author:** [@tassa](https://discuss.elastic.co/u/tassa)\ **Replies:** 14\ **Last updated:** [May 20, 2015, 5:58am UTC](https://discuss.elastic.co/t/grok-in-realtime-fails-but-in-debuuger-works-happy/552 "2015-05-20T05:58:30Z")
Hello Everyone im trying to parse my fotigate firewal logs with Grok Plese help im getting this failure message "grokparsefailuresysloginput" (kibana gui) but Grok debuggr works fine im using logstash 1.5.… --- ## [How to add field for the duration of a session?](https://discuss.elastic.co/t/how-to-add-field-for-the-duration-of-a-session/916)
**Author:** [@yeroc](https://discuss.elastic.co/u/yeroc)\ **Replies:** 3\ **Last updated:** [May 19, 2015, 8:14pm UTC](https://discuss.elastic.co/t/how-to-add-field-for-the-duration-of-a-session/916 "2015-05-19T20:14:41Z")
We have some logs where some critical information (such as username) is logged once at the beginning of a user's session (these are web applicaiton logs) and we'd like to associate this information in a field for all the… --- ## [Tutorial for creating custom filter](https://discuss.elastic.co/t/tutorial-for-creating-custom-filter/922)
**Author:** [@Ducheol\_Kim](https://discuss.elastic.co/u/Ducheol_Kim)\ **Replies:** 1\ **Last updated:** [May 19, 2015, 7:57pm UTC](https://discuss.elastic.co/t/tutorial-for-creating-custom-filter/922 "2015-05-19T19:57:50Z")
Hi All. I'm new guys in logstash. I briefly understand what is logstash and need to create my own filter. Below is my situation. We have own solution and keep the data in Database. We built our 1st elastic server … --- ## [Upgraded to 1.5.0, config check throwing errors](https://discuss.elastic.co/t/upgraded-to-1-5-0-config-check-throwing-errors/820)
**Author:** [@matejzero](https://discuss.elastic.co/u/matejzero)\ **Replies:** 2\ **Last updated:** [May 19, 2015, 7:15pm UTC](https://discuss.elastic.co/t/upgraded-to-1-5-0-config-check-throwing-errors/820 "2015-05-19T19:15:43Z")
Hello! Today I tried to upgrade from 1.4.2 to 1.5.0 on one of my nodes, but config check throws error right away: SyntaxError: (eval):4072: syntax error, unexpected kNOT "grokpostfix\_success" not in \[tags\] an… --- ## [How is log storage working?](https://discuss.elastic.co/t/how-is-log-storage-working/913)
**Author:** [@Yzord](https://discuss.elastic.co/u/Yzord)\ **Replies:** 1\ **Last updated:** [May 19, 2015, 5:15pm UTC](https://discuss.elastic.co/t/how-is-log-storage-working/913 "2015-05-19T17:15:50Z")
I have an ELK server where all my other (50+) servers are sending their logs to it. It's a virtual machine and i gave it at first 60GB of storage which i divided. I monitor this server and i received a warning that my … --- ## [Problem with config?](https://discuss.elastic.co/t/problem-with-config/902)
**Author:** [@lesurfl](https://discuss.elastic.co/u/lesurfl)\ **Replies:** 2\ **Last updated:** [May 19, 2015, 2:10pm UTC](https://discuss.elastic.co/t/problem-with-config/902 "2015-05-19T14:10:12Z")
Hi, I've been testing out some grok patterns and I thought I had everything working but in production I get the following output: +---------------------------------------------------------+ | An unexpected error occu… --- ## [How can I make iteration & loop in logstash?](https://discuss.elastic.co/t/how-can-i-make-iteration-loop-in-logstash/891)
**Author:** [@kimjmin](https://discuss.elastic.co/u/kimjmin)\ **Replies:** 2\ **Last updated:** [May 19, 2015, 12:12pm UTC](https://discuss.elastic.co/t/how-can-i-make-iteration-loop-in-logstash/891 "2015-05-19T12:12:40Z")
Hi, I have string data looks like next. "detail":"B01=10,B02=10,BANK02=10,BANK01=50,F04=10" I want parse the value of "detail" field. First I split with character "," "detail":\["B01=10","B02=10","BANK02=10","BANK01… --- ## [I cant make work multi grok filter for a file type](https://discuss.elastic.co/t/i-cant-make-work-multi-grok-filter-for-a-file-type/852)
**Author:** [@Raphaelk](https://discuss.elastic.co/u/Raphaelk)\ **Replies:** 2\ **Last updated:** [May 19, 2015, 7:18am UTC](https://discuss.elastic.co/t/i-cant-make-work-multi-grok-filter-for-a-file-type/852 "2015-05-19T07:18:11Z")
Hi, I'm on my way to parse Cisco PIX 515 Log. In the log file, there is 3 kind of line I would like to parse. My grok filters are working individually. I tested them with the grokdebugger. But as soon as I put them toge… --- ## [Getting external values into logstash; environment filter does not seen to work](https://discuss.elastic.co/t/getting-external-values-into-logstash-environment-filter-does-not-seen-to-work/868)
**Author:** [@ptrei](https://discuss.elastic.co/u/ptrei)\ **Replies:** 1\ **Last updated:** [May 19, 2015, 5:39am UTC](https://discuss.elastic.co/t/getting-external-values-into-logstash-environment-filter-does-not-seen-to-work/868 "2015-05-19T05:39:05Z")
I have a compressed log files, and am piping them into LS using stdin: bzip2 -k -d --stdout logfile.bz2 | logstash -f myconfig.conf This works, but I'd really like to record in each parsed record in Elasticsearch the f… --- ## [Logstash Forwarder Error Log](https://discuss.elastic.co/t/logstash-forwarder-error-log/662)
**Author:** [@SJ04](https://discuss.elastic.co/u/SJ04)\ **Replies:** 6\ **Last updated:** [May 18, 2015, 6:36am UTC](https://discuss.elastic.co/t/logstash-forwarder-error-log/662 "2015-05-18T06:36:59Z")
Hi there, Thanks a lot in advance. I have configured the Logstash Forwarder on the cent os System but when i run the Logstash-forwarded . i get this error message in the Error Log OF logstash Forwarder. 2015/05/1… --- ## [Metrics filter suggestions](https://discuss.elastic.co/t/metrics-filter-suggestions/817)
**Author:** [@allenmchan](https://discuss.elastic.co/u/allenmchan)\ **Replies:** 0\ **Last updated:** [May 18, 2015, 2:38am UTC](https://discuss.elastic.co/t/metrics-filter-suggestions/817 "2015-05-18T02:38:10Z")
Until https://github.com/elastic/logstash/issues/2611 is resolved, i would like to use the metrics filter to push to ES the amount of syslog / lumberjack events processed by each of the logstash instances. Does anyone… --- ## [Logstash repos for 1.5](https://discuss.elastic.co/t/logstash-repos-for-1-5/714)
**Author:** [@allenmchan](https://discuss.elastic.co/u/allenmchan)\ **Replies:** 3\ **Last updated:** [May 18, 2015, 2:36am UTC](https://discuss.elastic.co/t/logstash-repos-for-1-5/714 "2015-05-18T02:36:13Z")
This page is still referring to 1.4 release for the repo http://www.elastic.co/guide/en/logstash/current/package-repositories.html Can you point me to the 1.5 repo config? --- ## [Error: Cannot use more than 1 filter worker](https://discuss.elastic.co/t/error-cannot-use-more-than-1-filter-worker/782)
**Author:** [@allenmchan](https://discuss.elastic.co/u/allenmchan)\ **Replies:** 2\ **Last updated:** [May 18, 2015, 2:28am UTC](https://discuss.elastic.co/t/error-cannot-use-more-than-1-filter-worker/782 "2015-05-18T02:28:17Z")
Has anyone seen this error before? First time i saw it on 1.5 GA "Error: Cannot use more than 1 filter worker because the following plugins don't work with more than one worker: multiline, multiline, multiline" --- ## [Elasticsearch Output and the raw field](https://discuss.elastic.co/t/elasticsearch-output-and-the-raw-field/754)
**Author:** [@elvarb](https://discuss.elastic.co/u/elvarb)\ **Replies:** 1\ **Last updated:** [May 15, 2015, 10:11pm UTC](https://discuss.elastic.co/t/elasticsearch-output-and-the-raw-field/754 "2015-05-15T22:11:03Z")
Now that the elasticsearch output no longer gives you a message.raw field by default From the changelog Logstash does not create a "message.raw" by default whic is usually not\_analyzed; this helps save disk space (#… --- ## [Simple file uploading using Logstash](https://discuss.elastic.co/t/simple-file-uploading-using-logstash/724)
**Author:** [@hemanth](https://discuss.elastic.co/u/hemanth)\ **Replies:** 3\ **Last updated:** [May 15, 2015, 11:50am UTC](https://discuss.elastic.co/t/simple-file-uploading-using-logstash/724 "2015-05-15T11:50:42Z")
Hi, I have a requirement to upload files from Logstash machine to a remote machine, without any processing. Input is a file, no filtering required, output is upload entire file to remote machine. Can I do that ? --He… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1263) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1265)