# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1086 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1087 --- ## [\[Solved\] Use grok'd timestamp across multiple messages](https://discuss.elastic.co/t/solved-use-grokd-timestamp-across-multiple-messages/64220)
**Author:** [@AtTheControls](https://discuss.elastic.co/u/AtTheControls)\ **Replies:** 4\ **Last updated:** [October 28, 2016, 3:49pm UTC](https://discuss.elastic.co/t/solved-use-grokd-timestamp-across-multiple-messages/64220 "2016-10-28T15:49:22Z")
Hello, I am currently building a pipeline to parse iostat data and send it to elasticsearch. I am using grok to find the lines I am interested. Some lines contain the timestamp, the other lines the device information f… --- ## [PeriodicPoller TimeoutError after upgrading to Logstash 5.0](https://discuss.elastic.co/t/periodicpoller-timeouterror-after-upgrading-to-logstash-5-0/64306)
**Author:** [@sjivan](https://discuss.elastic.co/u/sjivan)\ **Replies:** 0\ **Last updated:** [October 28, 2016, 2:52pm UTC](https://discuss.elastic.co/t/periodicpoller-timeouterror-after-upgrading-to-logstash-5-0/64306 "2016-10-28T14:52:56Z")
Hi, I upgraded from Logstash 2.2.2 to Logstash 5.0 and have this ERROR message outputted every 3-10 seconds. I did not encounter such errors with LS 2.2.2. I'm running on windows and my configuration has file input conf… --- ## [Creating single instance of Logstash process](https://discuss.elastic.co/t/creating-single-instance-of-logstash-process/60514)
**Author:** [@Sumantha\_Kannantha](https://discuss.elastic.co/u/Sumantha_Kannantha)\ **Replies:** 1\ **Last updated:** [September 15, 2016, 11:05am UTC](https://discuss.elastic.co/t/creating-single-instance-of-logstash-process/60514 "2016-09-15T11:05:53Z")
We use exec input pug-in to execute external command/script to collect custom monitors. These commands are expensive and want to run it only on one of the Logstash instance. For HA, we want another Logstash instance t… --- ## [Log4net parsing with Logstash](https://discuss.elastic.co/t/log4net-parsing-with-logstash/64281)
**Author:** [@mmehraban](https://discuss.elastic.co/u/mmehraban)\ **Replies:** 0\ **Last updated:** [October 28, 2016, 12:46pm UTC](https://discuss.elastic.co/t/log4net-parsing-with-logstash/64281 "2016-10-28T12:46:50Z")
I want to pars log4net file which I put a part of the log file here :slight\_smile: 2016-10-17 03:20:37,690 \[319299\] ERROR Global - Unhandled application error System.ArgumentOutOfRangeException: Specified argument was… --- ## [Having problem to load a text file that contain DATE fields using logstash](https://discuss.elastic.co/t/having-problem-to-load-a-text-file-that-contain-date-fields-using-logstash/63668)
**Author:** [@yoavb](https://discuss.elastic.co/u/yoavb)\ **Replies:** 3\ **Last updated:** [October 28, 2016, 11:09am UTC](https://discuss.elastic.co/t/having-problem-to-load-a-text-file-that-contain-date-fields-using-logstash/63668 "2016-10-28T11:09:04Z")
Hi, I am totaly new to the ELK world. I have tried to load a text file that contain several date fields using logstash. I have tried to configure the field START\_DATE and END\_DATE to use the following format : DD/MM/YY… --- ## [How to config logstash 5?](https://discuss.elastic.co/t/how-to-config-logstash-5/64245)
**Author:** [@Me\_Cloud](https://discuss.elastic.co/u/Me_Cloud)\ **Replies:** 4\ **Last updated:** [October 28, 2016, 10:48am UTC](https://discuss.elastic.co/t/how-to-config-logstash-5/64245 "2016-10-28T10:48:06Z")
Guys please help me, how to config in logstash 5? im new for this --- ## [\[Solved\] Logstash-input-rabbitmq keeps redelivering msg](https://discuss.elastic.co/t/solved-logstash-input-rabbitmq-keeps-redelivering-msg/64253)
**Author:** [@stachdude](https://discuss.elastic.co/u/stachdude)\ **Replies:** 1\ **Last updated:** [October 28, 2016, 8:58am UTC](https://discuss.elastic.co/t/solved-logstash-input-rabbitmq-keeps-redelivering-msg/64253 "2016-10-28T08:58:07Z")
Hi, I'm having this strange issue, where every msg I consume gets redelivered to input queue. Logstah versions I've tried = 2.3, 2.4, 5 GA I've tried every possible setting, and the only difference form every doc I… --- ## [\[Solved\] Environment variable references doesn't seem to work in output](https://discuss.elastic.co/t/solved-environment-variable-references-doesnt-seem-to-work-in-output/63005)
**Author:** [@stachdude](https://discuss.elastic.co/u/stachdude)\ **Replies:** 2\ **Last updated:** [October 28, 2016, 6:30am UTC](https://discuss.elastic.co/t/solved-environment-variable-references-doesnt-seem-to-work-in-output/63005 "2016-10-28T06:30:45Z")
Using 2.4.0 version of Logstash, I ve tried to use environment variable as index name setting of an elasticsearch output plugin as so: I set env var : export ES\_IDX="test" output part of config file output { elasti… --- ## [How to install logstash 5 in Centos 6 X64\_86](https://discuss.elastic.co/t/how-to-install-logstash-5-in-centos-6-x64-86/64225)
**Author:** [@xiehaitang](https://discuss.elastic.co/u/xiehaitang)\ **Replies:** 3\ **Last updated:** [October 28, 2016, 2:27am UTC](https://discuss.elastic.co/t/how-to-install-logstash-5-in-centos-6-x64-86/64225 "2016-10-28T02:27:42Z")
How to install logstash 5 in Centos 6 X64\_86 --- ## [How many ESClients built by logstash-output-elasticsearch in 5.0?](https://discuss.elastic.co/t/how-many-esclients-built-by-logstash-output-elasticsearch-in-5-0/64221)
**Author:** [@chenryn](https://discuss.elastic.co/u/chenryn)\ **Replies:** 0\ **Last updated:** [October 28, 2016, 12:33am UTC](https://discuss.elastic.co/t/how-many-esclients-built-by-logstash-output-elasticsearch-in-5-0/64221 "2016-10-28T00:33:18Z")
I remember that logstash-output-elasticsearch would built lots of ES clients in 2.2/2.3 NG pipeline, if we set the pipeline-workers a bit larger. This cause the indexing performance decreasing. I haven't see any progre… --- ## [HTTP\_POLLER Multiple urls (SOLVED) there was a bad URL](https://discuss.elastic.co/t/http-poller-multiple-urls-solved-there-was-a-bad-url/64195)
**Author:** [@eperry](https://discuss.elastic.co/u/eperry)\ **Replies:** 1\ **Last updated:** [October 27, 2016, 8:14pm UTC](https://discuss.elastic.co/t/http-poller-multiple-urls-solved-there-was-a-bad-url/64195 "2016-10-27T20:14:30Z")
Oh Logstash 2.4 this Syntax works "--configtest" , Gives me a thumbs up "OK" but when I run it it but when I run it I get the following. ..... ALSO If I comment out/remote either entry it works fine. Set… --- ## [StructuredArgument keyValue with object that has no json serializer stops logback appender entirely](https://discuss.elastic.co/t/structuredargument-keyvalue-with-object-that-has-no-json-serializer-stops-logback-appender-entirely/63922)
**Author:** [@naisbitt](https://discuss.elastic.co/u/naisbitt)\ **Replies:** 0\ **Last updated:** [October 25, 2016, 7:07pm UTC](https://discuss.elastic.co/t/structuredargument-keyvalue-with-object-that-has-no-json-serializer-stops-logback-appender-entirely/63922 "2016-10-25T19:07:39Z")
If you have a log statement that tries to create a keyValue with an object that is not JSON-serializable, the entire log appender is stopped. This seems like a pretty big flaw. One erroneous log statement should not be… --- ## [Replacing @timestamp with a custom field does not occur](https://discuss.elastic.co/t/replacing-timestamp-with-a-custom-field-does-not-occur/63627)
**Author:** [@the\_iobender](https://discuss.elastic.co/u/the_iobender)\ **Replies:** 6\ **Last updated:** [October 27, 2016, 3:20pm UTC](https://discuss.elastic.co/t/replacing-timestamp-with-a-custom-field-does-not-occur/63627 "2016-10-27T15:20:59Z")
Hello, I am currently using the HTTP-IN plugin to receive HTTP PUT requests from various systems. The systems have a field called CreationTime which I am trying to use instead of the @timestamp field. I have read few … --- ## [Multi-worker plugins](https://discuss.elastic.co/t/multi-worker-plugins/63962)
**Author:** [@dawiro](https://discuss.elastic.co/u/dawiro)\ **Replies:** 1\ **Last updated:** [October 27, 2016, 1:46pm UTC](https://discuss.elastic.co/t/multi-worker-plugins/63962 "2016-10-27T13:46:30Z")
Hi, Is there a list anywhere of plugins which are compatible and will become compatible with the multi-worker model used in logstash 2.3.x and later? Regards, David --- ## [Create a new index in elasticsearch for each log file by date](https://discuss.elastic.co/t/create-a-new-index-in-elasticsearch-for-each-log-file-by-date/63607)
**Author:** [@adhikz](https://discuss.elastic.co/u/adhikz)\ **Replies:** 7\ **Last updated:** [October 27, 2016, 1:38pm UTC](https://discuss.elastic.co/t/create-a-new-index-in-elasticsearch-for-each-log-file-by-date/63607 "2016-10-27T13:38:32Z")
Hi, I have managed to get my ELK stack up and ruining which filters logs using grok and passes data to elasticsearch and allows me to visualize in Kibana. Currently I have completed the above task by using one log fi… --- ## [Installed the Java JRE - now recieving errors](https://discuss.elastic.co/t/installed-the-java-jre-now-recieving-errors/64138)
**Author:** [@saunderstd](https://discuss.elastic.co/u/saunderstd)\ **Replies:** 1\ **Last updated:** [October 27, 2016, 12:30pm UTC](https://discuss.elastic.co/t/installed-the-java-jre-now-recieving-errors/64138 "2016-10-27T12:30:25Z")
Morning, I am having a issue. I installed JRE on the system and now I am having this error thrown when performing service logstash configtest; service logstash configtest The given configuration is invalid. Reason: Exp… --- ## [Filebeats/Logstash compatibility](https://discuss.elastic.co/t/filebeats-logstash-compatibility/64130)
**Author:** [@danieljamesscott](https://discuss.elastic.co/u/danieljamesscott)\ **Replies:** 2\ **Last updated:** [October 27, 2016, 10:56am UTC](https://discuss.elastic.co/t/filebeats-logstash-compatibility/64130 "2016-10-27T10:56:34Z")
Is there any documentation on version compatibility between LogStash and Filebeat for upgrading? For example, I have a currently running system using logstash 2.3, and Filebeat 1.2. How should I proceed with an upgra… --- ## [How do I decrypt logs encrypted in PBEWithMD5AndDES in logstash?](https://discuss.elastic.co/t/how-do-i-decrypt-logs-encrypted-in-pbewithmd5anddes-in-logstash/64122)
**Author:** [@malik](https://discuss.elastic.co/u/malik)\ **Replies:** 0\ **Last updated:** [October 27, 2016, 9:52am UTC](https://discuss.elastic.co/t/how-do-i-decrypt-logs-encrypted-in-pbewithmd5anddes-in-logstash/64122 "2016-10-27T09:52:52Z")
I have logs which are encrypted using Java jasypt-1.92 in PBEWithMD5AndDES algorithm. What I need is to decrypt these logs in Logstash before sending them to elasticsearch/filters-grok. While going through logstash doc… --- ## [What's apt repository for logstash 5.0](https://discuss.elastic.co/t/whats-apt-repository-for-logstash-5-0/64120)
**Author:** [@linlynn](https://discuss.elastic.co/u/linlynn)\ **Replies:** 0\ **Last updated:** [October 27, 2016, 9:42am UTC](https://discuss.elastic.co/t/whats-apt-repository-for-logstash-5-0/64120 "2016-10-27T09:42:00Z")
per this document https://www.elastic.co/guide/en/logstash/current/installing-logstash.html ,it looks like the repository still points 5.0 rc release --- ## [Pattern error - but which pattern? (logstash 2.4.0) (SOLVED)](https://discuss.elastic.co/t/pattern-error-but-which-pattern-logstash-2-4-0-solved/63861)
**Author:** [@al\_form2000](https://discuss.elastic.co/u/al_form2000)\ **Replies:** 3\ **Last updated:** [October 27, 2016, 9:22am UTC](https://discuss.elastic.co/t/pattern-error-but-which-pattern-logstash-2-4-0-solved/63861 "2016-10-27T09:22:45Z")
Greetings. I am trying (and failing) in getting custom patterns to work with logstash 2.4.0. Here is the relevant part of the conf file: #some parsing happens above... grok { patterns\_dir =\> \["/config\_dir/… --- ## [Does Logstash process single file or multiple files simultaneously?](https://discuss.elastic.co/t/does-logstash-process-single-file-or-multiple-files-simultaneously/64108)
**Author:** [@Mayank\_Agrawal](https://discuss.elastic.co/u/Mayank_Agrawal)\ **Replies:** 9\ **Last updated:** [October 27, 2016, 8:57am UTC](https://discuss.elastic.co/t/does-logstash-process-single-file-or-multiple-files-simultaneously/64108 "2016-10-27T08:57:15Z")
How does the logstash process multiple log files ??? What I mean is, does the logstash processes one file at a time (and then move on to another file) or multiple files simultaneously ??? --- ## [Logstash Forwarder not supported on AIX - what now?!](https://discuss.elastic.co/t/logstash-forwarder-not-supported-on-aix-what-now/33305)
**Author:** [@jjdepaul](https://discuss.elastic.co/u/jjdepaul)\ **Replies:** 12\ **Last updated:** [October 27, 2016, 8:50am UTC](https://discuss.elastic.co/t/logstash-forwarder-not-supported-on-aix-what-now/33305 "2016-10-27T08:50:06Z")
I just found out that Logstash Forwarder is not supported on the AIX platform - currently that's our application platform from which we need to source our logs and send them to the ELK stack on a Linux box. Our applica… --- ## [Logstash syslog output to Arcsight](https://discuss.elastic.co/t/logstash-syslog-output-to-arcsight/64084)
**Author:** [@s802645](https://discuss.elastic.co/u/s802645)\ **Replies:** 3\ **Last updated:** [October 27, 2016, 8:31am UTC](https://discuss.elastic.co/t/logstash-syslog-output-to-arcsight/64084 "2016-10-27T08:31:35Z")
We are using LS to collect syslogs from a firewall and forwarding to siem. this is our config syslog { appname =\> "" host =\> "x.x.x.x" port =\> "5… --- ## [Reversing of src/dst ip and port in cisco fwpattern file](https://discuss.elastic.co/t/reversing-of-src-dst-ip-and-port-in-cisco-fwpattern-file/64106)
**Author:** [@ssi](https://discuss.elastic.co/u/ssi)\ **Replies:** 0\ **Last updated:** [October 27, 2016, 7:44am UTC](https://discuss.elastic.co/t/reversing-of-src-dst-ip-and-port-in-cisco-fwpattern-file/64106 "2016-10-27T07:44:52Z")
Hi I've noticed what i consider an error in the logstash firewall pattern related to cisco asa. the syslogID in the cisco firewall raw log looks like this with respective relevant fields. We are testing opendns so DNS … --- ## [Delay in creation of .sincedb file](https://discuss.elastic.co/t/delay-in-creation-of-sincedb-file/64099)
**Author:** [@Mayank\_Agrawal](https://discuss.elastic.co/u/Mayank_Agrawal)\ **Replies:** 0\ **Last updated:** [October 27, 2016, 6:03am UTC](https://discuss.elastic.co/t/delay-in-creation-of-sincedb-file/64099 "2016-10-27T06:03:59Z")
To know logstash reading progress, I am monitoring .sincedb files. The issue is for large files (say 100-110 MB), .sincedb file is not being created within 0-10 minutes (it takes much more time to create it). I have trie… --- ## [Query on filter plugin](https://discuss.elastic.co/t/query-on-filter-plugin/64097)
**Author:** [@shivabharat](https://discuss.elastic.co/u/shivabharat)\ **Replies:** 1\ **Last updated:** [October 27, 2016, 6:14am UTC](https://discuss.elastic.co/t/query-on-filter-plugin/64097 "2016-10-27T06:14:19Z")
I have log file which dumps data as single entry . The fields are of fixed length. I am not sure which filter plugin to use to parse the file. Any pointers would be helpful 3004121506000299···························10… --- ## [Is it possible to create multiple index in single config file with different input locations?](https://discuss.elastic.co/t/is-it-possible-to-create-multiple-index-in-single-config-file-with-different-input-locations/64062)
**Author:** [@asac2](https://discuss.elastic.co/u/asac2)\ **Replies:** 2\ **Last updated:** [October 26, 2016, 11:34pm UTC](https://discuss.elastic.co/t/is-it-possible-to-create-multiple-index-in-single-config-file-with-different-input-locations/64062 "2016-10-26T23:34:11Z")
Is it possible to create multiple index in single config file with different input locations? --- ## [Check if csv parse failed/ Change record type](https://discuss.elastic.co/t/check-if-csv-parse-failed-change-record-type/64066)
**Author:** [@Jimmy\_Shephard](https://discuss.elastic.co/u/Jimmy_Shephard)\ **Replies:** 1\ **Last updated:** [October 26, 2016, 11:23pm UTC](https://discuss.elastic.co/t/check-if-csv-parse-failed-change-record-type/64066 "2016-10-26T23:23:38Z")
Hi, I need logic in my file input filter such that if a csv parse fails I can catch that and change the record type. This is the structure I have filter{ csv { add\_tag = \> 'OK' } //How would I check if tag above… --- ## [Conditional apply of filter](https://discuss.elastic.co/t/conditional-apply-of-filter/63537)
**Author:** [@Jimmy\_Shephard](https://discuss.elastic.co/u/Jimmy_Shephard)\ **Replies:** 3\ **Last updated:** [October 26, 2016, 10:34pm UTC](https://discuss.elastic.co/t/conditional-apply-of-filter/63537 "2016-10-26T22:34:00Z")
Hi, Trying to parse different kinds of log files and output to ES but seems like the condition gets ingnored by logsash when applying filters and I get some error. Any ideas what could be going wrong? this is the str… --- ## [Below logstash file is not creating the index with the name i specified](https://discuss.elastic.co/t/below-logstash-file-is-not-creating-the-index-with-the-name-i-specified/63986)
**Author:** [@asac2](https://discuss.elastic.co/u/asac2)\ **Replies:** 6\ **Last updated:** [October 26, 2016, 8:35pm UTC](https://discuss.elastic.co/t/below-logstash-file-is-not-creating-the-index-with-the-name-i-specified/63986 "2016-10-26T20:35:22Z")
input { stdin { type =\> "stdin-type" } file { path =\> "D:/Attachments/plunk-results.csv" start\_position =\> "beginning" } } filter { csv { columns =\> \["IHS\_URL","count","90Percentile","Average","Minimum","Maximum"\] sepa… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1085) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1087)