# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1121 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1122 --- ## [Configure log file, into json format](https://discuss.elastic.co/t/configure-log-file-into-json-format/56554)
**Author:** [@asoong-94](https://discuss.elastic.co/u/asoong-94)\ **Replies:** 5\ **Last updated:** [August 1, 2016, 5:13pm UTC](https://discuss.elastic.co/t/configure-log-file-into-json-format/56554 "2016-08-01T17:13:23Z")
an example of my current log files are: Timestamp: 7/15/2016 Message: HandlingInstanceID: 12345 Type: ErrorMessage MachineName: mymachine is there anyway to configure my configuration file so that when using the jso… --- ## [Logstash re-open logfile with SIGHUP](https://discuss.elastic.co/t/logstash-re-open-logfile-with-sighup/56904)
**Author:** [@mbaranski](https://discuss.elastic.co/u/mbaranski)\ **Replies:** 1\ **Last updated:** [August 1, 2016, 5:07pm UTC](https://discuss.elastic.co/t/logstash-re-open-logfile-with-sighup/56904 "2016-08-01T17:07:27Z")
I'm running logstash via the apt installed package on Ubuntu. I am sending a SIGHUP to the process and it's not re-opening its logfiles. dpkg -l | grep logstash ii logstash 1:2.3.4… --- ## [Logstash read new data in elasticsearch as input?](https://discuss.elastic.co/t/logstash-read-new-data-in-elasticsearch-as-input/56541)
**Author:** [@sen](https://discuss.elastic.co/u/sen)\ **Replies:** 7\ **Last updated:** [August 1, 2016, 3:36pm UTC](https://discuss.elastic.co/t/logstash-read-new-data-in-elasticsearch-as-input/56541 "2016-08-01T15:36:08Z")
Hello folks, I am using elasticsearch as an input of logstash. My script works well. However, when I insert new data in elasticsearch, after I run logstash, these new data are not read by logstash. I know that Elastics… --- ## [Best way to develop ruby scripts for logstash?](https://discuss.elastic.co/t/best-way-to-develop-ruby-scripts-for-logstash/56867)
**Author:** [@asp](https://discuss.elastic.co/u/asp)\ **Replies:** 1\ **Last updated:** [August 1, 2016, 3:18pm UTC](https://discuss.elastic.co/t/best-way-to-develop-ruby-scripts-for-logstash/56867 "2016-08-01T15:18:23Z")
Hi all, now we are at the point, where simple grok parsing of our logs it not enough. We need to do further aggregations and calculations. I decided to use the ruby filter plugin for that. Also the ruby filter is direc… --- ## [How to create Index Every five 5 Minute](https://discuss.elastic.co/t/how-to-create-index-every-five-5-minute/56796)
**Author:** [@sumit\_gupta\_sgt](https://discuss.elastic.co/u/sumit_gupta_sgt)\ **Replies:** 8\ **Last updated:** [August 1, 2016, 2:08pm UTC](https://discuss.elastic.co/t/how-to-create-index-every-five-5-minute/56796 "2016-08-01T14:08:51Z")
Hi All, Basically I am pulling data from SQL Server into ES.I am able to create Index daily, hourly, minute, second etc ..but i want to create every 5 minute new index ..anybody knows how i can do this? Is there any … --- ## [RabbitMQ output configuration - alternate-exchange - help needed](https://discuss.elastic.co/t/rabbitmq-output-configuration-alternate-exchange-help-needed/56601)
**Author:** [@vandamo](https://discuss.elastic.co/u/vandamo)\ **Replies:** 2\ **Last updated:** [August 1, 2016, 6:21am UTC](https://discuss.elastic.co/t/rabbitmq-output-configuration-alternate-exchange-help-needed/56601 "2016-08-01T06:21:39Z")
Hi, I'm trying out RabbitMQ (instead of Redis), and am typically creating the exchanges, bindings, queues in advance, then configuring logstash to match the existing config. There is one configuration I just tried now … --- ## [Logstash index is not creating](https://discuss.elastic.co/t/logstash-index-is-not-creating/56276)
**Author:** [@ramanamohan](https://discuss.elastic.co/u/ramanamohan)\ **Replies:** 16\ **Last updated:** [August 1, 2016, 9:44am UTC](https://discuss.elastic.co/t/logstash-index-is-not-creating/56276 "2016-08-01T09:44:30Z")
Hi, I am unable to get the Logstash index created with my Logstash conf file. Below is the configuration file, input { file { path =\> "/logs/exampledata\_01.log" type =\> "json" start\_position =\> beginning ign… --- ## [Logstash As a Service](https://discuss.elastic.co/t/logstash-as-a-service/56851)
**Author:** [@vikas.saini](https://discuss.elastic.co/u/vikas.saini)\ **Replies:** 2\ **Last updated:** [August 1, 2016, 7:44am UTC](https://discuss.elastic.co/t/logstash-as-a-service/56851 "2016-08-01T07:44:09Z")
How can I install logstash as a service without using NSSM . --- ## [When Logstash 5 stable version will be released](https://discuss.elastic.co/t/when-logstash-5-stable-version-will-be-released/56622)
**Author:** [@georgiana.ciuchi](https://discuss.elastic.co/u/georgiana.ciuchi)\ **Replies:** 1\ **Last updated:** [July 31, 2016, 8:27pm UTC](https://discuss.elastic.co/t/when-logstash-5-stable-version-will-be-released/56622 "2016-07-31T20:27:34Z")
Hello, having a look to the latest version for Logstash 5 and I have found out alpha-4 released on 30 June 2016. Do you have an idea when a stable release will be provided? Thanks a bunch --- ## [Force LogStash to read End of File?](https://discuss.elastic.co/t/force-logstash-to-read-end-of-file/56613)
**Author:** [@londonx](https://discuss.elastic.co/u/londonx)\ **Replies:** 1\ **Last updated:** [July 31, 2016, 8:26pm UTC](https://discuss.elastic.co/t/force-logstash-to-read-end-of-file/56613 "2016-07-31T20:26:48Z")
Hi was wondering if anyone had a cunning way to force LogStash to always read the end of a file , no need for a sincedb\_path file etc whether the logs are new or not after say a Logstash restart? The only current out-o… --- ## [Logstash pattern for repeated custom patterns](https://discuss.elastic.co/t/logstash-pattern-for-repeated-custom-patterns/56821)
**Author:** [@ishanim](https://discuss.elastic.co/u/ishanim)\ **Replies:** 0\ **Last updated:** [July 31, 2016, 12:28pm UTC](https://discuss.elastic.co/t/logstash-pattern-for-repeated-custom-patterns/56821 "2016-07-31T12:28:30Z")
I have the following log: hostname#show platform software status control-processor brief Load Average Slot Status 1-Min 5-Min 15-Min RP0 Healthy 0.21 0.11 0.08 RP1 Healthy 0.08 0.06 0.05 ESP0 Health… --- ## [\_grokparsefailure also after Grok Constructor](https://discuss.elastic.co/t/-grokparsefailure-also-after-grok-constructor/56178)
**Author:** [@Hans](https://discuss.elastic.co/u/Hans)\ **Replies:** 6\ **Last updated:** [July 31, 2016, 6:37am UTC](https://discuss.elastic.co/t/-grokparsefailure-also-after-grok-constructor/56178 "2016-07-31T06:37:30Z")
HI, I have a challenge that I have been trying to resolve now for some time however not making positive progress. I have logs that I would like to filter and did Incremental Grok Construct and all is matching, however wh… --- ## [Timestamp Generation Query](https://discuss.elastic.co/t/timestamp-generation-query/56741)
**Author:** [@dawiro](https://discuss.elastic.co/u/dawiro)\ **Replies:** 1\ **Last updated:** [July 31, 2016, 12:26am UTC](https://discuss.elastic.co/t/timestamp-generation-query/56741 "2016-07-31T00:26:27Z")
Hi, When our logs hit logstash they already include an @timestamp field containing the timestamp of the log line. I also want to generate a timestamp to mark when the message was indexed... How do I generate a timestam… --- ## [Logstash in RH setup documentation from tar](https://discuss.elastic.co/t/logstash-in-rh-setup-documentation-from-tar/56736)
**Author:** [@justx](https://discuss.elastic.co/u/justx)\ **Replies:** 1\ **Last updated:** [July 31, 2016, 12:23am UTC](https://discuss.elastic.co/t/logstash-in-rh-setup-documentation-from-tar/56736 "2016-07-31T00:23:21Z")
I am having quite a bit of trouble locating information on installing Logstash 2.3.2 on RH using the tar file. I do not have the option to use a rpm. One of the largest issues I have current is the init.d scripts and run… --- ## [How to process messages strictly in the order they arrive?](https://discuss.elastic.co/t/how-to-process-messages-strictly-in-the-order-they-arrive/56790)
**Author:** [@peter\_rietzler](https://discuss.elastic.co/u/peter_rietzler)\ **Replies:** 4\ **Last updated:** [July 30, 2016, 4:45pm UTC](https://discuss.elastic.co/t/how-to-process-messages-strictly-in-the-order-they-arrive/56790 "2016-07-30T16:45:47Z")
Hi everybody. I've got an inputfile that contains data in the following format (one json object per line) {id: "1", status: "Running"} {id: "1", status: "Finished"} I'm shipping this via Filebeat to Logstash and then … --- ## [Rspec Testing for grok pattern filers in Logstash configuration](https://discuss.elastic.co/t/rspec-testing-for-grok-pattern-filers-in-logstash-configuration/56784)
**Author:** [@vijaycaws](https://discuss.elastic.co/u/vijaycaws)\ **Replies:** 1\ **Last updated:** [July 30, 2016, 12:21pm UTC](https://discuss.elastic.co/t/rspec-testing-for-grok-pattern-filers-in-logstash-configuration/56784 "2016-07-30T12:21:22Z")
I am new to ELK, Is there any one who has experience with testing of Logstash config with rspec...please let me know, it would be a great help...Thank you --- ## [Testing log stash filter against data](https://discuss.elastic.co/t/testing-log-stash-filter-against-data/56325)
**Author:** [@141984](https://discuss.elastic.co/u/141984)\ **Replies:** 3\ **Last updated:** [July 30, 2016, 12:18pm UTC](https://discuss.elastic.co/t/testing-log-stash-filter-against-data/56325 "2016-07-30T12:18:41Z")
Hi All, When i try to test a log stash filter, against some data, like below /opt/logstash/bin/logstash -f /etc/logstash/conf.d/11-tomcat-filter.conf \< /home/gibu\_test/data4 All i get in the output is: Setting… --- ## [Logstash Field Types Not Working Intermittently](https://discuss.elastic.co/t/logstash-field-types-not-working-intermittently/56490)
**Author:** [@JellyPanda](https://discuss.elastic.co/u/JellyPanda)\ **Replies:** 4\ **Last updated:** [July 30, 2016, 12:09pm UTC](https://discuss.elastic.co/t/logstash-field-types-not-working-intermittently/56490 "2016-07-30T12:09:11Z")
Hi Guys, I have seen this type of issue once or twice before but i can't seem to get it sorted my side still. I have a field defined as an INT in my patterns, which today has thrown out the error (in Kibana): Visualiz… --- ## [Status field in JSON filter must be numeric not string](https://discuss.elastic.co/t/status-field-in-json-filter-must-be-numeric-not-string/56304)
**Author:** [@chriswood123](https://discuss.elastic.co/u/chriswood123)\ **Replies:** 2\ **Last updated:** [July 30, 2016, 9:41am UTC](https://discuss.elastic.co/t/status-field-in-json-filter-must-be-numeric-not-string/56304 "2016-07-30T09:41:41Z")
Logstash version: 1.5.4 When passing parseable JSON to logstash where the field "status" is a string I get the error "failed action with response of 400, dropping action". Passing the "status" field as a numeric type… --- ## [Timestamp mapping in CEF](https://discuss.elastic.co/t/timestamp-mapping-in-cef/56510)
**Author:** [@Cabbage](https://discuss.elastic.co/u/Cabbage)\ **Replies:** 1\ **Last updated:** [July 30, 2016, 9:27am UTC](https://discuss.elastic.co/t/timestamp-mapping-in-cef/56510 "2016-07-30T09:27:07Z")
Hi! I have a problem with mapping timestamp field. I have log with events like this CEF:0|SI|Audit|1.2|SECA0023|Report Started|3|rt=Jul 25 2016 11:20:28 msg=Report RSRZLLG0 Started duser=SYS shost= cs2=000 cs2Label=cl… --- ## [Logstash JSON codec](https://discuss.elastic.co/t/logstash-json-codec/56553)
**Author:** [@piyush](https://discuss.elastic.co/u/piyush)\ **Replies:** 2\ **Last updated:** [July 30, 2016, 9:22am UTC](https://discuss.elastic.co/t/logstash-json-codec/56553 "2016-07-30T09:22:24Z")
Dear Team, I am working on processing a JSON file and need some clarification. I read earlier threads and tried using codecs (json, json\_lines and json as filter too) but as output i am getting individual lines of the fi… --- ## [Using update API for upserts](https://discuss.elastic.co/t/using-update-api-for-upserts/56787)
**Author:** [@shantanuo](https://discuss.elastic.co/u/shantanuo)\ **Replies:** 0\ **Last updated:** [July 30, 2016, 4:56am UTC](https://discuss.elastic.co/t/using-update-api-for-upserts/56787 "2016-07-30T04:56:12Z")
I have this python code that works as expected. I will like to know how to modify this code to suit the "output" parameters. import copy with open('myfile.txt', 'w') as f: for doc1 in df\_json: import json … --- ## [How to scale logstash-kafka](https://discuss.elastic.co/t/how-to-scale-logstash-kafka/55193)
**Author:** [@Maxwell\_Flanders](https://discuss.elastic.co/u/Maxwell_Flanders)\ **Replies:** 1\ **Last updated:** [July 30, 2016, 1:41am UTC](https://discuss.elastic.co/t/how-to-scale-logstash-kafka/55193 "2016-07-30T01:41:11Z")
We are using the logstash-kafka input with this configuration: kafka { #Kafka Consumer Group ("logstash" is default, specifying for clarity) group\_id =\> "logstash" white\_list =\> "topic1,topic2,topic3,...,to… --- ## [Download/install json\_encode plugin filter](https://discuss.elastic.co/t/download-install-json-encode-plugin-filter/56777)
**Author:** [@asoong-94](https://discuss.elastic.co/u/asoong-94)\ **Replies:** 0\ **Last updated:** [July 29, 2016, 9:41pm UTC](https://discuss.elastic.co/t/download-install-json-encode-plugin-filter/56777 "2016-07-29T21:41:15Z")
does anyone have any tips for downloading/installing the json\_encode plug-in on windows7? I keep getting : "Installation aborted, verification failed for logstash-filter-json\_encode" --- ## [Different index name for data from filebeat and packetbeat](https://discuss.elastic.co/t/different-index-name-for-data-from-filebeat-and-packetbeat/56768)
**Author:** [@Saurabh\_Jambhule](https://discuss.elastic.co/u/Saurabh_Jambhule)\ **Replies:** 1\ **Last updated:** [July 29, 2016, 8:07pm UTC](https://discuss.elastic.co/t/different-index-name-for-data-from-filebeat-and-packetbeat/56768 "2016-07-29T20:07:49Z")
I have taken input from beats in logstash and now I want to forward it to the elasticsearch. In addition to this, I want to make sure that data from packetbeat and filebeat should have different index name in ES like, fo… --- ## [How to add Nessus CSV report to logstash](https://discuss.elastic.co/t/how-to-add-nessus-csv-report-to-logstash/50470)
**Author:** [@Abe\_Bazouie](https://discuss.elastic.co/u/Abe_Bazouie)\ **Replies:** 7\ **Last updated:** [July 29, 2016, 8:02pm UTC](https://discuss.elastic.co/t/how-to-add-nessus-csv-report-to-logstash/50470 "2016-07-29T20:02:05Z")
Hi guys I am kind of new in ELK. I am trying to add CSV file which is Nessus report to logstash in order to analyze them. What I have done is I create a .conf file in logstash but when I try to run logstash -f config… --- ## [Upgrade from 1.5 to 2.0: Pattern not defined](https://discuss.elastic.co/t/upgrade-from-1-5-to-2-0-pattern-not-defined/34312)
**Author:** [@LansK](https://discuss.elastic.co/u/LansK)\ **Replies:** 7\ **Last updated:** [July 29, 2016, 6:22pm UTC](https://discuss.elastic.co/t/upgrade-from-1-5-to-2-0-pattern-not-defined/34312 "2016-07-29T18:22:39Z")
I am upgrading from the 1.5 branch to 2.0. I am getting the following errors on startup: "Default settings used: Filter workers: 1 The error reported is: pattern %{HOST:thehost} not defined " I think %{HOST} is a … --- ## [Match and Replace unicode characters?](https://discuss.elastic.co/t/match-and-replace-unicode-characters/56656)
**Author:** [@Maxwell\_Flanders](https://discuss.elastic.co/u/Maxwell_Flanders)\ **Replies:** 1\ **Last updated:** [July 29, 2016, 6:20pm UTC](https://discuss.elastic.co/t/match-and-replace-unicode-characters/56656 "2016-07-29T18:20:47Z")
I am attempting to perform a gsub on a unicode character, and sub it out with something else. However, I'm having trouble matching it. The unicode character is \\uFFFD (from http://www.ssec.wisc.edu/~tomw/java/unicode.htm… --- ## [Communication between Logstash and Redis is encrypted or not?](https://discuss.elastic.co/t/communication-between-logstash-and-redis-is-encrypted-or-not/56369)
**Author:** [@caoping](https://discuss.elastic.co/u/caoping)\ **Replies:** 2\ **Last updated:** [July 29, 2016, 3:04pm UTC](https://discuss.elastic.co/t/communication-between-logstash-and-redis-is-encrypted-or-not/56369 "2016-07-29T15:04:31Z")
I have many log sources with a huge number of log data, so I'd like to add Redis as message broker to avoid back-pressure and data loss. My architecture is: Filebeat -\> Logstash -\> Redis -\> Logstash Indexer -\> Elastics… --- ## [Logstash Parse Date Issue](https://discuss.elastic.co/t/logstash-parse-date-issue/56278)
**Author:** [@Kennedy\_Kan1](https://discuss.elastic.co/u/Kennedy_Kan1)\ **Replies:** 1\ **Last updated:** [July 29, 2016, 2:07pm UTC](https://discuss.elastic.co/t/logstash-parse-date-issue/56278 "2016-07-29T14:07:17Z")
I am using logstash to read some logs. I have a log file which the Timestamp only consist of time field, i.e. 08:28:20,500, but no date field. I would like to map it with the datetime of today. How should I do that with … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1120) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1122)