# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1194 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1195 --- ## [How to compress your file output](https://discuss.elastic.co/t/how-to-compress-your-file-output/41036)
**Author:** [@KabeloShongwe](https://discuss.elastic.co/u/KabeloShongwe)\ **Replies:** 2\ **Last updated:** [February 8, 2016, 6:55am UTC](https://discuss.elastic.co/t/how-to-compress-your-file-output/41036 "2016-02-08T06:55:05Z")
Hi i'm using the file output plugin to write logs to file. I'm having a problem getting the compression plugin. --- ## [Getting warning in Nohup.out file for logstash](https://discuss.elastic.co/t/getting-warning-in-nohup-out-file-for-logstash/41153)
**Author:** [@vikas\_gopal](https://discuss.elastic.co/u/vikas_gopal)\ **Replies:** 0\ **Last updated:** [February 8, 2016, 1:46am UTC](https://discuss.elastic.co/t/getting-warning-in-nohup-out-file-for-logstash/41153 "2016-02-08T01:46:07Z")
Hi Experts, I am getting below message on my logstash server . I am using LS1.5.4 version .Please suggest what it is and how i can remove or resolve this problem? {:timestamp=\>"2016-02-04T16:27:14.292000+0530", :messa… --- ## ["Suddenly" no more default mapping on new logstash-YYYY.MM.DD indexes](https://discuss.elastic.co/t/suddenly-no-more-default-mapping-on-new-logstash-yyyy-mm-dd-indexes/41151)
**Author:** [@jammann](https://discuss.elastic.co/u/jammann)\ **Replies:** 0\ **Last updated:** [February 7, 2016, 11:40pm UTC](https://discuss.elastic.co/t/suddenly-no-more-default-mapping-on-new-logstash-yyyy-mm-dd-indexes/41151 "2016-02-07T23:40:30Z")
Hi all Few days ago, I added a second node to my ELK test installation to get some redundancy. Shut everything down, changed ES (1.5.0) config, restarted and waited until green, started ES on the second node, changed r… --- ## [Logstash index without timestemp](https://discuss.elastic.co/t/logstash-index-without-timestemp/41141)
**Author:** [@ggilat](https://discuss.elastic.co/u/ggilat)\ **Replies:** 7\ **Last updated:** [February 7, 2016, 8:51pm UTC](https://discuss.elastic.co/t/logstash-index-without-timestemp/41141 "2016-02-07T20:51:51Z")
hello all, i'm ELK newbie and i having a few problems with log reading. i'm trying to read a csv file that have no times-temp field in it. for that i built configuration file using this guide help (https://kevinki… --- ## [Send perfmon csv file information to graphite as TSD via logstash](https://discuss.elastic.co/t/send-perfmon-csv-file-information-to-graphite-as-tsd-via-logstash/40914)
**Author:** [@maddipatla](https://discuss.elastic.co/u/maddipatla)\ **Replies:** 9\ **Last updated:** [February 7, 2016, 4:02pm UTC](https://discuss.elastic.co/t/send-perfmon-csv-file-information-to-graphite-as-tsd-via-logstash/40914 "2016-02-07T16:02:56Z")
We have this requirement to gather performance counter information in a dump and then have the information sent to graphite. We do not have the option of sending the information real-time. I have tried the powershell s… --- ## [Grok Pattern that captures lines\\events that starts with a specific text](https://discuss.elastic.co/t/grok-pattern-that-captures-lines-events-that-starts-with-a-specific-text/41138)
**Author:** [@techrawther](https://discuss.elastic.co/u/techrawther)\ **Replies:** 1\ **Last updated:** [February 7, 2016, 2:21pm UTC](https://discuss.elastic.co/t/grok-pattern-that-captures-lines-events-that-starts-with-a-specific-text/41138 "2016-02-07T14:21:38Z")
I have a pattern word that appears multiple times in my logs. I want to set a tag for those lines which starts with a nonspace + the pattern text + Any data. Following is a snippet from my log Changeset: ht… --- ## [How can i remove field?](https://discuss.elastic.co/t/how-can-i-remove-field/40972)
**Author:** [@Mykolaichenko](https://discuss.elastic.co/u/Mykolaichenko)\ **Replies:** 2\ **Last updated:** [February 7, 2016, 7:22am UTC](https://discuss.elastic.co/t/how-can-i-remove-field/40972 "2016-02-07T07:22:23Z")
Hi! I'd like to remove inner filed "message"becouse its epmty. How can i make this? My rule: if \[message\] == "" { drop { } } But its not working. Can anyone take advice for me? \[image\] --- ## [Posting @mention to slack channel](https://discuss.elastic.co/t/posting-mention-to-slack-channel/41079)
**Author:** [@bharath\_meka](https://discuss.elastic.co/u/bharath_meka)\ **Replies:** 2\ **Last updated:** [February 7, 2016, 6:37am UTC](https://discuss.elastic.co/t/posting-mention-to-slack-channel/41079 "2016-02-07T06:37:17Z")
Hi, We have a custom output slack plugin which post some alert messages to a slack channel. I would like to prefix the alert message with a "@userid" so that the slack sends an email to the user that he is mentioned in… --- ## [Kv filter plugin when there are spaces in key and value](https://discuss.elastic.co/t/kv-filter-plugin-when-there-are-spaces-in-key-and-value/41120)
**Author:** [@techrawther](https://discuss.elastic.co/u/techrawther)\ **Replies:** 4\ **Last updated:** [February 6, 2016, 4:23pm UTC](https://discuss.elastic.co/t/kv-filter-plugin-when-there-are-spaces-in-key-and-value/41120 "2016-02-06T16:23:02Z")
I am facing challenges in using kv filter plugin. I wanted to split data based on colon : . I have spaces in the key and also value. I wanted to have the data filtered with the key (entire key) with the spaces removed … --- ## [Help parsing nested JSON with dynamic top key](https://discuss.elastic.co/t/help-parsing-nested-json-with-dynamic-top-key/41069)
**Author:** [@thesilence](https://discuss.elastic.co/u/thesilence)\ **Replies:** 1\ **Last updated:** [February 6, 2016, 2:50pm UTC](https://discuss.elastic.co/t/help-parsing-nested-json-with-dynamic-top-key/41069 "2016-02-06T14:50:19Z")
Hi everyone, I am looking for help to tackle a fairly difficult problem for an unexperienced logstash user. I have an application that pushes JSON messages to redis. The message format is the following: I need t… --- ## [Help with Grok pattern](https://discuss.elastic.co/t/help-with-grok-pattern/41117)
**Author:** [@techrawther](https://discuss.elastic.co/u/techrawther)\ **Replies:** 3\ **Last updated:** [February 6, 2016, 1:11pm UTC](https://discuss.elastic.co/t/help-with-grok-pattern/41117 "2016-02-06T13:11:35Z")
I am using grok. I am stuck with an issue as mentioned below. I am facing issue when there is multiple colon ":" present in the logs . It is looking for GREEDYDATA and is not returning the first matching ":" I am usin… --- ## [Logstash 2.1.0 is compatible with redis 3.0.0?](https://discuss.elastic.co/t/logstash-2-1-0-is-compatible-with-redis-3-0-0/40968)
**Author:** [@Kishore](https://discuss.elastic.co/u/Kishore)\ **Replies:** 2\ **Last updated:** [February 5, 2016, 12:58pm UTC](https://discuss.elastic.co/t/logstash-2-1-0-is-compatible-with-redis-3-0-0/40968 "2016-02-05T12:58:50Z")
Hi, logstash2.1.0 is compatiable with redis 3.0.0? please let me know since i am facing lot of issues with this version of redis. --- ## [Locale automatically being set in date filter](https://discuss.elastic.co/t/locale-automatically-being-set-in-date-filter/41050)
**Author:** [@punit\_naik1](https://discuss.elastic.co/u/punit_naik1)\ **Replies:** 9\ **Last updated:** [February 5, 2016, 12:43pm UTC](https://discuss.elastic.co/t/locale-automatically-being-set-in-date-filter/41050 "2016-02-05T12:43:53Z")
I have a timestamp in my log and its already in the timezone that I want. But when I apply a date filter to it to convert it into a timestamp, it automatically is converted into my local time which I don't want because i… --- ## [Don´t see any filtered events in kibana](https://discuss.elastic.co/t/don-t-see-any-filtered-events-in-kibana/40853)
**Author:** [@golauty](https://discuss.elastic.co/u/golauty)\ **Replies:** 1\ **Last updated:** [February 5, 2016, 12:11pm UTC](https://discuss.elastic.co/t/don-t-see-any-filtered-events-in-kibana/40853 "2016-02-05T12:11:21Z")
Hi, i have a very weird problem. I am sending all my jenkins logs via redis to logstash. Now every single event that has a grokparsing failure is shown in kibana and the events that i am usually tracking and evalute d… --- ## [How to parse Elasticsearch logs?](https://discuss.elastic.co/t/how-to-parse-elasticsearch-logs/41029)
**Author:** [@punit\_naik1](https://discuss.elastic.co/u/punit_naik1)\ **Replies:** 3\ **Last updated:** [February 5, 2016, 11:40am UTC](https://discuss.elastic.co/t/how-to-parse-elasticsearch-logs/41029 "2016-02-05T11:40:16Z")
I have an elasticsearch log and one line from that looks like this: \[2016-02-02 00:03:17,558\]\[INFO \]\[cluster.service \] \[Sugar Man\] added {\[factordb-d0b9dfee4570f86e--3b3c30d8-1529e1c2cec--8000\]\[5ayv3rbgQT6AK7i… --- ## [Redis output plugin batch behavior](https://discuss.elastic.co/t/redis-output-plugin-batch-behavior/40655)
**Author:** [@Bruno\_Lavoie](https://discuss.elastic.co/u/Bruno_Lavoie)\ **Replies:** 0\ **Last updated:** [February 1, 2016, 5:35pm UTC](https://discuss.elastic.co/t/redis-output-plugin-batch-behavior/40655 "2016-02-01T17:35:14Z")
Hello, If we use redis output and enable batch output with a specified size and dynamic list name like this: output { redis { host =\> "127.0.0.1" data\_type =\> "list" key =\> "%{r… --- ## [Question about Logstash plugin pack scope?](https://discuss.elastic.co/t/question-about-logstash-plugin-pack-scope/41049)
**Author:** [@paulkeogh](https://discuss.elastic.co/u/paulkeogh)\ **Replies:** 0\ **Last updated:** [February 5, 2016, 11:20am UTC](https://discuss.elastic.co/t/question-about-logstash-plugin-pack-scope/41049 "2016-02-05T11:20:14Z")
I have installed Logstash onto a staging server and am using the bin/plugin pack command to create an archive of installed plugins for installation on a non-Internet connected production server. The command works fine … --- ## [WARNING: Default JAVA\_OPTS](https://discuss.elastic.co/t/warning-default-java-opts/41042)
**Author:** [@carmelom](https://discuss.elastic.co/u/carmelom)\ **Replies:** 0\ **Last updated:** [February 5, 2016, 10:06am UTC](https://discuss.elastic.co/t/warning-default-java-opts/41042 "2016-02-05T10:06:14Z")
Hello, I am using logstash-2.2.0 version. I am trying to run logstash in a Linux machine but I got this error: WARNING: Default JAVA\_OPTS will be overridden by the JAVA\_OPTS defined in the environment. Environment JAV… --- ## [Lowercase a field's value](https://discuss.elastic.co/t/lowercase-a-fields-value/41025)
**Author:** [@Kartik\_Ramachandran](https://discuss.elastic.co/u/Kartik_Ramachandran)\ **Replies:** 1\ **Last updated:** [February 5, 2016, 8:00am UTC](https://discuss.elastic.co/t/lowercase-a-fields-value/41025 "2016-02-05T08:00:19Z")
I am trying to lowercase the value of only 1 filed in logstash. From the old threads, i see that the lowercase filter does not help. So, i tried using ruby { code =\> "event.to\_hash.each{|k, v| v.downcase! if v.… --- ## [Logstash 2.2.0 vs Logstash 2.2.0 All Plugins](https://discuss.elastic.co/t/logstash-2-2-0-vs-logstash-2-2-0-all-plugins/40981)
**Author:** [@anhlqn](https://discuss.elastic.co/u/anhlqn)\ **Replies:** 4\ **Last updated:** [February 5, 2016, 5:35am UTC](https://discuss.elastic.co/t/logstash-2-2-0-vs-logstash-2-2-0-all-plugins/40981 "2016-02-05T05:35:27Z")
Hi everyone, Does anyone know what is the difference between the two packages? What plugins are not available in Logstash 2.2.0.tar.gz? Any pros or cons for using the Logstash 2.2.0 All Plugins vs. the other one? Th… --- ## [How to filter out certain lines from log file in logstash](https://discuss.elastic.co/t/how-to-filter-out-certain-lines-from-log-file-in-logstash/41010)
**Author:** [@subbu.nv](https://discuss.elastic.co/u/subbu.nv)\ **Replies:** 2\ **Last updated:** [February 4, 2016, 10:41pm UTC](https://discuss.elastic.co/t/how-to-filter-out-certain-lines-from-log-file-in-logstash/41010 "2016-02-04T22:41:49Z")
Hi, i am current have a log file with following lines \*\*\*\*\* TESTINGG Finished Start Time: 02:05:33 Oct 11 2015 Temp: 46 C: 46 C: 43 C: 40 C: 40 C: 53 C: 59 C End Time: 01:35:45 Oct 13 2015 Temp: 52 C: 49 C: 49 C: … --- ## [Sending Logs to Slack](https://discuss.elastic.co/t/sending-logs-to-slack/41000)
**Author:** [@cotcadaver](https://discuss.elastic.co/u/cotcadaver)\ **Replies:** 3\ **Last updated:** [February 4, 2016, 8:38pm UTC](https://discuss.elastic.co/t/sending-logs-to-slack/41000 "2016-02-04T20:38:27Z")
Hey guys, I am having trouble understanding why my instance of logstash isn't sending the files to slack. I was hoping a pair of fresh eyes may be able to find my mistakes. I am simply using an Ubuntu server and am wanti… --- ## [Reading multiple statusfile content and mapping to nested object](https://discuss.elastic.co/t/reading-multiple-statusfile-content-and-mapping-to-nested-object/40737)
**Author:** [@techrawther](https://discuss.elastic.co/u/techrawther)\ **Replies:** 1\ **Last updated:** [February 4, 2016, 8:13pm UTC](https://discuss.elastic.co/t/reading-multiple-statusfile-content-and-mapping-to-nested-object/40737 "2016-02-04T20:13:11Z")
I have a scenarios where multiple status file will be written on a NAS location continuously until a process (build) completes. folder1/status.txt folder2/status.txt folder3/status.txt folder4/status.txt ......./.....… --- ## [Logstash Queue](https://discuss.elastic.co/t/logstash-queue/40865)
**Author:** [@abinay](https://discuss.elastic.co/u/abinay)\ **Replies:** 22\ **Last updated:** [February 4, 2016, 7:11pm UTC](https://discuss.elastic.co/t/logstash-queue/40865 "2016-02-04T19:11:00Z")
Hii I am using logstash to parse my logs . This is my architecture . Filebeat ships logs from server to Logstash. Logstash parses the logs and sends it to Elasticsearch . Elasticsearch's logs can be viewed using kibana .… --- ## [Replacing @timestamp](https://discuss.elastic.co/t/replacing-timestamp/40993)
**Author:** [@tikejhya](https://discuss.elastic.co/u/tikejhya)\ **Replies:** 1\ **Last updated:** [February 4, 2016, 6:05pm UTC](https://discuss.elastic.co/t/replacing-timestamp/40993 "2016-02-04T18:05:22Z")
I am trying to replace @timestamp with actual logfile timestamp i find it not working with change in pattern: for instance: this works for 01/01/2016: match =\> \[ "timestamp" , "dd/MMM/yyyy:HH:mm:ss\] But not with … --- ## [Get line number of the log file line being processed](https://discuss.elastic.co/t/get-line-number-of-the-log-file-line-being-processed/40960)
**Author:** [@nabbaraju](https://discuss.elastic.co/u/nabbaraju)\ **Replies:** 3\ **Last updated:** [February 4, 2016, 2:48pm UTC](https://discuss.elastic.co/t/get-line-number-of-the-log-file-line-being-processed/40960 "2016-02-04T14:48:42Z")
I am setting up a log file processor using ELK stash. I need a unique ID so I can use it as document\_id - there can be duplicate lines in the app access log so all I need is the file name and the line number which is un… --- ## [How to load CSV file in logstash](https://discuss.elastic.co/t/how-to-load-csv-file-in-logstash/40950)
**Author:** [@Lovika\_Jain](https://discuss.elastic.co/u/Lovika_Jain)\ **Replies:** 3\ **Last updated:** [February 4, 2016, 12:57pm UTC](https://discuss.elastic.co/t/how-to-load-csv-file-in-logstash/40950 "2016-02-04T12:57:45Z")
Hi I am trying to load CSV file in logstash but it is not reading the file and not creating the index in elasticsearch My Config file input { file { type =\> "csv" path =\> "/root/installables/\*.csv" start… --- ## [Will logstash-filter-geoip automatically pickup new db file?](https://discuss.elastic.co/t/will-logstash-filter-geoip-automatically-pickup-new-db-file/24079)
**Author:** [@rtoma](https://discuss.elastic.co/u/rtoma)\ **Replies:** 2\ **Last updated:** [February 4, 2016, 12:43pm UTC](https://discuss.elastic.co/t/will-logstash-filter-geoip-automatically-pickup-new-db-file/24079 "2016-02-04T12:43:29Z")
We have a Maxmind site license and download updates every week. Logstash (1.5.0rc) config has a geoip filter with database parameter to point to the downloaded database. But does the filter detect a new db? Or does it … --- ## [Error parsing nested JSON](https://discuss.elastic.co/t/error-parsing-nested-json/40934)
**Author:** [@logstashid](https://discuss.elastic.co/u/logstashid)\ **Replies:** 2\ **Last updated:** [February 4, 2016, 9:25am UTC](https://discuss.elastic.co/t/error-parsing-nested-json/40934 "2016-02-04T09:25:53Z")
REDACTED upon request as this post contained proprietary information. --- ## [Issue with Logstash-input-s3 for CloudTrail Global log bucket as input](https://discuss.elastic.co/t/issue-with-logstash-input-s3-for-cloudtrail-global-log-bucket-as-input/40931)
**Author:** [@apurvara](https://discuss.elastic.co/u/apurvara)\ **Replies:** 0\ **Last updated:** [February 4, 2016, 6:35am UTC](https://discuss.elastic.co/t/issue-with-logstash-input-s3-for-cloudtrail-global-log-bucket-as-input/40931 "2016-02-04T06:35:45Z")
I'm trying to use LS2.1.1 with ES2.1.1 for AWS CloudTrail Log Analysis from S3 Bucket using logstash-input-s3 and logstash-codec-cloudtrail plugin. I am facing problem when trying to start my Logstash service where as my… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1193) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1195)