# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1005 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1006 --- ## [Ruby code with operation (modulo)](https://discuss.elastic.co/t/ruby-code-with-operation-modulo/81339)
**Author:** [@henrilabarre](https://discuss.elastic.co/u/henrilabarre)\ **Replies:** 1\ **Last updated:** [April 5, 2017, 5:40pm UTC](https://discuss.elastic.co/t/ruby-code-with-operation-modulo/81339 "2017-04-05T17:40:45Z")
Hi I use a CSV (and the logstash csv filter) and with one field I need to create a id based on this php code where $siren value is in my csv (named SIREN, for example 010441582), notice the use of php modulo (%) ret… --- ## [Update/rewrite the value in a field?](https://discuss.elastic.co/t/update-rewrite-the-value-in-a-field/81250)
**Author:** [@Josh\_Reichardt](https://discuss.elastic.co/u/Josh_Reichardt)\ **Replies:** 2\ **Last updated:** [April 5, 2017, 5:29pm UTC](https://discuss.elastic.co/t/update-rewrite-the-value-in-a-field/81250 "2017-04-05T17:29:36Z")
I have a log that has the pattern similar to the following in a field called host: ip-x-x-x-x.us-west-2.compute.internal I'm wondering if there is a good way to apply a regex to the value in the field so I obtain jus… --- ## [Matching @timestamp syslog](https://discuss.elastic.co/t/matching-timestamp-syslog/81225)
**Author:** [@Jose\_Campos](https://discuss.elastic.co/u/Jose_Campos)\ **Replies:** 4\ **Last updated:** [April 5, 2017, 4:25pm UTC](https://discuss.elastic.co/t/matching-timestamp-syslog/81225 "2017-04-05T16:25:56Z")
Hi there Some days ago my data were matching with the timestamp, but something happened that It collapsed. Here is how it was. And here how it is today. Can you help me with this? Greetings. --- ## [Logstash 5.3 Persistent Queue issue](https://discuss.elastic.co/t/logstash-5-3-persistent-queue-issue/81203)
**Author:** [@devsibwarra](https://discuss.elastic.co/u/devsibwarra)\ **Replies:** 11\ **Last updated:** [April 5, 2017, 3:16pm UTC](https://discuss.elastic.co/t/logstash-5-3-persistent-queue-issue/81203 "2017-04-05T15:16:47Z")
Versions \* OS: Ubuntu 14.04.5 LTS \* Filebeat 5.3.0 from https://artifacts.elastic.co/packages/5.x/apt/ \* Logstash 1:5.3.0-1 from https://artifacts.elastic.co/packages/5.x/apt/ \* Oracle Java(TM) SE Runtime Environment (bu… --- ## [\[Input file plugin\] sincedb default path changed since plugin version 3.1.1, so from logstash 5.0.0 and on](https://discuss.elastic.co/t/input-file-plugin-sincedb-default-path-changed-since-plugin-version-3-1-1-so-from-logstash-5-0-0-and-on/81307)
**Author:** [@fortikeco](https://discuss.elastic.co/u/fortikeco)\ **Replies:** 0\ **Last updated:** [April 5, 2017, 12:18pm UTC](https://discuss.elastic.co/t/input-file-plugin-sincedb-default-path-changed-since-plugin-version-3-1-1-so-from-logstash-5-0-0-and-on/81307 "2017-04-05T12:18:55Z")
Hi, I noticed the user and code documentation of the default sincedb path is not up-to-date. I had to lookup the source code to see what was going on. Since the commit of July 26, 2016 the behaviour of the default si… --- ## [Validate file with logstash-file-input plugin](https://discuss.elastic.co/t/validate-file-with-logstash-file-input-plugin/81068)
**Author:** [@shabirmean](https://discuss.elastic.co/u/shabirmean)\ **Replies:** 1\ **Last updated:** [April 5, 2017, 12:50pm UTC](https://discuss.elastic.co/t/validate-file-with-logstash-file-input-plugin/81068 "2017-04-05T12:50:14Z")
Hi, I am working on a project that uses Logstash to parse some csv files. For this the Logstash “file-input-plugin” is used where new files are read from a path and the parsing is done. However, I am trying to impl… --- ## [Logstash exec Output delete all files with extension](https://discuss.elastic.co/t/logstash-exec-output-delete-all-files-with-extension/81279)
**Author:** [@Mouss](https://discuss.elastic.co/u/Mouss)\ **Replies:** 1\ **Last updated:** [April 5, 2017, 9:15am UTC](https://discuss.elastic.co/t/logstash-exec-output-delete-all-files-with-extension/81279 "2017-04-05T09:15:03Z")
Hello ! I want to make an exec in output to delete all files with extension .conf i tried "rm \*.conf " but didn't work Thank you for your help I will be very grateful --- ## [Custom filter plugin creation error: logstash](https://discuss.elastic.co/t/custom-filter-plugin-creation-error-logstash/80128)
**Author:** [@vaishnavi](https://discuss.elastic.co/u/vaishnavi)\ **Replies:** 2\ **Last updated:** [April 4, 2017, 1:44pm UTC](https://discuss.elastic.co/t/custom-filter-plugin-creation-error-logstash/80128 "2017-04-04T13:44:57Z")
Hi I need to create a custom filter plugin in logstash to create new ruby events... bundle install was success but while running bundle exec rspec, I get the following errors you might need to reinstall the gem which … --- ## [How to Parse messages in csv format with only numbers with different column headings?](https://discuss.elastic.co/t/how-to-parse-messages-in-csv-format-with-only-numbers-with-different-column-headings/81075)
**Author:** [@cabishek188](https://discuss.elastic.co/u/cabishek188)\ **Replies:** 3\ **Last updated:** [April 4, 2017, 10:17pm UTC](https://discuss.elastic.co/t/how-to-parse-messages-in-csv-format-with-only-numbers-with-different-column-headings/81075 "2017-04-04T22:17:44Z")
Hi all, I want to be able to parse messages with just numbers. I have lot of .csv files with just numbers and different headings (some headings are the same across csv files) For eg. Col1, Col2, Col3 1,20,30 2,20,3… --- ## [Logstash Rspec Encoding as US-ASCII](https://discuss.elastic.co/t/logstash-rspec-encoding-as-us-ascii/81112)
**Author:** [@Conall\_Prendergast](https://discuss.elastic.co/u/Conall_Prendergast)\ **Replies:** 1\ **Last updated:** [April 4, 2017, 10:04pm UTC](https://discuss.elastic.co/t/logstash-rspec-encoding-as-us-ascii/81112 "2017-04-04T22:04:46Z")
Hi, I am running an Rspec test case which takes in some json files and matches them against an expected output after being put through my filter. However, I am getting an error during the validate\_valie phase where… --- ## [Filebeat write to Redis so fast that Logstash couldn't keep up reading from Redis](https://discuss.elastic.co/t/filebeat-write-to-redis-so-fast-that-logstash-couldnt-keep-up-reading-from-redis/57998)
**Author:** [@tanxiaolong](https://discuss.elastic.co/u/tanxiaolong)\ **Replies:** 6\ **Last updated:** [April 4, 2017, 8:06pm UTC](https://discuss.elastic.co/t/filebeat-write-to-redis-so-fast-that-logstash-couldnt-keep-up-reading-from-redis/57998 "2017-04-04T20:06:37Z")
Hi all, my logstash is 2.3.4 elasticsearch is 2.3.4 filebeat is 1.2.3 redis is 3.2.0 my os is CentOS my situation is as title said, big logs wrote from Filebeat to Redis, just one Filebeat process out there doing th… --- ## [Filebeat =\> logstash index](https://discuss.elastic.co/t/filebeat-logstash-index/81201)
**Author:** [@craftytech1](https://discuss.elastic.co/u/craftytech1)\ **Replies:** 2\ **Last updated:** [April 4, 2017, 7:04pm UTC](https://discuss.elastic.co/t/filebeat-logstash-index/81201 "2017-04-04T19:04:32Z")
I configured my filebeat client as per the instructions: https://www.elastic.co/guide/en/beats/filebeat/current/configuration-filebeat-options.html I also configured logstash as per the instructions as well: https://ww… --- ## [HELP - Logstash isn't working!](https://discuss.elastic.co/t/help-logstash-isnt-working/80614)
**Author:** [@safasalwan](https://discuss.elastic.co/u/safasalwan)\ **Replies:** 17\ **Last updated:** [April 4, 2017, 3:23pm UTC](https://discuss.elastic.co/t/help-logstash-isnt-working/80614 "2017-04-04T15:23:09Z")
Hey guys, , I'm in the process of installing Logstash, but I'm stuck halfway. This is what's happening: What am I supposed to do? I really need to install it ASAP. Thanks, Safa --- ## [How to use dictionary with multidimensional array in Logstash?](https://discuss.elastic.co/t/how-to-use-dictionary-with-multidimensional-array-in-logstash/52585)
**Author:** [@solidz](https://discuss.elastic.co/u/solidz)\ **Replies:** 3\ **Last updated:** [April 4, 2017, 3:19pm UTC](https://discuss.elastic.co/t/how-to-use-dictionary-with-multidimensional-array-in-logstash/52585 "2017-04-04T15:19:28Z")
Hi, I realize a project to collect NetFlow metric on Cisco router with Logstash. With the NetFlow protocol, I collect host address, and only the OID of interfaces (like "1") and not names interfaces (like "Gi8" for e… --- ## [Logstash reverse field DNS name](https://discuss.elastic.co/t/logstash-reverse-field-dns-name/81123)
**Author:** [@Kim-Kruse-Hansen](https://discuss.elastic.co/u/Kim-Kruse-Hansen)\ **Replies:** 1\ **Last updated:** [April 4, 2017, 2:38pm UTC](https://discuss.elastic.co/t/logstash-reverse-field-dns-name/81123 "2017-04-04T14:38:59Z")
Hi Suppose I have fields with fully qualified DNS names. fx myhost.company.com fx myhost.company.biz We have millions of these names. Suppose I want to search for all DNS names ending in \*.biz. This leads me to … --- ## [Logstash Unix Timestamp convert to milliseconds](https://discuss.elastic.co/t/logstash-unix-timestamp-convert-to-milliseconds/81067)
**Author:** [@tanong\_montree](https://discuss.elastic.co/u/tanong_montree)\ **Replies:** 1\ **Last updated:** [April 4, 2017, 12:44pm UTC](https://discuss.elastic.co/t/logstash-unix-timestamp-convert-to-milliseconds/81067 "2017-04-04T12:44:10Z")
Hello, my name is Tongsak. Ask how to convert the time unity x into a day, month, second. For example, 284465 1486008515154023 1486008515154023 1486008574188849 8 6 776 628 0 - - - - - - 4 2406: 3100: 1020: 1 :: 814 2404… --- ## [Date format for YYYY-mm-dd HH:mm:ss,SSS?](https://discuss.elastic.co/t/date-format-for-yyyy-mm-dd-hh-mm-ss-sss/80702)
**Author:** [@ZillaG](https://discuss.elastic.co/u/ZillaG)\ **Replies:** 10\ **Last updated:** [April 4, 2017, 1:41pm UTC](https://discuss.elastic.co/t/date-format-for-yyyy-mm-dd-hh-mm-ss-sss/80702 "2017-04-04T13:41:25Z")
I looked at the Elasticsearch date format docs after reading this post and can't find the format for a date of the form YYYY-mm-dd HH:mm:ss,SSSS, e.g., 2017-03-29 10:00:00,123, where there's a comma for the milliseconds … --- ## [How to divert the logstash logs?](https://discuss.elastic.co/t/how-to-divert-the-logstash-logs/80922)
**Author:** [@tomer](https://discuss.elastic.co/u/tomer)\ **Replies:** 6\ **Last updated:** [April 4, 2017, 1:12pm UTC](https://discuss.elastic.co/t/how-to-divert-the-logstash-logs/80922 "2017-04-04T13:12:49Z")
I am getting my LogStash logs to /var/log/logstash while I want to send them to /log/logstash. I saw many places writing how to do so but still I am a little confused. From now I am guessing I have to change the log4j … --- ## [Codec json is not working in logstash 5.2.1](https://discuss.elastic.co/t/codec-json-is-not-working-in-logstash-5-2-1/80376)
**Author:** [@mdafrozhcl](https://discuss.elastic.co/u/mdafrozhcl)\ **Replies:** 2\ **Last updated:** [April 4, 2017, 12:47pm UTC](https://discuss.elastic.co/t/codec-json-is-not-working-in-logstash-5-2-1/80376 "2017-04-04T12:47:12Z")
I have installed Logstash 5.2.1 and trying to do basic json output and input test. can you let me know if it is installation issue ? but rubydebug codec is working fine. logstash -e 'input { stdin { } } output { stdout… --- ## [Unable to output to Kafka Topic](https://discuss.elastic.co/t/unable-to-output-to-kafka-topic/79960)
**Author:** [@ademaria](https://discuss.elastic.co/u/ademaria)\ **Replies:** 3\ **Last updated:** [April 4, 2017, 11:52am UTC](https://discuss.elastic.co/t/unable-to-output-to-kafka-topic/79960 "2017-04-04T11:52:08Z")
Hello, I am trying to output to a kafka topic but it does not seem to work properly. I am using logstash 5.2.2 with the latest logstash-output-kafka the test setup is: input { file { path =\> "/var/log/dpkg.lo… --- ## [Expected one of #](https://discuss.elastic.co/t/expected-one-of/81114)
**Author:** [@Vivek\_Samaga](https://discuss.elastic.co/u/Vivek_Samaga)\ **Replies:** 7\ **Last updated:** [April 4, 2017, 10:17am UTC](https://discuss.elastic.co/t/expected-one-of/81114 "2017-04-04T10:17:49Z")
Am using conditionals for output my config looks as below output{ if\[type\] == "access\_log" { elasticsearch{ hosts =\> \["10.190.188.174:9200"\] index =\> "logstash-%{+YYYY.MM.dd}" } } else if \[type\] == "BPM" { elasticsear… --- ## [Tcp input data\_timeout in logstash-5.0-beta1 removed](https://discuss.elastic.co/t/tcp-input-data-timeout-in-logstash-5-0-beta1-removed/62027)
**Author:** [@wq\_Huang](https://discuss.elastic.co/u/wq_Huang)\ **Replies:** 1\ **Last updated:** [April 4, 2017, 9:56am UTC](https://discuss.elastic.co/t/tcp-input-data-timeout-in-logstash-5-0-beta1-removed/62027 "2017-04-04T09:56:12Z")
Hi guys,I saw the following info in tcp input: config :data\_timeout, :validate =\> :number, :default =\> -1, :deprecated =\> "This setting is not used by this plugin. It will be removed soon." I wonder what the reason … --- ## [Nginx log\_format grok help](https://discuss.elastic.co/t/nginx-log-format-grok-help/81106)
**Author:** [@LinuxEA-Mark](https://discuss.elastic.co/u/LinuxEA-Mark)\ **Replies:** 0\ **Last updated:** [April 4, 2017, 9:23am UTC](https://discuss.elastic.co/t/nginx-log-format-grok-help/81106 "2017-04-04T09:23:16Z")
My friends (ops old driver): For logstash I need some help nginx log log\_format as follows: '$proxy\_add\_x\_forwarded\_for $remote\_user \[$time\_local\] "$request" $http\_host' '$body\_bytes\_sent $request\_body "$http\_re… --- ## [SyslogServer to ELK](https://discuss.elastic.co/t/syslogserver-to-elk/80963)
**Author:** [@mun](https://discuss.elastic.co/u/mun)\ **Replies:** 1\ **Last updated:** [April 4, 2017, 9:09am UTC](https://discuss.elastic.co/t/syslogserver-to-elk/80963 "2017-04-04T09:09:20Z")
Hello, Can anyone help me on this issue? I have a syslog server and inside these server, there are several folder which are 1. snmp, 2. sophos fw, 3. netflow data & 4. syslog clients. I grab those data from diff… --- ## [Elasticsearch Logstash importing very large JSON files](https://discuss.elastic.co/t/elasticsearch-logstash-importing-very-large-json-files/37015)
**Author:** [@bgow](https://discuss.elastic.co/u/bgow)\ **Replies:** 3\ **Last updated:** [April 4, 2017, 8:29am UTC](https://discuss.elastic.co/t/elasticsearch-logstash-importing-very-large-json-files/37015 "2017-04-04T08:29:36Z")
I'm trying to import some very large JSON files (up to 80GB / file) into Elasticsearch and have tried a couple different approaches but neither is giving me an efficient working solution: 1) Using the BULK API - I rece… --- ## [Problem with xml filter](https://discuss.elastic.co/t/problem-with-xml-filter/80725)
**Author:** [@flalar](https://discuss.elastic.co/u/flalar)\ **Replies:** 11\ **Last updated:** [April 4, 2017, 7:37am UTC](https://discuss.elastic.co/t/problem-with-xml-filter/80725 "2017-04-04T07:37:35Z")
We're having issues setting a date pulled from a xml document as @timestamp in the date filter. input xml looks like this \ \ServerTasks-5017\ \Success\ \2… --- ## [Forcing Types in Elasticsearch](https://discuss.elastic.co/t/forcing-types-in-elasticsearch/79370)
**Author:** [@lisadeng](https://discuss.elastic.co/u/lisadeng)\ **Replies:** 4\ **Last updated:** [April 4, 2017, 7:23am UTC](https://discuss.elastic.co/t/forcing-types-in-elasticsearch/79370 "2017-04-04T07:23:13Z")
I'm using the http\_poller plugin right now to monitor websites. I noticed that when I query a website that refuses a connection, it doesn't give me a status code field in the error log. Right now I'm trying to visualiz… --- ## [Help, I'm blocked, LOADERROR](https://discuss.elastic.co/t/help-im-blocked-loaderror/81066)
**Author:** [@The\_Juke\_Box\_Hero](https://discuss.elastic.co/u/The_Juke_Box_Hero)\ **Replies:** 2\ **Last updated:** [April 4, 2017, 5:48am UTC](https://discuss.elastic.co/t/help-im-blocked-loaderror/81066 "2017-04-04T05:48:44Z")
Sorry to bother, but I'm new to logstash, well to the whole elastic stack. I'm trying to connect to to my wordpress database using jdbc, my configuration seems ok, I was getting some config errors but I managed to sort s… --- ## [Logstash Fields not resolving (PagerDuty plugin)](https://discuss.elastic.co/t/logstash-fields-not-resolving-pagerduty-plugin/81046)
**Author:** [@anthony\_engi](https://discuss.elastic.co/u/anthony_engi)\ **Replies:** 1\ **Last updated:** [April 4, 2017, 5:12am UTC](https://discuss.elastic.co/t/logstash-fields-not-resolving-pagerduty-plugin/81046 "2017-04-04T05:12:09Z")
The pagerduty plugin is not working for me. It cannot resolve the fields, so I cannot make it programmatic and was wondering if anyone can assist me? input { http\_poller { urls =\> { pagerTest =\> "http://ex… --- ## [Add\_field to write object & array values](https://discuss.elastic.co/t/add-field-to-write-object-array-values/1095)
**Author:** [@kyunam](https://discuss.elastic.co/u/kyunam)\ **Replies:** 1\ **Last updated:** [April 4, 2017, 3:56am UTC](https://discuss.elastic.co/t/add-field-to-write-object-array-values/1095 "2017-04-04T03:56:15Z")
Hi! I'm trying to create geo\_point field in a filter. I can create a string representation of geo\_point fine. But I can't figure out how to write object representation & array representation of geo\_point by using add\_f… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1004) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1006)