# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1016 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1017 --- ## [Persistent queue Ruby exception wrong argument type String](https://discuss.elastic.co/t/persistent-queue-ruby-exception-wrong-argument-type-string/79103)
**Author:** [@Exocomp](https://discuss.elastic.co/u/Exocomp)\ **Replies:** 0\ **Last updated:** [March 17, 2017, 10:58pm UTC](https://discuss.elastic.co/t/persistent-queue-ruby-exception-wrong-argument-type-string/79103 "2017-03-17T22:58:32Z")
Hi, I'm getting an error only when persistent queues are enabled. I enabled persistent queues with the following settings: queue.type: persisted path.queue: /etc/logstash/data/queue queue.max\_bytes: 8gb queue.check… --- ## [What is wrong with my logstash config?](https://discuss.elastic.co/t/what-is-wrong-with-my-logstash-config/79098)
**Author:** [@andre22](https://discuss.elastic.co/u/andre22)\ **Replies:** 2\ **Last updated:** [March 17, 2017, 9:47pm UTC](https://discuss.elastic.co/t/what-is-wrong-with-my-logstash-config/79098 "2017-03-17T21:47:09Z")
Hi, Logstash complains that this config file is wrong, but I don't see where: input { beats { port =\> 5044 } } filter { if "knxmonitor" in \[tags\] { grok { match =\> { "message" =\> "\\A%{TIMESTAMP\_… --- ## [Logstash 5.1.2 - TLS 1.2](https://discuss.elastic.co/t/logstash-5-1-2-tls-1-2/79094)
**Author:** [@fmhwong](https://discuss.elastic.co/u/fmhwong)\ **Replies:** 0\ **Last updated:** [March 17, 2017, 8:23pm UTC](https://discuss.elastic.co/t/logstash-5-1-2-tls-1-2/79094 "2017-03-17T20:23:50Z")
I am running Logstash 5.1.2 with the following configuration. input { lumberjack { port =\> "5043" ssl\_certificate =\> "logstash.crt" ssl\_key =\> "logstash.key" } } However, our securit… --- ## [Fortinet MIB's](https://discuss.elastic.co/t/fortinet-mibs/79092)
**Author:** [@sweath](https://discuss.elastic.co/u/sweath)\ **Replies:** 0\ **Last updated:** [March 17, 2017, 8:09pm UTC](https://discuss.elastic.co/t/fortinet-mibs/79092 "2017-03-17T20:09:11Z")
One of the steps to get snmptraps for Logstash running is importing MIB's from certain devices in your network. I'm running into an issue when trying to use the import.rb script on my FORTIGATE-FORTINET-MIB.mib file. The… --- ## [Grok Pattern for Discourse logs](https://discuss.elastic.co/t/grok-pattern-for-discourse-logs/79085)
**Author:** [@jackzampolin](https://discuss.elastic.co/u/jackzampolin)\ **Replies:** 0\ **Last updated:** [March 17, 2017, 7:24pm UTC](https://discuss.elastic.co/t/grok-pattern-for-discourse-logs/79085 "2017-03-17T19:24:15Z")
Hey! I have a discourse site too! I would like to ingest the logs with logstash, but they log in a custom format. Since this is the elastic community and also on discourse I thought I would see if anyone had a grok pat… --- ## [Logstash influxdb issues](https://discuss.elastic.co/t/logstash-influxdb-issues/78782)
**Author:** [@V\_Kurien](https://discuss.elastic.co/u/V_Kurien)\ **Replies:** 7\ **Last updated:** [March 17, 2017, 4:26pm UTC](https://discuss.elastic.co/t/logstash-influxdb-issues/78782 "2017-03-17T16:26:53Z")
Hi all I'm having some issues with using the influxdb plugin for logstash. Here is the config that I am using: === input { file { path =\> "/var/log/flowlog/flowlog.log" type =\> "conntrack-flowlog" start\_pos… --- ## [Blocked outputs](https://discuss.elastic.co/t/blocked-outputs/78881)
**Author:** [@tcouto](https://discuss.elastic.co/u/tcouto)\ **Replies:** 2\ **Last updated:** [March 17, 2017, 4:00pm UTC](https://discuss.elastic.co/t/blocked-outputs/78881 "2017-03-17T16:00:42Z")
Recently I had an issue where my Elastic Cloud instance was not responding well. Logstash was throwing a lot of errors about not being able to send events, and I noticed that some of my other outputs weren't getting proc… --- ## [List json parsing](https://discuss.elastic.co/t/list-json-parsing/78993)
**Author:** [@E.Eric](https://discuss.elastic.co/u/E.Eric)\ **Replies:** 2\ **Last updated:** [March 17, 2017, 1:52pm UTC](https://discuss.elastic.co/t/list-json-parsing/78993 "2017-03-17T13:52:21Z")
Hello, I have a probleme to create a list of json object. this is my config: input { file {#settings...} } filter { mutate { add\_field =\> {"book\_title" =\> "%{Book\_title}"} add\_field =\> {"book\_price" =\> "%{Book\_pri… --- ## [Consider only first IP on apache access log](https://discuss.elastic.co/t/consider-only-first-ip-on-apache-access-log/79040)
**Author:** [@nick76](https://discuss.elastic.co/u/nick76)\ **Replies:** 2\ **Last updated:** [March 17, 2017, 2:57pm UTC](https://discuss.elastic.co/t/consider-only-first-ip-on-apache-access-log/79040 "2017-03-17T14:57:21Z")
Hello all, I need an help about a small problem I've: in my access log I have a list of IPs (the end user, ha proxy, varnish ....) one after the other. in order to filter correctly I wish to retain only client IP. what… --- ## [Customized geoip not working with logstash-5.0.1 v](https://discuss.elastic.co/t/customized-geoip-not-working-with-logstash-5-0-1-v/78653)
**Author:** [@srikanth\_muddu](https://discuss.elastic.co/u/srikanth_muddu)\ **Replies:** 3\ **Last updated:** [March 17, 2017, 2:47pm UTC](https://discuss.elastic.co/t/customized-geoip-not-working-with-logstash-5-0-1-v/78653 "2017-03-17T14:47:38Z")
I have customized geoip with private networks using mmutils and works fine in logstash 2.3.4 v ,but in logstash 5.0.1 it throws an error ..how can i reslove this ? please do help Error: with 5.0.1.v \[2017-03-15T00:… --- ## [Parse different datetime values into timestamp](https://discuss.elastic.co/t/parse-different-datetime-values-into-timestamp/79029)
**Author:** [@Grokster](https://discuss.elastic.co/u/Grokster)\ **Replies:** 3\ **Last updated:** [March 17, 2017, 2:35pm UTC](https://discuss.elastic.co/t/parse-different-datetime-values-into-timestamp/79029 "2017-03-17T14:35:17Z")
Hello, bare with me, I'm fairly new to the stack and have a simple question. I have the following log lines in different files: 2017-03-17 22:25:04 My log and sometimes: 2017-03-17 22:25:04,123 another log line.… --- ## [Logstash bind to port 514](https://discuss.elastic.co/t/logstash-bind-to-port-514/44022)
**Author:** [@mikula](https://discuss.elastic.co/u/mikula)\ **Replies:** 12\ **Last updated:** [March 17, 2017, 2:23pm UTC](https://discuss.elastic.co/t/logstash-bind-to-port-514/44022 "2017-03-17T14:23:52Z")
Hello, Is there a way to run logstash as nonroot user and to use port 514 (syslog plugin)? I can not reconfigure all clients to other port... Thank you . AM --- ## [How to run logstash weekly](https://discuss.elastic.co/t/how-to-run-logstash-weekly/79034)
**Author:** [@Dan](https://discuss.elastic.co/u/Dan)\ **Replies:** 2\ **Last updated:** [March 17, 2017, 2:04pm UTC](https://discuss.elastic.co/t/how-to-run-logstash-weekly/79034 "2017-03-17T14:04:47Z")
Hi I want to run my logstash on weekly basis. How can i schedule my logstash.Currently i am using below as input its running for each second. "\* \* \* \*" How can i make this to run for every one week. Thanks in Advan… --- ## [Log4j plugin java.lang.NullPointerException](https://discuss.elastic.co/t/log4j-plugin-java-lang-nullpointerexception/51763)
**Author:** [@daytimemoon](https://discuss.elastic.co/u/daytimemoon)\ **Replies:** 3\ **Last updated:** [March 17, 2017, 1:48pm UTC](https://discuss.elastic.co/t/log4j-plugin-java-lang-nullpointerexception/51763 "2017-03-17T13:48:22Z")
I'm using ELK 2.3.3,logstash 2.3.2 . When I'm trying to use log4j as input,logstash reports java.lang.NullPointerException. here is my logstash config: input { log4j { type =\> "log4j-type" port =\> 4560 } }… --- ## [Best practice to increase memory for containerized logstash](https://discuss.elastic.co/t/best-practice-to-increase-memory-for-containerized-logstash/79007)
**Author:** [@hust](https://discuss.elastic.co/u/hust)\ **Replies:** 0\ **Last updated:** [March 17, 2017, 11:09am UTC](https://discuss.elastic.co/t/best-practice-to-increase-memory-for-containerized-logstash/79007 "2017-03-17T11:09:46Z")
We are running logstash within a Docker Container to monitor RabbitMQ. Due to some huge messages we have to increase the memory with which logstash is operating. According to https://www.elastic.co/guide/en/logstash/cu… --- ## [How to get the IP address of the HTTP request sender using logstash?](https://discuss.elastic.co/t/how-to-get-the-ip-address-of-the-http-request-sender-using-logstash/78891)
**Author:** [@gourab.jtv](https://discuss.elastic.co/u/gourab.jtv)\ **Replies:** 8\ **Last updated:** [March 17, 2017, 9:11am UTC](https://discuss.elastic.co/t/how-to-get-the-ip-address-of-the-http-request-sender-using-logstash/78891 "2017-03-17T09:11:37Z")
I am trying to find the IP address of the HTTP request sender (using Logstash HTTP input plugin) in logstash. curl -XPUT 'http://127.0.0.1:8080/twitter/tweet/1' -d 'hello' My config file for logstash is: input { h… --- ## [Bulk export of several directories of csv files to elasticsearch](https://discuss.elastic.co/t/bulk-export-of-several-directories-of-csv-files-to-elasticsearch/78474)
**Author:** [@Natheer\_Alabsi](https://discuss.elastic.co/u/Natheer_Alabsi)\ **Replies:** 3\ **Last updated:** [March 17, 2017, 8:19am UTC](https://discuss.elastic.co/t/bulk-export-of-several-directories-of-csv-files-to-elasticsearch/78474 "2017-03-17T08:19:36Z")
I have several directories and each has several subdirectories and each has csv files within them. I would like to export all these csv files inside all directories to elasticsearch using logstash and automatically take … --- ## [Tagging incoming logs based on source](https://discuss.elastic.co/t/tagging-incoming-logs-based-on-source/78921)
**Author:** [@cisaksen](https://discuss.elastic.co/u/cisaksen)\ **Replies:** 1\ **Last updated:** [March 17, 2017, 7:30am UTC](https://discuss.elastic.co/t/tagging-incoming-logs-based-on-source/78921 "2017-03-17T07:30:26Z")
4 Web servers - 3 are production and 1 is a test server. All 4 are sending multiple logs to logstash via filebeats. All are using the same log format. What is the best way to tag each log record as they are processed… --- ## [Logstash error while runing conf file](https://discuss.elastic.co/t/logstash-error-while-runing-conf-file/77775)
**Author:** [@leenanarmeta](https://discuss.elastic.co/u/leenanarmeta)\ **Replies:** 3\ **Last updated:** [March 17, 2017, 6:23am UTC](https://discuss.elastic.co/t/logstash-error-while-runing-conf-file/77775 "2017-03-17T06:23:31Z")
HI ,iam getting error while running conf file.... here iam using logstash for inserting MYSQL data into elasticsearch............ this is the error iam getting this is my conf file............... --- ## [Split "event\_data" to only show the original data field](https://discuss.elastic.co/t/split-event-data-to-only-show-the-original-data-field/78561)
**Author:** [@peanut](https://discuss.elastic.co/u/peanut)\ **Replies:** 2\ **Last updated:** [March 16, 2017, 10:43pm UTC](https://discuss.elastic.co/t/split-event-data-to-only-show-the-original-data-field/78561 "2017-03-16T22:43:24Z")
Good afternoon elastic community, Is it best practice to mutate all the fields with the "event\_data" attached to it to its original data field name? If so, what is the easiest way to do it in a filter? I am forwarding … --- ## [GELF multiline codec doesn't work](https://discuss.elastic.co/t/gelf-multiline-codec-doesnt-work/78929)
**Author:** [@Josh\_Reichardt](https://discuss.elastic.co/u/Josh_Reichardt)\ **Replies:** 0\ **Last updated:** [March 16, 2017, 8:16pm UTC](https://discuss.elastic.co/t/gelf-multiline-codec-doesnt-work/78929 "2017-03-16T20:16:18Z")
I am attempting to set up the multiline input for GELF but it doesn't seem to work and I'm not sure why. Is there a way to debug the multiline input, or does anybody know otherwise why this wouldn't be working? Here's… --- ## [Convert UTC time to specified timezone](https://discuss.elastic.co/t/convert-utc-time-to-specified-timezone/78928)
**Author:** [@matohl](https://discuss.elastic.co/u/matohl)\ **Replies:** 0\ **Last updated:** [March 16, 2017, 8:09pm UTC](https://discuss.elastic.co/t/convert-utc-time-to-specified-timezone/78928 "2017-03-16T20:09:00Z")
Hi, I have a log with events from different timezones, where the timestamps are all UTC. I also have the names of the timezones, e.g. "Europe/Madrid". I want to check if the event occurred during office hours in the lo… --- ## [Newbie assistance with grok](https://discuss.elastic.co/t/newbie-assistance-with-grok/78758)
**Author:** [@Lexa](https://discuss.elastic.co/u/Lexa)\ **Replies:** 2\ **Last updated:** [March 16, 2017, 8:05pm UTC](https://discuss.elastic.co/t/newbie-assistance-with-grok/78758 "2017-03-16T20:05:51Z")
Hello, I am new to logstash and struggling with custom grok. I have a log file that has a series of bracket bound fields that I am trying to parse with kv. However, clearly I do not understand how to do this as, i… --- ## [How to remove quotation marks](https://discuss.elastic.co/t/how-to-remove-quotation-marks/78919)
**Author:** [@andre22](https://discuss.elastic.co/u/andre22)\ **Replies:** 1\ **Last updated:** [March 16, 2017, 6:57pm UTC](https://discuss.elastic.co/t/how-to-remove-quotation-marks/78919 "2017-03-16T18:57:45Z")
Hi, the logs I am trying to parse contain some fields in quotation marks: 2016-11-01 00:27:17.034 "157.55.39.145" - .... What I am trying is the following: grok { match =\> { "message" =\> "\\A%{TIMESTAMP\_ISO8601… --- ## [Rotate index every 6 hours](https://discuss.elastic.co/t/rotate-index-every-6-hours/77851)
**Author:** [@danielmotaleite](https://discuss.elastic.co/u/danielmotaleite)\ **Replies:** 4\ **Last updated:** [March 16, 2017, 6:52pm UTC](https://discuss.elastic.co/t/rotate-index-every-6-hours/77851 "2017-03-16T18:52:28Z")
Hi i have 2 daily indices that are about 30-40Gb every day and even spread by 6 shards are big. When i have to restart one node, they take a long time to recover. So i want to rotate then not by day, but by every 6 h… --- ## [Redis - logstash - influxdb](https://discuss.elastic.co/t/redis-logstash-influxdb/78804)
**Author:** [@azeiner](https://discuss.elastic.co/u/azeiner)\ **Replies:** 2\ **Last updated:** [March 16, 2017, 6:01pm UTC](https://discuss.elastic.co/t/redis-logstash-influxdb/78804 "2017-03-16T18:01:02Z")
i've got a redis db, logstash and both elasticsearch an influxdb. i'm transferring keys from redis to elasticsearch, which works fine and want to test influxdb which does not work at all. has anybody a valid connector … --- ## [Logstash to Kafka with SASL](https://discuss.elastic.co/t/logstash-to-kafka-with-sasl/78918)
**Author:** [@Markovich](https://discuss.elastic.co/u/Markovich)\ **Replies:** 0\ **Last updated:** [March 16, 2017, 5:58pm UTC](https://discuss.elastic.co/t/logstash-to-kafka-with-sasl/78918 "2017-03-16T17:58:38Z")
Hello everyone! I'm new to logstash! I need to send logs from hdfs-namenode to kafka broker with kerberos. Here is my configs and full log of logstash. Please tell me what's wrong? I think logstash is not using ja… --- ## [5.2.2 and netflow codec: IP addresses appear as strings](https://discuss.elastic.co/t/5-2-2-and-netflow-codec-ip-addresses-appear-as-strings/78904)
**Author:** [@Bergante](https://discuss.elastic.co/u/Bergante)\ **Replies:** 0\ **Last updated:** [March 16, 2017, 4:17pm UTC](https://discuss.elastic.co/t/5-2-2-and-netflow-codec-ip-addresses-appear-as-strings/78904 "2017-03-16T16:17:28Z")
Hi I have just installed the ELK 5.2.2 suite. It's running fine. I have added two Netflow v9 exporters in two Mikrotik routers and configured the netflow codec according to the documentation: input { udp { … --- ## [Writing to no\_measurement](https://discuss.elastic.co/t/writing-to-no-measurement/78901)
**Author:** [@bob\_monty](https://discuss.elastic.co/u/bob_monty)\ **Replies:** 0\ **Last updated:** [March 16, 2017, 4:10pm UTC](https://discuss.elastic.co/t/writing-to-no-measurement/78901 "2017-03-16T16:10:58Z")
Greetings i have a situation where there's a ton of data is being written to the no\_measurement measurement in influx, although there is alot of data being written to measurements that i expect. I'm new to this so if som… --- ## [Logstash not sending data to elastic from kafka](https://discuss.elastic.co/t/logstash-not-sending-data-to-elastic-from-kafka/78866)
**Author:** [@Hussain\_Kothari](https://discuss.elastic.co/u/Hussain_Kothari)\ **Replies:** 3\ **Last updated:** [March 16, 2017, 3:59pm UTC](https://discuss.elastic.co/t/logstash-not-sending-data-to-elastic-from-kafka/78866 "2017-03-16T15:59:25Z")
Hi, im trying to add data in elastic with logstash and data is input from kafka this is the simple config im using on logstash server in conf file /etc/logstash/conf.d/1-config.conf input{ kafka{ bootstrap\_serve… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1015) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1017)