# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=970 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 971 --- ## [Zabbix output to logstash](https://discuss.elastic.co/t/zabbix-output-to-logstash/88401)
**Author:** [@shrwn](https://discuss.elastic.co/u/shrwn)\ **Replies:** 3\ **Last updated:** [June 6, 2017, 4:33pm UTC](https://discuss.elastic.co/t/zabbix-output-to-logstash/88401 "2017-06-06T16:33:00Z")
Hi, we have multiple node servers that are connecting with master server through logstash. Zabbix agent is installed on almost all nodes. Can i use zabbix to send data to logstash? If so, how can i do that? Is ther… --- ## [Error in installing logstash-plugin](https://discuss.elastic.co/t/error-in-installing-logstash-plugin/88442)
**Author:** [@sahar\_q](https://discuss.elastic.co/u/sahar_q)\ **Replies:** 7\ **Last updated:** [June 6, 2017, 4:26pm UTC](https://discuss.elastic.co/t/error-in-installing-logstash-plugin/88442 "2017-06-06T16:26:47Z")
Command: bin/logstash-plugin install logstash-codec-gzip\_lines ERROR: Installation Aborted, message: Bundler could not find compatible versions for gem "logstash-core-plugin-api": In snapshot (Gemfile.lock): … --- ## [Kafka input plugin: How many Logstash consumer threads to be used for many logstash consumers](https://discuss.elastic.co/t/kafka-input-plugin-how-many-logstash-consumer-threads-to-be-used-for-many-logstash-consumers/88437)
**Author:** [@sgarap](https://discuss.elastic.co/u/sgarap)\ **Replies:** 0\ **Last updated:** [June 6, 2017, 2:57pm UTC](https://discuss.elastic.co/t/kafka-input-plugin-how-many-logstash-consumer-threads-to-be-used-for-many-logstash-consumers/88437 "2017-06-06T14:57:13Z")
Hi, Continuing the discussion from Multiple logstash reading from a single kafka topic: I have 6 Kafka Instances running. I created a topic with 18 Partitions and 3 as a replication factor. On the other hand I have 6 l… --- ## [The best way to parse CDRs](https://discuss.elastic.co/t/the-best-way-to-parse-cdrs/87899)
**Author:** [@abelgana](https://discuss.elastic.co/u/abelgana)\ **Replies:** 1\ **Last updated:** [June 6, 2017, 2:23pm UTC](https://discuss.elastic.co/t/the-best-way-to-parse-cdrs/87899 "2017-06-06T14:23:15Z")
Hi, I have a CDR file that I would like to parse. The file looks like this: Thu Jun 1 06:00:00 2017 NAS-Identifier = 'TOR' Acct-Status-Type = Stop NAS-IP-Address = 10.2.1.16 NAS-Por… --- ## [Logstash and elasticsearch](https://discuss.elastic.co/t/logstash-and-elasticsearch/88393)
**Author:** [@baha1](https://discuss.elastic.co/u/baha1)\ **Replies:** 8\ **Last updated:** [June 6, 2017, 11:39am UTC](https://discuss.elastic.co/t/logstash-and-elasticsearch/88393 "2017-06-06T11:39:44Z")
Hi there, i am using EL and LS of the same version 5.4.0 and filbeat version 5.4.1.there is the configuration. --------------elasticsearch.yml cluster.name: elasticsearch node.name: node1 node.attr.rack: r1 … --- ## [Load data into Elastic Search using Log-stash](https://discuss.elastic.co/t/load-data-into-elastic-search-using-log-stash/88052)
**Author:** [@BunnyV](https://discuss.elastic.co/u/BunnyV)\ **Replies:** 8\ **Last updated:** [June 2, 2017, 1:02pm UTC](https://discuss.elastic.co/t/load-data-into-elastic-search-using-log-stash/88052 "2017-06-02T13:02:49Z")
Well the question may be already exists or there will be lots of solutions are available, but here the requirement is different. Here I have a huge file around 44GB and it is increasing day by day. Q1:- Can we load t… --- ## [Logstash Organize Conf folder](https://discuss.elastic.co/t/logstash-organize-conf-folder/88344)
**Author:** [@adonahoe](https://discuss.elastic.co/u/adonahoe)\ **Replies:** 2\ **Last updated:** [June 6, 2017, 10:57am UTC](https://discuss.elastic.co/t/logstash-organize-conf-folder/88344 "2017-06-06T10:57:14Z")
Hello everyone, I was wondering if/how to setup /etc/logstash/conf.d to be more organize. For instance if I have multiple configuration files and want to organize them by subdirectory how do I do that? Is that even pos… --- ## [Conditional for empty field](https://discuss.elastic.co/t/conditional-for-empty-field/87932)
**Author:** [@Lukas\_Tilch](https://discuss.elastic.co/u/Lukas_Tilch)\ **Replies:** 2\ **Last updated:** [June 6, 2017, 9:13am UTC](https://discuss.elastic.co/t/conditional-for-empty-field/87932 "2017-06-06T09:13:25Z")
Hello I'm trying to parse a date in a field which sometimes contains no data I'm getting this error log when I try to parse it. \[2017-06-01T16:43:23,036\]\[WARN \]\[logstash.outputs.elasticsearch\] Failed action. {:status=\>4… --- ## [Logstash : Reached open files limit](https://discuss.elastic.co/t/logstash-reached-open-files-limit/87903)
**Author:** [@Beuhlet\_Reseau](https://discuss.elastic.co/u/Beuhlet_Reseau)\ **Replies:** 0\ **Last updated:** [June 1, 2017, 12:13pm UTC](https://discuss.elastic.co/t/logstash-reached-open-files-limit/87903 "2017-06-01T12:13:44Z")
Hello, I have a common problem with other people about opening file with logstash 5.2. I work on RH 7.1 with rpm packages. \[logstash.inputs.file \] Reached open files limit: 4095, set by the 'max\_open\_files' optio… --- ## [Logstash not parsing](https://discuss.elastic.co/t/logstash-not-parsing/88223)
**Author:** [@tomer](https://discuss.elastic.co/u/tomer)\ **Replies:** 0\ **Last updated:** [June 5, 2017, 5:49am UTC](https://discuss.elastic.co/t/logstash-not-parsing/88223 "2017-06-05T05:49:25Z")
After building a new machine of Logstash and copying the same exact "filter" used in the last machine with Logstash. From some reason Logstash is not parsing the logs correctly. Info: Filebeat is sending three types of… --- ## [Not shipping logs from Cloud Foundry to logstash](https://discuss.elastic.co/t/not-shipping-logs-from-cloud-foundry-to-logstash/88332)
**Author:** [@pratheek\_k](https://discuss.elastic.co/u/pratheek_k)\ **Replies:** 0\ **Last updated:** [June 5, 2017, 9:05pm UTC](https://discuss.elastic.co/t/not-shipping-logs-from-cloud-foundry-to-logstash/88332 "2017-06-05T21:05:36Z")
Here is my logstash conf: Conf: input { beats { port =\> 5044 } tcp{ port =\> 5000 } At cloud foundry end i have binded logstash drain service: cf cups logstash-drain -l syslog://\[logserver\]:5000 cf bind-service \[app-nam… --- ## [Skip one o more tabs & in grok?](https://discuss.elastic.co/t/skip-one-o-more-tabs-in-grok/88299)
**Author:** [@Nikhilpawar1985](https://discuss.elastic.co/u/Nikhilpawar1985)\ **Replies:** 1\ **Last updated:** [June 5, 2017, 7:02pm UTC](https://discuss.elastic.co/t/skip-one-o-more-tabs-in-grok/88299 "2017-06-05T19:02:22Z")
can someone please advise how to skip one or more tabs coming in input log line ? I/P - \[2017-06-05 12:16:08,353\] :|: INFO :|: lvprdsnbfe1.lv.jabodo.com :|: :|: :|:\\n\\t\\t\\t\\t :|: :|: c.m.c.PeriodicProd… --- ## [Template length doesn't fit cleanly into flowset](https://discuss.elastic.co/t/template-length-doesnt-fit-cleanly-into-flowset/88314)
**Author:** [@tono](https://discuss.elastic.co/u/tono)\ **Replies:** 0\ **Last updated:** [June 5, 2017, 6:18pm UTC](https://discuss.elastic.co/t/template-length-doesnt-fit-cleanly-into-flowset/88314 "2017-06-05T18:18:09Z")
Hello, I upgraded to the latest version of logstash (5.4.1) in order to fix an "out of range" error that I was getting when processing netflow packets. The "out of range" error went away after upgrading the previous … --- ## [Using Grok fields in stdout](https://discuss.elastic.co/t/using-grok-fields-in-stdout/88311)
**Author:** [@dolanmk](https://discuss.elastic.co/u/dolanmk)\ **Replies:** 0\ **Last updated:** [June 5, 2017, 5:53pm UTC](https://discuss.elastic.co/t/using-grok-fields-in-stdout/88311 "2017-06-05T17:53:38Z")
Hi! My goal is to be able to stdout a message that uses the fields I have creating using Grok. I have pasted my output code for my config file. The way I am currently trying is not working. I'm not entirely sure what… --- ## [Date filter: Text Time zone names (z) cannot be parsed](https://discuss.elastic.co/t/date-filter-text-time-zone-names-z-cannot-be-parsed/88304)
**Author:** [@oraclept](https://discuss.elastic.co/u/oraclept)\ **Replies:** 1\ **Last updated:** [June 5, 2017, 5:36pm UTC](https://discuss.elastic.co/t/date-filter-text-time-zone-names-z-cannot-be-parsed/88304 "2017-06-05T17:36:26Z")
Hi - I am having a. May 30, 2017 10:13:48 PM EST date and time format. b. grokking with (?\\s?%{MONTH}\\s+%{MONTHDAY},\\s+%{YEAR}\\s?%{HOUR}:%{MINUTE}:%{SECOND}\\s+\[\[Aa\]|\[Pp\]\]\[Mm\]?\\s+%{TZ}) - Grok works perfect. c… --- ## [How to create dynamic columns for a CSV plugin](https://discuss.elastic.co/t/how-to-create-dynamic-columns-for-a-csv-plugin/88297)
**Author:** [@Deepikakamalla](https://discuss.elastic.co/u/Deepikakamalla)\ **Replies:** 0\ **Last updated:** [June 5, 2017, 4:13pm UTC](https://discuss.elastic.co/t/how-to-create-dynamic-columns-for-a-csv-plugin/88297 "2017-06-05T16:13:28Z")
Hello, We are trying to get the columns of a CSV file dynamically into elasticsearch using logstash csv plugin. Here we observed is everytime we need to manually specify the field names in csv filter plugin. Let us… --- ## [Sequel Deprecation warning](https://discuss.elastic.co/t/sequel-deprecation-warning/88287)
**Author:** [@dwjvaughan](https://discuss.elastic.co/u/dwjvaughan)\ **Replies:** 0\ **Last updated:** [June 5, 2017, 2:43pm UTC](https://discuss.elastic.co/t/sequel-deprecation-warning/88287 "2017-06-05T14:43:31Z")
Hi all, Have recently upgraded logstash to 5.3.2 and I'm now getting this warning: SEQUEL DEPRECATION WARNING: Database#identifier\_output\_method= is deprecated and will be removed in Sequel 5. Explicitly load the id… --- ## [Logstash JVM heap running out of space](https://discuss.elastic.co/t/logstash-jvm-heap-running-out-of-space/87565)
**Author:** [@Exocomp](https://discuss.elastic.co/u/Exocomp)\ **Replies:** 5\ **Last updated:** [June 5, 2017, 2:12pm UTC](https://discuss.elastic.co/t/logstash-jvm-heap-running-out-of-space/87565 "2017-06-05T14:12:48Z")
Need some advice on how to resolve this issue. I setup an ELK box (1 box) that is working great, except every 4 days or so logstash runs out of JVM heap and then doesn't do much processing and I have to do a restart to … --- ## [Type Geo\_Point](https://discuss.elastic.co/t/type-geo-point/88239)
**Author:** [@1623664](https://discuss.elastic.co/u/1623664)\ **Replies:** 5\ **Last updated:** [June 5, 2017, 12:33pm UTC](https://discuss.elastic.co/t/type-geo-point/88239 "2017-06-05T12:33:14Z")
input { jdbc { jdbc\_driver\_library =\> "/home/cuckoo/cuckoo/logstash-5.2.2/mariadb-java-client-1.5.9.jar" jdbc\_driver\_class =\> "org.mariadb.jdbc.Driver" jdbc\_connection\_string =\> "jdbc:mariadb://100.1100.100.100:3306… --- ## [Logstash Kafka Input plugin : no message read from kafka topic](https://discuss.elastic.co/t/logstash-kafka-input-plugin-no-message-read-from-kafka-topic/83875)
**Author:** [@gugansankar](https://discuss.elastic.co/u/gugansankar)\ **Replies:** 6\ **Last updated:** [June 5, 2017, 11:05am UTC](https://discuss.elastic.co/t/logstash-kafka-input-plugin-no-message-read-from-kafka-topic/83875 "2017-06-05T11:05:53Z")
Planned to upgrade my ELK setup to 5.x along with that I tried to add kafka message queue between forwarder and logstash server on the new environment. There is no issues on forwarders sent message to kafka. I verified … --- ## [Logstash service not starting](https://discuss.elastic.co/t/logstash-service-not-starting/87849)
**Author:** [@moorthyrajesh](https://discuss.elastic.co/u/moorthyrajesh)\ **Replies:** 11\ **Last updated:** [June 5, 2017, 9:49am UTC](https://discuss.elastic.co/t/logstash-service-not-starting/87849 "2017-06-05T09:49:57Z")
i have an issue similar to https://www.elastic.co/guide/en/logstash/current/running-logstash.html#running-logstash-upstart os: Ununtu Mate 14.04 logstash service not starting sudo initctl start logstash result: u… --- ## [Logstash grok parse failure](https://discuss.elastic.co/t/logstash-grok-parse-failure/87550)
**Author:** [@Marcello\_A](https://discuss.elastic.co/u/Marcello_A)\ **Replies:** 2\ **Last updated:** [June 5, 2017, 8:16am UTC](https://discuss.elastic.co/t/logstash-grok-parse-failure/87550 "2017-06-05T08:16:54Z")
Hi All, I tried to import a modified combined apache logs on a logstash instance and I have this sample row: 10.10.10.10 54338 - \[29/May/2017:16:21:34 +0200\] "GET /test.html HTTP/1.1" 200 682 8763 "https://mysite.com/t… --- ## [\[Solved\] How to get raw log in logstash](https://discuss.elastic.co/t/solved-how-to-get-raw-log-in-logstash/88215)
**Author:** [@tatdat](https://discuss.elastic.co/u/tatdat)\ **Replies:** 2\ **Last updated:** [June 5, 2017, 7:51am UTC](https://discuss.elastic.co/t/solved-how-to-get-raw-log-in-logstash/88215 "2017-06-05T07:51:05Z")
Hi, i have this case. I send log from filebeat -\> logstash. (5.4). Now i need to get raw log in field "message" only { "\_index": "fb-test-2017.20", "\_type": "ssh", "\_id": "AVwPYDgrVEOhQqT8cqhk", "\_version": 1, … --- ## [Error parsing csv](https://discuss.elastic.co/t/error-parsing-csv/87893)
**Author:** [@mitabrev](https://discuss.elastic.co/u/mitabrev)\ **Replies:** 3\ **Last updated:** [June 5, 2017, 7:03am UTC](https://discuss.elastic.co/t/error-parsing-csv/87893 "2017-06-05T07:03:14Z")
Hi, I'm checking my logstash logs and see many entries with WARN "Error parsing csv". Example: \[2017-06-01T13:09:16,380\]\[WARN \]\[logstash.filters.csv \] Error parsing csv {:field=\>"message", :source=\>"8:0.0/0.0\\",\\… --- ## [Unavailable\_shards\_exception, reason: primary shard is not active](https://discuss.elastic.co/t/unavailable-shards-exception-reason-primary-shard-is-not-active/56590)
**Author:** [@caoping](https://discuss.elastic.co/u/caoping)\ **Replies:** 1\ **Last updated:** [June 5, 2017, 6:50am UTC](https://discuss.elastic.co/t/unavailable-shards-exception-reason-primary-shard-is-not-active/56590 "2017-06-05T06:50:16Z")
Anyone can help me on that? It's pretty weird. I'm using a simple architecture: Filebeat -\> Logstash -\> Elasticsearch. Besides, only one instance of logstash and elasticsearch. When I started them, they worked well. Ho… --- ## [Logstash not importing data into elasticsearch](https://discuss.elastic.co/t/logstash-not-importing-data-into-elasticsearch/88197)
**Author:** [@amilus](https://discuss.elastic.co/u/amilus)\ **Replies:** 1\ **Last updated:** [June 5, 2017, 3:02am UTC](https://discuss.elastic.co/t/logstash-not-importing-data-into-elasticsearch/88197 "2017-06-05T03:02:13Z")
i'm importing data from twitter through logstash into elasticsearch, it was working just fine but since a \> few days it stopped working, and im getting this error, could someone help me please \[2017-06-04T18:25:39,… --- ## [Latest version of Elastic Stack and Suricata examples](https://discuss.elastic.co/t/latest-version-of-elastic-stack-and-suricata-examples/88184)
**Author:** [@AndrewW](https://discuss.elastic.co/u/AndrewW)\ **Replies:** 2\ **Last updated:** [June 5, 2017, 3:00am UTC](https://discuss.elastic.co/t/latest-version-of-elastic-stack-and-suricata-examples/88184 "2017-06-05T03:00:59Z")
I've got the latest version of suricata 3.2.1 working with EVE json logs I followed the woefully out of date instruction on https://redmine.openinfosecfoundation.org/projects/suricata/wiki/\_Logstash\_Kibana\_and\_Suricata\_J… --- ## [Syslog beats separate output](https://discuss.elastic.co/t/syslog-beats-separate-output/87653)
**Author:** [@VamPikmin](https://discuss.elastic.co/u/VamPikmin)\ **Replies:** 9\ **Last updated:** [June 5, 2017, 1:25am UTC](https://discuss.elastic.co/t/syslog-beats-separate-output/87653 "2017-06-05T01:25:18Z")
Hi there, I'm trying to learn logstash. I have three inputs in logstash listening on different ports. This is what the config looks like input { udp { tags =\> \["netflow"\] type =\> "netflow" port =\> 5046 … --- ## [Logstash creates index which is 2 dates ahead](https://discuss.elastic.co/t/logstash-creates-index-which-is-2-dates-ahead/88004)
**Author:** [@Steve\_1](https://discuss.elastic.co/u/Steve_1)\ **Replies:** 6\ **Last updated:** [June 5, 2017, 12:37am UTC](https://discuss.elastic.co/t/logstash-creates-index-which-is-2-dates-ahead/88004 "2017-06-05T00:37:36Z")
In my case log record timestamp doesn't have date value, just time, assuming that date value is current date. logstash .conf: input { beats { port =\> 5044 } } filter { if \[type\] == "filebeat" { grok { … --- ## [Logstash slows down over time](https://discuss.elastic.co/t/logstash-slows-down-over-time/81072)
**Author:** [@jbeck](https://discuss.elastic.co/u/jbeck)\ **Replies:** 20\ **Last updated:** [June 4, 2017, 11:01pm UTC](https://discuss.elastic.co/t/logstash-slows-down-over-time/81072 "2017-06-04T23:01:04Z")
I'm having a weird problem with logstash and I'd like some help debugging it. Basically our logstash cluster slows down over a period of a couple of weeks. What I mean by slows down is that it starts rejecting messages… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=969) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=971)