# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=834 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 835 --- ## [Expected one of #, {, ,, \] Error](https://discuss.elastic.co/t/expected-one-of-error/117140)
**Author:** [@Prakash\_Punj](https://discuss.elastic.co/u/Prakash_Punj)\ **Replies:** 3\ **Last updated:** [January 26, 2018, 3:06pm UTC](https://discuss.elastic.co/t/expected-one-of-error/117140 "2018-01-26T15:06:17Z")
Getting below error: \[2018-01-25T21:29:32,237\]\[ERROR\]\[logstash.agent \] Failed to execute action {:action=\>LogStash::PipelineAction::Create/pipeline\_id:main, :exception=\>"LogStash::ConfigurationError", :message… --- ## [RabbitMQ between provider and consumer](https://discuss.elastic.co/t/rabbitmq-between-provider-and-consumer/116709)
**Author:** [@Andrzej\_Rosiek](https://discuss.elastic.co/u/Andrzej_Rosiek)\ **Replies:** 5\ **Last updated:** [January 26, 2018, 2:48pm UTC](https://discuss.elastic.co/t/rabbitmq-between-provider-and-consumer/116709 "2018-01-26T14:48:37Z")
Hi, I have configuration like: nginx \> logstash \> rabbitmq \> logstash \> file provider output config: output { rabbitmq { id =\> "nginx" host =\> "rabbitmq" exchange =\> "nginx" exchang… --- ## [Logstash as service restarts every 30sec, also doesnt log](https://discuss.elastic.co/t/logstash-as-service-restarts-every-30sec-also-doesnt-log/117185)
**Author:** [@c4alexc4](https://discuss.elastic.co/u/c4alexc4)\ **Replies:** 4\ **Last updated:** [January 26, 2018, 2:41pm UTC](https://discuss.elastic.co/t/logstash-as-service-restarts-every-30sec-also-doesnt-log/117185 "2018-01-26T14:41:01Z")
Hey, I was trying out different things with logstash while starting it from command line using sudo /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/config.conf Everything worked fine there. I would like to us… --- ## [How to figure out at which line Logstash is in a file?](https://discuss.elastic.co/t/how-to-figure-out-at-which-line-logstash-is-in-a-file/116759)
**Author:** [@Precision](https://discuss.elastic.co/u/Precision)\ **Replies:** 5\ **Last updated:** [January 26, 2018, 11:52am UTC](https://discuss.elastic.co/t/how-to-figure-out-at-which-line-logstash-is-in-a-file/116759 "2018-01-26T11:52:13Z")
How can I figure out which file is currently processed by Logstash, and at which position Logstash actually is. I have configured 5 files for input, and usually its working fine. But since hours no one writes to those f… --- ## [Error: Your application used more memory than the safety cap of 4G](https://discuss.elastic.co/t/error-your-application-used-more-memory-than-the-safety-cap-of-4g/115285)
**Author:** [@jose\_rebelo](https://discuss.elastic.co/u/jose_rebelo)\ **Replies:** 6\ **Last updated:** [January 26, 2018, 10:02am UTC](https://discuss.elastic.co/t/error-your-application-used-more-memory-than-the-safety-cap-of-4g/115285 "2018-01-26T10:02:49Z")
Hello, we have are dealing with this error since ever... (basically after installing and start feeding it) We changed jmx and jms from initial 1Gb to 2Gb and then to 4Gb and we are consistently dealing with this error … --- ## [Running a Ruby external script to enrich event](https://discuss.elastic.co/t/running-a-ruby-external-script-to-enrich-event/117067)
**Author:** [@Anabella\_Cristaldi](https://discuss.elastic.co/u/Anabella_Cristaldi)\ **Replies:** 2\ **Last updated:** [January 26, 2018, 7:09am UTC](https://discuss.elastic.co/t/running-a-ruby-external-script-to-enrich-event/117067 "2018-01-26T07:09:47Z")
Hi All, I need to add some field in the events I'm processing with information about the country a phone number belogns to. So, I'm parsing ok the phone numbers in the events and I want to run ruby code in order to add… --- ## [Does ruby script in logstash not support inifile and iniparse?](https://discuss.elastic.co/t/does-ruby-script-in-logstash-not-support-inifile-and-iniparse/117154)
**Author:** [@Chunbo\_Liao](https://discuss.elastic.co/u/Chunbo_Liao)\ **Replies:** 0\ **Last updated:** [January 26, 2018, 6:15am UTC](https://discuss.elastic.co/t/does-ruby-script-in-logstash-not-support-inifile-and-iniparse/117154 "2018-01-26T06:15:06Z")
I want to use inifile(iniparse is ok too) in ruby code (in my logstash conf file). my code is: ruby{ init =\> "require 'inifile'; @@causeList = IniFile.load('./causeList.ini');" code =\>" if !(event.get('message').spl… --- ## [Could not run Logstash as Service](https://discuss.elastic.co/t/could-not-run-logstash-as-service/117024)
**Author:** [@inhinyera16](https://discuss.elastic.co/u/inhinyera16)\ **Replies:** 2\ **Last updated:** [January 26, 2018, 5:04am UTC](https://discuss.elastic.co/t/could-not-run-logstash-as-service/117024 "2018-01-26T05:04:31Z")
I installed logstash using rpm. When i run the logstash in command line, my filebeat can connect to it. But when i try to run it using the same config and settings file, its not working. I can verify that the service … --- ## [Help with aggregating events](https://discuss.elastic.co/t/help-with-aggregating-events/116926)
**Author:** [@TehMuffinMoo](https://discuss.elastic.co/u/TehMuffinMoo)\ **Replies:** 3\ **Last updated:** [January 26, 2018, 3:35am UTC](https://discuss.elastic.co/t/help-with-aggregating-events/116926 "2018-01-26T03:35:10Z")
Hey Guys, My first time posting here, but I’m hoping somebody may be able to assist in pointing me in the right direction. I’ve got OpenVPN logs going into my log server and there’s a number of events which get logged … --- ## [How do I include field-names into CSV file?](https://discuss.elastic.co/t/how-do-i-include-field-names-into-csv-file/117113)
**Author:** [@alexus](https://discuss.elastic.co/u/alexus)\ **Replies:** 1\ **Last updated:** [January 25, 2018, 8:29pm UTC](https://discuss.elastic.co/t/how-do-i-include-field-names-into-csv-file/117113 "2018-01-25T20:29:16Z")
Hello, I'm using Elasticsearch input plugin with Csv output plugin as part of my pipeline. now, generated CSV file has skip fields that are NULL in elasticsearch's document and that presents an issue as I can't really … --- ## [Changing grok pattern gradually](https://discuss.elastic.co/t/changing-grok-pattern-gradually/115333)
**Author:** [@YvorL](https://discuss.elastic.co/u/YvorL)\ **Replies:** 3\ **Last updated:** [January 25, 2018, 9:33pm UTC](https://discuss.elastic.co/t/changing-grok-pattern-gradually/115333 "2018-01-25T21:33:04Z")
Hi! I'd like to expand the logging to have more granular data available. However, the log format can't be updated on every instance, also in some cases, a backup restore will reset the changes. My issue is if I extend t… --- ## [Logstash remove unwanted fields](https://discuss.elastic.co/t/logstash-remove-unwanted-fields/116805)
**Author:** [@saikrishnagaddipati](https://discuss.elastic.co/u/saikrishnagaddipati)\ **Replies:** 1\ **Last updated:** [January 25, 2018, 9:08pm UTC](https://discuss.elastic.co/t/logstash-remove-unwanted-fields/116805 "2018-01-25T21:08:27Z")
I am using logstaah 5.6.4 I am trying to parse the below message field in logstash and add a field "error\_code" but logstash adds all the fields(bytes, syslog\_hostname, method, method2). How to configure logstash to st… --- ## [No sincedb file](https://discuss.elastic.co/t/no-sincedb-file/116764)
**Author:** [@Precision](https://discuss.elastic.co/u/Precision)\ **Replies:** 1\ **Last updated:** [January 25, 2018, 9:06pm UTC](https://discuss.elastic.co/t/no-sincedb-file/116764 "2018-01-25T21:06:53Z")
I have Logstash successfully working with the file input plugin, but there is no .sincedb\* file on my system. Nada. I have Logstash 5.6.6. Can this be? --- ## [Match none or multiple words in parentheses in grok](https://discuss.elastic.co/t/match-none-or-multiple-words-in-parentheses-in-grok/116692)
**Author:** [@meggenberger](https://discuss.elastic.co/u/meggenberger)\ **Replies:** 2\ **Last updated:** [January 25, 2018, 9:05pm UTC](https://discuss.elastic.co/t/match-none-or-multiple-words-in-parentheses-in-grok/116692 "2018-01-25T21:05:53Z")
I have a question about grok. I'm a total beginner and was searching but couldn't find a solution. I have log files with data that is in parentheses. Sometimes there is no data in it. the log lines look like 2017-11-24 … --- ## [Writing to Metadata with a custom pattern in grok](https://discuss.elastic.co/t/writing-to-metadata-with-a-custom-pattern-in-grok/116688)
**Author:** [@c4alexc4](https://discuss.elastic.co/u/c4alexc4)\ **Replies:** 1\ **Last updated:** [January 25, 2018, 9:02pm UTC](https://discuss.elastic.co/t/writing-to-metadata-with-a-custom-pattern-in-grok/116688 "2018-01-25T21:02:36Z")
Hey, I have more of a bugreport rather than a question: I want to use grok to write data into metadata fields. While this works with predefined patterns it doesnt work with custom patterns. Example: %{NUMBER:\[@metad… --- ## [Question Regarding \`index\` in logstash elasticsearch output plugin](https://discuss.elastic.co/t/question-regarding-index-in-logstash-elasticsearch-output-plugin/116954)
**Author:** [@shreyask](https://discuss.elastic.co/u/shreyask)\ **Replies:** 2\ **Last updated:** [January 25, 2018, 8:21pm UTC](https://discuss.elastic.co/t/question-regarding-index-in-logstash-elasticsearch-output-plugin/116954 "2018-01-25T20:21:01Z")
I am using logstash elasticsearch output plugin to create time based indices. I create index patterns in the following way currently: output { # other settings index =\> "server-netlogs-%{+YYYY.MM.dd}" } The events wh… --- ## [How to get file plugin work with the codec I developed](https://discuss.elastic.co/t/how-to-get-file-plugin-work-with-the-codec-i-developed/117108)
**Author:** [@Nathan.W](https://discuss.elastic.co/u/Nathan.W)\ **Replies:** 0\ **Last updated:** [January 25, 2018, 7:33pm UTC](https://discuss.elastic.co/t/how-to-get-file-plugin-work-with-the-codec-i-developed/117108 "2018-01-25T19:33:56Z")
I'm developing my own codec. I'm new to jruby, so after reading some posts, I figured out how to write the major codec work in Java and call the Java library from jruby code. The issue is, this codec won't work with fil… --- ## [Constant warnings in logstash-plain.log](https://discuss.elastic.co/t/constant-warnings-in-logstash-plain-log/117068)
**Author:** [@80c1141eb6d0c564dd90](https://discuss.elastic.co/u/80c1141eb6d0c564dd90)\ **Replies:** 1\ **Last updated:** [January 25, 2018, 4:27pm UTC](https://discuss.elastic.co/t/constant-warnings-in-logstash-plain-log/117068 "2018-01-25T16:27:45Z")
What are this constant warnings in logstash-plain.log: \[2018-01-25T18:38:13,210\]\[WARN \]\[org.logstash.dissect.Dissector\] Dissector mapping, field found in event but it was empty {"field"=\>"message", "event"=\>{"span-name"… --- ## ["Invalid format" message for a date format that always worked. Why the sudden date parsing error?](https://discuss.elastic.co/t/invalid-format-message-for-a-date-format-that-always-worked-why-the-sudden-date-parsing-error/114966)
**Author:** [@dorj1234](https://discuss.elastic.co/u/dorj1234)\ **Replies:** 6\ **Last updated:** [January 25, 2018, 2:38pm UTC](https://discuss.elastic.co/t/invalid-format-message-for-a-date-format-that-always-worked-why-the-sudden-date-parsing-error/114966 "2018-01-25T14:38:05Z")
Hello, I have had csv files for a long time parsing correctly, but something changed and I get the error below. My "run\_date" is a simple field that looks like this in the data files: "2017-12-28" It always worked. I … --- ## [High Availability for Logstash](https://discuss.elastic.co/t/high-availability-for-logstash/116742)
**Author:** [@Anjan\_Pratap](https://discuss.elastic.co/u/Anjan_Pratap)\ **Replies:** 5\ **Last updated:** [January 25, 2018, 2:24pm UTC](https://discuss.elastic.co/t/high-availability-for-logstash/116742 "2018-01-25T14:24:54Z")
Hi Everyone, I have ELK configured with 4 ES Cluster Nodes , one logstash and one Kibana Node. I want to have redundant logstash. Didnt find any solid procedure to configure logstash in cluster. I am using filebeat and… --- ## [Nested JSON use field as value](https://discuss.elastic.co/t/nested-json-use-field-as-value/116831)
**Author:** [@zacesa](https://discuss.elastic.co/u/zacesa)\ **Replies:** 3\ **Last updated:** [January 25, 2018, 11:55am UTC](https://discuss.elastic.co/t/nested-json-use-field-as-value/116831 "2018-01-25T11:55:22Z")
Hi, I have a json input as per below; "stats": { "src": { "port": { "6666": 1 }, "rep": 1, "country": { "US": 1 }, "ip": { "1.1.1.1": { "rep": 1, "uuid": "NULL", "country": "US… --- ## [Where should I place jar file containing de-serializer class for Kafka input plugin?](https://discuss.elastic.co/t/where-should-i-place-jar-file-containing-de-serializer-class-for-kafka-input-plugin/117021)
**Author:** [@elasticheart](https://discuss.elastic.co/u/elasticheart)\ **Replies:** 0\ **Last updated:** [January 25, 2018, 10:17am UTC](https://discuss.elastic.co/t/where-should-i-place-jar-file-containing-de-serializer-class-for-kafka-input-plugin/117021 "2018-01-25T10:17:41Z")
Hi everyone, I am using ELK GA 6.0.0. I am consuming mesages from my Kafka topic, below is my configuration; input { kafka{ group\_id =\> "group\_1" topics =\> \["topic\_1"\] bootstrap\_servers =\> "192.168.0.1:9092" c… --- ## [Log stash Configuration Error](https://discuss.elastic.co/t/log-stash-configuration-error/116280)
**Author:** [@MultiplierMultiplier](https://discuss.elastic.co/u/MultiplierMultiplier)\ **Replies:** 5\ **Last updated:** [January 25, 2018, 10:09am UTC](https://discuss.elastic.co/t/log-stash-configuration-error/116280 "2018-01-25T10:09:40Z")
When I start Logstash I get the below error. I am trying to separate my Gelf and Filebeat input into different indices. Can anyone offer any help? Inputs: input { gelf { host =\> "10.16.0.5" port =\> 20001 … --- ## [Convert a json nested string array to a flat string](https://discuss.elastic.co/t/convert-a-json-nested-string-array-to-a-flat-string/116731)
**Author:** [@hmed](https://discuss.elastic.co/u/hmed)\ **Replies:** 3\ **Last updated:** [January 25, 2018, 10:08am UTC](https://discuss.elastic.co/t/convert-a-json-nested-string-array-to-a-flat-string/116731 "2018-01-25T10:08:59Z")
Hello, I have this JSON { "users":"\[{"id":1,"name":"bob"},{"id":2,"name":"alice"},{"id":3,"name":"david"}\]", "date":"23/01/2017" } even if I'm using the logstash filter: filter { split { field =\> "users" } } t… --- ## [Date config issue and removing GMT](https://discuss.elastic.co/t/date-config-issue-and-removing-gmt/115107)
**Author:** [@spectre](https://discuss.elastic.co/u/spectre)\ **Replies:** 3\ **Last updated:** [January 25, 2018, 10:08am UTC](https://discuss.elastic.co/t/date-config-issue-and-removing-gmt/115107 "2018-01-25T10:08:07Z")
Hello All, I've been asked to find a log storage solution and i found one i love... :slight\_smile: The install of everything went great with zero errors and my initial imports were working but the Kibana functionality … --- ## [How to run and install logstash 6.1.2 on windows 7?](https://discuss.elastic.co/t/how-to-run-and-install-logstash-6-1-2-on-windows-7/117016)
**Author:** [@abhi210797](https://discuss.elastic.co/u/abhi210797)\ **Replies:** 2\ **Last updated:** [January 25, 2018, 10:07am UTC](https://discuss.elastic.co/t/how-to-run-and-install-logstash-6-1-2-on-windows-7/117016 "2018-01-25T10:07:25Z")
Do not install Logstash into a directory path that contains colon (:slight\_smile: characters. What does it mean? --- ## [Logstash not binding to port 5044](https://discuss.elastic.co/t/logstash-not-binding-to-port-5044/116557)
**Author:** [@JamesMcManus](https://discuss.elastic.co/u/JamesMcManus)\ **Replies:** 3\ **Last updated:** [January 25, 2018, 10:07am UTC](https://discuss.elastic.co/t/logstash-not-binding-to-port-5044/116557 "2018-01-25T10:07:01Z")
Hey everyone, Been fiddling around getting ELK stack running over the last couple days but have had issues getting other machines on the local network to send filebeat logs to logstash due to port 5044 not listening. I… --- ## [Wrong timestamp from Windows filebeat](https://discuss.elastic.co/t/wrong-timestamp-from-windows-filebeat/116892)
**Author:** [@uirobot](https://discuss.elastic.co/u/uirobot)\ **Replies:** 2\ **Last updated:** [January 25, 2018, 7:52am UTC](https://discuss.elastic.co/t/wrong-timestamp-from-windows-filebeat/116892 "2018-01-25T07:52:00Z")
Hi! I have a problem with timestamp in logstash. Timestamp in logs set to 01.01.2018, I don't know why. Here is my config: input { beats { port =\> "5044" } } filter { grok { match =\> \["message", "%{TIME:eventt… --- ## [Scrolling vs Sizing vs Batching](https://discuss.elastic.co/t/scrolling-vs-sizing-vs-batching/116386)
**Author:** [@seanziee](https://discuss.elastic.co/u/seanziee)\ **Replies:** 3\ **Last updated:** [January 25, 2018, 7:42am UTC](https://discuss.elastic.co/t/scrolling-vs-sizing-vs-batching/116386 "2018-01-25T07:42:51Z")
Hi, I've been reading as much as I can, but I am still lost to a couple of terms and their interaction/difference: Scroll -\> I don't understand this concept and how it interacts with size size (for an ES input for … --- ## [Netflow module on ELK 6.1.2](https://discuss.elastic.co/t/netflow-module-on-elk-6-1-2/116980)
**Author:** [@kencrozier](https://discuss.elastic.co/u/kencrozier)\ **Replies:** 0\ **Last updated:** [January 25, 2018, 6:55am UTC](https://discuss.elastic.co/t/netflow-module-on-elk-6-1-2/116980 "2018-01-25T06:55:41Z")
I have Netflow v9 coming in from a Fortinet firewall and can see a list of the native flows in the dashboards, but they all have this warning \_\_netflow\_ip\_version\_not\_recognized, \_\_netflow\_direction\_not\_recognized howe… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=833) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=835)