# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1174 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1175 --- ## [Logstash is creating indexes named %{\[@metadata\]\[beat\]}-2016.03.28 and I can't figure out why](https://discuss.elastic.co/t/logstash-is-creating-indexes-named-metadata-beat-2016-03-28-and-i-cant-figure-out-why/45633)
**Author:** [@Matthew\_Prinvale](https://discuss.elastic.co/u/Matthew_Prinvale)\ **Replies:** 2\ **Last updated:** [March 29, 2016, 2:21am UTC](https://discuss.elastic.co/t/logstash-is-creating-indexes-named-metadata-beat-2016-03-28-and-i-cant-figure-out-why/45633 "2016-03-29T02:21:22Z")
here's my logstash config output: output { … --- ## [Logstash 2.2.0 hangs at "Logstash startup completed" while attempting to ingest CSV](https://discuss.elastic.co/t/logstash-2-2-0-hangs-at-logstash-startup-completed-while-attempting-to-ingest-csv/41639)
**Author:** [@MattW224](https://discuss.elastic.co/u/MattW224)\ **Replies:** 3\ **Last updated:** [March 29, 2016, 1:31am UTC](https://discuss.elastic.co/t/logstash-2-2-0-hangs-at-logstash-startup-completed-while-attempting-to-ingest-csv/41639 "2016-03-29T01:31:31Z")
I installed the ELK stack from Elastic's repositories, and am encountering problems while attempting to ingest a 1 GB CSV file on an Amazon Linux (AWS) instance. This is my first attempt at ELK, so there's probably somet… --- ## [Duplicate postgres logs appearing in output](https://discuss.elastic.co/t/duplicate-postgres-logs-appearing-in-output/45455)
**Author:** [@Alexander\_Gray\_II](https://discuss.elastic.co/u/Alexander_Gray_II)\ **Replies:** 1\ **Last updated:** [March 28, 2016, 8:19pm UTC](https://discuss.elastic.co/t/duplicate-postgres-logs-appearing-in-output/45455 "2016-03-28T20:19:04Z")
I posted here: stackoverflow but I think this forum is probably better suited to helping me solve my problem. I basically have a postgres server generating logs like so: postgresql-Sat.csv postgresql-Sun.csv postgresql… --- ## [Is it possible to load LogStash configs like sudoers.d/ works?](https://discuss.elastic.co/t/is-it-possible-to-load-logstash-configs-like-sudoers-d-works/45403)
**Author:** [@Philip\_Dudley](https://discuss.elastic.co/u/Philip_Dudley)\ **Replies:** 3\ **Last updated:** [March 28, 2016, 7:30pm UTC](https://discuss.elastic.co/t/is-it-possible-to-load-logstash-configs-like-sudoers-d-works/45403 "2016-03-28T19:30:44Z")
Hello everyone, and I'm not sure if my question will make sense at first. I installed LogStash using the RPM on a RHEL 6 box. It created a /etc/logstash/conf.d/ folder. I'm wondering if I can have a single LogStash co… --- ## [Logstash adding duplicate rows for every run](https://discuss.elastic.co/t/logstash-adding-duplicate-rows-for-every-run/44775)
**Author:** [@vikram\_yerneni](https://discuss.elastic.co/u/vikram_yerneni)\ **Replies:** 10\ **Last updated:** [March 28, 2016, 6:58pm UTC](https://discuss.elastic.co/t/logstash-adding-duplicate-rows-for-every-run/44775 "2016-03-28T18:58:39Z")
Hello World, We implemented a solution to push Microsoft SQL Data in form of simple rows from SQL Server to Logstash which will index the data to Elasticsearch. The data (SQL rows and columns) are getting successfully by… --- ## [Debug code of file config to get log](https://discuss.elastic.co/t/debug-code-of-file-config-to-get-log/45573)
**Author:** [@Inbeo\_Beo](https://discuss.elastic.co/u/Inbeo_Beo)\ **Replies:** 1\ **Last updated:** [March 28, 2016, 5:04pm UTC](https://discuss.elastic.co/t/debug-code-of-file-config-to-get-log/45573 "2016-03-28T17:04:20Z")
Hi all I make a file config to get log from apache. input { file{ type =\> "apache-access" path =\> " /var/log/logstash/apache.log" start\_position =\> "beginning" … --- ## [Sql\_last\_start error](https://discuss.elastic.co/t/sql-last-start-error/45048)
**Author:** [@jkdenton](https://discuss.elastic.co/u/jkdenton)\ **Replies:** 4\ **Last updated:** [March 28, 2016, 2:31pm UTC](https://discuss.elastic.co/t/sql-last-start-error/45048 "2016-03-28T14:31:40Z")
I get the following error when attempting to use a column value or when using the default timestamp: Java::ComMicrosoftSqlserverJdbc::SQLServerException: Incorrect syntax near ':'.: SELECT \* from table where id \> :sql\_… --- ## [Undefined method \`close' for nil:NilClass](https://discuss.elastic.co/t/undefined-method-close-for-nil-nilclass/45564)
**Author:** [@KiraHort](https://discuss.elastic.co/u/KiraHort)\ **Replies:** 4\ **Last updated:** [March 28, 2016, 1:50pm UTC](https://discuss.elastic.co/t/undefined-method-close-for-nil-nilclass/45564 "2016-03-28T13:50:24Z")
Hi everyone. I'm a beginner in Logstash and I have some problem with it. It's, returning "The error reported is: undefined method \`close' for nil:NilClass". The full report is: The error reported is: undefined method… --- ## [Remove default fields from logstash output](https://discuss.elastic.co/t/remove-default-fields-from-logstash-output/45576)
**Author:** [@valjohn1647](https://discuss.elastic.co/u/valjohn1647)\ **Replies:** 1\ **Last updated:** [March 28, 2016, 11:04am UTC](https://discuss.elastic.co/t/remove-default-fields-from-logstash-output/45576 "2016-03-28T11:04:41Z")
Hi all Im going to use the logstash to send my logfiles to s3 and later Archie it via Glacier . But my issue was logstash by default add the @timestamp and "host" parameter details to the output log Eg: Inp… --- ## [How to express the condition "field A has value B or C"?](https://discuss.elastic.co/t/how-to-express-the-condition-field-a-has-value-b-or-c/45558)
**Author:** [@okwap](https://discuss.elastic.co/u/okwap)\ **Replies:** 1\ **Last updated:** [March 28, 2016, 8:35am UTC](https://discuss.elastic.co/t/how-to-express-the-condition-field-a-has-value-b-or-c/45558 "2016-03-28T08:35:37Z")
I have tried \[A\] in \["B", "C"\] but it does not work. --- ## [Can I create Sincedb file on daily basis?](https://discuss.elastic.co/t/can-i-create-sincedb-file-on-daily-basis/43831)
**Author:** [@vikas\_gopal](https://discuss.elastic.co/u/vikas_gopal)\ **Replies:** 5\ **Last updated:** [March 28, 2016, 7:28am UTC](https://discuss.elastic.co/t/can-i-create-sincedb-file-on-daily-basis/43831 "2016-03-28T07:28:26Z")
Hi Experts, I want to create sincedb file on daily basis from LS1.5.4 . Actually this will help me to re-index today's data only as I am creating index on daily basis. So my plan is to delete today's sincedb file and r… --- ## [Start Logstash from systemctl doesn't populate elasticsearch](https://discuss.elastic.co/t/start-logstash-from-systemctl-doesnt-populate-elasticsearch/45526)
**Author:** [@mchomicz](https://discuss.elastic.co/u/mchomicz)\ **Replies:** 2\ **Last updated:** [March 28, 2016, 12:40am UTC](https://discuss.elastic.co/t/start-logstash-from-systemctl-doesnt-populate-elasticsearch/45526 "2016-03-28T00:40:03Z")
I'm new to ELK, and I am trying to set up multiple index sources (from netflow sources and from snmp via collectd). Seems like the info polled from SNMP is missing ...fields don't show up) I have the following setup: C… --- ## [Optimizing my current GROK Filter](https://discuss.elastic.co/t/optimizing-my-current-grok-filter/45289)
**Author:** [@Sameer\_Panicker](https://discuss.elastic.co/u/Sameer_Panicker)\ **Replies:** 1\ **Last updated:** [March 27, 2016, 8:02am UTC](https://discuss.elastic.co/t/optimizing-my-current-grok-filter/45289 "2016-03-27T08:02:50Z")
I am new to GROK work. As of now, I have created a filter which extracts all the data. But I need to extract only the SessionID, Method, RequestURI and Content values only. This is my request log EventId : 1, Level … --- ## [Does it make sense to pipeline few Logstashs one after another?](https://discuss.elastic.co/t/does-it-make-sense-to-pipeline-few-logstashs-one-after-another/45503)
**Author:** [@mrohad](https://discuss.elastic.co/u/mrohad)\ **Replies:** 7\ **Last updated:** [March 26, 2016, 6:03pm UTC](https://discuss.elastic.co/t/does-it-make-sense-to-pipeline-few-logstashs-one-after-another/45503 "2016-03-26T18:03:35Z")
We would like to read data using our own input plug-in. The data that we receive should go through the log4j-input plug-in. We were thinking to have two Logstashs running in a sequence. the output of the first logstash w… --- ## [Logstash 2.2.2 fails to start on Ubuntu 14.04 LTS with IBM JDK 8](https://discuss.elastic.co/t/logstash-2-2-2-fails-to-start-on-ubuntu-14-04-lts-with-ibm-jdk-8/45469)
**Author:** [@tejas](https://discuss.elastic.co/u/tejas)\ **Replies:** 1\ **Last updated:** [March 25, 2016, 8:24pm UTC](https://discuss.elastic.co/t/logstash-2-2-2-fails-to-start-on-ubuntu-14-04-lts-with-ibm-jdk-8/45469 "2016-03-25T20:24:32Z")
When I start logstash 2.2.2, it fails to run. The error message from the log file is The signal INT is in use by the JVM and will not work correctly on this platform. Logstash Version: $ /opt/logstash/bin/logstash ve… --- ## [Two logstash Servers with Kafka](https://discuss.elastic.co/t/two-logstash-servers-with-kafka/45238)
**Author:** [@kirill\_polishchuk](https://discuss.elastic.co/u/kirill_polishchuk)\ **Replies:** 2\ **Last updated:** [March 25, 2016, 4:41pm UTC](https://discuss.elastic.co/t/two-logstash-servers-with-kafka/45238 "2016-03-25T16:41:31Z")
Hello I have two logstash servers as front-end, with Kafka Output and two logstash servers as back-end with Kafka Input. The front-ends are putting all events into one topic, theoretically the back-ends shoul… --- ## [How to convert json input to Syslog output](https://discuss.elastic.co/t/how-to-convert-json-input-to-syslog-output/45165)
**Author:** [@herofish](https://discuss.elastic.co/u/herofish)\ **Replies:** 1\ **Last updated:** [March 25, 2016, 1:54pm UTC](https://discuss.elastic.co/t/how-to-convert-json-input-to-syslog-output/45165 "2016-03-25T13:54:38Z")
Hello, I have a working Logstash server collecting logs from Windows hosts with the nxlog agent (sending json format). The Logstash server is taking the Windows agent logs and forwarding them to two destinations, a… --- ## [ArgumentError: comparison of Time with nil failed](https://discuss.elastic.co/t/argumenterror-comparison-of-time-with-nil-failed/45239)
**Author:** [@marcmga](https://discuss.elastic.co/u/marcmga)\ **Replies:** 1\ **Last updated:** [March 25, 2016, 1:42pm UTC](https://discuss.elastic.co/t/argumenterror-comparison-of-time-with-nil-failed/45239 "2016-03-25T13:42:56Z")
hello, I got this error in logstash. ArgumentError: comparison of Time with nil failed \>= at org/jruby/RubyComparable.java:155 \>= at org/jruby/RubyTime.java:511 filter at /opt/logst… --- ## [Converting term array into individually name fields](https://discuss.elastic.co/t/converting-term-array-into-individually-name-fields/45397)
**Author:** [@Dani](https://discuss.elastic.co/u/Dani)\ **Replies:** 2\ **Last updated:** [March 25, 2016, 1:36pm UTC](https://discuss.elastic.co/t/converting-term-array-into-individually-name-fields/45397 "2016-03-25T13:36:09Z")
I'm trying to mutate my string into an array and then put each term into a separate field. at the end of my grok, I have this pattern (?\(\[^:\])\*) I've got the following mutate split =\> \["info", "::"\] add\_field… --- ## [Geo\_point: adding a field from other fields gets wrong values](https://discuss.elastic.co/t/geo-point-adding-a-field-from-other-fields-gets-wrong-values/45443)
**Author:** [@sen](https://discuss.elastic.co/u/sen)\ **Replies:** 2\ **Last updated:** [March 25, 2016, 1:33pm UTC](https://discuss.elastic.co/t/geo-point-adding-a-field-from-other-fields-gets-wrong-values/45443 "2016-03-25T13:33:38Z")
Hello everyone, In my configuration file in logstash i want to add a field from other fields in order to get their values in order to use tilemap on kibana. I achieved to create this field and to convert it as a geo\_po… --- ## [Log files are not current](https://discuss.elastic.co/t/log-files-are-not-current/45391)
**Author:** [@rvaedex23](https://discuss.elastic.co/u/rvaedex23)\ **Replies:** 8\ **Last updated:** [March 25, 2016, 1:33pm UTC](https://discuss.elastic.co/t/log-files-are-not-current/45391 "2016-03-25T13:33:08Z")
I am running the Kibana GUI and the log files are not current: This the output of the first log: March 24th 2016, 15:17:08.747 message:2015-08-13 02:36:24,038 DEBUG \[AptService\] \[org.springframework.scheduling.quartz… --- ## [Remove field in output plugin](https://discuss.elastic.co/t/remove-field-in-output-plugin/45353)
**Author:** [@zulquarnain0110](https://discuss.elastic.co/u/zulquarnain0110)\ **Replies:** 2\ **Last updated:** [March 25, 2016, 1:25pm UTC](https://discuss.elastic.co/t/remove-field-in-output-plugin/45353 "2016-03-25T13:25:43Z")
Hi Everyone, I have a problem of day change. I google and saw the following URL My problem is that I don't want to save "index\_day" tag in elasticsearch. Is there any way to remove it or ignore it. --- ## [Logstash not sending to ES when date parsing fails](https://discuss.elastic.co/t/logstash-not-sending-to-es-when-date-parsing-fails/45243)
**Author:** [@dcroonen](https://discuss.elastic.co/u/dcroonen)\ **Replies:** 2\ **Last updated:** [March 25, 2016, 12:33pm UTC](https://discuss.elastic.co/t/logstash-not-sending-to-es-when-date-parsing-fails/45243 "2016-03-25T12:33:17Z")
Hi I have an epoch timestamp which I want to get into the @timestamp field in Elasticsearch/Kibana. When I use the code: date { match =\> \[ "slowlog\_timestamp","UNIX" \] remove\_field =\> \[ "slowlog\_timestamp" \] } I ge… --- ## [Logstash : High CPU usage and stop logging](https://discuss.elastic.co/t/logstash-high-cpu-usage-and-stop-logging/44534)
**Author:** [@yanpeipan](https://discuss.elastic.co/u/yanpeipan)\ **Replies:** 6\ **Last updated:** [March 25, 2016, 10:23am UTC](https://discuss.elastic.co/t/logstash-high-cpu-usage-and-stop-logging/44534 "2016-03-25T10:23:34Z")
my jsstack log: https://gist.github.com/yanpeipan/d9fd90f484d0cd0c9099#file-jsstack-txt my logstash config: https://gist.github.com/yanpeipan/d9fd90f484d0cd0c9099#file-logstash-nginx-conf This problem confuse me fo… --- ## [Can logstash log event when parsing error?](https://discuss.elastic.co/t/can-logstash-log-event-when-parsing-error/45410)
**Author:** [@okwap](https://discuss.elastic.co/u/okwap)\ **Replies:** 1\ **Last updated:** [March 25, 2016, 10:15am UTC](https://discuss.elastic.co/t/can-logstash-log-event-when-parsing-error/45410 "2016-03-25T10:15:25Z")
Hi all, To explain my question, assume the following scenario: I use logstash to handle huge logs, and I notice that /var/log/logstash/logstash.log has an parsing error message: {:timestamp=\>"2016-03-25T10:00:21… --- ## [Grok filter logstash config](https://discuss.elastic.co/t/grok-filter-logstash-config/45306)
**Author:** [@bishaka](https://discuss.elastic.co/u/bishaka)\ **Replies:** 6\ **Last updated:** [March 25, 2016, 10:09am UTC](https://discuss.elastic.co/t/grok-filter-logstash-config/45306 "2016-03-25T10:09:37Z")
Hi, Can somebody help me out here? So i am trying to use grok filter on a particular log file and I am having some trouble. I used http://grokdebug.herokuapp.com/ to make a pattern, but I am having trouble with it so… --- ## [Set logstash to only collect data every X seconds](https://discuss.elastic.co/t/set-logstash-to-only-collect-data-every-x-seconds/45368)
**Author:** [@ssi](https://discuss.elastic.co/u/ssi)\ **Replies:** 2\ **Last updated:** [March 25, 2016, 5:25am UTC](https://discuss.elastic.co/t/set-logstash-to-only-collect-data-every-x-seconds/45368 "2016-03-25T05:25:18Z")
is there an option for logstash to only collect to elasticsearch every X second i indexing netflow do display network bandwith usage and getting the data every other second or every 5 seconds is enough for us to see if… --- ## [Can't get a field type IP](https://discuss.elastic.co/t/cant-get-a-field-type-ip/45255)
**Author:** [@johncst](https://discuss.elastic.co/u/johncst)\ **Replies:** 11\ **Last updated:** [March 24, 2016, 8:44pm UTC](https://discuss.elastic.co/t/cant-get-a-field-type-ip/45255 "2016-03-24T20:44:26Z")
I'm trying to get an IP field so I can then geoip it. I've looked at the blog posts and other sites, and tried everything, but still no working IP fields. I looked at the internal blog posts, and some seem to minimize th… --- ## [Logstash csv check fields for numeric and mutate to integer](https://discuss.elastic.co/t/logstash-csv-check-fields-for-numeric-and-mutate-to-integer/45293)
**Author:** [@rbkumar88](https://discuss.elastic.co/u/rbkumar88)\ **Replies:** 3\ **Last updated:** [March 24, 2016, 7:03pm UTC](https://discuss.elastic.co/t/logstash-csv-check-fields-for-numeric-and-mutate-to-integer/45293 "2016-03-24T19:03:11Z")
Hi, I have a csv filter which generates the fields. I want to check all the generated fields (whether all the characters are numbers), if then mutate those fields to Integer. Regards Bharath --- ## [Problem with logstash and beats](https://discuss.elastic.co/t/problem-with-logstash-and-beats/37376)
**Author:** [@Max\_Just](https://discuss.elastic.co/u/Max_Just)\ **Replies:** 5\ **Last updated:** [March 24, 2016, 6:48pm UTC](https://discuss.elastic.co/t/problem-with-logstash-and-beats/37376 "2016-03-24T18:48:43Z")
Hi, I have got the follow errors in logstash: {:timestamp=\>"2015-12-16T06:42:13.227000+0300", :message=\>"Beats input: unhandled exception", :exception=\>#\, :backtr… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1173) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1175)