# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=868 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 869 --- ## [Using logstash+http\_poller to loop through JSON structure, run 2nd poller](https://discuss.elastic.co/t/using-logstash-http-poller-to-loop-through-json-structure-run-2nd-poller/109121)
**Author:** [@dorj1234](https://discuss.elastic.co/u/dorj1234)\ **Replies:** 1\ **Last updated:** [November 27, 2017, 6:30am UTC](https://discuss.elastic.co/t/using-logstash-http-poller-to-loop-through-json-structure-run-2nd-poller/109121 "2017-11-27T06:30:05Z")
Hello, I have an http\_poller working and getting me an array of JSON structures. Is it possible in logstash to loop through one of the fields and for each one run a second http\_poller? For example, let's say my struct… --- ## [How to get step by steps results as logstash processes the conf file](https://discuss.elastic.co/t/how-to-get-step-by-steps-results-as-logstash-processes-the-conf-file/109073)
**Author:** [@pgervais](https://discuss.elastic.co/u/pgervais)\ **Replies:** 4\ **Last updated:** [November 26, 2017, 10:00pm UTC](https://discuss.elastic.co/t/how-to-get-step-by-steps-results-as-logstash-processes-the-conf-file/109073 "2017-11-26T22:00:30Z")
Is there a way to get step by step results at stdout as each filter processes the input stream? Specifically, i'm parsing csv data. I assume that the csv input stream is converted in the order that input filter is defi… --- ## [Logstash not able to start. Error undefined method set\_validate\_after\_inactivity](https://discuss.elastic.co/t/logstash-not-able-to-start-error-undefined-method-set-validate-after-inactivity/109148)
**Author:** [@Abel\_Gbeli](https://discuss.elastic.co/u/Abel_Gbeli)\ **Replies:** 0\ **Last updated:** [November 26, 2017, 8:21pm UTC](https://discuss.elastic.co/t/logstash-not-able-to-start-error-undefined-method-set-validate-after-inactivity/109148 "2017-11-26T20:21:08Z")
Hi i'm beginner in Logstash and i have some problem with it. I am Using Logstash-6.0.0 with windows 10. The service is not getting started. Its breaking with following error mentioned below : \[2017-11-26T20:00:07,621\]\[… --- ## [Logstash Index problem after logrotate](https://discuss.elastic.co/t/logstash-index-problem-after-logrotate/108463)
**Author:** [@Vafa\_Ronaghi](https://discuss.elastic.co/u/Vafa_Ronaghi)\ **Replies:** 12\ **Last updated:** [November 26, 2017, 7:58pm UTC](https://discuss.elastic.co/t/logstash-index-problem-after-logrotate/108463 "2017-11-26T19:58:26Z")
Logstash is having problem to read log files after log rotation and send it to Elasticsearch. This issue occures when a log file with only one line of log is getting rotated and the new log file also has only one line. … --- ## [Reading the heading (1st line) of CSV file through logstash](https://discuss.elastic.co/t/reading-the-heading-1st-line-of-csv-file-through-logstash/108930)
**Author:** [@kiranilla](https://discuss.elastic.co/u/kiranilla)\ **Replies:** 7\ **Last updated:** [November 26, 2017, 6:00pm UTC](https://discuss.elastic.co/t/reading-the-heading-1st-line-of-csv-file-through-logstash/108930 "2017-11-26T18:00:03Z")
HI, Eg: sample CSV file Incident ID,Status,Resolved By,Resolution Breached,Resolution Month,Resolution Date,Resolution Date & Time, IM02370568,Closed,GUNTURS2,FALSE,1,1/4/2016,1/4/2016 10:35 IM02370648,Closed,PRASADR6,… --- ## [This error message Standard Nginx Error Log?](https://discuss.elastic.co/t/this-error-message-standard-nginx-error-log/109102)
**Author:** [@omid](https://discuss.elastic.co/u/omid)\ **Replies:** 2\ **Last updated:** [November 26, 2017, 7:37am UTC](https://discuss.elastic.co/t/this-error-message-standard-nginx-error-log/109102 "2017-11-26T07:37:38Z")
this error message Standard Nginx Error Log ?????????? 2017/11/25 06:25:18 \[error\] 14872#14872: \*16949093 FastCGI sent in stderr: "PHP message: PHP Notice: Undefined variable: params in /usr/share/nginx/www/mysitename… --- ## [Basic http\_poller question with REST API](https://discuss.elastic.co/t/basic-http-poller-question-with-rest-api/109085)
**Author:** [@dorj1234](https://discuss.elastic.co/u/dorj1234)\ **Replies:** 2\ **Last updated:** [November 25, 2017, 8:30pm UTC](https://discuss.elastic.co/t/basic-http-poller-question-with-rest-api/109085 "2017-11-25T20:30:20Z")
Hello, I read all posts for similar issues and still unable to get the output correctly from http\_poller and REST API. Not sure what I am missing and how to display the metadata or the JSON fields correctly. When this … --- ## [Logstash 6 not working after fresh service install](https://discuss.elastic.co/t/logstash-6-not-working-after-fresh-service-install/108236)
**Author:** [@Tomschi](https://discuss.elastic.co/u/Tomschi)\ **Replies:** 3\ **Last updated:** [November 25, 2017, 4:43pm UTC](https://discuss.elastic.co/t/logstash-6-not-working-after-fresh-service-install/108236 "2017-11-25T16:43:24Z")
I installed Logstash 6 on a CentOS 7 using the repo variant. After starting Logstash over systemd as service the logstash process consumes over 100% CPU and is not running correctly. OS: CentOS 7 Java: openjdk 8 I al… --- ## [High CPU usage with clean Logstash install](https://discuss.elastic.co/t/high-cpu-usage-with-clean-logstash-install/106598)
**Author:** [@Ender](https://discuss.elastic.co/u/Ender)\ **Replies:** 5\ **Last updated:** [November 25, 2017, 4:42pm UTC](https://discuss.elastic.co/t/high-cpu-usage-with-clean-logstash-install/106598 "2017-11-25T16:42:03Z")
Hi, I'm new on Elastic Stack but there is a problem with my logstash service. Without any configuration (or with a basic configuration), there is a heavy CPU usage when i start logstash, my server hanging: \[logstash\_… --- ## [Trying to import a stock price file into Elastic](https://discuss.elastic.co/t/trying-to-import-a-stock-price-file-into-elastic/108716)
**Author:** [@sharon1](https://discuss.elastic.co/u/sharon1)\ **Replies:** 3\ **Last updated:** [November 25, 2017, 10:04am UTC](https://discuss.elastic.co/t/trying-to-import-a-stock-price-file-into-elastic/108716 "2017-11-25T10:04:27Z")
Goodday, I am a newbie and still learning. Am trying to import a stock price file into Elastic. Spent some time to setup a Logstash config file, but didn't succeed. This is the format of the stock price file: 01/08/… --- ## [Could not find path of jruby in](https://discuss.elastic.co/t/could-not-find-path-of-jruby-in/108963)
**Author:** [@Krunal\_kalaria](https://discuss.elastic.co/u/Krunal_kalaria)\ **Replies:** 2\ **Last updated:** [November 25, 2017, 6:36am UTC](https://discuss.elastic.co/t/could-not-find-path-of-jruby-in/108963 "2017-11-25T06:36:16Z")
Hello, I have an issue with logstash i am using logstash 6.0.0 and when i running the configuration file then its give some error like following: \[logstash\] help me to resolve this issue i am using windows server R2 2… --- ## [Logstash csv generates an error when \\n is in quoted field](https://discuss.elastic.co/t/logstash-csv-generates-an-error-when-n-is-in-quoted-field/108927)
**Author:** [@yahoo](https://discuss.elastic.co/u/yahoo)\ **Replies:** 4\ **Last updated:** [November 24, 2017, 9:04pm UTC](https://discuss.elastic.co/t/logstash-csv-generates-an-error-when-n-is-in-quoted-field/108927 "2017-11-24T21:04:15Z")
The csv plugin for logstash 5.6.3 is generating the following error: logstash.filters.csv MalformedCSVError: Unclosed quoted field "tags" =\> \[ \[0\] "\_csvparsefailure" The input data are: field1, field2 aaa… --- ## [Logstash Http Input overwriting field](https://discuss.elastic.co/t/logstash-http-input-overwriting-field/109065)
**Author:** [@dj50x](https://discuss.elastic.co/u/dj50x)\ **Replies:** 2\ **Last updated:** [November 24, 2017, 8:19pm UTC](https://discuss.elastic.co/t/logstash-http-input-overwriting-field/109065 "2017-11-24T20:19:10Z")
So I recently changed from a TCP to a HTTP listener on logstash. I use Nxlog to transform my logs to Json and send it to a http listener on logstash with the codec json. One of my fields is simply 'host', which the hostn… --- ## [Parse failure Apache accesslog with empty remote user](https://discuss.elastic.co/t/parse-failure-apache-accesslog-with-empty-remote-user/107886)
**Author:** [@RomainD](https://discuss.elastic.co/u/RomainD)\ **Replies:** 4\ **Last updated:** [November 24, 2017, 5:12pm UTC](https://discuss.elastic.co/t/parse-failure-apache-accesslog-with-empty-remote-user/107886 "2017-11-24T17:12:50Z")
Hello, I have a \_grokparsefailure when the remote user is empty, let's show you the configurations files: Apache v2.4 Logstash v5.6.3 apache2.conf, my custom log format, %u is the remote user : LogFormat "%a %l %u \[… --- ## [Logstash 2.4.0 gets Sequel::DatabaseConnectionError in JDBC Input](https://discuss.elastic.co/t/logstash-2-4-0-gets-sequel-databaseconnectionerror-in-jdbc-input/109038)
**Author:** [@AlessandroC985](https://discuss.elastic.co/u/AlessandroC985)\ **Replies:** 3\ **Last updated:** [November 24, 2017, 3:00pm UTC](https://discuss.elastic.co/t/logstash-2-4-0-gets-sequel-databaseconnectionerror-in-jdbc-input/109038 "2017-11-24T15:00:56Z")
Hi, I have a docker image of ELK 4 stack with logstash 2.4.0 installed. In that container I have configured logstash with the following configuration: input { beats { ssl =\> false host =\> "0.0.0.0" port … --- ## [Grok csv filter](https://discuss.elastic.co/t/grok-csv-filter/108839)
**Author:** [@mbvelo](https://discuss.elastic.co/u/mbvelo)\ **Replies:** 13\ **Last updated:** [November 24, 2017, 1:00pm UTC](https://discuss.elastic.co/t/grok-csv-filter/108839 "2017-11-24T13:00:07Z")
Please assist with CSV Filter i tried using columns and Source nothing is happening to my event. csv { source =\> "payData.features" } csv { columns=\> "" } \[csv\] --- ## [Extract string from JSON and put into new value](https://discuss.elastic.co/t/extract-string-from-json-and-put-into-new-value/108808)
**Author:** [@sdndude](https://discuss.elastic.co/u/sdndude)\ **Replies:** 7\ **Last updated:** [November 24, 2017, 12:56pm UTC](https://discuss.elastic.co/t/extract-string-from-json-and-put-into-new-value/108808 "2017-11-24T12:56:05Z")
I am still fairly new to Logstash and I am starting to get on the tips of my skates when comes to understanding how to do what I need to. I am receiving JSON from a device (via HTTP payload) and it is super easy to slap… --- ## [Logstash multiline codec ignore last event / line](https://discuss.elastic.co/t/logstash-multiline-codec-ignore-last-event-line/109006)
**Author:** [@metju312](https://discuss.elastic.co/u/metju312)\ **Replies:** 1\ **Last updated:** [November 24, 2017, 12:52pm UTC](https://discuss.elastic.co/t/logstash-multiline-codec-ignore-last-event-line/109006 "2017-11-24T12:52:53Z")
Hello, Logstash multiline codec ignore my last event (line) until send next packege of logs. In extremely case - if log have only one line it looks like there is no event at all. My logstash.conf: input { } http… --- ## [Not able to parse custom logs having multi line xml](https://discuss.elastic.co/t/not-able-to-parse-custom-logs-having-multi-line-xml/107743)
**Author:** [@Husain\_Khan](https://discuss.elastic.co/u/Husain_Khan)\ **Replies:** 14\ **Last updated:** [November 24, 2017, 12:39pm UTC](https://discuss.elastic.co/t/not-able-to-parse-custom-logs-having-multi-line-xml/107743 "2017-11-24T12:39:53Z")
I have following log file, 5d563f04-b5d8-4b8d-b3ac-df26028c3719 SoapRequest CheckUserPassword \ \crt-mon\ \2016.11.01\ \01:23:04 CET\ \11.6\ \… --- ## [How to delete/hide fields in logstash filter](https://discuss.elastic.co/t/how-to-delete-hide-fields-in-logstash-filter/109021)
**Author:** [@Makra](https://discuss.elastic.co/u/Makra)\ **Replies:** 1\ **Last updated:** [November 24, 2017, 12:19pm UTC](https://discuss.elastic.co/t/how-to-delete-hide-fields-in-logstash-filter/109021 "2017-11-24T12:19:56Z")
Hi How should i delete the fields "\_id", "\_type", "\_index", "\_score" in logstash filter so that these fields does not show up in the kibana dashboard ? In the mutate filter, remove field does not delete these fields … --- ## [TCP output hangs while sending syslog](https://discuss.elastic.co/t/tcp-output-hangs-while-sending-syslog/109025)
**Author:** [@vishal0708gautam](https://discuss.elastic.co/u/vishal0708gautam)\ **Replies:** 1\ **Last updated:** [November 24, 2017, 12:10pm UTC](https://discuss.elastic.co/t/tcp-output-hangs-while-sending-syslog/109025 "2017-11-24T12:10:09Z")
Hello Experts, We facing an issue, while sending syslog event output from TCP, some times TCP hangs and then outputs the syslog event. The major problem here is that if any event comes in between for output, because TCP… --- ## [Remove header information added by logstash](https://discuss.elastic.co/t/remove-header-information-added-by-logstash/108892)
**Author:** [@vishal0708gautam](https://discuss.elastic.co/u/vishal0708gautam)\ **Replies:** 4\ **Last updated:** [November 24, 2017, 12:08pm UTC](https://discuss.elastic.co/t/remove-header-information-added-by-logstash/108892 "2017-11-24T12:08:26Z")
Hello experts, We are using logstash as syslog event receiver and forwarder using UDP & TCP and it is doing as expected but on forwarding syslog events to output, logstash is adding its own header, we want to remove thi… --- ## [One Logstash file with multiple logs entries/pipelines](https://discuss.elastic.co/t/one-logstash-file-with-multiple-logs-entries-pipelines/108991)
**Author:** [@Milad\_Hamid](https://discuss.elastic.co/u/Milad_Hamid)\ **Replies:** 7\ **Last updated:** [November 24, 2017, 11:43am UTC](https://discuss.elastic.co/t/one-logstash-file-with-multiple-logs-entries-pipelines/108991 "2017-11-24T11:43:52Z")
Hi, I have two different logs source ( Syslogs;logstash-syslog.conf, Apache logs;logstash-apache.conf), and i want to get the syslogs and apache logs into logstash and then ship them to elasticsearch with one logstash i… --- ## [Reading the properties file from input section of logstash](https://discuss.elastic.co/t/reading-the-properties-file-from-input-section-of-logstash/108931)
**Author:** [@kiranilla](https://discuss.elastic.co/u/kiranilla)\ **Replies:** 3\ **Last updated:** [November 24, 2017, 10:52am UTC](https://discuss.elastic.co/t/reading-the-properties-file-from-input-section-of-logstash/108931 "2017-11-24T10:52:17Z")
Hi, I am able to read the properties file from filter section of logstash but not from input section. Could you please help me out in reading the .properties file from input section of logstash. Thanks in advance. Re… --- ## [Ruby maths operation error](https://discuss.elastic.co/t/ruby-maths-operation-error/108966)
**Author:** [@Shekhar\_Pandey](https://discuss.elastic.co/u/Shekhar_Pandey)\ **Replies:** 4\ **Last updated:** [November 24, 2017, 9:11am UTC](https://discuss.elastic.co/t/ruby-maths-operation-error/108966 "2017-11-24T09:11:46Z")
Hi.. ruby { code =\> "event\['discount'\] = \[\[event\['mrp'\].to\_i - event\['price'\].to\_i\].to\_f / event\['mrp'\].to\_f \] \* 100" } But the above code some what wrong....can anybody explain where I am wrong... Th… --- ## [LS5.3: Variable codec setting](https://discuss.elastic.co/t/ls5-3-variable-codec-setting/108986)
**Author:** [@AlexW](https://discuss.elastic.co/u/AlexW)\ **Replies:** 4\ **Last updated:** [November 24, 2017, 9:09am UTC](https://discuss.elastic.co/t/ls5-3-variable-codec-setting/108986 "2017-11-24T09:09:27Z")
Hi! I wanted to optimize my Logstash config. The old one looked like this: output { if ("json\_extracted" in \[tags\]) { elasticsearch { codec =\> "json" hosts =\> \<%= $es\_hosts.prefix('"').suffix('"') %\> … --- ## [Logstash reject the metrics](https://discuss.elastic.co/t/logstash-reject-the-metrics/108953)
**Author:** [@Robin\_Guo](https://discuss.elastic.co/u/Robin_Guo)\ **Replies:** 1\ **Last updated:** [November 24, 2017, 6:30am UTC](https://discuss.elastic.co/t/logstash-reject-the-metrics/108953 "2017-11-24T06:30:38Z")
Hi folks, I've been running new ELK stack(5.5.1) for a month, Recently we found some problems on ELK side. we lost some metrics over time on Grafana. It's so bad. and It's super slow to load fields when I try to cust… --- ## [Logstash : convert date to unix time problem](https://discuss.elastic.co/t/logstash-convert-date-to-unix-time-problem/108711)
**Author:** [@linsie](https://discuss.elastic.co/u/linsie)\ **Replies:** 11\ **Last updated:** [November 24, 2017, 5:42am UTC](https://discuss.elastic.co/t/logstash-convert-date-to-unix-time-problem/108711 "2017-11-24T05:42:43Z")
logstash verison : 5.0.1 i want to convert the time : 20171122194855.000000480 to unix time how can i do in logstash filter ? thanks so much. --- ## [Error in logstash.agent - Bad file descriptor (Errno::EBADF)](https://discuss.elastic.co/t/error-in-logstash-agent-bad-file-descriptor-errno-ebadf/108961)
**Author:** [@arbabu89](https://discuss.elastic.co/u/arbabu89)\ **Replies:** 0\ **Last updated:** [November 24, 2017, 4:31am UTC](https://discuss.elastic.co/t/error-in-logstash-agent-bad-file-descriptor-errno-ebadf/108961 "2017-11-24T04:31:00Z")
Hi I am trying to load data from MS SQL server to Elastic search using logstash and got error in logstash agent this is my logstash config file - input { jdbc { jdbc\_driver\_library =\> "C:\\local\\lib\\sqljdbc42.jar" j… --- ## [Adding custom pattern failed](https://discuss.elastic.co/t/adding-custom-pattern-failed/108686)
**Author:** [@dominicdeng](https://discuss.elastic.co/u/dominicdeng)\ **Replies:** 6\ **Last updated:** [November 24, 2017, 1:53am UTC](https://discuss.elastic.co/t/adding-custom-pattern-failed/108686 "2017-11-24T01:53:52Z")
Hi experts I am a newer to Kibana(6.x) , when I tried to add custom patterns with the instruction 'https://www.elastic.co/guide/en/kibana/current/grokdebugger-getting-started.html' but failed. \[1\] Can anyone help me… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=867) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=869)