# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=549 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 550 --- ## [Logstash writing into wrong index](https://discuss.elastic.co/t/logstash-writing-into-wrong-index/180421)
**Author:** [@chandukreddi](https://discuss.elastic.co/u/chandukreddi)\ **Replies:** 11\ **Last updated:** [May 13, 2019, 5:55pm UTC](https://discuss.elastic.co/t/logstash-writing-into-wrong-index/180421 "2019-05-13T17:55:51Z")
Hello Team, I have 2 conf files you can see below 1st conf file using output index alias "stats\_write\_swift\_proxy\_alias" to process proxy-nodes-monitor-stats-.csv & proxy-nodes-latency-stats-.csv files but I see this a… --- ## [Unable to parse datestamp](https://discuss.elastic.co/t/unable-to-parse-datestamp/180746)
**Author:** [@Nikhil\_Pillai](https://discuss.elastic.co/u/Nikhil_Pillai)\ **Replies:** 2\ **Last updated:** [May 13, 2019, 4:30pm UTC](https://discuss.elastic.co/t/unable-to-parse-datestamp/180746 "2019-05-13T16:30:49Z")
Hi, Unable to parse datestamp as per the following log pattern. \[03/05/19 05:38:00:000 IST\] ERROR scheduler.EDISFileReaderTask: @@ Scheduler StepNumber 3 (incoming folder is empty OR it doesn't have EXCEL files) @@ Pa… --- ## [NoMethodError: undefined method \`\[\]' for nil:NilClass](https://discuss.elastic.co/t/nomethoderror-undefined-method-for-nil-nilclass/180696)
**Author:** [@pirimoglu](https://discuss.elastic.co/u/pirimoglu)\ **Replies:** 5\ **Last updated:** [May 13, 2019, 4:13pm UTC](https://discuss.elastic.co/t/nomethoderror-undefined-method-for-nil-nilclass/180696 "2019-05-13T16:13:40Z")
Hi, I try to use logstash.filters.aggregate plugin. I get an aggregate exception. Aggregate exception occurred {:error=\>#\, :code=\>"\\n\\t\\t\\tmap\['user'\]\['scopes'\] \<\<… --- ## [Dynamic Field Parsing](https://discuss.elastic.co/t/dynamic-field-parsing/180225)
**Author:** [@kdotson](https://discuss.elastic.co/u/kdotson)\ **Replies:** 5\ **Last updated:** [May 13, 2019, 3:07pm UTC](https://discuss.elastic.co/t/dynamic-field-parsing/180225 "2019-05-13T15:07:33Z")
Hello, I have a csv file which has a few fields that have quoted dynamic entries. I used the csv module to break out the fields and now I am trying to account for the embedded subfields. There are two different scenar… --- ## [Creation of new nested object in Ruby Filter yields no object in ElasticSearch](https://discuss.elastic.co/t/creation-of-new-nested-object-in-ruby-filter-yields-no-object-in-elasticsearch/180575)
**Author:** [@ben.dixon](https://discuss.elastic.co/u/ben.dixon)\ **Replies:** 2\ **Last updated:** [May 13, 2019, 2:55pm UTC](https://discuss.elastic.co/t/creation-of-new-nested-object-in-ruby-filter-yields-no-object-in-elasticsearch/180575 "2019-05-13T14:55:28Z")
I've got a logstash pipeline that takes discrete log trace events and writes them to a trace-oriented document in ElasticSearch. Currently I had been storing only the timestamp of the event in an array field in the docum… --- ## [Field type in logstash not applied to the elastic index](https://discuss.elastic.co/t/field-type-in-logstash-not-applied-to-the-elastic-index/180761)
**Author:** [@Shawcs](https://discuss.elastic.co/u/Shawcs)\ **Replies:** 2\ **Last updated:** [May 13, 2019, 2:49pm UTC](https://discuss.elastic.co/t/field-type-in-logstash-not-applied-to-the-elastic-index/180761 "2019-05-13T14:49:25Z")
Hi, I have an issue with some of my field type. In logstash I convert some fields to be in integer: mutate { add\_field =\> { "api\_transaction\_duration" =\> "%{\[json\]\[transactionElement\]\[duration\]}" "api… --- ## [Logstash configuration - comment breaks the logstash configuration](https://discuss.elastic.co/t/logstash-configuration-comment-breaks-the-logstash-configuration/180802)
**Author:** [@pastechecker](https://discuss.elastic.co/u/pastechecker)\ **Replies:** 3\ **Last updated:** [May 13, 2019, 2:06pm UTC](https://discuss.elastic.co/t/logstash-configuration-comment-breaks-the-logstash-configuration/180802 "2019-05-13T14:06:24Z")
Hello. I am running Logstash 6.7.0 My config is: input { stdin { } } filter { json { source =\> "message" } if \[Severity\] == "INFO" { #comment line 5 mutate { add\_field =\> { "@timestamp" =\> "%{timestamp}" #comm… --- ## [Can logstash handle 15 file beats sending heavy traffic concurrently?](https://discuss.elastic.co/t/can-logstash-handle-15-file-beats-sending-heavy-traffic-concurrently/180784)
**Author:** [@hannkeat](https://discuss.elastic.co/u/hannkeat)\ **Replies:** 2\ **Last updated:** [May 13, 2019, 1:56pm UTC](https://discuss.elastic.co/t/can-logstash-handle-15-file-beats-sending-heavy-traffic-concurrently/180784 "2019-05-13T13:56:49Z")
As per above. Wish to setup 15 file beat and ship data to logstash, is single logstash able to handle ? if not what is the setup will be ? Currently running 3 master node. Thanks is advance. --- ## [Give logstash two different log files with filebeat but i only get one in Elasticsearch](https://discuss.elastic.co/t/give-logstash-two-different-log-files-with-filebeat-but-i-only-get-one-in-elasticsearch/180749)
**Author:** [@Axel\_Kruger](https://discuss.elastic.co/u/Axel_Kruger)\ **Replies:** 6\ **Last updated:** [May 13, 2019, 1:54pm UTC](https://discuss.elastic.co/t/give-logstash-two-different-log-files-with-filebeat-but-i-only-get-one-in-elasticsearch/180749 "2019-05-13T13:54:01Z")
Hey, i'v got the problem that i have only one input in to logstash via filebeat but two difrent logfiles, a liferaylog an a apache Acceess\_log. i can see that filebeat analyse that two files and send it tp logstash. IN… --- ## [Where do I put the template file for the elasticsearch filter?](https://discuss.elastic.co/t/where-do-i-put-the-template-file-for-the-elasticsearch-filter/180819)
**Author:** [@jdmcalee](https://discuss.elastic.co/u/jdmcalee)\ **Replies:** 2\ **Last updated:** [May 13, 2019, 1:47pm UTC](https://discuss.elastic.co/t/where-do-i-put-the-template-file-for-the-elasticsearch-filter/180819 "2019-05-13T13:47:35Z")
I'm trying to add a new filter to a pipeline with the following config: elasticsearch { index =\> "customers" query\_template =\> "ticket-customer-search.json" docinfo\_fields =\> { "\_id" =\> "customerId" } } It … --- ## [Jmx input plugin number of threads](https://discuss.elastic.co/t/jmx-input-plugin-number-of-threads/180786)
**Author:** [@sahere37](https://discuss.elastic.co/u/sahere37)\ **Replies:** 0\ **Last updated:** [May 13, 2019, 10:23am UTC](https://discuss.elastic.co/t/jmx-input-plugin-number-of-threads/180786 "2019-05-13T10:23:48Z")
Hi all, I am going to use JMX input plugin in Logstash, in the jmx setting, we can set number of threads as "nb\_thread". what is best value for this field? it is noted that number of cpu cores is 4. could you please adv… --- ## [File not closed with JDBC input and CSV output](https://discuss.elastic.co/t/file-not-closed-with-jdbc-input-and-csv-output/180566)
**Author:** [@romariol](https://discuss.elastic.co/u/romariol)\ **Replies:** 4\ **Last updated:** [May 13, 2019, 7:42am UTC](https://discuss.elastic.co/t/file-not-closed-with-jdbc-input-and-csv-output/180566 "2019-05-13T07:42:55Z")
Hi, I'm using a JDBC input to fetch data from a specific table. I want to put this data in a new file. For this I'm using the CSV output. What I'm seeing, is that the file remains open. I want this file to be closed, be… --- ## [Logstash netflow codec v5 First/EndSwitched timestamps shown with .999 precision](https://discuss.elastic.co/t/logstash-netflow-codec-v5-first-endswitched-timestamps-shown-with-999-precision/180768)
**Author:** [@Zeeshan\_Alam](https://discuss.elastic.co/u/Zeeshan_Alam)\ **Replies:** 0\ **Last updated:** [May 13, 2019, 9:19am UTC](https://discuss.elastic.co/t/logstash-netflow-codec-v5-first-endswitched-timestamps-shown-with-999-precision/180768 "2019-05-13T09:19:17Z")
Logstash: 6.7.0 Plugin: logstash-codec-netflow (3.14.1) OS: RHEL 7 There is a problem with the first\_switched and last\_switched timestamps as the documents are not showing millisecond precision. All my netflow events… --- ## [Netflow decode error](https://discuss.elastic.co/t/netflow-decode-error/180766)
**Author:** [@Haresh\_Perera](https://discuss.elastic.co/u/Haresh_Perera)\ **Replies:** 0\ **Last updated:** [May 13, 2019, 9:09am UTC](https://discuss.elastic.co/t/netflow-decode-error/180766 "2019-05-13T09:09:48Z")
ogstash.codecs.netflow \] Can't (yet) decode flowset id 258 from source id 258, because no template to decode it with has been received. This message will usually go away after 1 minute. --- ## [How to run logstash in unix as background service](https://discuss.elastic.co/t/how-to-run-logstash-in-unix-as-background-service/180757)
**Author:** [@mandar.raj](https://discuss.elastic.co/u/mandar.raj)\ **Replies:** 0\ **Last updated:** [May 13, 2019, 8:25am UTC](https://discuss.elastic.co/t/how-to-run-logstash-in-unix-as-background-service/180757 "2019-05-13T08:25:09Z")
How to run logstash in unix as background service? --- ## [ALERT TRIGGERING FOR EVERY 30MINUTES](https://discuss.elastic.co/t/alert-triggering-for-every-30minutes/180318)
**Author:** [@Ashok2](https://discuss.elastic.co/u/Ashok2)\ **Replies:** 7\ **Last updated:** [May 13, 2019, 7:25am UTC](https://discuss.elastic.co/t/alert-triggering-for-every-30minutes/180318 "2019-05-13T07:25:59Z")
Hi, I have the logstash config file in which i have written the mail alert for particular text present then automatically send an email with the message but i keep on receiving lots of mail continuously. I need to hav… --- ## [Logstash-output-email-plugin installation error](https://discuss.elastic.co/t/logstash-output-email-plugin-installation-error/180745)
**Author:** [@cppatel](https://discuss.elastic.co/u/cppatel)\ **Replies:** 0\ **Last updated:** [May 13, 2019, 6:46am UTC](https://discuss.elastic.co/t/logstash-output-email-plugin-installation-error/180745 "2019-05-13T06:46:28Z")
Hi all, When I try to install the logstash-output-email-4.1.1.zip plugin it's give me following error. I have downloaded it directly from github. It also includes gem to run. Please help me with the installation. Insta… --- ## [Logstash filter: Convert all fields in json to integer](https://discuss.elastic.co/t/logstash-filter-convert-all-fields-in-json-to-integer/180662)
**Author:** [@Neeraj\_Jain](https://discuss.elastic.co/u/Neeraj_Jain)\ **Replies:** 2\ **Last updated:** [May 13, 2019, 6:41am UTC](https://discuss.elastic.co/t/logstash-filter-convert-all-fields-in-json-to-integer/180662 "2019-05-13T06:41:53Z")
Hi, I have a json data of below format (in rubydebug output). I want to convert all the counter values to integer. with mutate filter plugin, i have to convert each counter to integer. However, this is a shortened data.… --- ## [Class JDBC not found](https://discuss.elastic.co/t/class-jdbc-not-found/180700)
**Author:** [@KashiPashoria\_Tonny](https://discuss.elastic.co/u/KashiPashoria_Tonny)\ **Replies:** 7\ **Last updated:** [May 13, 2019, 5:17am UTC](https://discuss.elastic.co/t/class-jdbc-not-found/180700 "2019-05-13T05:17:19Z")
I'm trying to ingest data using logstash 1.5 to an ES 1.5 index. Every time I run logstash gives error D:\\ES\\logstash-1.5.1\\bin\>logstash -f logstash\_conf.conf io/console not supported; tty will not be manipulated The … --- ## [Parse timestamp not being indexed](https://discuss.elastic.co/t/parse-timestamp-not-being-indexed/180718)
**Author:** [@bkasrai](https://discuss.elastic.co/u/bkasrai)\ **Replies:** 2\ **Last updated:** [May 13, 2019, 12:49am UTC](https://discuss.elastic.co/t/parse-timestamp-not-being-indexed/180718 "2019-05-13T00:49:02Z")
I am attempting to parse log data using GROK and KV filters and I am running into an issue with the date\\timestamp field. When I add the timestamp to my GROK statement, it is picked up and added fine when being sent to E… --- ## [Logstash monitoring not working](https://discuss.elastic.co/t/logstash-monitoring-not-working/180706)
**Author:** [@siddaram\_kj](https://discuss.elastic.co/u/siddaram_kj)\ **Replies:** 0\ **Last updated:** [May 12, 2019, 4:47pm UTC](https://discuss.elastic.co/t/logstash-monitoring-not-working/180706 "2019-05-12T16:47:08Z")
i have using logstash verion 7.0 , and i have configure the logstash with trial licence and xpack.monitoring .enabled set to true. The logstash monitoring indices are getting created in the elasticsearch cluster but wher… --- ## [Logstash Docker default port](https://discuss.elastic.co/t/logstash-docker-default-port/180704)
**Author:** [@toni](https://discuss.elastic.co/u/toni)\ **Replies:** 0\ **Last updated:** [May 12, 2019, 4:13pm UTC](https://discuss.elastic.co/t/logstash-docker-default-port/180704 "2019-05-12T16:13:51Z")
Hi, I am trying to use the official Logstash Docker image in Cloud Foundry to forward application logs to an ElasticSearch cluster via http input and a CF syslog drain (https://docs.cloudfoundry.org/devguide/services/lo… --- ## [Logstash kafka input not working](https://discuss.elastic.co/t/logstash-kafka-input-not-working/180171)
**Author:** [@RonenNess](https://discuss.elastic.co/u/RonenNess)\ **Replies:** 3\ **Last updated:** [May 12, 2019, 11:54am UTC](https://discuss.elastic.co/t/logstash-kafka-input-not-working/180171 "2019-05-12T11:54:51Z")
Hello, I have this very simple logstash config: input { kafka { topics =\> \["analytics"\] bootstrap\_servers =\> "HSTSCSRP2:9092" auto\_offset\_reset =\> "earliest" } } output { elasticse… --- ## [LogStash could not index to Elactic search](https://discuss.elastic.co/t/logstash-could-not-index-to-elactic-search/180642)
**Author:** [@Kailaash\_k](https://discuss.elastic.co/u/Kailaash_k)\ **Replies:** 1\ **Last updated:** [May 11, 2019, 9:27pm UTC](https://discuss.elastic.co/t/logstash-could-not-index-to-elactic-search/180642 "2019-05-11T21:27:20Z")
LogStash is able to get input from filebeat but its not able to log it in elasticsearch. Lostash.conf : ##log stash get input from here with file beat input { beats { type =\> "test" port =\> "6000" } } output to … --- ## [\[NAGIOS\] Nagios integration](https://discuss.elastic.co/t/nagios-nagios-integration/180375)
**Author:** [@Alessio\_Creo](https://discuss.elastic.co/u/Alessio_Creo)\ **Replies:** 2\ **Last updated:** [May 11, 2019, 5:52am UTC](https://discuss.elastic.co/t/nagios-nagios-integration/180375 "2019-05-11T05:52:11Z")
Hi there, I need to send alert to a Nagios XI server in a Logatsh pipeline, in particular when a field of my document take a specific value I want to send a notification to Nagios server, that is not in the same machine… --- ## [Logstash with Jdbc input plugin error](https://discuss.elastic.co/t/logstash-with-jdbc-input-plugin-error/180633)
**Author:** [@danish\_hyt](https://discuss.elastic.co/u/danish_hyt)\ **Replies:** 0\ **Last updated:** [May 11, 2019, 5:39am UTC](https://discuss.elastic.co/t/logstash-with-jdbc-input-plugin-error/180633 "2019-05-11T05:39:54Z")
I am tring to connect mysql workbench 6.3 CE with logstash via jdbc plugin but i am having error which i am unable to figure out research on it but found no fix so that is why i am posting it here. OS: Windows 10 Logst… --- ## [Squid parsing stopped to parse all fields](https://discuss.elastic.co/t/squid-parsing-stopped-to-parse-all-fields/180477)
**Author:** [@mauriciothomsen](https://discuss.elastic.co/u/mauriciothomsen)\ **Replies:** 1\ **Last updated:** [May 10, 2019, 11:35pm UTC](https://discuss.elastic.co/t/squid-parsing-stopped-to-parse-all-fields/180477 "2019-05-10T23:35:00Z")
Hello everybody. \*\*I have a Logstash job loading data from a kafka topic, running for many months without any issue. The only change done on the last 3 weeks was the upgrade to ELK 7.0. Still, after the upgrade the load… --- ## [Log timestamp filed converting as time filter field](https://discuss.elastic.co/t/log-timestamp-filed-converting-as-time-filter-field/180609)
**Author:** [@chandukreddi](https://discuss.elastic.co/u/chandukreddi)\ **Replies:** 2\ **Last updated:** [May 10, 2019, 8:57pm UTC](https://discuss.elastic.co/t/log-timestamp-filed-converting-as-time-filter-field/180609 "2019-05-10T20:57:34Z")
Hello Experts, Need help to covert my log timestamp into time filter field. I have log entry like below: Mar 5 03:23:26 0-1 proxy-server: 127.0.0.1 127.0.0.1 05/Mar/2019/11/23/26 GET /v1/ACC\_1/.trash-01f2f42f-8cc0-4… --- ## [Logstash won't start after clean shutdown](https://discuss.elastic.co/t/logstash-wont-start-after-clean-shutdown/180387)
**Author:** [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Replies:** 2\ **Last updated:** [May 10, 2019, 8:42pm UTC](https://discuss.elastic.co/t/logstash-wont-start-after-clean-shutdown/180387 "2019-05-10T20:42:19Z")
After we upgraded to ELK version 7.0.1 Everytime when we shutdown logstash it won't start until I reboot the machine Here is the message. There is no other log gets register in log file. It is so annoying. just simple… --- ## [Grok filter issue with data on Logstah config](https://discuss.elastic.co/t/grok-filter-issue-with-data-on-logstah-config/180337)
**Author:** [@Sharma3007](https://discuss.elastic.co/u/Sharma3007)\ **Replies:** 3\ **Last updated:** [May 10, 2019, 7:50pm UTC](https://discuss.elastic.co/t/grok-filter-issue-with-data-on-logstah-config/180337 "2019-05-10T19:50:12Z")
Hi Team, I am facing issue while using grok filter pattern for our log: here is my log message: 2019-05-09 08:18:41.6499|INFO|Processor|8552|5825|Content|Calling for job id 5776, logid 17586 LogStash.config : inp… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=548) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=550)