# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1166 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1167 --- ## [Import CSV - Column names are also present in the values](https://discuss.elastic.co/t/import-csv-column-names-are-also-present-in-the-values/47383)
**Author:** [@Denny](https://discuss.elastic.co/u/Denny)\ **Replies:** 1\ **Last updated:** [April 15, 2016, 6:05am UTC](https://discuss.elastic.co/t/import-csv-column-names-are-also-present-in-the-values/47383 "2016-04-15T06:05:38Z")
Hi, I've used Logstash to import a csv-file which works great and all was looking good but then I tried another csv-file and then I saw something strange. In Kibana I saw that the headers of the file are also in the val… --- ## [@timestamp should match with aupm\_timestamp is not working for me](https://discuss.elastic.co/t/timestamp-should-match-with-aupm-timestamp-is-not-working-for-me/47311)
**Author:** [@Vivek\_Raj](https://discuss.elastic.co/u/Vivek_Raj)\ **Replies:** 8\ **Last updated:** [April 14, 2016, 11:02pm UTC](https://discuss.elastic.co/t/timestamp-should-match-with-aupm-timestamp-is-not-working-for-me/47311 "2016-04-14T23:02:03Z")
error --- ## [Clients sending invalid docs](https://discuss.elastic.co/t/clients-sending-invalid-docs/47261)
**Author:** [@mhughes](https://discuss.elastic.co/u/mhughes)\ **Replies:** 3\ **Last updated:** [April 14, 2016, 11:00pm UTC](https://discuss.elastic.co/t/clients-sending-invalid-docs/47261 "2016-04-14T23:00:13Z")
I have a bunch of clients that are sending documents directly to my ES cluster via bulkIndex. Those documents are valid for 1.x, but are now invalid with 2.x. Specifically, a lot of the documents had periods in field n… --- ## [Expensive if statement in Logstash config](https://discuss.elastic.co/t/expensive-if-statement-in-logstash-config/47420)
**Author:** [@anhlqn](https://discuss.elastic.co/u/anhlqn)\ **Replies:** 0\ **Last updated:** [April 14, 2016, 5:44pm UTC](https://discuss.elastic.co/t/expensive-if-statement-in-logstash-config/47420 "2016-04-14T17:44:15Z")
Hi all, I'm using Logstash 2.3.0 to receive netflow data from nProbe. There are some custom application ports that would be named as Unknown protocol, so I use if/else if in Logstash to check for destination port and r… --- ## [Logstash does not process logs in time and so the application sending log message backs up](https://discuss.elastic.co/t/logstash-does-not-process-logs-in-time-and-so-the-application-sending-log-message-backs-up/47115)
**Author:** [@Ramdev\_Wudali](https://discuss.elastic.co/u/Ramdev_Wudali)\ **Replies:** 1\ **Last updated:** [April 14, 2016, 5:31pm UTC](https://discuss.elastic.co/t/logstash-does-not-process-logs-in-time-and-so-the-application-sending-log-message-backs-up/47115 "2016-04-14T17:31:26Z")
Hi: I have a java application that sends an warning message whenever it hits an error during its analyzing a document. I am processing several millions of these documents in a day, and the application is generating … --- ## [Error while ingest data](https://discuss.elastic.co/t/error-while-ingest-data/47043)
**Author:** [@Amit2015](https://discuss.elastic.co/u/Amit2015)\ **Replies:** 2\ **Last updated:** [April 14, 2016, 5:24pm UTC](https://discuss.elastic.co/t/error-while-ingest-data/47043 "2016-04-14T17:24:41Z")
I am trying to ingest logs and getting error that - The signal HUP is in used by JVM and will not work currectly on this platform. I am using this command : logstash -e 'input { stdin { } } output { elasticsearch { … --- ## [Template not being applied to elasticsearch output](https://discuss.elastic.co/t/template-not-being-applied-to-elasticsearch-output/47308)
**Author:** [@nocode](https://discuss.elastic.co/u/nocode)\ **Replies:** 5\ **Last updated:** [April 14, 2016, 5:16pm UTC](https://discuss.elastic.co/t/template-not-being-applied-to-elasticsearch-output/47308 "2016-04-14T17:16:36Z")
In Nginx, I've converted my log\_format to json and sending the logs via Filebeat to logstash. I'm trying to set some of my fields to not\_analyzed but nothing I'm doing is working. Almost all the fields from nginx logs … --- ## [Logstash metrics for different types?](https://discuss.elastic.co/t/logstash-metrics-for-different-types/47299)
**Author:** [@jclose](https://discuss.elastic.co/u/jclose)\ **Replies:** 2\ **Last updated:** [April 14, 2016, 3:37pm UTC](https://discuss.elastic.co/t/logstash-metrics-for-different-types/47299 "2016-04-14T15:37:20Z")
I'd like to use the metrics filter to give me output of metrics for various type data that is coming in. How would I do that? For example, if Logstash is reading in, and the type can be "http", "dns", or "dhcp", I'd l… --- ## [Iteration/Ruby help in Logstash](https://discuss.elastic.co/t/iteration-ruby-help-in-logstash/47409)
**Author:** [@Dustman1977](https://discuss.elastic.co/u/Dustman1977)\ **Replies:** 0\ **Last updated:** [April 14, 2016, 3:19pm UTC](https://discuss.elastic.co/t/iteration-ruby-help-in-logstash/47409 "2016-04-14T15:19:57Z")
I'm Parsing an XML stream that I'm parsing with Logstash. I have one section of the message that is repetitive and in varying lengths 0 to 24 is the highest I've seen so far: \ \5.243618,20.591172,off\<… --- ## [Sighup doesnt work for reloading conf](https://discuss.elastic.co/t/sighup-doesnt-work-for-reloading-conf/47403)
**Author:** [@randude](https://discuss.elastic.co/u/randude)\ **Replies:** 0\ **Last updated:** [April 14, 2016, 2:34pm UTC](https://discuss.elastic.co/t/sighup-doesnt-work-for-reloading-conf/47403 "2016-04-14T14:34:47Z")
Hi, I'm using logstash 2.3.0 on my linux machine. I change one of the conf files in the conf.d folder and then i send a sighup signal with the kill command. Logstash doesnt seem to reload the configuration. However, when… --- ## [Grok match an fqdn](https://discuss.elastic.co/t/grok-match-an-fqdn/47373)
**Author:** [@cwgueco](https://discuss.elastic.co/u/cwgueco)\ **Replies:** 3\ **Last updated:** [April 14, 2016, 11:46am UTC](https://discuss.elastic.co/t/grok-match-an-fqdn/47373 "2016-04-14T11:46:11Z")
Hello, I am a newbie for grok and patterns. I have been trying to grok parse an fqdn entry which has variable length - multiple dots (.) and uses dashes (-) and underscore (\_\_) for the hostnames. The DNS entries could … --- ## [Update fields of logstash index on ElasticSearch](https://discuss.elastic.co/t/update-fields-of-logstash-index-on-elasticsearch/47364)
**Author:** [@oth1113](https://discuss.elastic.co/u/oth1113)\ **Replies:** 2\ **Last updated:** [April 14, 2016, 9:37am UTC](https://discuss.elastic.co/t/update-fields-of-logstash-index-on-elasticsearch/47364 "2016-04-14T09:37:27Z")
On ElasticSearch I have logstash index that are generated every day, I had changed the types of some of my fields, they had changed in the new indexes that was generated but not on the old index. Knowing that I can not d… --- ## [Send data to elasticsearch index from kafka that is named the same as the topic](https://discuss.elastic.co/t/send-data-to-elasticsearch-index-from-kafka-that-is-named-the-same-as-the-topic/47365)
**Author:** [@amb](https://discuss.elastic.co/u/amb)\ **Replies:** 2\ **Last updated:** [April 14, 2016, 9:36am UTC](https://discuss.elastic.co/t/send-data-to-elasticsearch-index-from-kafka-that-is-named-the-same-as-the-topic/47365 "2016-04-14T09:36:35Z")
As it says in the title I want to pull out the topic name from the incoming event and write to an index in elasticsearchthat is named the same as that topic name. input { kafka { zk\_connect =\> "zookeeper:2181" group… --- ## [Bad UTF-8 encoding Latin characters with groovy](https://discuss.elastic.co/t/bad-utf-8-encoding-latin-characters-with-groovy/47361)
**Author:** [@bigtn](https://discuss.elastic.co/u/bigtn)\ **Replies:** 0\ **Last updated:** [April 14, 2016, 8:54am UTC](https://discuss.elastic.co/t/bad-utf-8-encoding-latin-characters-with-groovy/47361 "2016-04-14T08:54:14Z")
hi, im using groovy to append an array of nested object on elasticsearch from a csv file when groovy insert the first object there is no encoding problem, otherwise when he append the array with others objects i hav… --- ## [Logstash grok filter for removing comments](https://discuss.elastic.co/t/logstash-grok-filter-for-removing-comments/47099)
**Author:** [@rameeelastic](https://discuss.elastic.co/u/rameeelastic)\ **Replies:** 2\ **Last updated:** [April 14, 2016, 7:34am UTC](https://discuss.elastic.co/t/logstash-grok-filter-for-removing-comments/47099 "2016-04-14T07:34:23Z")
I have a websphere ESB log that I am trying to parse and the log file contains certain entries that are comments that needs to be removed. I am actually using dateparsefailure and grokparsefailure conditions to drop thes… --- ## [Dynamic configuration of logstash](https://discuss.elastic.co/t/dynamic-configuration-of-logstash/42358)
**Author:** [@harshafrnd4u](https://discuss.elastic.co/u/harshafrnd4u)\ **Replies:** 11\ **Last updated:** [April 14, 2016, 7:08am UTC](https://discuss.elastic.co/t/dynamic-configuration-of-logstash/42358 "2016-04-14T07:08:03Z")
Hi, Is there anyway to load logstash configuration dynamically using database or some filesystem. ? The usecase is lets say i have multiple apaches where I am getting data from and I am getting data with their ips (eac… --- ## [How to import my Vendor's MIBs?](https://discuss.elastic.co/t/how-to-import-my-vendors-mibs/45186)
**Author:** [@stefansaye](https://discuss.elastic.co/u/stefansaye)\ **Replies:** 2\ **Last updated:** [April 14, 2016, 6:01am UTC](https://discuss.elastic.co/t/how-to-import-my-vendors-mibs/45186 "2016-04-14T06:01:05Z")
hi all, how to import my Vendor's MIBs ? If anyone knows how to do with it , please help. Thanks. --- ## [Upload data to elasteacsearch with logstash is stuck](https://discuss.elastic.co/t/upload-data-to-elasteacsearch-with-logstash-is-stuck/45587)
**Author:** [@chenbe2204](https://discuss.elastic.co/u/chenbe2204)\ **Replies:** 2\ **Last updated:** [April 14, 2016, 5:55am UTC](https://discuss.elastic.co/t/upload-data-to-elasteacsearch-with-logstash-is-stuck/45587 "2016-04-14T05:55:02Z")
Hi , I'm using logstash 2.2.0 to upload json files to elasticsearch (version 2.2). I already used the conf files of the logstash I built to upload data , and it worked !! Now when I'm trying again it's stuck , the con… --- ## [How do I use JIRA with ELK?](https://discuss.elastic.co/t/how-do-i-use-jira-with-elk/46914)
**Author:** [@Caba\_Aba\_Baba](https://discuss.elastic.co/u/Caba_Aba_Baba)\ **Replies:** 5\ **Last updated:** [April 14, 2016, 2:54am UTC](https://discuss.elastic.co/t/how-do-i-use-jira-with-elk/46914 "2016-04-14T02:54:37Z")
I want to be able to query all of my JIRA tickets in Kibana. Is there a proper way to do this or is it not really supported? Currently I just query for any JIRA ticket modified in the past minute, dedupe, and put this in… --- ## [Translate Dictionary Path Updating of Changes](https://discuss.elastic.co/t/translate-dictionary-path-updating-of-changes/47303)
**Author:** [@neu5ron](https://discuss.elastic.co/u/neu5ron)\ **Replies:** 1\ **Last updated:** [April 14, 2016, 1:03am UTC](https://discuss.elastic.co/t/translate-dictionary-path-updating-of-changes/47303 "2016-04-14T01:03:18Z")
In the article for https://www.elastic.co/blog/bro-ids-elastic-stack (great article by the way) may want to note to users that if they are continuously updating a file, that the dictionary\_path in translate points at, th… --- ## [Creating a LogStash plugin with Java instead of Ruby](https://discuss.elastic.co/t/creating-a-logstash-plugin-with-java-instead-of-ruby/38080)
**Author:** [@adaviding](https://discuss.elastic.co/u/adaviding)\ **Replies:** 2\ **Last updated:** [April 14, 2016, 12:08am UTC](https://discuss.elastic.co/t/creating-a-logstash-plugin-with-java-instead-of-ruby/38080 "2016-04-14T00:08:09Z")
I want to create a LogStash plugin that utilizes code written in Java and I need some help. Has anybody done this before? I am not comfortable with Ruby. Unfortunately, I can see that LogStash plugins must be writt… --- ## [Date Format Error with Full ISO Date](https://discuss.elastic.co/t/date-format-error-with-full-iso-date/47153)
**Author:** [@CPoston](https://discuss.elastic.co/u/CPoston)\ **Replies:** 6\ **Last updated:** [April 13, 2016, 9:09pm UTC](https://discuss.elastic.co/t/date-format-error-with-full-iso-date/47153 "2016-04-13T21:09:17Z")
From logstash I'm getting (removed extraneous info) : :timestamp=\>"2016-04-12T12:14:08.428000-0500" :message=\>"Failed action. " "reason"=\>"Invalid format: "2016-04-12T12:14:04.7872034-05:00" is malformed at "-04-12T12:… --- ## [NoMethodError after upgrading ELK stack](https://discuss.elastic.co/t/nomethoderror-after-upgrading-elk-stack/47146)
**Author:** [@Michael\_Mullay](https://discuss.elastic.co/u/Michael_Mullay)\ **Replies:** 2\ **Last updated:** [April 13, 2016, 8:28pm UTC](https://discuss.elastic.co/t/nomethoderror-after-upgrading-elk-stack/47146 "2016-04-13T20:28:58Z")
Hi all, Just this past Thursday I upgraded our ELK stack from: logstash-1.4.2-1 elasticsearch-1.3.2 Kibana 3 To: logstash-2.3.0-1 elasticsearch-2.3.1 kibana-4.5.0-1 This was via local rpm installs since we are PC… --- ## [How to correctly use translate in Logstash?](https://discuss.elastic.co/t/how-to-correctly-use-translate-in-logstash/47067)
**Author:** [@nateuni](https://discuss.elastic.co/u/nateuni)\ **Replies:** 1\ **Last updated:** [April 13, 2016, 1:55pm UTC](https://discuss.elastic.co/t/how-to-correctly-use-translate-in-logstash/47067 "2016-04-13T13:55:33Z")
This is an example of my data in Kibana and what I am doing in Logstash with groks and some background info can be found here. I have categories, subcategories and products which can be seen in the Kibana screen captu… --- ## [How to correctly parse and then enrich log data in Logstash?](https://discuss.elastic.co/t/how-to-correctly-parse-and-then-enrich-log-data-in-logstash/46244)
**Author:** [@nateuni](https://discuss.elastic.co/u/nateuni)\ **Replies:** 12\ **Last updated:** [April 13, 2016, 1:45pm UTC](https://discuss.elastic.co/t/how-to-correctly-parse-and-then-enrich-log-data-in-logstash/46244 "2016-04-13T13:45:16Z")
I am working on my thesis and I have this setup: Webserver-\>Filebeats-\>Logstash-\>Kafka-\>ELK Server I have logs that contain the product info in the GET request from the webserver, depending on what was being served. An… --- ## [Multiline not picking up events immediately](https://discuss.elastic.co/t/multiline-not-picking-up-events-immediately/46823)
**Author:** [@Sameer\_Panicker](https://discuss.elastic.co/u/Sameer_Panicker)\ **Replies:** 16\ **Last updated:** [April 13, 2016, 12:44pm UTC](https://discuss.elastic.co/t/multiline-not-picking-up-events-immediately/46823 "2016-04-13T12:44:03Z")
This is my conf file - file{ path =\> "C:/Logs/\*" codec =\> multiline { pattern =\> "%{WORD:EventId} : %{NUMBER:EventID}, %{WORD:Level} : %{WORD:EventLevel}, %{WORD:Message} : %{WORD:Operation}, %{WORD:Payload} : \[%{WORD… --- ## [Logstash beats plugin](https://discuss.elastic.co/t/logstash-beats-plugin/47236)
**Author:** [@sunilmchaudhari](https://discuss.elastic.co/u/sunilmchaudhari)\ **Replies:** 0\ **Last updated:** [April 13, 2016, 11:03am UTC](https://discuss.elastic.co/t/logstash-beats-plugin/47236 "2016-04-13T11:03:29Z")
Hi, How would I install logstash plugin for filebeat, where there is no internet access? I have logstash-1.5.3 in production environment. Prod env has no access to repositories. Please guide on this. Can I copy plugins … --- ## [\[SOLVED\] How to filter log from syslog input](https://discuss.elastic.co/t/solved-how-to-filter-log-from-syslog-input/47097)
**Author:** [@tatdat](https://discuss.elastic.co/u/tatdat)\ **Replies:** 2\ **Last updated:** [April 13, 2016, 10:17am UTC](https://discuss.elastic.co/t/solved-how-to-filter-log-from-syslog-input/47097 "2016-04-13T10:17:37Z")
i have logstash instanse using syslog input plugin for get log from access point . I try to save log to file (using file output) and i get some line log (json format) {"tags":\["\_grokparsefailure\_sysloginput"\],"message… --- ## [Create nested field](https://discuss.elastic.co/t/create-nested-field/47222)
**Author:** [@boernd](https://discuss.elastic.co/u/boernd)\ **Replies:** 2\ **Last updated:** [April 13, 2016, 10:05am UTC](https://discuss.elastic.co/t/create-nested-field/47222 "2016-04-13T10:05:51Z")
Hi, is there a better way to create a nested field than my two step approach? Did not get it to work otherwise. mutate { add\_field =\> { "riemann" =\> { } "riemannstate" =\> "critical" "riemannservice" =\> "testservic… --- ## [Default path of Geodb in logstash](https://discuss.elastic.co/t/default-path-of-geodb-in-logstash/47210)
**Author:** [@shan](https://discuss.elastic.co/u/shan)\ **Replies:** 5\ **Last updated:** [April 13, 2016, 8:44am UTC](https://discuss.elastic.co/t/default-path-of-geodb-in-logstash/47210 "2016-04-13T08:44:48Z")
Hi, I need to know where Geodb files are referring in logstash ? I have to place the new GeoIP.dat file. Can anyone help on this Thanks [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1165) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1167)