# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1139 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1140 --- ## [Distributing DB input across multiple Logstash instances](https://discuss.elastic.co/t/distributing-db-input-across-multiple-logstash-instances/52788)
**Author:** [@ankh](https://discuss.elastic.co/u/ankh)\ **Replies:** 1\ **Last updated:** [June 19, 2016, 10:21am UTC](https://discuss.elastic.co/t/distributing-db-input-across-multiple-logstash-instances/52788 "2016-06-19T10:21:40Z")
Very much a n00b here. We're experimenting with an ELK install to index XML stored in an RDBMS. We're performing a fair amount of parsing on it to define certain fields of interest to be displayed in Kibana. With a s… --- ## [How do I send request headers using HTTP input plugin?](https://discuss.elastic.co/t/how-do-i-send-request-headers-using-http-input-plugin/53009)
**Author:** [@iamthealex](https://discuss.elastic.co/u/iamthealex)\ **Replies:** 4\ **Last updated:** [June 18, 2016, 3:50pm UTC](https://discuss.elastic.co/t/how-do-i-send-request-headers-using-http-input-plugin/53009 "2016-06-18T15:50:14Z")
The LogStash HTTP output plugin has a headers field that one can use to add custom headers to the http request. Why isn't there a headers field for the LogStash HTTP input plugin? --- ## ["File" fail to read a file (sometimes)](https://discuss.elastic.co/t/file-fail-to-read-a-file-sometimes/53204)
**Author:** [@Magic](https://discuss.elastic.co/u/Magic)\ **Replies:** 0\ **Last updated:** [June 18, 2016, 7:16am UTC](https://discuss.elastic.co/t/file-fail-to-read-a-file-sometimes/53204 "2016-06-18T07:16:05Z")
Hi, My platform is windows and I fail to process some big (actually not so big) file with file input plugin. Below is my file input configuration segment: input { file { path =\> \["d:/X/zzzzz.csv"\] codec =\> pla… --- ## [Geohash with packetbeat and logstash](https://discuss.elastic.co/t/geohash-with-packetbeat-and-logstash/53136)
**Author:** [@Kennedy\_Kan1](https://discuss.elastic.co/u/Kennedy_Kan1)\ **Replies:** 3\ **Last updated:** [June 18, 2016, 6:43am UTC](https://discuss.elastic.co/t/geohash-with-packetbeat-and-logstash/53136 "2016-06-18T06:43:10Z")
I have used packetbeat and logstash to read data and have downloaded the geolitecity.dat, but I could still not make any tile map in kibana as it still limits me to the option of client location as the geohash even I hav… --- ## [Logstash JMS Input Plugin - Message Selector](https://discuss.elastic.co/t/logstash-jms-input-plugin-message-selector/53137)
**Author:** [@ckayay](https://discuss.elastic.co/u/ckayay)\ **Replies:** 1\ **Last updated:** [June 18, 2016, 1:38am UTC](https://discuss.elastic.co/t/logstash-jms-input-plugin-message-selector/53137 "2016-06-18T01:38:39Z")
Is it possible to apply Message Selectors (only get messages older than 5 mins) to JMS Input plugin? --- ## [Changing logstash location to a different drive on Windows](https://discuss.elastic.co/t/changing-logstash-location-to-a-different-drive-on-windows/53168)
**Author:** [@George\_Nussbaum](https://discuss.elastic.co/u/George_Nussbaum)\ **Replies:** 1\ **Last updated:** [June 18, 2016, 1:25am UTC](https://discuss.elastic.co/t/changing-logstash-location-to-a-different-drive-on-windows/53168 "2016-06-18T01:25:10Z")
Hello, I just started using the ELK stack so I'm very new and trying to figure it out. One question that was asked by my manager is if there is a way to have logging live on a different drive than the local drive as o… --- ## [Is JMS output even possible?](https://discuss.elastic.co/t/is-jms-output-even-possible/53177)
**Author:** [@pjoyce](https://discuss.elastic.co/u/pjoyce)\ **Replies:** 3\ **Last updated:** [June 17, 2016, 8:16pm UTC](https://discuss.elastic.co/t/is-jms-output-even-possible/53177 "2016-06-17T20:16:22Z")
I'm hoping someone here can provide just a basic example of using the logstash-output-jms plugin for logstash. The site itself has no info and I can find no config examples for output anywhere. Is this a valid plugin ? … --- ## [Filter is not working](https://discuss.elastic.co/t/filter-is-not-working/53068)
**Author:** [@nanshan](https://discuss.elastic.co/u/nanshan)\ **Replies:** 5\ **Last updated:** [June 17, 2016, 7:05pm UTC](https://discuss.elastic.co/t/filter-is-not-working/53068 "2016-06-17T19:05:58Z")
I am following this tutorial; https://www.elastic.co/guide/en/logstash/current/advanced-pipeline.html The grok pattern is not working. I can not get the result as it expected in the article --- ## [Extract time stamp from the log data](https://discuss.elastic.co/t/extract-time-stamp-from-the-log-data/52917)
**Author:** [@Sri\_ram](https://discuss.elastic.co/u/Sri_ram)\ **Replies:** 3\ **Last updated:** [June 17, 2016, 5:33pm UTC](https://discuss.elastic.co/t/extract-time-stamp-from-the-log-data/52917 "2016-06-17T17:33:16Z")
Hello, I am trying to extract the time stamp from the header of log data. Here is my log data looks like: Page: 1 Active Internet connections Report: Total ( 223) Time: Mon Mar 17 00:14:32 EST 2014 Proto |R… --- ## [Logstash: problem for querying elasticsearch](https://discuss.elastic.co/t/logstash-problem-for-querying-elasticsearch/52977)
**Author:** [@sen](https://discuss.elastic.co/u/sen)\ **Replies:** 15\ **Last updated:** [June 17, 2016, 2:25pm UTC](https://discuss.elastic.co/t/logstash-problem-for-querying-elasticsearch/52977 "2016-06-17T14:25:08Z")
Hello everyone, Through logstash, I want to query elasticsearch in order to get fields from previous events and do some computation with fields of my current event and add new fields. Here is what I did: input file: {… --- ## [LOGSTASH Config file error, doesn't like host assigning but it requires one](https://discuss.elastic.co/t/logstash-config-file-error-doesnt-like-host-assigning-but-it-requires-one/53142)
**Author:** [@brandonmcgrath1](https://discuss.elastic.co/u/brandonmcgrath1)\ **Replies:** 5\ **Last updated:** [June 17, 2016, 1:20pm UTC](https://discuss.elastic.co/t/logstash-config-file-error-doesnt-like-host-assigning-but-it-requires-one/53142 "2016-06-17T13:20:55Z")
I'm trying to configure logstash and this is the error im getting: {:timestamp=\>"2016-06-17T11:36:27.128000+0100", :message=\>"Could not start TCP server: Address in use", :host=\>"elk.etechdc.local", :port=\>3515, :level… --- ## [Logstash re-read input file everytime there is modification](https://discuss.elastic.co/t/logstash-re-read-input-file-everytime-there-is-modification/53022)
**Author:** [@shiva](https://discuss.elastic.co/u/shiva)\ **Replies:** 3\ **Last updated:** [June 17, 2016, 12:40pm UTC](https://discuss.elastic.co/t/logstash-re-read-input-file-everytime-there-is-modification/53022 "2016-06-17T12:40:17Z")
I have an input file named test.csv and I use it as some kind of configuration. How do i force logstash to re read the file from beginning, everytime there is modification in the test.csv file --- ## [Logstash not reading changes to config file (file input)](https://discuss.elastic.co/t/logstash-not-reading-changes-to-config-file-file-input/53151)
**Author:** [@fetchmelogs](https://discuss.elastic.co/u/fetchmelogs)\ **Replies:** 2\ **Last updated:** [June 17, 2016, 12:02pm UTC](https://discuss.elastic.co/t/logstash-not-reading-changes-to-config-file-file-input/53151 "2016-06-17T12:02:33Z")
Hi, I have been struggling with this a bit so any help would be much appreciated. I am running logstash 2.2.4. First Problem: I am parsing csv files locallyand needed to make a configuration change to my filte… --- ## [Convert string to date in logstash config file](https://discuss.elastic.co/t/convert-string-to-date-in-logstash-config-file/52976)
**Author:** [@sarbjeet](https://discuss.elastic.co/u/sarbjeet)\ **Replies:** 3\ **Last updated:** [June 17, 2016, 11:32am UTC](https://discuss.elastic.co/t/convert-string-to-date-in-logstash-config-file/52976 "2016-06-17T11:32:08Z")
hi... my csv file is:- input { stdin {} } filter { csv { separator =\> " " columns =\> \["frame.time","col.Source","col.Destination","col.Protocol","frame.len"\] } mutate {rename =\> {"frame.time" =\> "Time"}} mutate… --- ## [Logstash-hipchat messages are not displaying in hipchat](https://discuss.elastic.co/t/logstash-hipchat-messages-are-not-displaying-in-hipchat/52611)
**Author:** [@babeesh](https://discuss.elastic.co/u/babeesh)\ **Replies:** 11\ **Last updated:** [June 17, 2016, 10:55am UTC](https://discuss.elastic.co/t/logstash-hipchat-messages-are-not-displaying-in-hipchat/52611 "2016-06-17T10:55:44Z")
Hi, I have redirected logs recieving in logstash to hipchat. output { if \[type\]== 'eventlog' { hipchat { room\_id =\> '\*\*\*\*' token =\> '\*\*\*\*' } elasticsearch { hosts =\> \['172.30… --- ## [Date Formatting of](https://discuss.elastic.co/t/date-formatting-of/53087)
**Author:** [@kunalp](https://discuss.elastic.co/u/kunalp)\ **Replies:** 5\ **Last updated:** [June 17, 2016, 10:39am UTC](https://discuss.elastic.co/t/date-formatting-of/53087 "2016-06-17T10:39:02Z")
I need parse date in logstash of format \[03/Apr/2016:10:35:57 +0530\] . This is from Weblogic Access logs. How to do it. Regards, Kunal --- ## [Logstash config for file containing both json and text](https://discuss.elastic.co/t/logstash-config-for-file-containing-both-json-and-text/52503)
**Author:** [@ankush\_goswami](https://discuss.elastic.co/u/ankush_goswami)\ **Replies:** 2\ **Last updated:** [June 17, 2016, 9:24am UTC](https://discuss.elastic.co/t/logstash-config-for-file-containing-both-json-and-text/52503 "2016-06-17T09:24:06Z")
Hi, I am new to logstash and want to know how to parse a file containing multiple format such as :--- {"name":"travel.train","hostname":"mktldstickets01","pid":4241,"meta":{"route":"/search","type":"search","date":"2… --- ## [Start\_position =\> "end" is not working for me](https://discuss.elastic.co/t/start-position-end-is-not-working-for-me/53098)
**Author:** [@rupal](https://discuss.elastic.co/u/rupal)\ **Replies:** 2\ **Last updated:** [June 17, 2016, 9:05am UTC](https://discuss.elastic.co/t/start-position-end-is-not-working-for-me/53098 "2016-06-17T09:05:07Z")
I need real time data monitoring in such a way that all existing as well as new csv files should be monitored. Both files should get monitored from beginning and any monitored files if are modified in future should get m… --- ## [Elasticsearch not immediately available for search through Logstash](https://discuss.elastic.co/t/elasticsearch-not-immediately-available-for-search-through-logstash/53111)
**Author:** [@sen](https://discuss.elastic.co/u/sen)\ **Replies:** 3\ **Last updated:** [June 17, 2016, 8:32am UTC](https://discuss.elastic.co/t/elasticsearch-not-immediately-available-for-search-through-logstash/53111 "2016-06-17T08:32:49Z")
Hello everyone, I want to send queries to Elasticsearch through the Elasticsearch plugin within Logstash for every event in process. However, Logstash sends requests to Elasticsearch in bulk and indexed events are no… --- ## [Can not get log after install](https://discuss.elastic.co/t/can-not-get-log-after-install/53103)
**Author:** [@kayyazka](https://discuss.elastic.co/u/kayyazka)\ **Replies:** 3\ **Last updated:** [June 17, 2016, 7:54am UTC](https://discuss.elastic.co/t/can-not-get-log-after-install/53103 "2016-06-17T07:54:19Z")
Hi All, This first time i have install EL, I am finish install ELK but can not get log error. Only appear the very first time install next did not coming back How should i do to fix it? Thank you in advance --- ## [SOLVED: Grok + Dynamic field name](https://discuss.elastic.co/t/solved-grok-dynamic-field-name/52842)
**Author:** [@Clement\_Ros](https://discuss.elastic.co/u/Clement_Ros)\ **Replies:** 2\ **Last updated:** [June 17, 2016, 7:43am UTC](https://discuss.elastic.co/t/solved-grok-dynamic-field-name/52842 "2016-06-17T07:43:07Z")
Hi everybody, I am creating filters for monitoring a new server. I have a amavis log that perform spamassassin tests on email. The problem is that there are a hundred tests and the tests in the log are never the sam… --- ## [Newbee with logstash an JSON](https://discuss.elastic.co/t/newbee-with-logstash-an-json/52714)
**Author:** [@paracetamol](https://discuss.elastic.co/u/paracetamol)\ **Replies:** 6\ **Last updated:** [June 17, 2016, 7:13am UTC](https://discuss.elastic.co/t/newbee-with-logstash-an-json/52714 "2016-06-17T07:13:04Z")
Hi, I'm polling a JSON with th following config file: input { http\_poller { urls =\> { myresource =\> "http://192.168.32.40/json/daily.json" } request\_timeout =\> 60 interval =\> 60 codec… --- ## [Continuous monitoring of csv files using logstash creating one issue of duplicate records](https://discuss.elastic.co/t/continuous-monitoring-of-csv-files-using-logstash-creating-one-issue-of-duplicate-records/53003)
**Author:** [@rupal](https://discuss.elastic.co/u/rupal)\ **Replies:** 2\ **Last updated:** [June 17, 2016, 6:47am UTC](https://discuss.elastic.co/t/continuous-monitoring-of-csv-files-using-logstash-creating-one-issue-of-duplicate-records/53003 "2016-06-17T06:47:07Z")
When we are using start\_position parameter as "beginning" and running logstash conf file. Its start monitoring records but as and when I update same csv file(while continuous monitoring) with one new record, All of the r… --- ## [Attempting to use drop filter to discard events that match a type and string](https://discuss.elastic.co/t/attempting-to-use-drop-filter-to-discard-events-that-match-a-type-and-string/52921)
**Author:** [@skynardo](https://discuss.elastic.co/u/skynardo)\ **Replies:** 2\ **Last updated:** [June 17, 2016, 6:08am UTC](https://discuss.elastic.co/t/attempting-to-use-drop-filter-to-discard-events-that-match-a-type-and-string/52921 "2016-06-17T06:08:07Z")
This seems to not be working. Messages that I want dropped (postgres messages that contain the string "password") are being sent to Elasticsearch. Can someone help me with the error in my ways? elasticsearch-output.co… --- ## [Join two documents](https://discuss.elastic.co/t/join-two-documents/53015)
**Author:** [@Chartreuse](https://discuss.elastic.co/u/Chartreuse)\ **Replies:** 1\ **Last updated:** [June 17, 2016, 5:55am UTC](https://discuss.elastic.co/t/join-two-documents/53015 "2016-06-17T05:55:23Z")
Hi, I have two .csv files. I have a column in each document that is key for me ElasticSearch. When logstash indexes data overwrites the oldest values. Is it possible to keep these values in a table? For example : Cs… --- ## [Where do I assign Logstash a port?](https://discuss.elastic.co/t/where-do-i-assign-logstash-a-port/53001)
**Author:** [@brandonmcgrath1](https://discuss.elastic.co/u/brandonmcgrath1)\ **Replies:** 1\ **Last updated:** [June 17, 2016, 5:41am UTC](https://discuss.elastic.co/t/where-do-i-assign-logstash-a-port/53001 "2016-06-17T05:41:42Z")
As the title suggests, where is it that you can assign logstash a port? I've noticed that in filebeat, winlogbeat and so on there is an output section with logstash default to 5044, but is that the only place that offers… --- ## [Possible to parse the message in Logstash and build indexes/fields from it](https://discuss.elastic.co/t/possible-to-parse-the-message-in-logstash-and-build-indexes-fields-from-it/53058)
**Author:** [@kasi](https://discuss.elastic.co/u/kasi)\ **Replies:** 1\ **Last updated:** [June 17, 2016, 5:32am UTC](https://discuss.elastic.co/t/possible-to-parse-the-message-in-logstash-and-build-indexes-fields-from-it/53058 "2016-06-17T05:32:24Z")
We have JSON messages that are getting forwarded to LogStash from filebeat, We would like to parse the JSON messages and convert them into fields and add them as part of the message and would also like to create index b… --- ## [Testing your logstash configuration](https://discuss.elastic.co/t/testing-your-logstash-configuration/53061)
**Author:** [@boazshor](https://discuss.elastic.co/u/boazshor)\ **Replies:** 1\ **Last updated:** [June 17, 2016, 5:31am UTC](https://discuss.elastic.co/t/testing-your-logstash-configuration/53061 "2016-06-17T05:31:24Z")
Hi, I'm new to Logstash. I'm working with Logstash 2.3.2. I'm trying to write unit tests for my configuration files. I wrote a simple test for a simple filter, but when I try to run them using bin\\rspec, I'm getting the… --- ## [How to get rid of wrongly named index that must be lower case](https://discuss.elastic.co/t/how-to-get-rid-of-wrongly-named-index-that-must-be-lower-case/52550)
**Author:** [@ikovalev](https://discuss.elastic.co/u/ikovalev)\ **Replies:** 10\ **Last updated:** [June 17, 2016, 2:15am UTC](https://discuss.elastic.co/t/how-to-get-rid-of-wrongly-named-index-that-must-be-lower-case/52550 "2016-06-17T02:15:12Z")
Wrong index name esm\_DMZ\_results was put into Logstash config file /opt/logstash/first-pipeline.conf . After that elasticsearch log /data/elastic/logs/elastic\_concept.log start showing "Invalid index name \[esm\_DMZ\_r… --- ## [Logstash and NXLOG Config?](https://discuss.elastic.co/t/logstash-and-nxlog-config/52996)
**Author:** [@brandonmcgrath1](https://discuss.elastic.co/u/brandonmcgrath1)\ **Replies:** 1\ **Last updated:** [June 16, 2016, 8:52pm UTC](https://discuss.elastic.co/t/logstash-and-nxlog-config/52996 "2016-06-16T20:52:35Z")
Im trying to configure NXLOG and Logstash to send info to the ELK Stack. Im having trouble setting it up, my error is: :message=\>"Pipeline aborted due to error", :exception=\>#\