# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1179 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1180 --- ## [File input not working - logstash 2.2.2](https://discuss.elastic.co/t/file-input-not-working-logstash-2-2-2/44335)
**Author:** [@Giulio](https://discuss.elastic.co/u/Giulio)\ **Replies:** 4\ **Last updated:** [March 15, 2016, 8:27am UTC](https://discuss.elastic.co/t/file-input-not-working-logstash-2-2-2/44335 "2016-03-15T08:27:53Z")
Hi First of all, sorry for my poor english, I will try to explain my problem as better as I can. I have just begun to use logstash + elasticsearch and I have a problem with file input. Each program is running in a v… --- ## [Redundancy in logging "Flushing buffer at interval"](https://discuss.elastic.co/t/redundancy-in-logging-flushing-buffer-at-interval/43888)
**Author:** [@NiteshBarnwal](https://discuss.elastic.co/u/NiteshBarnwal)\ **Replies:** 4\ **Last updated:** [March 15, 2016, 8:21am UTC](https://discuss.elastic.co/t/redundancy-in-logging-flushing-buffer-at-interval/43888 "2016-03-15T08:21:30Z")
I am seeing below logs being displayed continuously in debug mode. What may be the issue? Flushing buffer at interval {:instance=\>"#\ **Author:** [@guojingjing](https://discuss.elastic.co/u/guojingjing)\ **Replies:** 0\ **Last updated:** [March 15, 2016, 7:10am UTC](https://discuss.elastic.co/t/logstash-and-cloud-foundry/44397 "2016-03-15T07:10:00Z") The cloud services Logging and kibana-me-logs can be used for developing cloud apps, but both services do not seem to be customizable, e.g. for logstash service I do not know how to access the configuration files or impl… --- ## [Hi, I am new to ELK. Trying to parse the date inside the multiline messages as timestamp. Let me know what config file to be used](https://discuss.elastic.co/t/hi-i-am-new-to-elk-trying-to-parse-the-date-inside-the-multiline-messages-as-timestamp-let-me-know-what-config-file-to-be-used/43792)
**Author:** [@nkpaul](https://discuss.elastic.co/u/nkpaul)\ **Replies:** 7\ **Last updated:** [March 15, 2016, 6:33am UTC](https://discuss.elastic.co/t/hi-i-am-new-to-elk-trying-to-parse-the-date-inside-the-multiline-messages-as-timestamp-let-me-know-what-config-file-to-be-used/43792 "2016-03-15T06:33:20Z")
Message file will look like: I. 2016/03/08 09:22:34. Hi My name is Nil. This is a test message I. 2016/03/08 09:22:35. This is the message 2 I. 2016/03/08 09:22:36. This is multiline message 3 message 3 I. 2016/03/0… --- ## [Ship logs to Logstash from public&private IP addresses](https://discuss.elastic.co/t/ship-logs-to-logstash-from-public-private-ip-addresses/44163)
**Author:** [@cazorla19](https://discuss.elastic.co/u/cazorla19)\ **Replies:** 1\ **Last updated:** [March 15, 2016, 6:32am UTC](https://discuss.elastic.co/t/ship-logs-to-logstash-from-public-private-ip-addresses/44163 "2016-03-15T06:32:48Z")
Hi. I'm trying to meet Logstash with all servers I have by Filebeat. The subject: some servers with private IP's (like 192.168.0.0/24) and some servers with public IP's (like 11.22.33.44). Personally Logstash is a privat… --- ## [Logstash configtest Failing](https://discuss.elastic.co/t/logstash-configtest-failing/44371)
**Author:** [@Robert\_Groos](https://discuss.elastic.co/u/Robert_Groos)\ **Replies:** 1\ **Last updated:** [March 15, 2016, 12:49am UTC](https://discuss.elastic.co/t/logstash-configtest-failing/44371 "2016-03-15T00:49:38Z")
I have a fairly simple logstash config file. However upon running 'service logstash configtest' i get the following error: " Error: Expected one of #, -, ", ', } at line 12, column 21 (byte 280) after filter { if \[type\]… --- ## [Webhdfs output plugin does not create new files](https://discuss.elastic.co/t/webhdfs-output-plugin-does-not-create-new-files/36538)
**Author:** [@Olaf\_Thormaehlen](https://discuss.elastic.co/u/Olaf_Thormaehlen)\ **Replies:** 3\ **Last updated:** [March 14, 2016, 10:16pm UTC](https://discuss.elastic.co/t/webhdfs-output-plugin-does-not-create-new-files/36538 "2016-03-14T22:16:09Z")
Hi, we have problems writing to hdfs via the webhdfs plugin. It does not create new files but will wirte to a file that does already exist. This prevents using tags like date, time, host and so on. Does someone have … --- ## [Logstash is losing messages](https://discuss.elastic.co/t/logstash-is-losing-messages/44359)
**Author:** [@woytekm](https://discuss.elastic.co/u/woytekm)\ **Replies:** 3\ **Last updated:** [March 14, 2016, 9:16pm UTC](https://discuss.elastic.co/t/logstash-is-losing-messages/44359 "2016-03-14T21:16:50Z")
Hello, i have built ELK solution for processing/storing/visualizing syslog data from certain number of highly utilized Cisco ASA firewalls. My elasticsearch cluster setup is: 6 data nodes, one master and one client node.… --- ## [Logs that do not match the Filter with IF statement](https://discuss.elastic.co/t/logs-that-do-not-match-the-filter-with-if-statement/44247)
**Author:** [@Hans](https://discuss.elastic.co/u/Hans)\ **Replies:** 5\ **Last updated:** [March 14, 2016, 6:44pm UTC](https://discuss.elastic.co/t/logs-that-do-not-match-the-filter-with-if-statement/44247 "2016-03-14T18:44:59Z")
I have different filters for different fields in the messages and separate them with if statements. Now I have been trying to add one final section where none of the logs match and try to filter these through generic fie… --- ## [CSV filter - Create Index per log timestamp](https://discuss.elastic.co/t/csv-filter-create-index-per-log-timestamp/44234)
**Author:** [@thunderbirdgit](https://discuss.elastic.co/u/thunderbirdgit)\ **Replies:** 8\ **Last updated:** [March 14, 2016, 6:31pm UTC](https://discuss.elastic.co/t/csv-filter-create-index-per-log-timestamp/44234 "2016-03-14T18:31:26Z")
Hi I want to create ES index based on the dates matching from the logfile. I am using logstash CSV filter to process the logs. For instance, the log data appears like below 2016-02-21 00:02:32.238,123.abc.com,data 20… --- ## [How does geoip filter works when geo database file updated](https://discuss.elastic.co/t/how-does-geoip-filter-works-when-geo-database-file-updated/44044)
**Author:** [@Bruno\_Lavoie](https://discuss.elastic.co/u/Bruno_Lavoie)\ **Replies:** 2\ **Last updated:** [March 14, 2016, 4:53pm UTC](https://discuss.elastic.co/t/how-does-geoip-filter-works-when-geo-database-file-updated/44044 "2016-03-14T16:53:42Z")
Hello, We're looking to make our GeoLite Database being updated automatically at regular interval. How is behaving LS - geoip filter when file is updated? is it crashing? is the new database taken automatically o… --- ## [Problems with logstash codecs (json, rubydebug)](https://discuss.elastic.co/t/problems-with-logstash-codecs-json-rubydebug/44252)
**Author:** [@tetlika](https://discuss.elastic.co/u/tetlika)\ **Replies:** 10\ **Last updated:** [March 14, 2016, 2:56pm UTC](https://discuss.elastic.co/t/problems-with-logstash-codecs-json-rubydebug/44252 "2016-03-14T14:56:59Z")
hi, we're using logstash 2.2.2 I'm trying to feed logstash with kibana logs, kibana log looks like: {"type":"log","@timestamp":"2016-03-13T10:05:30+00:00","tags":\["status","plugin:elasticsearch","info"\],"pid":15856,"na… --- ## [Logstash - v:2.2.2 : input file path not working](https://discuss.elastic.co/t/logstash-v-2-2-2-input-file-path-not-working/44317)
**Author:** [@andreealib](https://discuss.elastic.co/u/andreealib)\ **Replies:** 4\ **Last updated:** [March 14, 2016, 12:52pm UTC](https://discuss.elastic.co/t/logstash-v-2-2-2-input-file-path-not-working/44317 "2016-03-14T12:52:57Z")
I have the following stack installed on my Windows 8: elasticsearch 2.20 logstash 2.2.2 kibana 4.4.1 I have a csv file from which I want to load the data into elasticsearch. My configuration file is: input { file … --- ## [Using logstash to get data out of elasticsearch to a csv file](https://discuss.elastic.co/t/using-logstash-to-get-data-out-of-elasticsearch-to-a-csv-file/44230)
**Author:** [@Hideyoshi\_Maeda](https://discuss.elastic.co/u/Hideyoshi_Maeda)\ **Replies:** 2\ **Last updated:** [March 14, 2016, 11:55am UTC](https://discuss.elastic.co/t/using-logstash-to-get-data-out-of-elasticsearch-to-a-csv-file/44230 "2016-03-14T11:55:54Z")
Hello, I am new to using logstash and am struggling to get data out of elasticsearch using logstash as a csv. To create some sample data, we can first add a basic csv into elasticsearch... the head of the sample csv ca… --- ## [Logstash client feeding inputs to another Logstash](https://discuss.elastic.co/t/logstash-client-feeding-inputs-to-another-logstash/44278)
**Author:** [@mohit\_sharma](https://discuss.elastic.co/u/mohit_sharma)\ **Replies:** 1\ **Last updated:** [March 14, 2016, 11:52am UTC](https://discuss.elastic.co/t/logstash-client-feeding-inputs-to-another-logstash/44278 "2016-03-14T11:52:51Z")
I have an ELK stack setup working well. Next, I have installed only Logstash clients on specific systems from where I am expecting logs to be generated for ELK processing. How can I configure/write an input plugin for Lo… --- ## [Logstash: Configuration for websphere](https://discuss.elastic.co/t/logstash-configuration-for-websphere/44295)
**Author:** [@krushnat\_khawale](https://discuss.elastic.co/u/krushnat_khawale)\ **Replies:** 1\ **Last updated:** [March 14, 2016, 8:54am UTC](https://discuss.elastic.co/t/logstash-configuration-for-websphere/44295 "2016-03-14T08:54:38Z")
Following is my configuration file for logstash to read data from websphere logs. input { file { type =\> "bolbo" path =\> \[ "D:/Users/abced/Documents/My Received Files/1003/1003/gca1.log" \] … --- ## [Error from command logstash](https://discuss.elastic.co/t/error-from-command-logstash/44290)
**Author:** [@Inbeo\_Beo](https://discuss.elastic.co/u/Inbeo_Beo)\ **Replies:** 4\ **Last updated:** [March 14, 2016, 8:47am UTC](https://discuss.elastic.co/t/error-from-command-logstash/44290 "2016-03-14T08:47:10Z")
Dear all when i config logstash-forwarder with command line: sudo service logstash-forwarder start i have had answer --\> nohup: appending output to \`nohup.out I don't i understand why? Could you support me Than… --- ## [How to use Logstash automatically to store data from MySQL](https://discuss.elastic.co/t/how-to-use-logstash-automatically-to-store-data-from-mysql/44291)
**Author:** [@Jemli\_Fathi](https://discuss.elastic.co/u/Jemli_Fathi)\ **Replies:** 2\ **Last updated:** [March 14, 2016, 8:26am UTC](https://discuss.elastic.co/t/how-to-use-logstash-automatically-to-store-data-from-mysql/44291 "2016-03-14T08:26:47Z")
I used Logstash to load data from MySQL table into elasticsearch, but, the question is how can I configure logstash to do that automatically and to use it to store many tables or a whole database not a single table eve… --- ## [Why is my Logstash plugin PR being ignored](https://discuss.elastic.co/t/why-is-my-logstash-plugin-pr-being-ignored/24396)
**Author:** [@jrgns](https://discuss.elastic.co/u/jrgns)\ **Replies:** 5\ **Last updated:** [March 14, 2016, 7:10am UTC](https://discuss.elastic.co/t/why-is-my-logstash-plugin-pr-being-ignored/24396 "2016-03-14T07:10:53Z")
Hey I submitted a PR to the mutate filter in the beginning of the year. I've signed the CLA and GitHub shows that the PR can be merged. I've also asked for feedback on the PR on why it hasn't been merged yet, but I've … --- ## [Throughput metrics for Lagstash and Elasticsearch](https://discuss.elastic.co/t/throughput-metrics-for-lagstash-and-elasticsearch/44039)
**Author:** [@usahitya](https://discuss.elastic.co/u/usahitya)\ **Replies:** 5\ **Last updated:** [March 14, 2016, 7:07am UTC](https://discuss.elastic.co/t/throughput-metrics-for-lagstash-and-elasticsearch/44039 "2016-03-14T07:07:23Z")
Hi Community, I would like to know the metrics about Logstash and Kibana. How much throughput can tolerate by Logstash ? means, how much data it can receive from redis or any shiper? , How much data it can send to th… --- ## [CSV Filter in Logstash 2.2](https://discuss.elastic.co/t/csv-filter-in-logstash-2-2/41937)
**Author:** [@Dustman1977](https://discuss.elastic.co/u/Dustman1977)\ **Replies:** 4\ **Last updated:** [March 13, 2016, 2:16pm UTC](https://discuss.elastic.co/t/csv-filter-in-logstash-2-2/41937 "2016-03-13T14:16:03Z")
I just upgraded from Logstash 1.7 to Logstash 2.2. most things are working except for my csv filters. I get XML data and some of the fields contain csv values. In Logstash 1.7 my filter looked like this: csv { … --- ## [Multiple dates in log file defining the @timestamp date](https://discuss.elastic.co/t/multiple-dates-in-log-file-defining-the-timestamp-date/44233)
**Author:** [@Hans](https://discuss.elastic.co/u/Hans)\ **Replies:** 5\ **Last updated:** [March 13, 2016, 8:35am UTC](https://discuss.elastic.co/t/multiple-dates-in-log-file-defining-the-timestamp-date/44233 "2016-03-13T08:35:56Z")
Hi, I have a challenge, in the log-file of mine, I have three dates and would like to define the 3rd one as the @timestamp that is reflected in the ES and Kibana searches. What filter can I apply for this as the date one… --- ## [Elasticsearch Filter - how to query term in a specific index](https://discuss.elastic.co/t/elasticsearch-filter-how-to-query-term-in-a-specific-index/44168)
**Author:** [@MikeM](https://discuss.elastic.co/u/MikeM)\ **Replies:** 2\ **Last updated:** [March 12, 2016, 10:44pm UTC](https://discuss.elastic.co/t/elasticsearch-filter-how-to-query-term-in-a-specific-index/44168 "2016-03-12T22:44:38Z")
I'm trying to use the Elasticsearch filter in my logstash configuration to search a specific index for a term. I need help building the query. Would the following be valid? { "indices" : { "indices" : \[ … --- ## [Need Help in converting Julian date in to Calender date](https://discuss.elastic.co/t/need-help-in-converting-julian-date-in-to-calender-date/44140)
**Author:** [@Pavithrananda\_Prabhu](https://discuss.elastic.co/u/Pavithrananda_Prabhu)\ **Replies:** 1\ **Last updated:** [March 12, 2016, 9:01pm UTC](https://discuss.elastic.co/t/need-help-in-converting-julian-date-in-to-calender-date/44140 "2016-03-12T21:01:10Z")
Need help in converting Julian date in to Calender date Below is the sample Syslog message from the mainframe N C000000 PROD 15307 23:58:19.27 JOB78042 00000004 $HASP101 OGIVJS11 HELD would like to convert 15… --- ## [Multiline Capture Group](https://discuss.elastic.co/t/multiline-capture-group/44083)
**Author:** [@Cole\_Shores](https://discuss.elastic.co/u/Cole_Shores)\ **Replies:** 3\ **Last updated:** [March 12, 2016, 11:36am UTC](https://discuss.elastic.co/t/multiline-capture-group/44083 "2016-03-12T11:36:14Z")
(^Mar\[\\s\\S\]?Caused) ... or more specifically (^%{MONTH}\[\\s\\S\]?Caused) doesn't work to capture a multiline log. Is there a way to capture a multiline log from a start and ending delimiter. Or is there a multiline filter… --- ## [Im Using logstash 2.2 . Im getting below logs](https://discuss.elastic.co/t/im-using-logstash-2-2-im-getting-below-logs/43979)
**Author:** [@Sudharsankotti](https://discuss.elastic.co/u/Sudharsankotti)\ **Replies:** 1\ **Last updated:** [March 12, 2016, 2:18am UTC](https://discuss.elastic.co/t/im-using-logstash-2-2-im-getting-below-logs/43979 "2016-03-12T02:18:21Z")
"/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.5.1-java/lib/logstash/outputs/elasticsearch/http\_client.rb:60:in start\_sniffing!'", "org/jruby/ext/thread/Mutex.java:149:insynchronize'", "/opt/… --- ## [Does Elastic Cloud provide Logstash SaaS as well?](https://discuss.elastic.co/t/does-elastic-cloud-provide-logstash-saas-as-well/44148)
**Author:** [@Ignasi\_Fosch](https://discuss.elastic.co/u/Ignasi_Fosch)\ **Replies:** 1\ **Last updated:** [March 12, 2016, 1:53am UTC](https://discuss.elastic.co/t/does-elastic-cloud-provide-logstash-saas-as-well/44148 "2016-03-12T01:53:57Z")
I tried to found this question in the forum unsuccessfully, but maybe I was not really good at that, so my apologies if there is such question already there. Just evaluating Elastic Cloud as ELK SaaS, and wondered if i… --- ## [Logstash Installation](https://discuss.elastic.co/t/logstash-installation/44206)
**Author:** [@piyush](https://discuss.elastic.co/u/piyush)\ **Replies:** 2\ **Last updated:** [March 12, 2016, 1:06am UTC](https://discuss.elastic.co/t/logstash-installation/44206 "2016-03-12T01:06:25Z")
Hi Team, I successfully tested Beats (Filebeat,topbeat,packetbeat) =\> ElasticSearch now i am trying for Beats =\> Logstash =\> Elasticsearch. But i am getting error at beat end, though all traffic is allowed. I found som… --- ## [Assistance with grok filter](https://discuss.elastic.co/t/assistance-with-grok-filter/43954)
**Author:** [@Don\_Pich](https://discuss.elastic.co/u/Don_Pich)\ **Replies:** 3\ **Last updated:** [March 11, 2016, 8:44pm UTC](https://discuss.elastic.co/t/assistance-with-grok-filter/43954 "2016-03-11T20:44:42Z")
So my grok is working great, but my index is not creating new fields as I would expect (new field as in 'entryDateTime', 'user', and 'event'). First off, here is the data in my log file: 2016-03-09T16:22:39-06:00|dpich… --- ## [Logstash combining all the grok pattern returns](https://discuss.elastic.co/t/logstash-combining-all-the-grok-pattern-returns/44058)
**Author:** [@athreya\_vc](https://discuss.elastic.co/u/athreya_vc)\ **Replies:** 2\ **Last updated:** [March 11, 2016, 4:35pm UTC](https://discuss.elastic.co/t/logstash-combining-all-the-grok-pattern-returns/44058 "2016-03-11T16:35:56Z")
Hi, I have a logfile like this 2016-03-10 04:49:41,586 p=8250 u=root | TASK \[filetester : Display messages\] \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\* 2016-03-10 04:49:41,619 p=8250 u=root | skipping: \[MYLABSERV… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1178) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1180)