# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=863 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 864 --- ## [Best way to index IIS logs, Weblogic logs, etc](https://discuss.elastic.co/t/best-way-to-index-iis-logs-weblogic-logs-etc/109422)
**Author:** [@mrc](https://discuss.elastic.co/u/mrc)\ **Replies:** 3\ **Last updated:** [December 4, 2017, 7:30pm UTC](https://discuss.elastic.co/t/best-way-to-index-iis-logs-weblogic-logs-etc/109422 "2017-12-04T19:30:42Z")
Hello, I was able to successfully index Weblogic HTTP access logs into ElasticSearch thru Logstash. I am now tasked to do IIS logs - I was under the impression that when I process the IIS logs the new fields will be add… --- ## [Logstash input with Kerberos Kafka Stream](https://discuss.elastic.co/t/logstash-input-with-kerberos-kafka-stream/110202)
**Author:** [@vmandulapalli](https://discuss.elastic.co/u/vmandulapalli)\ **Replies:** 0\ **Last updated:** [December 4, 2017, 7:24pm UTC](https://discuss.elastic.co/t/logstash-input-with-kerberos-kafka-stream/110202 "2017-12-04T19:24:58Z")
Hi, I'm trying to create a logstash pipeline with kafka input plugin, kafka streams are protected with kerberos security. Provided configuration settings for jaas.conf, krb5.conf, and .keytab file, getting below error… --- ## [Logstash stops when running as a service on Redhat](https://discuss.elastic.co/t/logstash-stops-when-running-as-a-service-on-redhat/110193)
**Author:** [@amruth](https://discuss.elastic.co/u/amruth)\ **Replies:** 0\ **Last updated:** [December 4, 2017, 5:45pm UTC](https://discuss.elastic.co/t/logstash-stops-when-running-as-a-service-on-redhat/110193 "2017-12-04T17:45:02Z")
Hi, I am running Logstash as a service on Redhat with the command 'systemctl start logstash'. I am sending logs to Elasticsearch using http output plugin. It worked for some time, but later it is stopped with the follow… --- ## [Logstash input connections from beats stay established and lead to "too many open files" errors](https://discuss.elastic.co/t/logstash-input-connections-from-beats-stay-established-and-lead-to-too-many-open-files-errors/110180)
**Author:** [@Antoine\_Benoit](https://discuss.elastic.co/u/Antoine_Benoit)\ **Replies:** 0\ **Last updated:** [December 4, 2017, 4:35pm UTC](https://discuss.elastic.co/t/logstash-input-connections-from-beats-stay-established-and-lead-to-too-many-open-files-errors/110180 "2017-12-04T16:35:20Z")
Hello, I am using the Elastic Stack 6 on 3 servers running Debian 9.1. One ("server2") has Elasticsearch, Logstash, Kibana installed All three have Filebeat and Metricbeat installed to collect data Logstash is config… --- ## [Logstash kafka with log windows sent format json](https://discuss.elastic.co/t/logstash-kafka-with-log-windows-sent-format-json/110152)
**Author:** [@abdallah\_belahcen](https://discuss.elastic.co/u/abdallah_belahcen)\ **Replies:** 0\ **Last updated:** [December 4, 2017, 1:30pm UTC](https://discuss.elastic.co/t/logstash-kafka-with-log-windows-sent-format-json/110152 "2017-12-04T13:30:03Z")
Hello i have problem with my filter. my log sent by NXLOG format JSON from windows Server to two shipper logstash, the shipper send the log to kafka. my problem kafka add some information and this information alter th… --- ## [GROK pattern extract](https://discuss.elastic.co/t/grok-pattern-extract/110146)
**Author:** [@Sumit\_Sijaria](https://discuss.elastic.co/u/Sumit_Sijaria)\ **Replies:** 6\ **Last updated:** [December 4, 2017, 1:27pm UTC](https://discuss.elastic.co/t/grok-pattern-extract/110146 "2017-12-04T13:27:25Z")
HI, I am extracting info from string - \[Fri Dec 01 09:00:00.983 2017\] app-pr.com app-onl-pr-lb app-v1-online-pr-node4 0.017070 200 /rest/internal/itemInfo/123123213/method\_call GROK pattern: \[%{DATA:timestamp}\] %{DAT… --- ## [Convert Incoming UTC timestamp to local time](https://discuss.elastic.co/t/convert-incoming-utc-timestamp-to-local-time/110125)
**Author:** [@chivas](https://discuss.elastic.co/u/chivas)\ **Replies:** 1\ **Last updated:** [December 4, 2017, 12:41pm UTC](https://discuss.elastic.co/t/convert-incoming-utc-timestamp-to-local-time/110125 "2017-12-04T12:41:28Z")
Newbie here, Is there a way to convert the incoming timestamp from my log file into my local timezone, e.g. my file contains the timestamp in UTC "2017-12-01T07:30:32.425064Z", is there a way to convert this into "2017… --- ## [\[include\_in\_all\] is not allowed for indices (after upgrading to 6.0.0)](https://discuss.elastic.co/t/include-in-all-is-not-allowed-for-indices-after-upgrading-to-6-0-0/109769)
**Author:** [@dulman](https://discuss.elastic.co/u/dulman)\ **Replies:** 2\ **Last updated:** [December 4, 2017, 12:13pm UTC](https://discuss.elastic.co/t/include-in-all-is-not-allowed-for-indices-after-upgrading-to-6-0-0/109769 "2017-12-04T12:13:03Z")
Hello, we have upgraded our Elastic Stack (Elasticsearch, Logstash, Kibana, Beats) to version 6.0.0. Our Logstash config: input { tcp { type =\> app port =\> 5140 } } output { elasticsearch { hosts=\>… --- ## [Googleanalytics plugin](https://discuss.elastic.co/t/googleanalytics-plugin/110137)
**Author:** [@artobstrel](https://discuss.elastic.co/u/artobstrel)\ **Replies:** 0\ **Last updated:** [December 4, 2017, 12:11pm UTC](https://discuss.elastic.co/t/googleanalytics-plugin/110137 "2017-12-04T12:11:57Z")
Can you please tell me how to correctly install the plugin logstash-input-googleanalytics on Logstash 6.0 Is it possible? --- ## [Logstash:5.4.0 sending cloudtrail logs from s3 to elasticsearch - logstash.instrument.periodicpoller.cgroup - Error, cannot retrieve cgroups information](https://discuss.elastic.co/t/logstash-5-4-0-sending-cloudtrail-logs-from-s3-to-elasticsearch-logstash-instrument-periodicpoller-cgroup-error-cannot-retrieve-cgroups-information/110120)
**Author:** [@berlin](https://discuss.elastic.co/u/berlin)\ **Replies:** 0\ **Last updated:** [December 4, 2017, 11:02am UTC](https://discuss.elastic.co/t/logstash-5-4-0-sending-cloudtrail-logs-from-s3-to-elasticsearch-logstash-instrument-periodicpoller-cgroup-error-cannot-retrieve-cgroups-information/110120 "2017-12-04T11:02:02Z")
Running Logstash in a Docker container version logstash:5.4.0 and I get the following log message after few seconds and after its sent few of the logs from s3 to ElasticSearch. Logstash appears to be frozen, no updates … --- ## [Slack with logstash and elastalert](https://discuss.elastic.co/t/slack-with-logstash-and-elastalert/109544)
**Author:** [@pir1981](https://discuss.elastic.co/u/pir1981)\ **Replies:** 3\ **Last updated:** [December 4, 2017, 10:48am UTC](https://discuss.elastic.co/t/slack-with-logstash-and-elastalert/109544 "2017-12-04T10:48:27Z")
Hi, I have an ELK stack with a hook to slack The logstash filter is the following if \[message\] =~ "Reallocated\_Sector\_Count" or \[message\] =~ "Reallocated\_Sector\_Ct" or \[message\] =~ "Reported\_Uncorrectable\_Errors… --- ## [Logstash YUM installation issue!](https://discuss.elastic.co/t/logstash-yum-installation-issue/110105)
**Author:** [@SJN8](https://discuss.elastic.co/u/SJN8)\ **Replies:** 1\ **Last updated:** [December 4, 2017, 9:47am UTC](https://discuss.elastic.co/t/logstash-yum-installation-issue/110105 "2017-12-04T09:47:42Z")
Hi All, I am trying to install logstash with YUM package method , But i am facing below issue , sudo yum update logstash Loaded plugins: aliases, changelog, fastestmirror, kabi, langpacks, tmprepo, verify, versionl… --- ## [Compare two fields](https://discuss.elastic.co/t/compare-two-fields/110088)
**Author:** [@pope843](https://discuss.elastic.co/u/pope843)\ **Replies:** 4\ **Last updated:** [December 4, 2017, 7:23am UTC](https://discuss.elastic.co/t/compare-two-fields/110088 "2017-12-04T07:23:35Z")
Hello Guys, Just want to know if below condition will work or possible? filter { ruby { code =\> " event.set('dateko', Time.now.strftime('%Y-%m-%d')) " } if \[dateko\] not in \[message\] { ....... } } --- ## [Delete by query using logstash](https://discuss.elastic.co/t/delete-by-query-using-logstash/109776)
**Author:** [@Sai\_Jana\_Selvakumar](https://discuss.elastic.co/u/Sai_Jana_Selvakumar)\ **Replies:** 1\ **Last updated:** [December 4, 2017, 6:30am UTC](https://discuss.elastic.co/t/delete-by-query-using-logstash/109776 "2017-12-04T06:30:47Z")
Is there any way to send delete by query API by means of elasticsearch output plugin or any other plugin in logstash? issue: I want to delete the index data by using on of the fields using logstash. --- ## [Logstash-6.0.0 displays an unhelpful NullPointerException Error in case of corrupted persistent queue](https://discuss.elastic.co/t/logstash-6-0-0-displays-an-unhelpful-nullpointerexception-error-in-case-of-corrupted-persistent-queue/108641)
**Author:** [@Dheeraj\_Gupta](https://discuss.elastic.co/u/Dheeraj_Gupta)\ **Replies:** 6\ **Last updated:** [December 4, 2017, 5:36am UTC](https://discuss.elastic.co/t/logstash-6-0-0-displays-an-unhelpful-nullpointerexception-error-in-case-of-corrupted-persistent-queue/108641 "2017-12-04T05:36:48Z")
I upgraded my logstash node to 6.0.0 from 5.6.0. Initially the node worked fine, but today I tried to restart it and the node refused to start. The error logged was \[2017-11-22T10:41:30,054\]\[ERROR\]\[logstash.agent … --- ## [How to make logstash publish from a certain line number onwards of the input file](https://discuss.elastic.co/t/how-to-make-logstash-publish-from-a-certain-line-number-onwards-of-the-input-file/109727)
**Author:** [@Kasun\_Siyambalapitiy](https://discuss.elastic.co/u/Kasun_Siyambalapitiy)\ **Replies:** 1\ **Last updated:** [December 4, 2017, 5:28am UTC](https://discuss.elastic.co/t/how-to-make-logstash-publish-from-a-certain-line-number-onwards-of-the-input-file/109727 "2017-12-04T05:28:40Z")
Hi all, I am using logstash to publish events of a log file which logs are constantly getting appended. But I need logstash to publish events from a certain line number onwards (say from 100th line onwards). How do I ac… --- ## [IPFix Custom Cisco ASR1k some not parsing](https://discuss.elastic.co/t/ipfix-custom-cisco-asr1k-some-not-parsing/107475)
**Author:** [@sudoritz](https://discuss.elastic.co/u/sudoritz)\ **Replies:** 2\ **Last updated:** [December 3, 2017, 7:17pm UTC](https://discuss.elastic.co/t/ipfix-custom-cisco-asr1k-some-not-parsing/107475 "2017-12-03T19:17:32Z")
buildig Custom asr1k IPFIX flows I looked at some template data / pcaps n such and got these to work with SPlunk Stream(BUT 12235/12236/12237 doesnt seem to parse right in it) So im trying to rule out and look at trying… --- ## [Logstash sending logs to wrong indices](https://discuss.elastic.co/t/logstash-sending-logs-to-wrong-indices/110024)
**Author:** [@GuillaumeN](https://discuss.elastic.co/u/GuillaumeN)\ **Replies:** 2\ **Last updated:** [December 3, 2017, 7:10pm UTC](https://discuss.elastic.co/t/logstash-sending-logs-to-wrong-indices/110024 "2017-12-03T19:10:02Z")
Hi! I'm having an issue with logstash, it's sending logs to the configured output indices of ALL the pipeline configuration files (I have 2 with 1 output index per file) even though it shouldn't. I'm using the ELK stac… --- ## [Dupilcate message on elasticsearch](https://discuss.elastic.co/t/dupilcate-message-on-elasticsearch/107182)
**Author:** [@pope843](https://discuss.elastic.co/u/pope843)\ **Replies:** 5\ **Last updated:** [December 3, 2017, 1:20am UTC](https://discuss.elastic.co/t/dupilcate-message-on-elasticsearch/107182 "2017-12-03T01:20:29Z")
Hello Guys, Need help regarding duplicate messages on my elasticsearch. I have below config. As you can see i'm using fingerprint plugin to fix duplicates but to no avail. input { beats { port =\> 5044 ssl =\> true s… --- ## [Systemd restarts logstash when using jdbc input](https://discuss.elastic.co/t/systemd-restarts-logstash-when-using-jdbc-input/109952)
**Author:** [@arisbanach](https://discuss.elastic.co/u/arisbanach)\ **Replies:** 5\ **Last updated:** [December 2, 2017, 9:34pm UTC](https://discuss.elastic.co/t/systemd-restarts-logstash-when-using-jdbc-input/109952 "2017-12-02T21:34:36Z")
Not sure if I've configured something wrong since this is my first time working with JDBC. I am trying to import a large database into Elasticsearch using Logstash and the logstash-input-jdbc plugin. Here is my pipeline: … --- ## [Logstash error in config file](https://discuss.elastic.co/t/logstash-error-in-config-file/110003)
**Author:** [@Shreya\_Chatterjee](https://discuss.elastic.co/u/Shreya_Chatterjee)\ **Replies:** 7\ **Last updated:** [December 2, 2017, 6:36pm UTC](https://discuss.elastic.co/t/logstash-error-in-config-file/110003 "2017-12-02T18:36:59Z")
I'm trying to feed data in csv files into elastic search using logstash. My logsatsh config file looks like this: input { file { path =\> "C:\\Users\\shreya\\Data\\RetailData.csv" start\_position =\> "b… --- ## [Unable to index one field..please help](https://discuss.elastic.co/t/unable-to-index-one-field-please-help/110019)
**Author:** [@Blason](https://discuss.elastic.co/u/Blason)\ **Replies:** 1\ **Last updated:** [December 2, 2017, 6:20pm UTC](https://discuss.elastic.co/t/unable-to-index-one-field-please-help/110019 "2017-12-02T18:20:41Z")
Hi All, I am using bind log parser to parse the logs and at the same time have created a translate\_plugin to create dictionary\_path and adding customised field but I am not sure how to index that field using logstash-te… --- ## [I cannot run Logstash on Raspberry pi3](https://discuss.elastic.co/t/i-cannot-run-logstash-on-raspberry-pi3/109789)
**Author:** [@AhmyOhlin](https://discuss.elastic.co/u/AhmyOhlin)\ **Replies:** 2\ **Last updated:** [December 2, 2017, 12:05pm UTC](https://discuss.elastic.co/t/i-cannot-run-logstash-on-raspberry-pi3/109789 "2017-12-02T12:05:40Z")
Hello all, i could install and run elasticsearch and Kibana on Raspberry pi 3 but iam struggling right now to run logstash. I did the same steps on windows and it works without problems. i run Logstash With this com… --- ## [Add\_field and copy data](https://discuss.elastic.co/t/add-field-and-copy-data/109773)
**Author:** [@mathurin68](https://discuss.elastic.co/u/mathurin68)\ **Replies:** 7\ **Last updated:** [December 2, 2017, 5:22am UTC](https://discuss.elastic.co/t/add-field-and-copy-data/109773 "2017-12-02T05:22:34Z")
Alright, I think I just about have what I'm looking for ... Any activity that comes over event\_data.CommandLine the doesn't have " " quotations around it. I want to rename the field and copy the data from the field i… --- ## [Logstash JDBC plugin not working without LIMIT set in statement](https://discuss.elastic.co/t/logstash-jdbc-plugin-not-working-without-limit-set-in-statement/109989)
**Author:** [@arisbanach](https://discuss.elastic.co/u/arisbanach)\ **Replies:** 0\ **Last updated:** [December 1, 2017, 11:50pm UTC](https://discuss.elastic.co/t/logstash-jdbc-plugin-not-working-without-limit-set-in-statement/109989 "2017-12-01T23:50:30Z")
I am not sure if there is another issue causing this, but I have done some testing and found an odd issue with my Logstash instance. I am trying to import a very large Postgresql database from a remote server and if I se… --- ## [Grok Config to populate Date with Name of File](https://discuss.elastic.co/t/grok-config-to-populate-date-with-name-of-file/109985)
**Author:** [@sconrod](https://discuss.elastic.co/u/sconrod)\ **Replies:** 0\ **Last updated:** [December 1, 2017, 9:30pm UTC](https://discuss.elastic.co/t/grok-config-to-populate-date-with-name-of-file/109985 "2017-12-01T21:30:09Z")
HI Magnus, I saw your previous notes on this here: Extract timestamp from filename and I have tried to populate my date field with the date that is in the name of the log file like this but it is not working. Suggestion… --- ## [Config files reloaded with -SIGHUP](https://discuss.elastic.co/t/config-files-reloaded-with-sighup/109984)
**Author:** [@zpraxis](https://discuss.elastic.co/u/zpraxis)\ **Replies:** 0\ **Last updated:** [December 1, 2017, 9:09pm UTC](https://discuss.elastic.co/t/config-files-reloaded-with-sighup/109984 "2017-12-01T21:09:24Z")
Hi all, I am new to Logstash, and was wondering which files are reloaded when a SIGHUP signal is sent to the process. As far as I can see, only .conf files are read for changes. I changed the name of the instance in th… --- ## [How Can I make my csv fields available on the Filter drop down in Kibana when discovering the data via the index?](https://discuss.elastic.co/t/how-can-i-make-my-csv-fields-available-on-the-filter-drop-down-in-kibana-when-discovering-the-data-via-the-index/107643)
**Author:** [@sconrod](https://discuss.elastic.co/u/sconrod)\ **Replies:** 4\ **Last updated:** [December 1, 2017, 8:11pm UTC](https://discuss.elastic.co/t/how-can-i-make-my-csv-fields-available-on-the-filter-drop-down-in-kibana-when-discovering-the-data-via-the-index/107643 "2017-12-01T20:11:19Z")
Below have pasted the .conf file and the mappings output: root@ubuntu-16:/etc/logstash/conf.d# cat bzip4-v4.conf input { file { path =\> "/opt/sample-data/csv/bzip4/\*.csv" start\_position =\> "beginning" sincedb\_path… --- ## [Could not find logstash.yml](https://discuss.elastic.co/t/could-not-find-logstash-yml/109834)
**Author:** [@saad](https://discuss.elastic.co/u/saad)\ **Replies:** 10\ **Last updated:** [December 1, 2017, 6:55pm UTC](https://discuss.elastic.co/t/could-not-find-logstash-yml/109834 "2017-12-01T18:55:56Z")
I installed losgatsh 6 in Centos 7 by using RPM installation. when I run bin/logstash -f /etc/logstash/conf.d/pan.conf I got the following error WARNING: Could not find logstash.yml which is typically located in $LS\_H… --- ## [Elapsed filter with multiple workers..does it work or not?](https://discuss.elastic.co/t/elapsed-filter-with-multiple-workers-does-it-work-or-not/108538)
**Author:** [@Nikhil\_Utane](https://discuss.elastic.co/u/Nikhil_Utane)\ **Replies:** 7\ **Last updated:** [December 1, 2017, 6:40pm UTC](https://discuss.elastic.co/t/elapsed-filter-with-multiple-workers-does-it-work-or-not/108538 "2017-12-01T18:40:15Z")
Hi, I tried searching but haven't got a definite answer. Aggregate filter clearly mentions that: "You should be very careful to set Logstash filter workers to 1 (-w 1 flag) for this filter to work correctly otherwise … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=862) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=864)