# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1100 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1101 --- ## [Logstash failing after upgrade into the version 2.x](https://discuss.elastic.co/t/logstash-failing-after-upgrade-into-the-version-2-x/59537)
**Author:** [@napoleon182](https://discuss.elastic.co/u/napoleon182)\ **Replies:** 18\ **Last updated:** [September 21, 2016, 1:55pm UTC](https://discuss.elastic.co/t/logstash-failing-after-upgrade-into-the-version-2-x/59537 "2016-09-21T13:55:49Z")
hi All, I recently upgraded logstash as per https://www.elastic.co/guide/en/logstash/current/installing-logstash.html#package-repositories and I'm unable to start the logstash service which is now failing with below erro… --- ## [Update multiple docs using Logstash](https://discuss.elastic.co/t/update-multiple-docs-using-logstash/61134)
**Author:** [@ydkadri](https://discuss.elastic.co/u/ydkadri)\ **Replies:** 0\ **Last updated:** [September 21, 2016, 1:04pm UTC](https://discuss.elastic.co/t/update-multiple-docs-using-logstash/61134 "2016-09-21T13:04:52Z")
Hi I have two data sources which will be ingested separately using Logstash into elasticsearch, one containing page data from larger documents, the other containing metadata around those same documents e.g. source 1… --- ## [\[SOLVED\] Grok and split fields](https://discuss.elastic.co/t/solved-grok-and-split-fields/61010)
**Author:** [@Romain\_PZK](https://discuss.elastic.co/u/Romain_PZK)\ **Replies:** 4\ **Last updated:** [September 21, 2016, 12:26pm UTC](https://discuss.elastic.co/t/solved-grok-and-split-fields/61010 "2016-09-21T12:26:33Z")
Hello, I want to put logs from a Fortigate firewall into elasticsearch. My fortigate sends logs to my syslog-ng, and logstash reads the log file. I use a grok to parse my log, but I don't understand how to split the "m… --- ## [Custom index not creating at ElasticSearch cluster](https://discuss.elastic.co/t/custom-index-not-creating-at-elasticsearch-cluster/61016)
**Author:** [@Phani\_Nadiminti](https://discuss.elastic.co/u/Phani_Nadiminti)\ **Replies:** 8\ **Last updated:** [September 21, 2016, 12:11pm UTC](https://discuss.elastic.co/t/custom-index-not-creating-at-elasticsearch-cluster/61016 "2016-09-21T12:11:29Z")
HI All, I am using ES 1.5.2 with logstash 2.4.0 . i have the following configurations to pull data from kafka topic and index to ES. input{ kafka { zk\_connect =\> ":2181" group\_id =\> "logstash" topic\_i… --- ## [Filtering Windows Event Logs](https://discuss.elastic.co/t/filtering-windows-event-logs/60889)
**Author:** [@ANU\_SARA\_VARGHESE](https://discuss.elastic.co/u/ANU_SARA_VARGHESE)\ **Replies:** 10\ **Last updated:** [September 21, 2016, 11:41am UTC](https://discuss.elastic.co/t/filtering-windows-event-logs/60889 "2016-09-21T11:41:16Z")
Hi, I am creating a POC of ELK for analysing windows event logs. I am not getting how to apply filters on these event logs. Is there any pattern defined to be directly used in the Grok filter like for Syslogs? If not… --- ## [Scale Logstash indexers for \>200,000 messages per minute?](https://discuss.elastic.co/t/scale-logstash-indexers-for-200-000-messages-per-minute/33878)
**Author:** [@jbiggley](https://discuss.elastic.co/u/jbiggley)\ **Replies:** 16\ **Last updated:** [September 21, 2016, 11:01am UTC](https://discuss.elastic.co/t/scale-logstash-indexers-for-200-000-messages-per-minute/33878 "2016-09-21T11:01:25Z")
First, that isn't a typo. In fact, I want to be able to scale LS to handle \>500,000 (maybe a million) message per minute once we get out of our POC. Our basic setup is as follows: On-prem LS server + Redis \> x1 AWS L… --- ## [Timezone causing problem when doing a search query to an index!](https://discuss.elastic.co/t/timezone-causing-problem-when-doing-a-search-query-to-an-index/61077)
**Author:** [@Kulasangar\_Gowrisang](https://discuss.elastic.co/u/Kulasangar_Gowrisang)\ **Replies:** 6\ **Last updated:** [September 21, 2016, 10:59am UTC](https://discuss.elastic.co/t/timezone-causing-problem-when-doing-a-search-query-to-an-index/61077 "2016-09-21T10:59:53Z")
I'm trying to find out the results from a search query (ie: searching results for the given date range) of a particular index. So that I could get the results in a daily basis. This is the query : http://localhost:9200… --- ## [Matching/Finding certain text using logstash filter](https://discuss.elastic.co/t/matching-finding-certain-text-using-logstash-filter/48411)
**Author:** [@spravn789](https://discuss.elastic.co/u/spravn789)\ **Replies:** 8\ **Last updated:** [September 21, 2016, 10:57am UTC](https://discuss.elastic.co/t/matching-finding-certain-text-using-logstash-filter/48411 "2016-09-21T10:57:01Z")
I have a text with \[INFO or ERROR or Warn\] concatenated with few other texts, which i am able to get under greedy data, But I want to split those Log Level values under new field. Sample Text: 2016 Apr 26 15:44:40:603 … --- ## [Failing to parse and aggregate the Existing Index](https://discuss.elastic.co/t/failing-to-parse-and-aggregate-the-existing-index/60920)
**Author:** [@pramod\_kumar](https://discuss.elastic.co/u/pramod_kumar)\ **Replies:** 1\ **Last updated:** [September 20, 2016, 6:07pm UTC](https://discuss.elastic.co/t/failing-to-parse-and-aggregate-the-existing-index/60920 "2016-09-20T18:07:00Z")
Hi , I am trying to parse the existing index with the logstash config file, But am Failing to parse and aggregate the Existing Index. I have attached the config file. please correct me if am doing something wrong. i… --- ## [How to connect mysql database by logstash?](https://discuss.elastic.co/t/how-to-connect-mysql-database-by-logstash/61004)
**Author:** [@ajay\_shanker](https://discuss.elastic.co/u/ajay_shanker)\ **Replies:** 2\ **Last updated:** [September 21, 2016, 4:20am UTC](https://discuss.elastic.co/t/how-to-connect-mysql-database-by-logstash/61004 "2016-09-21T04:20:53Z")
step 1.I installed logstash in c drive and create a simple-out.conf file with this code # file: simple-out.conf input { jdbc { # Postgres jdbc connection string to our database, mydb … --- ## [S3 Input Logstash, nothing shows up](https://discuss.elastic.co/t/s3-input-logstash-nothing-shows-up/61060)
**Author:** [@Allie\_Yang](https://discuss.elastic.co/u/Allie_Yang)\ **Replies:** 0\ **Last updated:** [September 20, 2016, 11:11pm UTC](https://discuss.elastic.co/t/s3-input-logstash-nothing-shows-up/61060 "2016-09-20T23:11:30Z")
I am new to aws s3. -1, When i use CLI from KLAM, and run my config, nothing shows up. -2, When i use the access and secret keys from KLAM, it has the error "Error: The AWS Access Key Id you provided does not exist in… --- ## [How do I create a global date based on file header](https://discuss.elastic.co/t/how-do-i-create-a-global-date-based-on-file-header/60928)
**Author:** [@TaylorSanchez](https://discuss.elastic.co/u/TaylorSanchez)\ **Replies:** 8\ **Last updated:** [September 20, 2016, 8:59pm UTC](https://discuss.elastic.co/t/how-do-i-create-a-global-date-based-on-file-header/60928 "2016-09-20T20:59:30Z")
I am parsing some sysstat (Linux sar commands) files. They sysstat files start with a header of the Linux kernel, hostname, date, and architecture. I was able to parse out what I wanted with this grok filter: patterns\_… --- ## [Where does Logstash store its data?](https://discuss.elastic.co/t/where-does-logstash-store-its-data/60930)
**Author:** [@ZillaG](https://discuss.elastic.co/u/ZillaG)\ **Replies:** 8\ **Last updated:** [September 20, 2016, 8:42pm UTC](https://discuss.elastic.co/t/where-does-logstash-store-its-data/60930 "2016-09-20T20:42:20Z")
I'm using a central Logstash server that gets logs from multiple remote Logstash clients, which use rsyslog to forward logs. Where does Logstash store the logs in the server? Can I setup an NFS mount and tell Logstash to… --- ## [Logstash on my windows 7 machine](https://discuss.elastic.co/t/logstash-on-my-windows-7-machine/61002)
**Author:** [@ajay\_shanker](https://discuss.elastic.co/u/ajay_shanker)\ **Replies:** 1\ **Last updated:** [September 20, 2016, 8:03pm UTC](https://discuss.elastic.co/t/logstash-on-my-windows-7-machine/61002 "2016-09-20T20:03:18Z")
I just installed elasticsearch and logstash on my windows 7 machine with jdk1.8.0 and I got this error: "io/console not supported; tty will not be manipulated The signal HUP is in use by the JVM and will not work corr… --- ## [Identify lines older than X days](https://discuss.elastic.co/t/identify-lines-older-than-x-days/60388)
**Author:** [@Alex\_6](https://discuss.elastic.co/u/Alex_6)\ **Replies:** 11\ **Last updated:** [September 20, 2016, 6:21pm UTC](https://discuss.elastic.co/t/identify-lines-older-than-x-days/60388 "2016-09-20T18:21:04Z")
Logstash 2.3.4 I have dates extracted from the log lines using the date filter. I want to drop any log line which is older than 5 days and not put it into Elasticsearch at all. Is there a standard filter which does thi… --- ## [Logstash parsing only if run as root](https://discuss.elastic.co/t/logstash-parsing-only-if-run-as-root/61032)
**Author:** [@rwagner](https://discuss.elastic.co/u/rwagner)\ **Replies:** 3\ **Last updated:** [September 20, 2016, 5:40pm UTC](https://discuss.elastic.co/t/logstash-parsing-only-if-run-as-root/61032 "2016-09-20T17:40:17Z")
Hi guys, my logstash parsing logs only if run as root. Follow the video link showing the problem. Logstash version: yum list installed | grep logstash Repodata is over 2 weeks old. Install yum-cron? Or… --- ## [SNMP Trap listener died - bind: name or service not known](https://discuss.elastic.co/t/snmp-trap-listener-died-bind-name-or-service-not-known/60936)
**Author:** [@agonzalez](https://discuss.elastic.co/u/agonzalez)\ **Replies:** 2\ **Last updated:** [September 20, 2016, 1:40pm UTC](https://discuss.elastic.co/t/snmp-trap-listener-died-bind-name-or-service-not-known/60936 "2016-09-20T13:40:01Z")
This was working before and now getting this error. Any idea? {:timestamp=\>"2016-09-19T20:39:59.548000+0000", :message=\>"SNMP Trap listener died", :exception=\>#, :backtrace=\>\["org/jruby/ext/socket/RubyUDPSocket.java:16… --- ## [How to take beats inputs in multiple codecs in logstash](https://discuss.elastic.co/t/how-to-take-beats-inputs-in-multiple-codecs-in-logstash/60633)
**Author:** [@Bret\_Wortman](https://discuss.elastic.co/u/Bret_Wortman)\ **Replies:** 7\ **Last updated:** [September 20, 2016, 12:37pm UTC](https://discuss.elastic.co/t/how-to-take-beats-inputs-in-multiple-codecs-in-logstash/60633 "2016-09-20T12:37:56Z")
I've got a set of systems that I'll be deploying filebeats to, and I need it to monitor some text sources (like /var/log/secure) and some json sources (JSON-ified Apache logs). I've got the following on the filebeats s… --- ## [How to build logstash into jar ..and make it executable](https://discuss.elastic.co/t/how-to-build-logstash-into-jar-and-make-it-executable/60888)
**Author:** [@srikanth\_muddu](https://discuss.elastic.co/u/srikanth_muddu)\ **Replies:** 3\ **Last updated:** [September 20, 2016, 12:36pm UTC](https://discuss.elastic.co/t/how-to-build-logstash-into-jar-and-make-it-executable/60888 "2016-09-20T12:36:58Z")
hi guys ,i want to build logstash into jar.,,can u guys suggest wat steps i need to follow... have done ES with maven ...but how for logstash ?? ...please do help --- ## [Jdbc: using statement\_filepath together with sql\_last\_value](https://discuss.elastic.co/t/jdbc-using-statement-filepath-together-with-sql-last-value/45748)
**Author:** [@titzi](https://discuss.elastic.co/u/titzi)\ **Replies:** 5\ **Last updated:** [September 20, 2016, 12:05pm UTC](https://discuss.elastic.co/t/jdbc-using-statement-filepath-together-with-sql-last-value/45748 "2016-09-20T12:05:34Z")
I want to use the jdbc plugin to sync data from MySQL to Elasticsearch. The MySQL query is quite long so I chose to put it in an external SQL file using the 'statement\_filepath' option in the Logstash config file. Now … --- ## [Understanding event loss with HTTP input and jmeter test](https://discuss.elastic.co/t/understanding-event-loss-with-http-input-and-jmeter-test/60977)
**Author:** [@amit\_oren](https://discuss.elastic.co/u/amit_oren)\ **Replies:** 2\ **Last updated:** [September 20, 2016, 9:09am UTC](https://discuss.elastic.co/t/understanding-event-loss-with-http-input-and-jmeter-test/60977 "2016-09-20T09:09:45Z")
Hi, I'm trying to configure & stress test an HTTP input agent. The agent for the sake of example cant be as simple as HTTP input and not much more. At around 9000 messages per second, I start seeing events that time o… --- ## [Logstash input file codec multiline not working as expected](https://discuss.elastic.co/t/logstash-input-file-codec-multiline-not-working-as-expected/55913)
**Author:** [@S\_Roy](https://discuss.elastic.co/u/S_Roy)\ **Replies:** 3\ **Last updated:** [September 20, 2016, 8:58am UTC](https://discuss.elastic.co/t/logstash-input-file-codec-multiline-not-working-as-expected/55913 "2016-09-20T08:58:52Z")
I am trying to process multiline messages in log using input codec multiline. I am getting stuck and sometimes giving grokparse error. Any recommendations? --Log lines start -- 2016-06-24 12:21:38||INFO||\ **Author:** [@ThomasBorno](https://discuss.elastic.co/u/ThomasBorno)\ **Replies:** 3\ **Last updated:** [September 20, 2016, 7:44am UTC](https://discuss.elastic.co/t/errors-with-geo-point/60904 "2016-09-20T07:44:32Z") Hey, I'm new to this platform. Been using ELK since last week. I'm encountering several problems concerning geo localisations. I have a CSV which is made out of the following Name,Adress,Latitude,Longitude I have… --- ## [Logstash grok filter not working](https://discuss.elastic.co/t/logstash-grok-filter-not-working/60914)
**Author:** [@Roberto\_Iglesias](https://discuss.elastic.co/u/Roberto_Iglesias)\ **Replies:** 5\ **Last updated:** [September 20, 2016, 6:29am UTC](https://discuss.elastic.co/t/logstash-grok-filter-not-working/60914 "2016-09-20T06:29:23Z")
Hi and thanks for reading. I'm trying to do my own postfix/imapd logs filter. This is what i made: %{SYSLOGTIMESTAMP:timestamp} %{WORD:mail} %{WORD:program}/?(%{WORD:subprogram})?\\\['?(%{BASE10NUM:id})?\\\]?:? %{GREEDYDAT… --- ## [Same pattern multiple times in a single logmessage](https://discuss.elastic.co/t/same-pattern-multiple-times-in-a-single-logmessage/60817)
**Author:** [@Mayank\_Agrawal](https://discuss.elastic.co/u/Mayank_Agrawal)\ **Replies:** 2\ **Last updated:** [September 20, 2016, 6:11am UTC](https://discuss.elastic.co/t/same-pattern-multiple-times-in-a-single-logmessage/60817 "2016-09-20T06:11:49Z")
One of the log messages is like this: 2016-07-20 09:01:39,708 28152772 INFO Received message key1=value1 key2=value2 key3=value3 Msg 1: word1 rslt=0x00 try=1 Msg 2: word2 rslt=0x01 try=4 Msg 3: word2 rslt=0x02 … --- ## [Parsing csv file which has some field/column containing separator as values](https://discuss.elastic.co/t/parsing-csv-file-which-has-some-field-column-containing-separator-as-values/60768)
**Author:** [@Immortal](https://discuss.elastic.co/u/Immortal)\ **Replies:** 3\ **Last updated:** [September 20, 2016, 6:02am UTC](https://discuss.elastic.co/t/parsing-csv-file-which-has-some-field-column-containing-separator-as-values/60768 "2016-09-20T06:02:59Z")
I have following type of entries in csv file: 16 , 5 , 53 , Logging/LoggingError , 2, error while opening file "a,b,c,d": , 10.10.4.219 , serviceA I am trying to parse it using csv pluging as follows: csv { co… --- ## [Streaming Media Log Analyze](https://discuss.elastic.co/t/streaming-media-log-analyze/60954)
**Author:** [@bv4wolf](https://discuss.elastic.co/u/bv4wolf)\ **Replies:** 2\ **Last updated:** [September 20, 2016, 1:43am UTC](https://discuss.elastic.co/t/streaming-media-log-analyze/60954 "2016-09-20T01:43:51Z")
Hi all, I am the new user with the ELK system. I've setup the ELK server and running OK. Now I want to analyze the streaming Media logs (Videos, Flash, etc...) is put the the client servers (Filebeats is installed … --- ## [Yet another @timestamp log file trouble... but it should be easy!](https://discuss.elastic.co/t/yet-another-timestamp-log-file-trouble-but-it-should-be-easy/561)
**Author:** [@alexolivan](https://discuss.elastic.co/u/alexolivan)\ **Replies:** 33\ **Last updated:** [September 19, 2016, 10:36pm UTC](https://discuss.elastic.co/t/yet-another-timestamp-log-file-trouble-but-it-should-be-easy/561 "2016-09-19T22:36:22Z")
Hi again forum! I never thoug it could be that hard... but I'm unable to properly handle log file acquired data... because a date problem. Google is full of entries around, but none solved my problem, and I'm affraid I… --- ## [Has anybody used date\_happened for the datadog logstash output plugin](https://discuss.elastic.co/t/has-anybody-used-date-happened-for-the-datadog-logstash-output-plugin/60949)
**Author:** [@iamthealex](https://discuss.elastic.co/u/iamthealex)\ **Replies:** 0\ **Last updated:** [September 19, 2016, 9:53pm UTC](https://discuss.elastic.co/t/has-anybody-used-date-happened-for-the-datadog-logstash-output-plugin/60949 "2016-09-19T21:53:55Z")
I am unable to find an example config. The documentation says string. I tried date\_happened =\> "1474319717" that didn't work I turned up verbosity on logstash I see that when I don't specify date\_happened in my filt… --- ## [How to make hash sum from string?](https://discuss.elastic.co/t/how-to-make-hash-sum-from-string/60897)
**Author:** [@KIVagant](https://discuss.elastic.co/u/KIVagant)\ **Replies:** 3\ **Last updated:** [September 19, 2016, 9:01pm UTC](https://discuss.elastic.co/t/how-to-make-hash-sum-from-string/60897 "2016-09-19T21:01:54Z")
Hello. If I have some string field, how I can make a hash sum from that in the Logstash configuration? Md5 or something else? [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1099) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1101)