# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1191 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1192 --- ## [Problems with logstash doc example](https://discuss.elastic.co/t/problems-with-logstash-doc-example/41720)
**Author:** [@guojingjing](https://discuss.elastic.co/u/guojingjing)\ **Replies:** 2\ **Last updated:** [February 15, 2016, 9:49am UTC](https://discuss.elastic.co/t/problems-with-logstash-doc-example/41720 "2016-02-15T09:49:25Z")
I'm an absolute beginner of Logstash. I ran the second example with logstash-apache.conf on https://www.elastic.co/guide/en/logstash/current/advanced-pipeline.html. It seems that my file was found (and possibly opened… --- ## [Delete older documents](https://discuss.elastic.co/t/delete-older-documents/41754)
**Author:** [@Croos](https://discuss.elastic.co/u/Croos)\ **Replies:** 1\ **Last updated:** [February 15, 2016, 9:45am UTC](https://discuss.elastic.co/t/delete-older-documents/41754 "2016-02-15T09:45:40Z")
It is possible to delete existing older document from elasticsearch by schedule using logstash? My current config file is following, input { jdbc { jdbc\_driver\_library =\> "mysql-connector-java-5.1.38-bin.jar" … --- ## [How to define specific mapping in logstash to parse data in certain format](https://discuss.elastic.co/t/how-to-define-specific-mapping-in-logstash-to-parse-data-in-certain-format/41736)
**Author:** [@Lovika\_Jain](https://discuss.elastic.co/u/Lovika_Jain)\ **Replies:** 1\ **Last updated:** [February 15, 2016, 6:36am UTC](https://discuss.elastic.co/t/how-to-define-specific-mapping-in-logstash-to-parse-data-in-certain-format/41736 "2016-02-15T06:36:24Z")
Hi, I need to parse my data like root:\[group:\[instance:\[hourly:{},monthly:{},daily:{}\]\]\] how to define mappings to get data like this? --- ## [Logstash not reading file in windows](https://discuss.elastic.co/t/logstash-not-reading-file-in-windows/41723)
**Author:** [@Smitha\_Bipin](https://discuss.elastic.co/u/Smitha_Bipin)\ **Replies:** 2\ **Last updated:** [February 14, 2016, 10:58pm UTC](https://discuss.elastic.co/t/logstash-not-reading-file-in-windows/41723 "2016-02-14T22:58:08Z")
Hi , I am a newbie to logstash and configured the logstash.conf as follows: input { file { path =\> "C:\\log4j\\log.out" start\_position =\> "beginning" } } filter { grok { match =\> { "message" =\> "%{COMBINEDA… --- ## [Multithreading management in multiline codec](https://discuss.elastic.co/t/multithreading-management-in-multiline-codec/41410)
**Author:** [@michelecappelletti](https://discuss.elastic.co/u/michelecappelletti)\ **Replies:** 1\ **Last updated:** [February 14, 2016, 4:18pm UTC](https://discuss.elastic.co/t/multithreading-management-in-multiline-codec/41410 "2016-02-14T16:18:34Z")
Hi, i've just moved multiline management from filter layer to input layer. In development enviroment i've no forwarders, the pipeline directly get from local file input. it's not clear from LS2.2 changelog if multilin… --- ## [Exception in thread "main" org.jruby.exceptions.RaiseException](https://discuss.elastic.co/t/exception-in-thread-main-org-jruby-exceptions-raiseexception/41475)
**Author:** [@Shrawan\_Bhagwat](https://discuss.elastic.co/u/Shrawan_Bhagwat)\ **Replies:** 1\ **Last updated:** [February 14, 2016, 4:14pm UTC](https://discuss.elastic.co/t/exception-in-thread-main-org-jruby-exceptions-raiseexception/41475 "2016-02-14T16:14:20Z")
Hi All, I am trying to logstash-1.1.10 using ./logstashctrl start command. I am getting the below error: Exception in thread "main" org.jruby.exceptions.RaiseException: (LoadError) Could not load FFI Provider: (Not… --- ## [Adding dynamic top keys to existing json](https://discuss.elastic.co/t/adding-dynamic-top-keys-to-existing-json/41479)
**Author:** [@bongo\_zg](https://discuss.elastic.co/u/bongo_zg)\ **Replies:** 1\ **Last updated:** [February 14, 2016, 4:12pm UTC](https://discuss.elastic.co/t/adding-dynamic-top-keys-to-existing-json/41479 "2016-02-14T16:12:22Z")
I am manually generating json file that looks typical like { "file1": "value1", "file2", "value2" } but, I would like to add dynamic top keys like \_id and \_index, but instead of writing them manually, I would like t… --- ## [Mysql-Slow-Log](https://discuss.elastic.co/t/mysql-slow-log/41480)
**Author:** [@novice](https://discuss.elastic.co/u/novice)\ **Replies:** 1\ **Last updated:** [February 14, 2016, 2:47pm UTC](https://discuss.elastic.co/t/mysql-slow-log/41480 "2016-02-14T14:47:26Z")
Hi everyone, I am using filebeats to forward mysql logs to logstash , and it breaks the query to different lines. I want the complete query as a single entry unlike a single line the way logstash does. What filter shou… --- ## [Help with regex/grok](https://discuss.elastic.co/t/help-with-regex-grok/41529)
**Author:** [@l1carter](https://discuss.elastic.co/u/l1carter)\ **Replies:** 1\ **Last updated:** [February 14, 2016, 2:45pm UTC](https://discuss.elastic.co/t/help-with-regex-grok/41529 "2016-02-14T14:45:19Z")
sample line: SOMETEXTBEFORE... UserName=jsmith, CmdSet=\[ CmdAV=show CmdArgAV=running-config CmdArgAV=interfaces CmdArgAV=Port-Channel1 CmdArgAV= \], Protocol=Tacacs, MatchedCommandSet=AllowAllCommands, SOMETEXTAFTER... w… --- ## [Logstash JDBC plugin question](https://discuss.elastic.co/t/logstash-jdbc-plugin-question/41586)
**Author:** [@jaminvp](https://discuss.elastic.co/u/jaminvp)\ **Replies:** 1\ **Last updated:** [February 14, 2016, 12:01pm UTC](https://discuss.elastic.co/t/logstash-jdbc-plugin-question/41586 "2016-02-14T12:01:34Z")
Greetings, Presume I have a database table containing status info about processing jobs. This table would have the following columns with an example rule: JobID - Jobname - RunDate - Status - StartTimestamp - EndTi… --- ## [Logstash 2.2.0 File Input ignores sincedb\_path option when running as a service](https://discuss.elastic.co/t/logstash-2-2-0-file-input-ignores-sincedb-path-option-when-running-as-a-service/41516)
**Author:** [@Eric\_Howard](https://discuss.elastic.co/u/Eric_Howard)\ **Replies:** 2\ **Last updated:** [February 14, 2016, 11:30am UTC](https://discuss.elastic.co/t/logstash-2-2-0-file-input-ignores-sincedb-path-option-when-running-as-a-service/41516 "2016-02-14T11:30:53Z")
This is with Logstash 2.2.0 I have the following input in my configuration file: input { file { path =\> /opt/logstash/file.txt start\_position =\> "beginning" sincedb\_path =\> "/opt/logstash/sinced… --- ## [Logstash / Filewatch double initialization](https://discuss.elastic.co/t/logstash-filewatch-double-initialization/41647)
**Author:** [@bdanziger](https://discuss.elastic.co/u/bdanziger)\ **Replies:** 1\ **Last updated:** [February 13, 2016, 6:39pm UTC](https://discuss.elastic.co/t/logstash-filewatch-double-initialization/41647 "2016-02-13T18:39:39Z")
I contributed to filewatch in the past, and was retesting Logstash 2.2.0 to see that the bug fixes were really working properly since they were integrated differently than I originally intended. So, the question. I am … --- ## [Sequel\_opts to pass securityMechanism as a parmeter to jdbc](https://discuss.elastic.co/t/sequel-opts-to-pass-securitymechanism-as-a-parmeter-to-jdbc/41539)
**Author:** [@kirankumarchava](https://discuss.elastic.co/u/kirankumarchava)\ **Replies:** 6\ **Last updated:** [February 13, 2016, 5:41pm UTC](https://discuss.elastic.co/t/sequel-opts-to-pass-securitymechanism-as-a-parmeter-to-jdbc/41539 "2016-02-13T17:41:18Z")
how can we pass the property securityMechanism using the sequel\_opts in logstash configuration. I tried sequel\_opts =\> {"securityMechanism" =\> "7"} but no use. Equivalent to pOpts.put("securityMechanism", sMech); Co… --- ## [GeoIP - can't get it working](https://discuss.elastic.co/t/geoip-cant-get-it-working/41635)
**Author:** [@bluethundr](https://discuss.elastic.co/u/bluethundr)\ **Replies:** 2\ **Last updated:** [February 13, 2016, 4:09pm UTC](https://discuss.elastic.co/t/geoip-cant-get-it-working/41635 "2016-02-13T16:09:22Z")
Hi all, I've followed this tutorial on how to add GeoIP to LS: DO GeoIP I've setup my apache logs to output JSON to Logstash. It's working pretty well. And since I added the json codec to the beats plugin, the lo… --- ## [Logstash output to EC2](https://discuss.elastic.co/t/logstash-output-to-ec2/41628)
**Author:** [@Amerikiwi](https://discuss.elastic.co/u/Amerikiwi)\ **Replies:** 1\ **Last updated:** [February 13, 2016, 11:28am UTC](https://discuss.elastic.co/t/logstash-output-to-ec2/41628 "2016-02-13T11:28:39Z")
How to you set up Logstash to output to an ec2 instance in AWS? I want to use Log stash to grab input log files from S3 bucket and send the output cvs files to an eC2 instance via Logstash. Basically use Logstash as a da… --- ## [What is best way to send logstash data to greylog?](https://discuss.elastic.co/t/what-is-best-way-to-send-logstash-data-to-greylog/41661)
**Author:** [@JoeKing](https://discuss.elastic.co/u/JoeKing)\ **Replies:** 1\ **Last updated:** [February 12, 2016, 10:30pm UTC](https://discuss.elastic.co/t/what-is-best-way-to-send-logstash-data-to-greylog/41661 "2016-02-12T22:30:40Z")
I am currently sending log data from Logstash to Greylog via Gelf UDP. Is there a better way to do this? Does Logstash have a way to send it via TCP in Gelf format? --- ## [Logstash Hive or HDFS Input](https://discuss.elastic.co/t/logstash-hive-or-hdfs-input/41666)
**Author:** [@Omer\_Uludag](https://discuss.elastic.co/u/Omer_Uludag)\ **Replies:** 0\ **Last updated:** [February 12, 2016, 10:22pm UTC](https://discuss.elastic.co/t/logstash-hive-or-hdfs-input/41666 "2016-02-12T22:22:20Z")
Hello together, is it possible with Logstash to get data from Hive or HDFS? I know that Elasticsearch can be directly integrated with Hadoop, but I am in this use case interested to use also Logstash. --- ## [Inserting Quotes with mutate, gsub](https://discuss.elastic.co/t/inserting-quotes-with-mutate-gsub/41630)
**Author:** [@maxwell](https://discuss.elastic.co/u/maxwell)\ **Replies:** 1\ **Last updated:** [February 12, 2016, 7:54pm UTC](https://discuss.elastic.co/t/inserting-quotes-with-mutate-gsub/41630 "2016-02-12T19:54:06Z")
I have a json document which is failing json parse because some of the fields contain quotes, messing up the json. My solution is to remove all of the quotes, and then add them back in myself around relevant fields. How… --- ## [Redmine output plugin](https://discuss.elastic.co/t/redmine-output-plugin/41409)
**Author:** [@tgdesrochers](https://discuss.elastic.co/u/tgdesrochers)\ **Replies:** 2\ **Last updated:** [February 10, 2016, 7:19pm UTC](https://discuss.elastic.co/t/redmine-output-plugin/41409 "2016-02-10T19:19:48Z")
I am trying to use the redmine output plugin to write tickets to my redmine database. my config works but my question is how to make the description field more robust. I am trying to use description =\> "%{message}" so… --- ## [Failed Parsing Date and MulitLine tomcat log](https://discuss.elastic.co/t/failed-parsing-date-and-mulitline-tomcat-log/41537)
**Author:** [@bilguuna](https://discuss.elastic.co/u/bilguuna)\ **Replies:** 5\ **Last updated:** [February 12, 2016, 4:01pm UTC](https://discuss.elastic.co/t/failed-parsing-date-and-mulitline-tomcat-log/41537 "2016-02-12T16:01:49Z")
Hello, I've been trying to read tomcat log and make some analysis such as number of errors and exceptions raised etc. Here is my conf: input { file { path =\> "/opt/logstash/localhost.log" start\_po… --- ## [Logstash 2.2.0 \> elasticsearch output connectivity issue](https://discuss.elastic.co/t/logstash-2-2-0-elasticsearch-output-connectivity-issue/41534)
**Author:** [@Sijo\_C](https://discuss.elastic.co/u/Sijo_C)\ **Replies:** 4\ **Last updated:** [February 12, 2016, 3:53pm UTC](https://discuss.elastic.co/t/logstash-2-2-0-elasticsearch-output-connectivity-issue/41534 "2016-02-12T15:53:50Z")
I have filebeat,logstash 2.2.0, ES 2.1.0 working in dev. Strange connectivity issue when moving to prod: ES cluster behind an elb (aws vpc), no TLS "curl prod-elb-xxx:9200" works from my logstash box (in datacenter)… --- ## [Dynamic attachments to email output](https://discuss.elastic.co/t/dynamic-attachments-to-email-output/41559)
**Author:** [@Ssawa](https://discuss.elastic.co/u/Ssawa)\ **Replies:** 2\ **Last updated:** [February 12, 2016, 3:49pm UTC](https://discuss.elastic.co/t/dynamic-attachments-to-email-output/41559 "2016-02-12T15:49:47Z")
Hey folks, I'm trying to use the email output to send an alert including a file attachment using something like the following: email { | … --- ## [Need help for regular expresion](https://discuss.elastic.co/t/need-help-for-regular-expresion/38636)
**Author:** [@jayaram](https://discuss.elastic.co/u/jayaram)\ **Replies:** 3\ **Last updated:** [February 12, 2016, 12:37pm UTC](https://discuss.elastic.co/t/need-help-for-regular-expresion/38636 "2016-02-12T12:37:00Z")
I have the following either of the Pattern in my log messages ABC\[\] ABC\[User(UserID)\] ABC\[User(UserID) Sponser(SponserName)\] UserID and SponserName are Alphanumeric, Ex: \[User(ABCDE12345) Sponser(Test Sponser)\] Ple… --- ## [Logstash test with rspec](https://discuss.elastic.co/t/logstash-test-with-rspec/41478)
**Author:** [@Sabbasth](https://discuss.elastic.co/u/Sabbasth)\ **Replies:** 2\ **Last updated:** [February 12, 2016, 11:07am UTC](https://discuss.elastic.co/t/logstash-test-with-rspec/41478 "2016-02-12T11:07:43Z")
Hi, I have issues getting fields set during filtering with rspec. I created a post on stackoverflow: Any clue on this strange behavior ? --- ## [Logstash + file\_concat Puppet 4.2 Compatibility](https://discuss.elastic.co/t/logstash-file-concat-puppet-4-2-compatibility/41602)
**Author:** [@CameronMcAuley](https://discuss.elastic.co/u/CameronMcAuley)\ **Replies:** 0\ **Last updated:** [February 12, 2016, 10:51am UTC](https://discuss.elastic.co/t/logstash-file-concat-puppet-4-2-compatibility/41602 "2016-02-12T10:51:57Z")
Hi there, Just wondering if anyone has been able to get the elasticsearch/logstash module working on Puppet version 4.2.1. I noticed that one of the dependancies for elasticsearch/logstash is theispavailability/file\_co… --- ## [Logstash how to store partial info from the log](https://discuss.elastic.co/t/logstash-how-to-store-partial-info-from-the-log/41495)
**Author:** [@Navneet\_Mathpal](https://discuss.elastic.co/u/Navneet_Mathpal)\ **Replies:** 1\ **Last updated:** [February 12, 2016, 10:23am UTC](https://discuss.elastic.co/t/logstash-how-to-store-partial-info-from-the-log/41495 "2016-02-12T10:23:34Z")
Hi, I came across a use case where I need to store some value from the log in two different places ex - parsed log is :- {"message" : "log\_message" , "source" : "internet" , "time" : "sometime" , "id" :"123" } N… --- ## [Error in logstash config file after adding gelf](https://discuss.elastic.co/t/error-in-logstash-config-file-after-adding-gelf/41566)
**Author:** [@JoeKing](https://discuss.elastic.co/u/JoeKing)\ **Replies:** 1\ **Last updated:** [February 12, 2016, 8:29am UTC](https://discuss.elastic.co/t/error-in-logstash-config-file-after-adding-gelf/41566 "2016-02-12T08:29:52Z")
I receive the following error after adding gelf in logstash config file. The gelf output plugin is listed as one of my plugins. Error: Expected one of #, input, filter, output at line 9, column 1 (byte 125) input { … --- ## [Additional // are seen in parsed message after logstash parsed syslog event](https://discuss.elastic.co/t/additional-are-seen-in-parsed-message-after-logstash-parsed-syslog-event/41541)
**Author:** [@gaurav1424](https://discuss.elastic.co/u/gaurav1424)\ **Replies:** 2\ **Last updated:** [February 12, 2016, 6:44am UTC](https://discuss.elastic.co/t/additional-are-seen-in-parsed-message-after-logstash-parsed-syslog-event/41541 "2016-02-12T06:44:40Z")
Hello All, This is my input string : SECU \\"OTN2.LAB\\":SHELF-1 001328 SHELF-1:16-02-05,00-15-00 And output parsed message looked like this : "syslog\_message" =\> "SECU \\\\"OTN2.LAB\\\\":SHELF-1 001328 Here is my filt… --- ## [Defining grok patterns](https://discuss.elastic.co/t/defining-grok-patterns/40700)
**Author:** [@Anmol\_Garg](https://discuss.elastic.co/u/Anmol_Garg)\ **Replies:** 21\ **Last updated:** [February 12, 2016, 6:30am UTC](https://discuss.elastic.co/t/defining-grok-patterns/40700 "2016-02-12T06:30:44Z")
In logstash, certain patterns are already defined like "COMBINEDAPACHELOG" etc. But what if I want to use my own regular expression? Where do I define it? And how do I use it? --- ## [Logstash pipeline example apache.log](https://discuss.elastic.co/t/logstash-pipeline-example-apache-log/41236)
**Author:** [@w0lverine](https://discuss.elastic.co/u/w0lverine)\ **Replies:** 4\ **Last updated:** [February 11, 2016, 11:49pm UTC](https://discuss.elastic.co/t/logstash-pipeline-example-apache-log/41236 "2016-02-11T23:49:42Z")
I am doing my first example of logstash located in the Documents and I am not getting a proper curl output from querying "200" from the apache log tutorial.log file I have configured my config file correctly(tripled che… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1190) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1192)