# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=991 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 992 --- ## [Sap log forwarding through syslog](https://discuss.elastic.co/t/sap-log-forwarding-through-syslog/84277)
**Author:** [@ArjunSISA](https://discuss.elastic.co/u/ArjunSISA)\ **Replies:** 0\ **Last updated:** [May 2, 2017, 1:54pm UTC](https://discuss.elastic.co/t/sap-log-forwarding-through-syslog/84277 "2017-05-02T13:54:43Z")
hi all, can anyone help me in recommending a way to forward sap logs through rsyslog/filebeat to logstash. sample: 2AUK20170407000007134567899124DeztidiascECCSERVICE1 SAPMyuY1 … --- ## [How to push two inputs data as a single event(docment) in logstash.yml](https://discuss.elastic.co/t/how-to-push-two-inputs-data-as-a-single-event-docment-in-logstash-yml/84272)
**Author:** [@kabali12345](https://discuss.elastic.co/u/kabali12345)\ **Replies:** 0\ **Last updated:** [May 2, 2017, 1:26pm UTC](https://discuss.elastic.co/t/how-to-push-two-inputs-data-as-a-single-event-docment-in-logstash-yml/84272 "2017-05-02T13:26:21Z")
Hi.....all How can i push two inputs data as a single event(docment) in logstash.yml For example I have two indexes that must be separated: // index =order\_item\` { "ID": 1, "Name": "Shoes", "Price": 9.99, … --- ## [Stdin AND Windows](https://discuss.elastic.co/t/stdin-and-windows/84262)
**Author:** [@Mario\_van\_Gemert](https://discuss.elastic.co/u/Mario_van_Gemert)\ **Replies:** 2\ **Last updated:** [May 2, 2017, 1:18pm UTC](https://discuss.elastic.co/t/stdin-and-windows/84262 "2017-05-02T13:18:02Z")
In order to read my files I always use a path in my input section, like this: input { file { path =\> \["D:\\ELK-5.3.0\\logstash-5.3.0\\filename.csv"\] start\_position =\> "beginning" sincedb\_path =\> … --- ## [How to aggregate web access logs every X seconds?](https://discuss.elastic.co/t/how-to-aggregate-web-access-logs-every-x-seconds/84267)
**Author:** [@PauloAsos](https://discuss.elastic.co/u/PauloAsos)\ **Replies:** 0\ **Last updated:** [May 2, 2017, 1:13pm UTC](https://discuss.elastic.co/t/how-to-aggregate-web-access-logs-every-x-seconds/84267 "2017-05-02T13:13:52Z")
We're trying to aggregate Web Access Logs: \* HTTP Method; \* PATH; \* HTTP Status Code; Aggregating the logs in buckets of X seconds by Response Time (ex: Median, Mean, Percentiles, etc), To only send only 1 summary (aggre… --- ## [Logstash is neither uploading csv nor creating index in elastic search](https://discuss.elastic.co/t/logstash-is-neither-uploading-csv-nor-creating-index-in-elastic-search/83870)
**Author:** [@kartikeya\_sharma\_13](https://discuss.elastic.co/u/kartikeya_sharma_13)\ **Replies:** 2\ **Last updated:** [May 2, 2017, 12:58pm UTC](https://discuss.elastic.co/t/logstash-is-neither-uploading-csv-nor-creating-index-in-elastic-search/83870 "2017-05-02T12:58:43Z")
Hi All, I am new to ELK stack, i read multiple articles about the same issue , but none of them were able to solve my problem . This is the logging which i am getting after starting the logstash. Now i have a csv file … --- ## [Email Output Plugin Issues variables not working](https://discuss.elastic.co/t/email-output-plugin-issues-variables-not-working/84261)
**Author:** [@Harneet463](https://discuss.elastic.co/u/Harneet463)\ **Replies:** 0\ **Last updated:** [May 2, 2017, 12:53pm UTC](https://discuss.elastic.co/t/email-output-plugin-issues-variables-not-working/84261 "2017-05-02T12:53:30Z")
input { file { type =\> "log" path =\> "z:\\Server.log" start\_position =\> "beginning" } file { type =\> "messagepublisher" path =\> "z:\\Server1.log" start\_position =\> "beginning" } jmx { … --- ## [Only the first pattern in the message field is worling, others are ignored](https://discuss.elastic.co/t/only-the-first-pattern-in-the-message-field-is-worling-others-are-ignored/84253)
**Author:** [@Darya\_Semenova](https://discuss.elastic.co/u/Darya_Semenova)\ **Replies:** 1\ **Last updated:** [May 2, 2017, 12:16pm UTC](https://discuss.elastic.co/t/only-the-first-pattern-in-the-message-field-is-worling-others-are-ignored/84253 "2017-05-02T12:16:26Z")
For some reason for the code like this: filter { if \[type\] == "syslog" and !("parsed\_by\_slurm\_filter" in \[tags\]) and !("parsed\_by\_xcat\_filter" in \[tags\]) { grok { patterns\_dir =\> \["./patterns"\] … --- ## [Netscaler grok pattern](https://discuss.elastic.co/t/netscaler-grok-pattern/84254)
**Author:** [@ssi](https://discuss.elastic.co/u/ssi)\ **Replies:** 0\ **Last updated:** [May 2, 2017, 11:55am UTC](https://discuss.elastic.co/t/netscaler-grok-pattern/84254 "2017-05-02T11:55:30Z")
Hi i've googled the following grok pattern for this as follows if \[type\] == "Netscaler" { grok { match =\> \[ "message", "\<%{POSINT:syslog\_pri}\> %{DATE}:%{TIME} %{SYSLOGHOST:syslog\_ho… --- ## [Matching multiple Grok patterns in a single Logstash config file](https://discuss.elastic.co/t/matching-multiple-grok-patterns-in-a-single-logstash-config-file/84249)
**Author:** [@binky](https://discuss.elastic.co/u/binky)\ **Replies:** 1\ **Last updated:** [May 2, 2017, 11:44am UTC](https://discuss.elastic.co/t/matching-multiple-grok-patterns-in-a-single-logstash-config-file/84249 "2017-05-02T11:44:41Z")
Hi there. I'm trying to send a couple of files through Filebeat to a ELK stack. The thing is that the log syntax is not the same, so I came up with two different patterns that mach each of them separately. I've trie… --- ## [Using my log timestamp in logstash sample log (first-pipeline.conf)](https://discuss.elastic.co/t/using-my-log-timestamp-in-logstash-sample-log-first-pipeline-conf/83854)
**Author:** [@hasan\_sharekhan](https://discuss.elastic.co/u/hasan_sharekhan)\ **Replies:** 4\ **Last updated:** [May 2, 2017, 11:37am UTC](https://discuss.elastic.co/t/using-my-log-timestamp-in-logstash-sample-log-first-pipeline-conf/83854 "2017-05-02T11:37:28Z")
I have just successfully completed first example (first-pipeline.conf) from Logstash tutorial by uploading logs from FileBeat to Logstash to ElasticSearch In Kibana while creating new Index "logstash-\*" I am not gettin… --- ## [NoMethodError: undefined method \`include?' for #\](https://discuss.elastic.co/t/nomethoderror-undefined-method-include-for-bignum-0x452129a4/84242)
**Author:** [@jwcn](https://discuss.elastic.co/u/jwcn)\ **Replies:** 0\ **Last updated:** [May 2, 2017, 10:53am UTC](https://discuss.elastic.co/t/nomethoderror-undefined-method-include-for-bignum-0x452129a4/84242 "2017-05-02T10:53:12Z")
Version: 2.4.1 Operating System: ubuntu16.04 input: kafka{ zk\_connect =\> "10.19.176.68:2181" topic\_id =\> "lion\_request-log" group\_id =\> "logstash" codec =\> plain reset\_beginning =\> false consumer\_threads =\> 16 … --- ## [Defining a custom ES index based on an existing field](https://discuss.elastic.co/t/defining-a-custom-es-index-based-on-an-existing-field/84244)
**Author:** [@ticktockhouse](https://discuss.elastic.co/u/ticktockhouse)\ **Replies:** 3\ **Last updated:** [May 2, 2017, 11:08am UTC](https://discuss.elastic.co/t/defining-a-custom-es-index-based-on-an-existing-field/84244 "2017-05-02T11:08:53Z")
I have a logstash pipeline designed to break container logs coming from coreos into separate indexes, but it's not quite working: My pipeline looks like this: input { beats { port =\> 5044 } } filter { grok… --- ## [Custom patterns in logstash that make logstash stop compiling](https://discuss.elastic.co/t/custom-patterns-in-logstash-that-make-logstash-stop-compiling/84236)
**Author:** [@Darya\_Semenova](https://discuss.elastic.co/u/Darya_Semenova)\ **Replies:** 0\ **Last updated:** [May 2, 2017, 10:39am UTC](https://discuss.elastic.co/t/custom-patterns-in-logstash-that-make-logstash-stop-compiling/84236 "2017-05-02T10:39:46Z")
I'd written a regular expression for the particular date format in my log (created ./patterns directory and file custom\_time\_pattern in it): Date format: 2013-11-05T12:05:46.123456+03:00 CUSTOMTIMESTAMP (\[0-9\]{4,4}… --- ## [How to read bz2 file from S3?](https://discuss.elastic.co/t/how-to-read-bz2-file-from-s3/84238)
**Author:** [@weibin.wu](https://discuss.elastic.co/u/weibin.wu)\ **Replies:** 1\ **Last updated:** [May 2, 2017, 10:45am UTC](https://discuss.elastic.co/t/how-to-read-bz2-file-from-s3/84238 "2017-05-02T10:45:40Z")
Hi Logstash: Can I know how can I read bz2 file from AWS S3 using logstash? --- ## [Log line missing](https://discuss.elastic.co/t/log-line-missing/84234)
**Author:** [@bouboune](https://discuss.elastic.co/u/bouboune)\ **Replies:** 0\ **Last updated:** [May 2, 2017, 10:08am UTC](https://discuss.elastic.co/t/log-line-missing/84234 "2017-05-02T10:08:41Z")
Hello, I'm using log4j(SocketAppender) to send the logs of my applications to logstash/elasticsearch. The problem is that some log line are missing, and in kibana the log line are not in the good order. I tried to use … --- ## [Already programmed patterns in regular expressions in logstash](https://discuss.elastic.co/t/already-programmed-patterns-in-regular-expressions-in-logstash/84223)
**Author:** [@Darya\_Semenova](https://discuss.elastic.co/u/Darya_Semenova)\ **Replies:** 2\ **Last updated:** [May 2, 2017, 9:42am UTC](https://discuss.elastic.co/t/already-programmed-patterns-in-regular-expressions-in-logstash/84223 "2017-05-02T09:42:10Z")
Hello, I'd like to know if it is possible to insert patterns that were previously written in new regular expressions. For example, will the code like that be possible when implementing my own patterns? EXPBEGIN ((\[A-… --- ## [Logstash pipeline workers instance variables](https://discuss.elastic.co/t/logstash-pipeline-workers-instance-variables/84231)
**Author:** [@Conall\_Prendergast](https://discuss.elastic.co/u/Conall_Prendergast)\ **Replies:** 0\ **Last updated:** [May 2, 2017, 9:39am UTC](https://discuss.elastic.co/t/logstash-pipeline-workers-instance-variables/84231 "2017-05-02T09:39:04Z")
Hi, I am writing a filter plugin that caches some information from events and may use that to populate other events. Something like marking a tcp handshake packets with details of the handshake (whether or not the hand… --- ## [Logstash environment dynamic varibale](https://discuss.elastic.co/t/logstash-environment-dynamic-varibale/84226)
**Author:** [@deepak\_purwar](https://discuss.elastic.co/u/deepak_purwar)\ **Replies:** 0\ **Last updated:** [May 2, 2017, 9:01am UTC](https://discuss.elastic.co/t/logstash-environment-dynamic-varibale/84226 "2017-05-02T09:01:41Z")
Heading Heading --- ## [Sequel::DatabaseConnectionError: Java::JavaSql::SQLException: Access denied for user ''@'localhost' (using password: NO)](https://discuss.elastic.co/t/sequel-databaseconnectionerror-java-javasql-sqlexception-access-denied-for-user-localhost-using-password-no/83713)
**Author:** [@shadowmaster63](https://discuss.elastic.co/u/shadowmaster63)\ **Replies:** 7\ **Last updated:** [May 2, 2017, 8:46am UTC](https://discuss.elastic.co/t/sequel-databaseconnectionerror-java-javasql-sqlexception-access-denied-for-user-localhost-using-password-no/83713 "2017-05-02T08:46:44Z")
If jdbc\_password\_filepath replace jdbc\_password no problem. logstash 5.3.1 Ubuntu 14.04.5 LTS \[2017-04-26T16:44:34,676\]\[INFO \]\[logstash.outputs.elasticsearch\] Running health check to see if an Elasticsearch connecti… --- ## [Ingesting table-like data](https://discuss.elastic.co/t/ingesting-table-like-data/83771)
**Author:** [@WiemDW](https://discuss.elastic.co/u/WiemDW)\ **Replies:** 5\ **Last updated:** [May 2, 2017, 8:38am UTC](https://discuss.elastic.co/t/ingesting-table-like-data/83771 "2017-05-02T08:38:44Z")
Hi, I am trying to ingest the following file format: \ =================== X 1 2 Y 1 3 And would like the following document format in Elasticsearch 5.x {'item': 'X', 'value': '1'} {'item… --- ## [Max Lines Multiline Codec](https://discuss.elastic.co/t/max-lines-multiline-codec/84216)
**Author:** [@Sam67000](https://discuss.elastic.co/u/Sam67000)\ **Replies:** 1\ **Last updated:** [May 2, 2017, 8:38am UTC](https://discuss.elastic.co/t/max-lines-multiline-codec/84216 "2017-05-02T08:38:39Z")
Hello EveryOne, I have a problem with one log. This log contains 600 lines and I need it to be store in ElasticSearch as a single document. So I use the multiline codec but it puts me : "Multiline codec max lines reac… --- ## [Unable to query .keyword fields from an index created using the default logstash template](https://discuss.elastic.co/t/unable-to-query-keyword-fields-from-an-index-created-using-the-default-logstash-template/84185)
**Author:** [@GuillaumeN](https://discuss.elastic.co/u/GuillaumeN)\ **Replies:** 9\ **Last updated:** [May 2, 2017, 5:10am UTC](https://discuss.elastic.co/t/unable-to-query-keyword-fields-from-an-index-created-using-the-default-logstash-template/84185 "2017-05-02T05:10:29Z")
Hi all, Using an ELK v5.3 stack, I'm trying to load and search syslogs from archived files. I'm using a simple logstash mapping file and my logs all to to a logstash-YYY.MM.DD index. This index uses the default elastic… --- ## [Converting string to date - getting parse error](https://discuss.elastic.co/t/converting-string-to-date-getting-parse-error/84073)
**Author:** [@sraghu1980](https://discuss.elastic.co/u/sraghu1980)\ **Replies:** 3\ **Last updated:** [May 2, 2017, 3:56am UTC](https://discuss.elastic.co/t/converting-string-to-date-getting-parse-error/84073 "2017-05-02T03:56:49Z")
Hi, I would like to convert date in string to date in integer. In below example I would like format of "TS" to be similar to that of @timestamp { "type" =\> "perf", ............. "@timestamp" =\> 2017-… --- ## [Host unreachable](https://discuss.elastic.co/t/host-unreachable/83488)
**Author:** [@Raja1](https://discuss.elastic.co/u/Raja1)\ **Replies:** 4\ **Last updated:** [May 2, 2017, 2:52am UTC](https://discuss.elastic.co/t/host-unreachable/83488 "2017-05-02T02:52:22Z")
Hi All, I am keep on getting below error after installing x-pack. Please find my configuration below and also i have disabled the sniffing. I am not understansing how its taking localhost. Please guide me to resolve… --- ## [SQS Input Not working - SOLVED](https://discuss.elastic.co/t/sqs-input-not-working-solved/82458)
**Author:** [@Wayne\_Taylor](https://discuss.elastic.co/u/Wayne_Taylor)\ **Replies:** 1\ **Last updated:** [May 1, 2017, 5:45pm UTC](https://discuss.elastic.co/t/sqs-input-not-working-solved/82458 "2017-05-01T17:45:26Z")
Hi, I have a very simple conf right now. Logstash 5.3.0. input{ sqs{ queue =\> "somename" } } If i run I get the following error: \[2017-04-14T20:08:55,084\]\[INFO \]\[logstash.pipeline \] Starting pipeline {"id"=\>"… --- ## [Run not\_analyzed querries on index populated with Logstash](https://discuss.elastic.co/t/run-not-analyzed-querries-on-index-populated-with-logstash/84106)
**Author:** [@GuillaumeN](https://discuss.elastic.co/u/GuillaumeN)\ **Replies:** 2\ **Last updated:** [May 1, 2017, 9:21pm UTC](https://discuss.elastic.co/t/run-not-analyzed-querries-on-index-populated-with-logstash/84106 "2017-05-01T21:21:52Z")
Hi, I'm a newbie with the ELK stack and although I've extensively searched Google for this, I just can't find the solution to my problem. I'm running ELK 5.3 on RHEL 7.3. I need to use the ELK stack to search in rece… --- ## [Elb logs grok inside grok](https://discuss.elastic.co/t/elb-logs-grok-inside-grok/82615)
**Author:** [@bob-bza](https://discuss.elastic.co/u/bob-bza)\ **Replies:** 5\ **Last updated:** [May 1, 2017, 9:06pm UTC](https://discuss.elastic.co/t/elb-logs-grok-inside-grok/82615 "2017-05-01T21:06:07Z")
This was my filter looks like for ELB log. filter { grok { \*\* match =\> \[ "message", "%{TIMESTAMP\_ISO8601:timestamp} %{NOTSPACE:elb\_name} %{IP:elb\_client\_ip}:%{INT:elb\_client\_port:int} (?:%{IP:elb\_backend\_ip}:%{NUMBER:e… --- ## [Logstash losing ES connection periodically](https://discuss.elastic.co/t/logstash-losing-es-connection-periodically/84045)
**Author:** [@rayl7786](https://discuss.elastic.co/u/rayl7786)\ **Replies:** 3\ **Last updated:** [May 1, 2017, 1:37pm UTC](https://discuss.elastic.co/t/logstash-losing-es-connection-periodically/84045 "2017-05-01T13:37:18Z")
I have been trying to narrow down what could be causing my Logstash to periodically not get a connection to my elasticsearch (Separate Machines). I keep on getting the following but there is no set time between the error… --- ## [Json filter to parse nested json](https://discuss.elastic.co/t/json-filter-to-parse-nested-json/83755)
**Author:** [@harshinim](https://discuss.elastic.co/u/harshinim)\ **Replies:** 3\ **Last updated:** [April 30, 2017, 10:02pm UTC](https://discuss.elastic.co/t/json-filter-to-parse-nested-json/83755 "2017-04-30T22:02:37Z")
I have the below json {"test1":1,"test2":2,"usage":{"test3":3,"test4":4}} and I want the output to be test1:1, test2:2, usage.test3:3, usage.test4:4 Also the filter needs to be dynamic, I don't know the name of the… --- ## [\[Ingest Node\] Unrecognized character escape ';' (code 59)](https://discuss.elastic.co/t/ingest-node-unrecognized-character-escape-code-59/84098)
**Author:** [@joaoantoniopereira](https://discuss.elastic.co/u/joaoantoniopereira)\ **Replies:** 3\ **Last updated:** [April 29, 2017, 11:31pm UTC](https://discuss.elastic.co/t/ingest-node-unrecognized-character-escape-code-59/84098 "2017-04-29T23:31:44Z")
Hi, I am trying to use ingest node to parse lines from filebeat. I used grok debugger to get to the pattern. My json file is like this: { "description": "Test pipeline", "processors": \[ { … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=990) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=992)