# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=971 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 972 --- ## [Logstash multiple indexing](https://discuss.elastic.co/t/logstash-multiple-indexing/88201)
**Author:** [@mostafa](https://discuss.elastic.co/u/mostafa)\ **Replies:** 1\ **Last updated:** [June 4, 2017, 10:04pm UTC](https://discuss.elastic.co/t/logstash-multiple-indexing/88201 "2017-06-04T22:04:53Z")
Dear All, I am using logstash for logging my F5 web logs and working fine. I want to add cisco ASA logs to the same log server, I see both logs on kibana. Is there a way to have 2 index one for F5 and another for cis… --- ## [Grok match to last index unti begin of UserAgent](https://discuss.elastic.co/t/grok-match-to-last-index-unti-begin-of-useragent/88189)
**Author:** [@khaleid\_lund](https://discuss.elastic.co/u/khaleid_lund)\ **Replies:** 0\ **Last updated:** [June 4, 2017, 1:00pm UTC](https://discuss.elastic.co/t/grok-match-to-last-index-unti-begin-of-useragent/88189 "2017-06-04T13:00:08Z")
Hi all I have message "sid-cmascioieiow89322&New\*Sou,th%20Skvn%20and%20ir&o,n%20Age,Mozilla/5.0 (Linux; Android 6.0; CHM-U01 Build/HonorCHM-U01) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.98 Mobile Safari… --- ## [I want to match two grok patterns in one config file?](https://discuss.elastic.co/t/i-want-to-match-two-grok-patterns-in-one-config-file/87878)
**Author:** [@Yaswanth](https://discuss.elastic.co/u/Yaswanth)\ **Replies:** 5\ **Last updated:** [June 4, 2017, 9:07am UTC](https://discuss.elastic.co/t/i-want-to-match-two-grok-patterns-in-one-config-file/87878 "2017-06-04T09:07:59Z")
Hi, I have two types of error messages in the below format: \[2017-05-25 01:00:00,647\]\[ERROR\]\[marvel.agent.exporter.local\] local exporter \[default\_local\] - failed to delete indices RemoteTransportException\[\[data-0\]\[10… --- ## [Logstash output to kafka does not working?](https://discuss.elastic.co/t/logstash-output-to-kafka-does-not-working/88176)
**Author:** [@zhaochl](https://discuss.elastic.co/u/zhaochl)\ **Replies:** 4\ **Last updated:** [June 4, 2017, 8:54am UTC](https://discuss.elastic.co/t/logstash-output-to-kafka-does-not-working/88176 "2017-06-04T08:54:15Z")
as question is, who can give me a example, my config as follows: input { #stdin { } kafka { bootstrap\_servers =\> "bi1:9092,bi2:9092,bi3:9092" topics =\> "test" reset\_beginning=\>true … --- ## [Regarding debugging connections between FB and LS?](https://discuss.elastic.co/t/regarding-debugging-connections-between-fb-and-ls/87725)
**Author:** [@sravankumarch](https://discuss.elastic.co/u/sravankumarch)\ **Replies:** 1\ **Last updated:** [June 3, 2017, 3:04pm UTC](https://discuss.elastic.co/t/regarding-debugging-connections-between-fb-and-ls/87725 "2017-06-03T15:04:46Z")
Hi friends i am new to elk, i configured my elk on 1 aws instance every thing configured perfectly, and i installed filebeat on another aws instances, but i am not getting any logs to my elk server could any one please h… --- ## [De\_dot not working with certain inputs](https://discuss.elastic.co/t/de-dot-not-working-with-certain-inputs/87891)
**Author:** [@1stewart](https://discuss.elastic.co/u/1stewart)\ **Replies:** 5\ **Last updated:** [June 3, 2017, 2:56pm UTC](https://discuss.elastic.co/t/de-dot-not-working-with-certain-inputs/87891 "2017-06-03T14:56:58Z")
Hello, I'm trying to input json from azure via beats, but due to some of the field names containing schema addresses it's erroring due to them containing '.'. I can't exclude the lines as azlog is producing the logs as… --- ## [Grok not working](https://discuss.elastic.co/t/grok-not-working/87946)
**Author:** [@Nikhilpawar1985](https://discuss.elastic.co/u/Nikhilpawar1985)\ **Replies:** 1\ **Last updated:** [June 2, 2017, 5:37pm UTC](https://discuss.elastic.co/t/grok-not-working/87946 "2017-06-02T17:37:30Z")
Hi , with below config input { beats { port =\> 5044 codec =\> multiline { pattern =\> "^%{TIMESTAMP\_ISO8601} " negate =\> true what =\> previous } } } filter { grok { match … --- ## [Is it possible to configure different tracking\_column for different jdbc queries in single logstash conf file](https://discuss.elastic.co/t/is-it-possible-to-configure-different-tracking-column-for-different-jdbc-queries-in-single-logstash-conf-file/87998)
**Author:** [@pks](https://discuss.elastic.co/u/pks)\ **Replies:** 2\ **Last updated:** [June 2, 2017, 5:15pm UTC](https://discuss.elastic.co/t/is-it-possible-to-configure-different-tracking-column-for-different-jdbc-queries-in-single-logstash-conf-file/87998 "2017-06-02T17:15:27Z")
I have a few queries running via logstash jdbc plugin with tracking\_column as a timestamp from db and using sql\_last\_value while querying. These are set to run every minute. This is working fine. use\_column\_value … --- ## [Multi line filter in logstash conf](https://discuss.elastic.co/t/multi-line-filter-in-logstash-conf/88035)
**Author:** [@Aylwinns\_Tan](https://discuss.elastic.co/u/Aylwinns_Tan)\ **Replies:** 2\ **Last updated:** [June 2, 2017, 5:03pm UTC](https://discuss.elastic.co/t/multi-line-filter-in-logstash-conf/88035 "2017-06-02T17:03:21Z")
Hi all, i'm trying to build a logstash.conf file for the following type of log files. Below is excerpt of an two events: ----Event 1---- Exception at 21 May 2017 (tab)Module: 6764 (tab)Level: Abort (tab)Code: 83hdjwu… --- ## [Logstash JDBC input connector](https://discuss.elastic.co/t/logstash-jdbc-input-connector/88117)
**Author:** [@asharma](https://discuss.elastic.co/u/asharma)\ **Replies:** 0\ **Last updated:** [June 2, 2017, 4:57pm UTC](https://discuss.elastic.co/t/logstash-jdbc-input-connector/88117 "2017-06-02T16:57:59Z")
I have multiple jdbc inputs in my logstash config file all of which use a jdbc connection to the same redshift database. whenever i trigger logstash, it fires all the input sql queries in parallel which is crashing my re… --- ## [Logstash mutate split](https://discuss.elastic.co/t/logstash-mutate-split/88097)
**Author:** [@Man-kyun\_Choi](https://discuss.elastic.co/u/Man-kyun_Choi)\ **Replies:** 1\ **Last updated:** [June 2, 2017, 4:32pm UTC](https://discuss.elastic.co/t/logstash-mutate-split/88097 "2017-06-02T16:32:10Z")
I installed ELK and try to configured logstash to parse our Web Application FIrewall security log. The WAF log is below May 31 13:47:11 2017 WAFK\_TAC (warning) log\_processor: \[WEBFRONT/0x00727001\] Violated Cross Site… --- ## [Unable to mutate values to integer\_range](https://discuss.elastic.co/t/unable-to-mutate-values-to-integer-range/86705)
**Author:** [@xanadsonf](https://discuss.elastic.co/u/xanadsonf)\ **Replies:** 3\ **Last updated:** [June 2, 2017, 4:00pm UTC](https://discuss.elastic.co/t/unable-to-mutate-values-to-integer-range/86705 "2017-06-02T16:00:13Z")
Hi all, I'm trying to "convert" two integer values to a range with mutate without success... Range is a recent functionality, I'm unable to find any working syntax in the documentation. values : 2 integers in \[eng\]\[r… --- ## [Logstash JDBC input plugin](https://discuss.elastic.co/t/logstash-jdbc-input-plugin/87994)
**Author:** [@asharma](https://discuss.elastic.co/u/asharma)\ **Replies:** 3\ **Last updated:** [June 2, 2017, 3:57pm UTC](https://discuss.elastic.co/t/logstash-jdbc-input-plugin/87994 "2017-06-02T15:57:45Z")
I am using the jdbc plugin to import data form AMAZON redshift to elasticsearch using logstash. I am processing incremental updates for a very big table which adds around 2 million rows every hour and has a timestamp a… --- ## [Logstash cannot find config file](https://discuss.elastic.co/t/logstash-cannot-find-config-file/45491)
**Author:** [@Shashi\_Ravula](https://discuss.elastic.co/u/Shashi_Ravula)\ **Replies:** 10\ **Last updated:** [June 2, 2017, 3:15pm UTC](https://discuss.elastic.co/t/logstash-cannot-find-config-file/45491 "2017-06-02T15:15:52Z")
Hi guys , I am trying to run logstash as a container on marathon framework (DCOS) taking input from Kafka . the marathon json file fetches the uri from gist which has the logstash.conf file . I can also see the conf f… --- ## [Ignore wrong date value](https://discuss.elastic.co/t/ignore-wrong-date-value/87896)
**Author:** [@mitabrev](https://discuss.elastic.co/u/mitabrev)\ **Replies:** 12\ **Last updated:** [June 2, 2017, 3:11pm UTC](https://discuss.elastic.co/t/ignore-wrong-date-value/87896 "2017-06-02T15:11:23Z")
Hi, Hope I'm in the corrent sections. If not, please forgive. My parsed log files contain some date fields, and in some cases, the filed is not filled with epoch time value but with an empty string or just an "0". l… --- ## [\[UPDATED\] Logstash output to elasticsearch in docker](https://discuss.elastic.co/t/updated-logstash-output-to-elasticsearch-in-docker/87789)
**Author:** [@Jaxon\_Kochel](https://discuss.elastic.co/u/Jaxon_Kochel)\ **Replies:** 1\ **Last updated:** [June 2, 2017, 2:32pm UTC](https://discuss.elastic.co/t/updated-logstash-output-to-elasticsearch-in-docker/87789 "2017-06-02T14:32:46Z")
I'm currently experiencing an issue getting my logstash output to get to elasticsearch inside docker containers I start elasticsearch like this: docker run -d --name elasticsearch -p 9200:9200 -p 9300:9300 docker.el… --- ## [Not able to configure logstash in windows getting jruby error](https://discuss.elastic.co/t/not-able-to-configure-logstash-in-windows-getting-jruby-error/88099)
**Author:** [@srivastavashubhi52](https://discuss.elastic.co/u/srivastavashubhi52)\ **Replies:** 0\ **Last updated:** [June 2, 2017, 2:15pm UTC](https://discuss.elastic.co/t/not-able-to-configure-logstash-in-windows-getting-jruby-error/88099 "2017-06-02T14:15:59Z")
Hi, I am not able to configure logstash in windows getting the below jruby error. Error: Could not find or load main class org.jruby.Main Please help me out. Actually I want to configure logstash with elastic search and… --- ## [Logstash goes down with ruby in filter](https://discuss.elastic.co/t/logstash-goes-down-with-ruby-in-filter/88073)
**Author:** [@abrejuin](https://discuss.elastic.co/u/abrejuin)\ **Replies:** 4\ **Last updated:** [June 2, 2017, 2:05pm UTC](https://discuss.elastic.co/t/logstash-goes-down-with-ruby-in-filter/88073 "2017-06-02T14:05:07Z")
Hello, I have a filter which were worked when I was in logstash version 2. Now Logstash is in version 5 and something goes wrong with a ruby part (I know there is a new behaviour since the v5). If I activate the part a… --- ## [Weird? dateparse fail , if corrected, messages vanish](https://discuss.elastic.co/t/weird-dateparse-fail-if-corrected-messages-vanish/88026)
**Author:** [@uschellh](https://discuss.elastic.co/u/uschellh)\ **Replies:** 0\ **Last updated:** [June 2, 2017, 7:57am UTC](https://discuss.elastic.co/t/weird-dateparse-fail-if-corrected-messages-vanish/88026 "2017-06-02T07:57:50Z")
Hello, turns out that as long there is a dateparse fail, kibana will show the logfile entry based on @timestamp. if there is no dateparse fail, kibana will show based on timestamp. timestamp was filled all the time. … --- ## [Is there any Logstash filter to convert a log-line to json](https://discuss.elastic.co/t/is-there-any-logstash-filter-to-convert-a-log-line-to-json/87922)
**Author:** [@Pranav1](https://discuss.elastic.co/u/Pranav1)\ **Replies:** 2\ **Last updated:** [June 2, 2017, 12:10pm UTC](https://discuss.elastic.co/t/is-there-any-logstash-filter-to-convert-a-log-line-to-json/87922 "2017-06-02T12:10:25Z")
I am getting logs having a log line which has key value pair separated by pipe. Example- "apple=1 | banana= 3 | mango=5" or "apple=1 | banana= 3 | mango=5 | tiger=7 | elephant=1" I have to convert this log into a… --- ## [Lostash grok parse failure](https://discuss.elastic.co/t/lostash-grok-parse-failure/87902)
**Author:** [@Rajat\_Singh](https://discuss.elastic.co/u/Rajat_Singh)\ **Replies:** 6\ **Last updated:** [June 2, 2017, 12:04pm UTC](https://discuss.elastic.co/t/lostash-grok-parse-failure/87902 "2017-06-02T12:04:37Z")
Hi folks, I had created a logstash pattern to parse the file, when i validating it grok debugger in first it is not working but when i am removing space and then again creating space, then it is working 10.150.86.92 - … --- ## [Undersized batches produced in ES output while persistent queue is enabled](https://discuss.elastic.co/t/undersized-batches-produced-in-es-output-while-persistent-queue-is-enabled/88076)
**Author:** [@shog](https://discuss.elastic.co/u/shog)\ **Replies:** 2\ **Last updated:** [June 2, 2017, 11:58am UTC](https://discuss.elastic.co/t/undersized-batches-produced-in-es-output-while-persistent-queue-is-enabled/88076 "2017-06-02T11:58:50Z")
I've described this problem in more details here https://github.com/elastic/logstash/issues/7243, but got no reply so far. TLDR problem is that; enabling persistent queue in logstash will result in undersized elasticse… --- ## [Error received using WebHDFS Logstash Output](https://discuss.elastic.co/t/error-received-using-webhdfs-logstash-output/38458)
**Author:** [@yachtsman60](https://discuss.elastic.co/u/yachtsman60)\ **Replies:** 6\ **Last updated:** [June 2, 2017, 11:58am UTC](https://discuss.elastic.co/t/error-received-using-webhdfs-logstash-output/38458 "2017-06-02T11:58:35Z")
Constantly receive the following error with no indication on how to resolve the issue and with a result that no events are actually pushed to my hdfs. Can someone please provide some guidance? Failed to flush outgoin… --- ## [How to convert existing timestamp format to another timestamp format using logstash?](https://discuss.elastic.co/t/how-to-convert-existing-timestamp-format-to-another-timestamp-format-using-logstash/87889)
**Author:** [@yash\_mangla](https://discuss.elastic.co/u/yash_mangla)\ **Replies:** 8\ **Last updated:** [June 2, 2017, 11:27am UTC](https://discuss.elastic.co/t/how-to-convert-existing-timestamp-format-to-another-timestamp-format-using-logstash/87889 "2017-06-02T11:27:49Z")
Hi Guyz, i have 2 different application logs. one is containing (application1) 16/02/2017 19:21:19:452 INFO - 9DDF7A1C53D518BC9C4F7DAE9045C70E:/RCOM\_PREPAID : \*\* Popped Stack Frame \[/RCOM\_PREPAID\] 16/02/2017 19:21:19… --- ## [Individual index for each csv](https://discuss.elastic.co/t/individual-index-for-each-csv/88071)
**Author:** [@Pokecallum](https://discuss.elastic.co/u/Pokecallum)\ **Replies:** 0\ **Last updated:** [June 2, 2017, 11:08am UTC](https://discuss.elastic.co/t/individual-index-for-each-csv/88071 "2017-06-02T11:08:26Z")
Here is my current logstash config, What I would like for each file is to have its own index starting with a string so for example string+Jan01 string+Jan02 so on Any help appreciated !! input { file { … --- ## [Dependable output in bulks to local and remote elasticsearch](https://discuss.elastic.co/t/dependable-output-in-bulks-to-local-and-remote-elasticsearch/88067)
**Author:** [@Ma.Na](https://discuss.elastic.co/u/Ma.Na)\ **Replies:** 0\ **Last updated:** [June 2, 2017, 10:48am UTC](https://discuss.elastic.co/t/dependable-output-in-bulks-to-local-and-remote-elasticsearch/88067 "2017-06-02T10:48:22Z")
I am running several instances of a local ELK Stack. Hence, logstash imports, filters and then outputs to a local elasticsearch: output { elasticsearch { hosts =\> \["localhost:9200"\] } Due to space constraints,… --- ## [Logstash not listening remote tcp port on Azure windows VM](https://discuss.elastic.co/t/logstash-not-listening-remote-tcp-port-on-azure-windows-vm/88062)
**Author:** [@Ravindra\_Barapatre](https://discuss.elastic.co/u/Ravindra_Barapatre)\ **Replies:** 0\ **Last updated:** [June 2, 2017, 10:17am UTC](https://discuss.elastic.co/t/logstash-not-listening-remote-tcp-port-on-azure-windows-vm/88062 "2017-06-02T10:17:33Z")
I have deployed following components on Azure VM. 1. logstash 5.4.0 2. ElasticSearch 5.4.0 3. kibana 5.4.0 It is running locally perfectly for TCP input plugin on VM. input { tcp { port =\> 52811 host =\> "0.0.0.0… --- ## [EET Time converted to UTC....... why?](https://discuss.elastic.co/t/eet-time-converted-to-utc-why/88021)
**Author:** [@sunilmchaudhari](https://discuss.elastic.co/u/sunilmchaudhari)\ **Replies:** 5\ **Last updated:** [June 2, 2017, 8:00am UTC](https://discuss.elastic.co/t/eet-time-converted-to-utc-why/88021 "2017-06-02T08:00:42Z")
Hi, I have below configuration to logstash indexer. grok { match =\> { "message" =\> "%{TIMESTAMP\_ISO8601:logTime}" } } log message: 2017-06-02 08:28:59,335 | INFO acknowledgeMessages Acknowl… --- ## [Can any one please let me know what is this error about](https://discuss.elastic.co/t/can-any-one-please-let-me-know-what-is-this-error-about/87926)
**Author:** [@sravankumarch](https://discuss.elastic.co/u/sravankumarch)\ **Replies:** 3\ **Last updated:** [June 2, 2017, 7:33am UTC](https://discuss.elastic.co/t/can-any-one-please-let-me-know-what-is-this-error-about/87926 "2017-06-02T07:33:21Z")
{:timestamp=\>"2017-06-01T10:18:40.210000+0000", :message=\>"Pipeline aborted due to error", :exception=\>#, :backtrace=\>\["/opt/logstash/vendo r/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/config/mixin.rb:87… --- ## [Error in template](https://discuss.elastic.co/t/error-in-template/87865)
**Author:** [@AadarshKumayan](https://discuss.elastic.co/u/AadarshKumayan)\ **Replies:** 13\ **Last updated:** [June 2, 2017, 7:10am UTC](https://discuss.elastic.co/t/error-in-template/87865 "2017-06-02T07:10:30Z")
Hi i have a problem , i am trying to run logsatsh it generaates the following error :message=\>"Error: Expected one of #, input, filter, output at line 34, column 1 (byte 855) after ", :level=\>:error} i have downloaded… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=970) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=972)