# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=697 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 698 --- ## [Logstash timestamp parse failed](https://discuss.elastic.co/t/logstash-timestamp-parse-failed/145713)
**Author:** [@Timofey\_Gusev](https://discuss.elastic.co/u/Timofey_Gusev)\ **Replies:** 1\ **Last updated:** [August 23, 2018, 6:21pm UTC](https://discuss.elastic.co/t/logstash-timestamp-parse-failed/145713 "2018-08-23T18:21:44Z")
I use logstash to store all logs from different nodes in one place. So, I use logback.xml on the node side and logstash udp listener on the other. In logback I use LogstashSocketAppender, so as I know I can't set up logs… --- ## [Logstash Pipeline Exception](https://discuss.elastic.co/t/logstash-pipeline-exception/145779)
**Author:** [@paul1243](https://discuss.elastic.co/u/paul1243)\ **Replies:** 0\ **Last updated:** [August 23, 2018, 5:05pm UTC](https://discuss.elastic.co/t/logstash-pipeline-exception/145779 "2018-08-23T17:05:31Z")
Hi- Am trying to trigger the .conf file on a ubuntu server to run as a background process, I see the below exception: \[ERROR\] 2018-08-23 18:50:19.264 \[main\] Logstash - java.lang.IllegalStateException: Logstash stopped… --- ## [Useragent filter plugin not working for subfields?](https://discuss.elastic.co/t/useragent-filter-plugin-not-working-for-subfields/145571)
**Author:** [@prprt](https://discuss.elastic.co/u/prprt)\ **Replies:** 7\ **Last updated:** [August 23, 2018, 2:34pm UTC](https://discuss.elastic.co/t/useragent-filter-plugin-not-working-for-subfields/145571 "2018-08-23T14:34:42Z")
I have the following Useragent filter plugin configuration in Logstash 6.3.2 which does work for my Apache logs, but not for Nginx or HAProxy logs. I have double checked that the referenced field names are correct. My A… --- ## [Logstash configuration : 6.3](https://discuss.elastic.co/t/logstash-configuration-6-3/145574)
**Author:** [@pavan1991](https://discuss.elastic.co/u/pavan1991)\ **Replies:** 10\ **Last updated:** [August 23, 2018, 2:21pm UTC](https://discuss.elastic.co/t/logstash-configuration-6-3/145574 "2018-08-23T14:21:28Z")
Hi Team, I am new to ELK stack and I am looking for a solution for the below problem. ELK Components : ElasticSearch Cluster ( 1 Master & 2 Data Nodes ) Kibana is Installed on same server as the master Logstash is a… --- ## [Logstash - Exchange 2013 Message Tracking](https://discuss.elastic.co/t/logstash-exchange-2013-message-tracking/145581)
**Author:** [@L33T](https://discuss.elastic.co/u/L33T)\ **Replies:** 1\ **Last updated:** [August 23, 2018, 12:39pm UTC](https://discuss.elastic.co/t/logstash-exchange-2013-message-tracking/145581 "2018-08-23T12:39:19Z")
Hi, I cannot get logstash to output exchange 2013 tracking logs properly. Can anyone help? Logstash Config: input { beats { port =\> 5044 type =\> "ex\_msg\_trk" } } filter { if \[type\] == "ex\_msg\_trk" { grok { m… --- ## [Another Aggregate filter issue](https://discuss.elastic.co/t/another-aggregate-filter-issue/145462)
**Author:** [@ITNinjaGirl](https://discuss.elastic.co/u/ITNinjaGirl)\ **Replies:** 1\ **Last updated:** [August 23, 2018, 12:23pm UTC](https://discuss.elastic.co/t/another-aggregate-filter-issue/145462 "2018-08-23T12:23:25Z")
Hi. I am new to using the Aggregate filter. What I have is rsyslog data coming from an ntopng system. I need to aggregate when a network device disconnects and then reconnects to determine the duration of the discon… --- ## [Logstash high CPU usage](https://discuss.elastic.co/t/logstash-high-cpu-usage/145497)
**Author:** [@mamta](https://discuss.elastic.co/u/mamta)\ **Replies:** 13\ **Last updated:** [August 23, 2018, 10:57am UTC](https://discuss.elastic.co/t/logstash-high-cpu-usage/145497 "2018-08-23T10:57:30Z")
Hi, I have installed ELK on redhat machine. ELK version is 5.6.8. I searched a lot on google and discuss elastic forum as well but not able to solve it. I have uploaded a screenshot of CPU usage. Logstash yaml file … --- ## [How can i use 'or' in logstash grok filter](https://discuss.elastic.co/t/how-can-i-use-or-in-logstash-grok-filter/145685)
**Author:** [@wonki](https://discuss.elastic.co/u/wonki)\ **Replies:** 0\ **Last updated:** [August 23, 2018, 8:58am UTC](https://discuss.elastic.co/t/how-can-i-use-or-in-logstash-grok-filter/145685 "2018-08-23T08:58:41Z")
sample my log format: first line -\> 320 634 second line -\> 236 - third line -\> 234 123 if i make grok filter like: %{NUMBER:response} %{NUMBER:byte} then second line makes error. i tried %{NUMBER:response} … --- ## [I can't find logstash in Monitoring](https://discuss.elastic.co/t/i-cant-find-logstash-in-monitoring/145680)
**Author:** [@justTeethWhite](https://discuss.elastic.co/u/justTeethWhite)\ **Replies:** 0\ **Last updated:** [August 23, 2018, 8:41am UTC](https://discuss.elastic.co/t/i-cant-find-logstash-in-monitoring/145680 "2018-08-23T08:41:42Z")
i change monitoring settings ,but it does not work,Logstash's version is 6.3.1,Kibana and elastisearch too. help --- ## [Logstash dont receive log](https://discuss.elastic.co/t/logstash-dont-receive-log/145642)
**Author:** [@Ilyas\_Ahsan](https://discuss.elastic.co/u/Ilyas_Ahsan)\ **Replies:** 2\ **Last updated:** [August 23, 2018, 7:39am UTC](https://discuss.elastic.co/t/logstash-dont-receive-log/145642 "2018-08-23T07:39:34Z")
Hi i have problem on logstash, my logstash dont receive any logstash. how do i debug this? because i have no get error this is my logstash input and output. input { syslog { port =\> 5043 add\_field =\> {"log\_type" =\>… --- ## [Could not index in elasticsearch](https://discuss.elastic.co/t/could-not-index-in-elasticsearch/145660)
**Author:** [@ArjunSISA](https://discuss.elastic.co/u/ArjunSISA)\ **Replies:** 1\ **Last updated:** [August 23, 2018, 7:13am UTC](https://discuss.elastic.co/t/could-not-index-in-elasticsearch/145660 "2018-08-23T07:13:20Z")
\[2018-08-23T11:11:52,626\]\[WARN \]\[logstash.outputs.elasticsearch\] Could not index event to Elasticsearch. {:status=\>400, :action=\>\["index", {:\_id=\>nil, :\_index=\>"logstash-syslog-2018.08.23", :\_type=\>"doc", :\_routing=\>nil… --- ## [Dynamically read the event data. Not getting expected result](https://discuss.elastic.co/t/dynamically-read-the-event-data-not-getting-expected-result/145664)
**Author:** [@Satej](https://discuss.elastic.co/u/Satej)\ **Replies:** 0\ **Last updated:** [August 23, 2018, 6:42am UTC](https://discuss.elastic.co/t/dynamically-read-the-event-data-not-getting-expected-result/145664 "2018-08-23T06:42:47Z")
I am reading following json data from file and adding it to kinesis input event data, {"logstashtestloggroup\_123456789": {"index": "testindex","indexproperty": "","patterns":"D:/logstash/config/Patterns/"} I want to re… --- ## [Ww is returning a week in the future - why?](https://discuss.elastic.co/t/ww-is-returning-a-week-in-the-future-why/144279)
**Author:** [@francisaugusto](https://discuss.elastic.co/u/francisaugusto)\ **Replies:** 2\ **Last updated:** [August 23, 2018, 6:06am UTC](https://discuss.elastic.co/t/ww-is-returning-a-week-in-the-future-why/144279 "2018-08-23T06:06:51Z")
Hi! I am experiencing a weird thing. I am using the following pattern on the output of logstash: index =\> "{{logowner}}-%{application}-%{+xxxx.ww}” However, I am getting errors saying that the index doesn't exist, and… --- ## [Logstash config file for elasticsearch query monitoring](https://discuss.elastic.co/t/logstash-config-file-for-elasticsearch-query-monitoring/145620)
**Author:** [@Derek\_Liu](https://discuss.elastic.co/u/Derek_Liu)\ **Replies:** 0\ **Last updated:** [August 22, 2018, 10:43pm UTC](https://discuss.elastic.co/t/logstash-config-file-for-elasticsearch-query-monitoring/145620 "2018-08-22T22:43:32Z")
I am trying to use packetbeat and logstash to monitor search queries on elasticsearch by following this article: https://www.elastic.co/blog/monitoring-the-search-queries I am using the config file thats provided in the… --- ## [How to load emails into ES instance](https://discuss.elastic.co/t/how-to-load-emails-into-es-instance/145614)
**Author:** [@Edward\_Pratt](https://discuss.elastic.co/u/Edward_Pratt)\ **Replies:** 0\ **Last updated:** [August 22, 2018, 9:12pm UTC](https://discuss.elastic.co/t/how-to-load-emails-into-es-instance/145614 "2018-08-22T21:12:14Z")
I've used this guide below to forward emails from a server to my ES instance using logstash imap plugin: Anyone have recommendations on getting the historical data (i.e. the rest of the emails that are in the inbox pr… --- ## [Ruby error in adding millisecond to timestamp copy](https://discuss.elastic.co/t/ruby-error-in-adding-millisecond-to-timestamp-copy/145190)
**Author:** [@perryparktung](https://discuss.elastic.co/u/perryparktung)\ **Replies:** 10\ **Last updated:** [August 22, 2018, 5:36pm UTC](https://discuss.elastic.co/t/ruby-error-in-adding-millisecond-to-timestamp-copy/145190 "2018-08-22T17:36:41Z")
Hi, There seems to be an error in my ruby script, but I have no idea what it is, as the log is showing nothing... Please help! part of config file: date { match =\> \["date", "yyyy-MM-dd HH:mm:ss.SSS"\] } grok { matc… --- ## [Problems with NXLOG and Logstash](https://discuss.elastic.co/t/problems-with-nxlog-and-logstash/145586)
**Author:** [@Irving\_Hernandez](https://discuss.elastic.co/u/Irving_Hernandez)\ **Replies:** 0\ **Last updated:** [August 22, 2018, 4:07pm UTC](https://discuss.elastic.co/t/problems-with-nxlog-and-logstash/145586 "2018-08-22T16:07:26Z")
I have 2 virtual Ubuntu machines: 1. Realize that both could be seen. 2. Ping to see if they are calling between them. "the previous image is of a virtual machine called Ubuntu 180" "the next image i… --- ## [Add nested field with logstash and JDBC](https://discuss.elastic.co/t/add-nested-field-with-logstash-and-jdbc/145585)
**Author:** [@M.alsioufi](https://discuss.elastic.co/u/M.alsioufi)\ **Replies:** 5\ **Last updated:** [August 22, 2018, 4:21pm UTC](https://discuss.elastic.co/t/add-nested-field-with-logstash-and-jdbc/145585 "2018-08-22T16:21:00Z")
Hi, I am using Logstash to update my Elasticsearch index. I have index called my\_files, and I want to update its records with "metadata" field. "metadata" should be a nested field that contains some other fields I want … --- ## [Grok error](https://discuss.elastic.co/t/grok-error/143834)
**Author:** [@FBurke](https://discuss.elastic.co/u/FBurke)\ **Replies:** 17\ **Last updated:** [August 22, 2018, 12:45pm UTC](https://discuss.elastic.co/t/grok-error/143834 "2018-08-22T12:45:05Z")
Hi All, I'm new to logstash / grok and I'm getting errors with my grok statement that I was hoping somebody could help me with? My log entry looks like this: 50.60.70.80 \[08/Aug/2018:16:25:48 +0100\] 50.60.70.72 /Commo… --- ## [Using preliminary version of Logstash 6.4.0](https://discuss.elastic.co/t/using-preliminary-version-of-logstash-6-4-0/145549)
**Author:** [@jesusgn90](https://discuss.elastic.co/u/jesusgn90)\ **Replies:** 0\ **Last updated:** [August 22, 2018, 12:31pm UTC](https://discuss.elastic.co/t/using-preliminary-version-of-logstash-6-4-0/145549 "2018-08-22T12:31:45Z")
Hello team! Recently I've been testing our Kibana plugin in order to adapt it to future 6.4.0 version. Currently I'm using Kibana dev from 6.4 branch, and launching an ES instance with ES snapshot command from Kibana's … --- ## [How to push same document into multiple indexes?](https://discuss.elastic.co/t/how-to-push-same-document-into-multiple-indexes/145121)
**Author:** [@Rohlik](https://discuss.elastic.co/u/Rohlik)\ **Replies:** 6\ **Last updated:** [August 22, 2018, 12:17pm UTC](https://discuss.elastic.co/t/how-to-push-same-document-into-multiple-indexes/145121 "2018-08-22T12:17:31Z")
Hello, I collecting logs via TCP input and I want specific log to be in logstash index and also in monitoring index. But logs with "time" in \[tags\] are only in monitoring index, not logstash. Why? My Logstash output co… --- ## [Removing nested keys using ruby filtering](https://discuss.elastic.co/t/removing-nested-keys-using-ruby-filtering/145511)
**Author:** [@Arnold\_Yahad](https://discuss.elastic.co/u/Arnold_Yahad)\ **Replies:** 0\ **Last updated:** [August 22, 2018, 7:53am UTC](https://discuss.elastic.co/t/removing-nested-keys-using-ruby-filtering/145511 "2018-08-22T07:53:24Z")
i have an index with a lot of spam fields (over 300). they are all nested and look like this: kv.amp-1-234 kv.amp-1-abc kv.amp-1-efg so i wanted to do filtering and use remove\_fields to get rid of them. wanted to use … --- ## [Multiple pipelines with docker not working](https://discuss.elastic.co/t/multiple-pipelines-with-docker-not-working/145181)
**Author:** [@pkaramol](https://discuss.elastic.co/u/pkaramol)\ **Replies:** 5\ **Last updated:** [August 22, 2018, 7:46am UTC](https://discuss.elastic.co/t/multiple-pipelines-with-docker-not-working/145181 "2018-08-22T07:46:22Z")
Using the following compose service to spin up an ELK stack logstash: build: context: logstash/ volumes: - ./logstash/pipelines.yml:/etc/logstash/pipelines.yml:ro # - ./logstash/config/logsta… --- ## [Grok Pattern for multiple word with hyphens and dots](https://discuss.elastic.co/t/grok-pattern-for-multiple-word-with-hyphens-and-dots/145493)
**Author:** [@Makra](https://discuss.elastic.co/u/Makra)\ **Replies:** 1\ **Last updated:** [August 22, 2018, 7:23am UTC](https://discuss.elastic.co/t/grok-pattern-for-multiple-word-with-hyphens-and-dots/145493 "2018-08-22T07:23:32Z")
What will be the GROK pattern for the following word which includes hyphens and dots ABC-DEF-GHI.ae90.tomcat-boot-server-logging Regards --- ## [How to run multiple logstash conf file in single machine using logstash 6.2.4](https://discuss.elastic.co/t/how-to-run-multiple-logstash-conf-file-in-single-machine-using-logstash-6-2-4/145492)
**Author:** [@priti](https://discuss.elastic.co/u/priti)\ **Replies:** 3\ **Last updated:** [August 22, 2018, 6:59am UTC](https://discuss.elastic.co/t/how-to-run-multiple-logstash-conf-file-in-single-machine-using-logstash-6-2-4/145492 "2018-08-22T06:59:50Z")
I have downloaded and extract logstash 6.2.4 on linux machine on /opt/mount1 drive . Created 5 conf files in bin directory. When i give path.data: /opt/mount1/logstash6.2.4/data in logstash.yml fiel i am able to run 2 … --- ## [How to add to a Set collection in Logstash Elasticsearch output plugin?](https://discuss.elastic.co/t/how-to-add-to-a-set-collection-in-logstash-elasticsearch-output-plugin/145482)
**Author:** [@pushanbhattacharya](https://discuss.elastic.co/u/pushanbhattacharya)\ **Replies:** 0\ **Last updated:** [August 22, 2018, 4:16am UTC](https://discuss.elastic.co/t/how-to-add-to-a-set-collection-in-logstash-elasticsearch-output-plugin/145482 "2018-08-22T04:16:39Z")
Hi, I am trying to add a field to an array (or collection) in elasticsearch and preferably delete a field from logstash event map using elasticsearch output plugin. How can I accomplish it? P.S. For my situation I cann… --- ## [Dateparsefailure for "yyyy-MM-dd-HH:mm:ss.SSSZZ" format](https://discuss.elastic.co/t/dateparsefailure-for-yyyy-mm-dd-hhss-ssszz-format/145295)
**Author:** [@JeffYe](https://discuss.elastic.co/u/JeffYe)\ **Replies:** 2\ **Last updated:** [August 22, 2018, 3:22am UTC](https://discuss.elastic.co/t/dateparsefailure-for-yyyy-mm-dd-hhss-ssszz-format/145295 "2018-08-22T03:22:26Z")
Hi, My log file has the format of timestamp like '2018-08-20-14:10:16.496+8:00...something...'. It looks like ISO8601 but it uses '-' instead of 'space' between date and time. I can't use ISO8601 to match my logs in th… --- ## [Filter Duplicate Logs](https://discuss.elastic.co/t/filter-duplicate-logs/145472)
**Author:** [@Naruhodou](https://discuss.elastic.co/u/Naruhodou)\ **Replies:** 1\ **Last updated:** [August 22, 2018, 12:32am UTC](https://discuss.elastic.co/t/filter-duplicate-logs/145472 "2018-08-22T00:32:21Z")
Hi, We built the pipeline of Filebeat+Logstash+Elasticserach+Kibana in a Kafka cluster. But when there is an issue happening in the cluster, same logs related to this issue keep generating before the issue is resolved. … --- ## [Exception in throttle](https://discuss.elastic.co/t/exception-in-throttle/145473)
**Author:** [@Naruhodou](https://discuss.elastic.co/u/Naruhodou)\ **Replies:** 1\ **Last updated:** [August 22, 2018, 12:31am UTC](https://discuss.elastic.co/t/exception-in-throttle/145473 "2018-08-22T00:31:49Z")
Hi, I got exceptions while trying to use throttle in Logstash, the exception is Exception in pipelineworker, the pipeline stopped processing new events, please check your filter configuration and restart Logstash. {:p… --- ## [Adding optional match pattern to grok for syslog messages](https://discuss.elastic.co/t/adding-optional-match-pattern-to-grok-for-syslog-messages/144250)
**Author:** [@nodesocket](https://discuss.elastic.co/u/nodesocket)\ **Replies:** 4\ **Last updated:** [August 21, 2018, 11:25pm UTC](https://discuss.elastic.co/t/adding-optional-match-pattern-to-grok-for-syslog-messages/144250 "2018-08-21T23:25:16Z")
Currently I am using the following logstash configuration to match syslog messages in the form of: Message: Aug 10 01:09:56.071 (info) smtpd: \[178cea3e-9c3a-11e8-93fa-f3ebd9db2b94\] \[SMTP\] \[EHLO\] EHLO SEKO-SBS.sekonorth… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=696) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=698)