# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=787 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 788 --- ## [Help deleting data](https://discuss.elastic.co/t/help-deleting-data/121851)
**Author:** [@keithh](https://discuss.elastic.co/u/keithh)\ **Replies:** 7\ **Last updated:** [April 4, 2018, 3:03pm UTC](https://discuss.elastic.co/t/help-deleting-data/121851 "2018-04-04T15:03:47Z")
Hi there, Forgive my ignorance in advance. I've inherited an old stack with elasticsearch 1.7. Previously, as has been documented in many threads, it was possible to delete logs using a chrome plugin, which has since … --- ## [Logstash metrics endpoint](https://discuss.elastic.co/t/logstash-metrics-endpoint/126749)
**Author:** [@pjh](https://discuss.elastic.co/u/pjh)\ **Replies:** 1\ **Last updated:** [April 4, 2018, 2:45pm UTC](https://discuss.elastic.co/t/logstash-metrics-endpoint/126749 "2018-04-04T14:45:33Z")
How can I use the logstash metrics endpoint? I can't see any detailed documentation on this. I have uncommented the following in the logstash.yml file but when I visit :9600/metrics there is just a 404 error. # -------… --- ## [Multiple format of date to single target](https://discuss.elastic.co/t/multiple-format-of-date-to-single-target/126709)
**Author:** [@Sridhar](https://discuss.elastic.co/u/Sridhar)\ **Replies:** 2\ **Last updated:** [April 4, 2018, 2:36pm UTC](https://discuss.elastic.co/t/multiple-format-of-date-to-single-target/126709 "2018-04-04T14:36:32Z")
Hi, I have the input with 2 date format in same column of csv file. 3-Mar-18 and Mar-18 below is the config file I created. -- date { match =\> \["rdate", "d-MMM-yy"\] target =\> "rdate" } date { match =\> \["rdate",… --- ## [How to create multiple indices from multiple oracle databases?](https://discuss.elastic.co/t/how-to-create-multiple-indices-from-multiple-oracle-databases/123033)
**Author:** [@heisjustgod](https://discuss.elastic.co/u/heisjustgod)\ **Replies:** 4\ **Last updated:** [April 4, 2018, 1:08pm UTC](https://discuss.elastic.co/t/how-to-create-multiple-indices-from-multiple-oracle-databases/123033 "2018-04-04T13:08:16Z")
I am trying to setup elasticsearch to a 50+ application, those applications hold their own oracle DB. I can use 'logstash' to feed DB-records using logstash.conf file. but it holds single INPUT section, How can I connec… --- ## [Grok search and match from file](https://discuss.elastic.co/t/grok-search-and-match-from-file/126737)
**Author:** [@kamaradski](https://discuss.elastic.co/u/kamaradski)\ **Replies:** 1\ **Last updated:** [April 4, 2018, 12:38pm UTC](https://discuss.elastic.co/t/grok-search-and-match-from-file/126737 "2018-04-04T12:38:33Z")
Hi, I'm logging events based on customer account numbers. However, at that point in the system, our software is not aware of the accountName. Now I know it's a bit of a workaround, but since it is not really possible t… --- ## [Logstash Drop Filter](https://discuss.elastic.co/t/logstash-drop-filter/126509)
**Author:** [@Nitish\_Upadhyay](https://discuss.elastic.co/u/Nitish_Upadhyay)\ **Replies:** 3\ **Last updated:** [April 4, 2018, 12:35pm UTC](https://discuss.elastic.co/t/logstash-drop-filter/126509 "2018-04-04T12:35:30Z")
Hello guys, I am new to logstash and was trying to drop few events which do not match a criteria. Even though logstash is dropping those events, but still it keeps on sending an acknowledgement about the deleted event … --- ## [Matching date filter with Month in UPPERCASE](https://discuss.elastic.co/t/matching-date-filter-with-month-in-uppercase/126538)
**Author:** [@anisen](https://discuss.elastic.co/u/anisen)\ **Replies:** 4\ **Last updated:** [April 4, 2018, 12:06pm UTC](https://discuss.elastic.co/t/matching-date-filter-with-month-in-uppercase/126538 "2018-04-04T12:06:14Z")
Hi, The date-timestamp column "time" in my log looks like: 09-FEB-18 05.32.53.045598000 AM With the below configuration in logstash, I am getting dateparse failure: date { match =\> \[ "time", "dd-MMM-yy HH.mm.ss.SSSS… --- ## [Mutate and Gsub advice](https://discuss.elastic.co/t/mutate-and-gsub-advice/126624)
**Author:** [@Zorkmid](https://discuss.elastic.co/u/Zorkmid)\ **Replies:** 1\ **Last updated:** [April 4, 2018, 10:20am UTC](https://discuss.elastic.co/t/mutate-and-gsub-advice/126624 "2018-04-04T10:20:49Z")
Hi All, I have event logs streaming from my SEIM to ES. The format of these logs are LEEF and have "|"'s and "=" signs that delimit the fields. What I'd like to is to use a filter to find and replace the "|"'s and "=" … --- ## [Fields not appearing in Kibana which are defined in Logstash](https://discuss.elastic.co/t/fields-not-appearing-in-kibana-which-are-defined-in-logstash/124537)
**Author:** [@falsekingpin](https://discuss.elastic.co/u/falsekingpin)\ **Replies:** 19\ **Last updated:** [April 4, 2018, 9:37am UTC](https://discuss.elastic.co/t/fields-not-appearing-in-kibana-which-are-defined-in-logstash/124537 "2018-04-04T09:37:11Z")
Hi all, I've defined some fields in logstash filter which is stored in /etc/logstash/conf.d/ The filter is as follows: input { beats { port =\> 5044 } } filter { if \[type\] == "log" { grok { match =\> { "message"… --- ## [Importing more than one xml file](https://discuss.elastic.co/t/importing-more-than-one-xml-file/126534)
**Author:** [@EloaCardoso](https://discuss.elastic.co/u/EloaCardoso)\ **Replies:** 4\ **Last updated:** [April 4, 2018, 9:10am UTC](https://discuss.elastic.co/t/importing-more-than-one-xml-file/126534 "2018-04-04T09:10:02Z")
Hi, I would like to know how I can import more than one xml file from logstash to elasticsearch. My configuration file is working just for the importing of one xml file. The input filter is written as follows: input … --- ## [Recommended way to maintain multiple logstash output plugin configuration](https://discuss.elastic.co/t/recommended-way-to-maintain-multiple-logstash-output-plugin-configuration/126678)
**Author:** [@Nikhil\_Utane](https://discuss.elastic.co/u/Nikhil_Utane)\ **Replies:** 2\ **Last updated:** [April 4, 2018, 8:55am UTC](https://discuss.elastic.co/t/recommended-way-to-maintain-multiple-logstash-output-plugin-configuration/126678 "2018-04-04T08:55:01Z")
Hi, Since this would be a common problem faced by many, would like to understand how you are solving it. We have multiple Elasticsearch setups and common code in git. This includes the logstash output plugin configurat… --- ## [Is it possible to create a master conf file and "overrides"](https://discuss.elastic.co/t/is-it-possible-to-create-a-master-conf-file-and-overrides/126699)
**Author:** [@dorj1234](https://discuss.elastic.co/u/dorj1234)\ **Replies:** 2\ **Last updated:** [April 4, 2018, 8:51am UTC](https://discuss.elastic.co/t/is-it-possible-to-create-a-master-conf-file-and-overrides/126699 "2018-04-04T08:51:21Z")
Hello, I have a logstash.conf file that I use for several indexes. The only difference between each conf file is the input path and the index name in the output to elasticsearch. I would like to create a single conf f… --- ## [\[Elastic\]\[Logstash\] Ask for help on parsing the Microsoft exchange logs](https://discuss.elastic.co/t/elastic-logstash-ask-for-help-on-parsing-the-microsoft-exchange-logs/126369)
**Author:** [@Robin\_Guo](https://discuss.elastic.co/u/Robin_Guo)\ **Replies:** 2\ **Last updated:** [April 4, 2018, 8:18am UTC](https://discuss.elastic.co/t/elastic-logstash-ask-for-help-on-parsing-the-microsoft-exchange-logs/126369 "2018-04-04T08:18:18Z")
Dear Elastic colleagues && folks, Recently there is a request from our security team that they need to collect the Microsoft exchange message tracking logs from our exchange servers, every line from original exchange l… --- ## [Logstash filter to split field into fields of arrays](https://discuss.elastic.co/t/logstash-filter-to-split-field-into-fields-of-arrays/126152)
**Author:** [@mad\_dog](https://discuss.elastic.co/u/mad_dog)\ **Replies:** 4\ **Last updated:** [April 4, 2018, 7:39am UTC](https://discuss.elastic.co/t/logstash-filter-to-split-field-into-fields-of-arrays/126152 "2018-04-04T07:39:20Z")
Hi, I'm struggling to process some logs in Logstash. So far I managed to convert them to something like this: "sessions": \[ "http: active 0", "http: started … --- ## [Import CSV into Elasticsearch with Logstash issue](https://discuss.elastic.co/t/import-csv-into-elasticsearch-with-logstash-issue/126514)
**Author:** [@dyl](https://discuss.elastic.co/u/dyl)\ **Replies:** 5\ **Last updated:** [April 4, 2018, 7:30am UTC](https://discuss.elastic.co/t/import-csv-into-elasticsearch-with-logstash-issue/126514 "2018-04-04T07:30:52Z")
Hello ELK community ! I use the ELK stack to parse CSV files and send them to Elasticsearch after parsing them with logstash. Unfortunately, I have a problem: When I send my files to the listening directory of the "in… --- ## [Logstash Http\_Poller : Connection Timed out while connecting to Azure application Insight API](https://discuss.elastic.co/t/logstash-http-poller-connection-timed-out-while-connecting-to-azure-application-insight-api/126679)
**Author:** [@Dinesh\_Ginwali](https://discuss.elastic.co/u/Dinesh_Ginwali)\ **Replies:** 0\ **Last updated:** [April 4, 2018, 6:40am UTC](https://discuss.elastic.co/t/logstash-http-poller-connection-timed-out-while-connecting-to-azure-application-insight-api/126679 "2018-04-04T06:40:07Z")
I am having similar issue while connecting to Azure Application Insight using Http\_Poller. I am using X-API-Key to connect to the URL. Same as what shown in Below discussion. This is not network issue, as i connecte… --- ## [Errors when running logstash](https://discuss.elastic.co/t/errors-when-running-logstash/126670)
**Author:** [@Chips](https://discuss.elastic.co/u/Chips)\ **Replies:** 1\ **Last updated:** [April 4, 2018, 5:58am UTC](https://discuss.elastic.co/t/errors-when-running-logstash/126670 "2018-04-04T05:58:42Z")
When I run the command bin/logstash -f 01-wazuh.conf two errors pop up and one warning. Although the file is on the specified path, oh well, the most important thing is how to fix the two lower errors? WARNING: Could no… --- ## [Aggregate multiple filebeat](https://discuss.elastic.co/t/aggregate-multiple-filebeat/126664)
**Author:** [@las](https://discuss.elastic.co/u/las)\ **Replies:** 1\ **Last updated:** [April 4, 2018, 4:22am UTC](https://discuss.elastic.co/t/aggregate-multiple-filebeat/126664 "2018-04-04T04:22:43Z")
There are multiple filebeat nodes that send data to logstash. The logstash needs to aggregate events together and send to elasticsearch. How logstash needs to be configured to avoid being a single point of failure. --- ## [Dead Letter Queue generates single byte file log](https://discuss.elastic.co/t/dead-letter-queue-generates-single-byte-file-log/126668)
**Author:** [@pandeesh](https://discuss.elastic.co/u/pandeesh)\ **Replies:** 0\ **Last updated:** [April 4, 2018, 4:09am UTC](https://discuss.elastic.co/t/dead-letter-queue-generates-single-byte-file-log/126668 "2018-04-04T04:09:04Z")
I am using logstash 6.2.3 and running two pipelines as given below: pipeline.id: main path.config: "/etc/logstash/conf.d/logstash\_es.conf" dead\_letter\_queue.enable: true dead\_letter\_queue.max\_bytes: 30g path.dead\_… --- ## [Autoscale logstash in AWS](https://discuss.elastic.co/t/autoscale-logstash-in-aws/125262)
**Author:** [@Jathin](https://discuss.elastic.co/u/Jathin)\ **Replies:** 4\ **Last updated:** [April 4, 2018, 1:20am UTC](https://discuss.elastic.co/t/autoscale-logstash-in-aws/125262 "2018-04-04T01:20:36Z")
i am using logstash-input-plugin with persistent queues and fowarding my logs to Elasticsearch and a small subset of logs are sent to s3. Is there any best practice for auto scaling my logstash instances. I am deploying… --- ## [Continuous Logstash Stack Trace in Log](https://discuss.elastic.co/t/continuous-logstash-stack-trace-in-log/126635)
**Author:** [@daved](https://discuss.elastic.co/u/daved)\ **Replies:** 0\ **Last updated:** [April 3, 2018, 8:07pm UTC](https://discuss.elastic.co/t/continuous-logstash-stack-trace-in-log/126635 "2018-04-03T20:07:54Z")
Hi, I'm seeing the following stack trace continuously repeated in my logstash-plain.log (I couldn't post the whole trace since it was over the character limit): \[2018-04-03T13:17:03,815\]\[ERROR\]\[logstash.pipeline … --- ## [Logstash rake boostrap fails on raspberry pi](https://discuss.elastic.co/t/logstash-rake-boostrap-fails-on-raspberry-pi/126622)
**Author:** [@cement](https://discuss.elastic.co/u/cement)\ **Replies:** 1\ **Last updated:** [April 3, 2018, 6:36pm UTC](https://discuss.elastic.co/t/logstash-rake-boostrap-fails-on-raspberry-pi/126622 "2018-04-03T18:36:11Z")
\# env | grep ES ES\_JAVA\_OPTS=-Xms512m -Xmx512m root@raspberry:/home/pi/logstash# rake bootstrap ---8\<--- # # There is insufficient memory for the Java Runtime Environment to continue. # Native memory allocation (mmap) fa… --- ## [Logstash Monitoring Error](https://discuss.elastic.co/t/logstash-monitoring-error/126482)
**Author:** [@wwalker](https://discuss.elastic.co/u/wwalker)\ **Replies:** 4\ **Last updated:** [April 3, 2018, 6:22pm UTC](https://discuss.elastic.co/t/logstash-monitoring-error/126482 "2018-04-03T18:22:58Z")
My ElasticStack decided to go haywire today and I really have no idea what went wrong. After reinstalling the services, Logstash now logs this error message non-stop: \[2018-04-02T19:33:52,358\]\[WARN \]\[logstash.outputs.e… --- ## [Create quarterly index in log stash config file](https://discuss.elastic.co/t/create-quarterly-index-in-log-stash-config-file/126521)
**Author:** [@DILIP\_SHARMA](https://discuss.elastic.co/u/DILIP_SHARMA)\ **Replies:** 1\ **Last updated:** [April 3, 2018, 5:04pm UTC](https://discuss.elastic.co/t/create-quarterly-index-in-log-stash-config-file/126521 "2018-04-03T17:04:47Z")
hi there is any way to create quarterly index in logstash config file , i know how to create daily / monthly / year index , there is any function which can give quarter in log stash , or we need to calculate that ? T… --- ## [Logstash syslog output](https://discuss.elastic.co/t/logstash-syslog-output/126605)
**Author:** [@rxp1997](https://discuss.elastic.co/u/rxp1997)\ **Replies:** 0\ **Last updated:** [April 3, 2018, 4:08pm UTC](https://discuss.elastic.co/t/logstash-syslog-output/126605 "2018-04-03T16:08:32Z")
Hi, logstash newbie in need of advice I am using logstash to ingest google pub/sub messages, which are single-line json messages, and want to forward them to a syslog server. Ingestion and writting the message to a fil… --- ## [Logstash TLS performance troubleshooting](https://discuss.elastic.co/t/logstash-tls-performance-troubleshooting/126599)
**Author:** [@bad\_sector](https://discuss.elastic.co/u/bad_sector)\ **Replies:** 0\ **Last updated:** [April 3, 2018, 3:36pm UTC](https://discuss.elastic.co/t/logstash-tls-performance-troubleshooting/126599 "2018-04-03T15:36:27Z")
I am suffering from poor TLS handshake performance while connecting to Logstash. While testing using openssl s\_client ... and providing client key it takes up to 40 seconds, to finish TLS handshake. This is machine that … --- ## [Logstash output struggle](https://discuss.elastic.co/t/logstash-output-struggle/126572)
**Author:** [@J\_B](https://discuss.elastic.co/u/J_B)\ **Replies:** 0\ **Last updated:** [April 3, 2018, 12:41pm UTC](https://discuss.elastic.co/t/logstash-output-struggle/126572 "2018-04-03T12:41:59Z")
Hello, iam new to whole ELK problematics and iam trying to redo "logstack" i inherited from scratch. Previous configuration was using logstash - graylog- elasticserach with custom filtering in logstash. Iam trying to use… --- ## [Logstash erreur d'exécution](https://discuss.elastic.co/t/logstash-erreur-dexecution/126444)
**Author:** [@Ibti\_Ssem](https://discuss.elastic.co/u/Ibti_Ssem)\ **Replies:** 25\ **Last updated:** [April 3, 2018, 12:26pm UTC](https://discuss.elastic.co/t/logstash-erreur-dexecution/126444 "2018-04-03T12:26:39Z")
lorsque j'ai exucute logstash j'ai une erreur pourquoi ? sachant que ce la première fois ça fonctionnait bien !! \[image\] --- ## [Logstash grok pattern for ping](https://discuss.elastic.co/t/logstash-grok-pattern-for-ping/125821)
**Author:** [@geokal](https://discuss.elastic.co/u/geokal)\ **Replies:** 11\ **Last updated:** [April 3, 2018, 12:09pm UTC](https://discuss.elastic.co/t/logstash-grok-pattern-for-ping/125821 "2018-04-03T12:09:03Z")
Hi everybody, have this sample from a log: Tue Mar 27 06:51:48 2018 PING www.google.com (172.217.169.100) 56(84) bytes of data. 64 bytes from sof02s31-in-f4.1e100.net (172.217.169.100): icmp\_seq=1 ttl=128 time=17… --- ## [8 hours difference in time zone](https://discuss.elastic.co/t/8-hours-difference-in-time-zone/126536)
**Author:** [@chenzilong444](https://discuss.elastic.co/u/chenzilong444)\ **Replies:** 9\ **Last updated:** [April 3, 2018, 11:29am UTC](https://discuss.elastic.co/t/8-hours-difference-in-time-zone/126536 "2018-04-03T11:29:09Z")
elasticsearch { hosts =\> localhost index =\> "logstash(%{+YYYY.MM.dd})" document\_type =\>"logstash" flush\_size=\>1000 idle\_flush\_time=\>15 } i didn‘’t set date in logstash @timestamp … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=786) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=788)