# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=939 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 940 --- ## [Logstash-output-icinga installation](https://discuss.elastic.co/t/logstash-output-icinga-installation/93775)
**Author:** [@muratungor](https://discuss.elastic.co/u/muratungor)\ **Replies:** 1\ **Last updated:** [July 26, 2017, 7:27am UTC](https://discuss.elastic.co/t/logstash-output-icinga-installation/93775 "2017-07-26T07:27:43Z")
Hi all, I'm new in this world. We have a logstash server behing firewall and proxy. I want to install logstash-output-icinga plugin on it. in the document it says "For plugins not bundled by default, it is easy to instal… --- ## [How to filter a time stamp with no year value](https://discuss.elastic.co/t/how-to-filter-a-time-stamp-with-no-year-value/94548)
**Author:** [@ssasporta](https://discuss.elastic.co/u/ssasporta)\ **Replies:** 2\ **Last updated:** [July 26, 2017, 5:46am UTC](https://discuss.elastic.co/t/how-to-filter-a-time-stamp-with-no-year-value/94548 "2017-07-26T05:46:35Z")
Hi, This is how my event logs looks like : \<\\> \[Jul 25 19:17:08\] \[\[ACTIVE\] I am trying to filter the time stamp. The grok looks like: \\\<\\\<%{LOGLEVEL:severity}\\\>\\\> \\\[%{PARTTIMESTAMP:timestamp}\\\] \\\[\\\[%{DATA… --- ## [S3 as intermediate stage in Elastic stack pipeline](https://discuss.elastic.co/t/s3-as-intermediate-stage-in-elastic-stack-pipeline/94057)
**Author:** [@hilt86](https://discuss.elastic.co/u/hilt86)\ **Replies:** 2\ **Last updated:** [July 26, 2017, 2:55am UTC](https://discuss.elastic.co/t/s3-as-intermediate-stage-in-elastic-stack-pipeline/94057 "2017-07-26T02:55:25Z")
Hi Folks, Thanks to all for creating the Elastic stack (ELK) - it enables small businesses like my own to have visibility of security events and performance data that is too costly or inflexible with proprietary tools.… --- ## [Filebeat logstash primary shard error](https://discuss.elastic.co/t/filebeat-logstash-primary-shard-error/94546)
**Author:** [@shankarbala](https://discuss.elastic.co/u/shankarbala)\ **Replies:** 1\ **Last updated:** [July 26, 2017, 12:47am UTC](https://discuss.elastic.co/t/filebeat-logstash-primary-shard-error/94546 "2017-07-26T00:47:19Z")
Hi All I have logstash 2.4.0 and filebeat 5.2.2...filebeat reads log files and send it to logstash..logstash send it to our ELK cluster running 2.2.0 here is my filebeat config ======== filebeat.prospectors: in… --- ## [High CPU on logstash cluster](https://discuss.elastic.co/t/high-cpu-on-logstash-cluster/94359)
**Author:** [@Nikhilpawar1985](https://discuss.elastic.co/u/Nikhilpawar1985)\ **Replies:** 4\ **Last updated:** [July 25, 2017, 9:41pm UTC](https://discuss.elastic.co/t/high-cpu-on-logstash-cluster/94359 "2017-07-25T21:41:17Z")
Hi , I am struggling with High CPU on logstash cluster . I have 4 nodes with 8 CPUs and 8 GB memory . Memory seems to used half of it. but CPU is always above 90% . Below is my logstash config input { beats… --- ## [Unable to parse out XML fields](https://discuss.elastic.co/t/unable-to-parse-out-xml-fields/94353)
**Author:** [@maverick88](https://discuss.elastic.co/u/maverick88)\ **Replies:** 2\ **Last updated:** [July 25, 2017, 9:29pm UTC](https://discuss.elastic.co/t/unable-to-parse-out-xml-fields/94353 "2017-07-25T21:29:11Z")
Hello - I've exported some Windows events to XML and need to parse them into fields, but I'm not having much luck. I'm able to parse each event into a separate log using the multiline filter, but nothing I seem to do is … --- ## [LogStash 5.5.0 start as a service does not read config file](https://discuss.elastic.co/t/logstash-5-5-0-start-as-a-service-does-not-read-config-file/94427)
**Author:** [@manojkumar](https://discuss.elastic.co/u/manojkumar)\ **Replies:** 1\ **Last updated:** [July 25, 2017, 9:26pm UTC](https://discuss.elastic.co/t/logstash-5-5-0-start-as-a-service-does-not-read-config-file/94427 "2017-07-25T21:26:55Z")
Hi, I am running logstash 5.5 on ubuntu 16.04. I SSH into my linux box using a cert and sudo to root user. My config file is under /etc/logstash/conf.d/logstash-apache.conf. a) When I run logstash from the command … --- ## [File input plugin](https://discuss.elastic.co/t/file-input-plugin/94488)
**Author:** [@ktomu](https://discuss.elastic.co/u/ktomu)\ **Replies:** 1\ **Last updated:** [July 25, 2017, 9:19pm UTC](https://discuss.elastic.co/t/file-input-plugin/94488 "2017-07-25T21:19:06Z")
hello, which is the best way to control if all my messages from a file (file input plugin) were processed by logstash? --- ## [Logstash 5.5 dynamic columns detection](https://discuss.elastic.co/t/logstash-5-5-dynamic-columns-detection/94336)
**Author:** [@Vinuja1](https://discuss.elastic.co/u/Vinuja1)\ **Replies:** 1\ **Last updated:** [July 25, 2017, 9:14pm UTC](https://discuss.elastic.co/t/logstash-5-5-dynamic-columns-detection/94336 "2017-07-25T21:14:38Z")
Hi there, The auto detection of the columns name in the version 5.5 is a great idea. However, I have some issues to use it. In fact, I have tried the following : filter { if \[type\] =~ "trade\_v7" { csv { separa… --- ## [Need to create a field for multiple type exceptions](https://discuss.elastic.co/t/need-to-create-a-field-for-multiple-type-exceptions/94349)
**Author:** [@Narmatha](https://discuss.elastic.co/u/Narmatha)\ **Replies:** 1\ **Last updated:** [July 25, 2017, 9:13pm UTC](https://discuss.elastic.co/t/need-to-create-a-field-for-multiple-type-exceptions/94349 "2017-07-25T21:13:19Z")
Hi I have configured logstash configuration to create an index in ES for all the logs present in three node cluster. Please find the below conf. input { file { path =\> "D:\\Development\_Avecto\\logstash-2.4.1\\bin\\… --- ## [Logstash =\> ES, Timezone and IndexName](https://discuss.elastic.co/t/logstash-es-timezone-and-indexname/94297)
**Author:** [@bastimm](https://discuss.elastic.co/u/bastimm)\ **Replies:** 3\ **Last updated:** [July 25, 2017, 9:10pm UTC](https://discuss.elastic.co/t/logstash-es-timezone-and-indexname/94297 "2017-07-25T21:10:36Z")
Hello, we have logfiles with a timestamp like "yyyyMMddHHmmss" with Europe/Berlin Timezone. This files were sent via FileBeat=\>Logstash=\>ES. Now we changed timezone in our Logstash config. Now the timestamp in ES is … --- ## [Logstash not able to break message into given fields](https://discuss.elastic.co/t/logstash-not-able-to-break-message-into-given-fields/94211)
**Author:** [@NITIN-BHAISARE](https://discuss.elastic.co/u/NITIN-BHAISARE)\ **Replies:** 2\ **Last updated:** [July 25, 2017, 9:04pm UTC](https://discuss.elastic.co/t/logstash-not-able-to-break-message-into-given-fields/94211 "2017-07-25T21:04:36Z")
I have a log which contains java stack traces and a line which is very simple like this: \[id\] 002 \[uri\] /legacy/biology/v2/status Grok Pattern: \[%{DATA:id}\]\\s%{JAVAFILE:track\_id}\\s\[%{DATA:uri}\]\\s%{DATA:path} I am using f… --- ## [Log with no date/time. Manual field not linking to @timestamp](https://discuss.elastic.co/t/log-with-no-date-time-manual-field-not-linking-to-timestamp/94160)
**Author:** [@Blake\_2112](https://discuss.elastic.co/u/Blake_2112)\ **Replies:** 3\ **Last updated:** [July 25, 2017, 9:01pm UTC](https://discuss.elastic.co/t/log-with-no-date-time-manual-field-not-linking-to-timestamp/94160 "2017-07-25T21:01:18Z")
I have a set of logs that I would like to input into ES and analyze. The logs are flowing into ES fine but my manual 'date' field is not properly linking to @timestamp. Thus, @timestamp is the ES ingestion time, not the … --- ## [How to grok a text containing json](https://discuss.elastic.co/t/how-to-grok-a-text-containing-json/94180)
**Author:** [@ssasporta](https://discuss.elastic.co/u/ssasporta)\ **Replies:** 1\ **Last updated:** [July 25, 2017, 8:58pm UTC](https://discuss.elastic.co/t/how-to-grok-a-text-containing-json/94180 "2017-07-25T20:58:06Z")
I don't really understand how to grok the below text ? The response itself also needs to be pars. At least I Need an example for it. Timestamp:2017-07-21 17:32:32.778 \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*… --- ## [Multiple Logstash instances vs Filebeats - Architecture](https://discuss.elastic.co/t/multiple-logstash-instances-vs-filebeats-architecture/94437)
**Author:** [@DougMcDonald](https://discuss.elastic.co/u/DougMcDonald)\ **Replies:** 1\ **Last updated:** [July 25, 2017, 8:54pm UTC](https://discuss.elastic.co/t/multiple-logstash-instances-vs-filebeats-architecture/94437 "2017-07-25T20:54:39Z")
I'm trying to establish the best architecture for our elastic stack implementation. We have two distinct networks (lets call them internal and external) and several web / db / application servers (approx 10) on each o… --- ## [Execute logstash as service on startup](https://discuss.elastic.co/t/execute-logstash-as-service-on-startup/94086)
**Author:** [@Danilo\_Giacchi](https://discuss.elastic.co/u/Danilo_Giacchi)\ **Replies:** 1\ **Last updated:** [July 25, 2017, 8:49pm UTC](https://discuss.elastic.co/t/execute-logstash-as-service-on-startup/94086 "2017-07-25T20:49:18Z")
Hi i am new to Logstash and i am using the version 5.5.0 and elasticsearch 5.5.0 on xubuntu 16.04. I'm trying to create a bridge between postgresql 9.6.0 and elasticsearch. I'm able to execute logstash with configuration… --- ## [Please Suggest the config file for the XML data to load in logstash](https://discuss.elastic.co/t/please-suggest-the-config-file-for-the-xml-data-to-load-in-logstash/93362)
**Author:** [@abhisek](https://discuss.elastic.co/u/abhisek)\ **Replies:** 10\ **Last updated:** [July 25, 2017, 8:29pm UTC](https://discuss.elastic.co/t/please-suggest-the-config-file-for-the-xml-data-to-load-in-logstash/93362 "2017-07-25T20:29:17Z")
--- ## [Importing Github api response in elasticsearch/logstash](https://discuss.elastic.co/t/importing-github-api-response-in-elasticsearch-logstash/94530)
**Author:** [@Elastic\_01](https://discuss.elastic.co/u/Elastic_01)\ **Replies:** 0\ **Last updated:** [July 25, 2017, 6:53pm UTC](https://discuss.elastic.co/t/importing-github-api-response-in-elasticsearch-logstash/94530 "2017-07-25T18:53:36Z")
Is there a way to add nested JSON objects from github api response to elasticsearch with/without logstash? --- ## [\[Attention\] Logstash 5.5.0 - Known Issue with Dissect Filter and Dead Letter Queue Input](https://discuss.elastic.co/t/attention-logstash-5-5-0-known-issue-with-dissect-filter-and-dead-letter-queue-input/92337)
**Author:** [@alvin](https://discuss.elastic.co/u/alvin)\ **Replies:** 1\ **Last updated:** [July 25, 2017, 4:06pm UTC](https://discuss.elastic.co/t/attention-logstash-5-5-0-known-issue-with-dissect-filter-and-dead-letter-queue-input/92337 "2017-07-25T16:06:51Z")
This notice addresses a known issue with the default bundled versions of the dissect filter and dead letter queue input plugins in the Logstash 5.5.0 release. The Problem When using the dissect filter or dead letter q… --- ## [Logstash output to jms , qpid broker queue getting below error](https://discuss.elastic.co/t/logstash-output-to-jms-qpid-broker-queue-getting-below-error/94480)
**Author:** [@Jagatjyoti\_Mohanty](https://discuss.elastic.co/u/Jagatjyoti_Mohanty)\ **Replies:** 2\ **Last updated:** [July 25, 2017, 1:44pm UTC](https://discuss.elastic.co/t/logstash-output-to-jms-qpid-broker-queue-getting-below-error/94480 "2017-07-25T13:44:56Z")
:exception=\>#, :backtrace=\>\["C:/logstash-5.4.1/vendor/bundle/jruby/1.9/gems/logstash-output-jms-3.0.1/lib/logstash/outputs/jms.rb:114:in receive'", "C:/logstash-5.4.1/logstash-core/lib/logstash/outputs/base.rb:92:inmulti… --- ## [Grok parser to return only numbers](https://discuss.elastic.co/t/grok-parser-to-return-only-numbers/94294)
**Author:** [@Ravi\_Shanker\_Reddy](https://discuss.elastic.co/u/Ravi_Shanker_Reddy)\ **Replies:** 3\ **Last updated:** [July 25, 2017, 12:06pm UTC](https://discuss.elastic.co/t/grok-parser-to-return-only-numbers/94294 "2017-07-25T12:06:41Z")
I have a field where I will get phone number. Here are the examples. 1-800-1234 1(800)1234 18001234 1 800 1234 If the format is matching anyone I want to save the number in ES like 18001234. Can anyone help for the g… --- ## [Logstash with jdbc input for Oracle DB](https://discuss.elastic.co/t/logstash-with-jdbc-input-for-oracle-db/94315)
**Author:** [@Jyothig](https://discuss.elastic.co/u/Jyothig)\ **Replies:** 1\ **Last updated:** [July 25, 2017, 11:56am UTC](https://discuss.elastic.co/t/logstash-with-jdbc-input-for-oracle-db/94315 "2017-07-25T11:56:25Z")
I updated jdbc details as below: jdbc\_driver\_library =\> "../ojdbc6.jar" jdbc\_driver\_class =\> "Java::oracle.jdbc.driver.OracleDriver" jdbc\_connection\_string =\> "jdbc:oracle:thin@//\<\\>:1521/\<\\>… --- ## [Get 10 characters before substring in logstash?](https://discuss.elastic.co/t/get-10-characters-before-substring-in-logstash/94439)
**Author:** [@Yaswanth](https://discuss.elastic.co/u/Yaswanth)\ **Replies:** 6\ **Last updated:** [July 25, 2017, 10:35am UTC](https://discuss.elastic.co/t/get-10-characters-before-substring-in-logstash/94439 "2017-07-25T10:35:42Z")
Hi, I want to get the first 10 characters from a string that ends with Exception. Example: I have field like this: "ERRORMESSAGE" =\> "local exporter \[default\_local\] - failed to delete indice s\\r\\nRemoteTransportExc… --- ## [UDP input plugin monitoring and elasticsearch output](https://discuss.elastic.co/t/udp-input-plugin-monitoring-and-elasticsearch-output/93540)
**Author:** [@Mr.Flakes](https://discuss.elastic.co/u/Mr.Flakes)\ **Replies:** 4\ **Last updated:** [July 25, 2017, 8:49am UTC](https://discuss.elastic.co/t/udp-input-plugin-monitoring-and-elasticsearch-output/93540 "2017-07-25T08:49:51Z")
Hello Logstash GURUS, so I have a logstash running live and basically the guy who set it up quit his job :frowning: Since I am totally new to this and have a system running live already that I need to take care of some … --- ## [Logstash - cannot delete "\]" sign](https://discuss.elastic.co/t/logstash-cannot-delete-sign/93974)
**Author:** [@rachel1](https://discuss.elastic.co/u/rachel1)\ **Replies:** 1\ **Last updated:** [July 25, 2017, 8:09am UTC](https://discuss.elastic.co/t/logstash-cannot-delete-sign/93974 "2017-07-25T08:09:41Z")
Hi! I'm trying to convert a text type field into a float one in Logstash and ElasticSearch mapping. field name: votingPercentage. My log looks as follow: "votingPercentage = 0.13\]" Since there's a "\]" sign in the end … --- ## [Logstash, local plugin installation fails if network connected without internet](https://discuss.elastic.co/t/logstash-local-plugin-installation-fails-if-network-connected-without-internet/93590)
**Author:** [@IHateBugs](https://discuss.elastic.co/u/IHateBugs)\ **Replies:** 2\ **Last updated:** [July 25, 2017, 7:39am UTC](https://discuss.elastic.co/t/logstash-local-plugin-installation-fails-if-network-connected-without-internet/93590 "2017-07-25T07:39:09Z")
installing a local plugin in logstash only works with internet connection or without any network connection. When trying to do it in a local network without internet i get the following error message to install the pl… --- ## [Dealing with repeated upstream fields in a Nginx log](https://discuss.elastic.co/t/dealing-with-repeated-upstream-fields-in-a-nginx-log/91352)
**Author:** [@Simon\_Oxwell](https://discuss.elastic.co/u/Simon_Oxwell)\ **Replies:** 1\ **Last updated:** [July 25, 2017, 1:53am UTC](https://discuss.elastic.co/t/dealing-with-repeated-upstream-fields-in-a-nginx-log/91352 "2017-07-25T01:53:50Z")
Hi, I'm trying to parse Nginx logs where Nginx is acting as a front-side proxy to multiple upstream servers in such a way that a failure for one upstream server to respond will cause Nginx to try another server. As suc… --- ## [Imap plugin in-depth](https://discuss.elastic.co/t/imap-plugin-in-depth/94019)
**Author:** [@seanziee](https://discuss.elastic.co/u/seanziee)\ **Replies:** 1\ **Last updated:** [July 25, 2017, 12:09am UTC](https://discuss.elastic.co/t/imap-plugin-in-depth/94019 "2017-07-25T00:09:54Z")
Hi, I've been successfully using the IMAP plugin for the past few months and now I'm doing some tweaking and would love to have some insight on the inner workings. These questions originate from using an office365 ac… --- ## [Finding Patterns - Logstash 5.5](https://discuss.elastic.co/t/finding-patterns-logstash-5-5/94394)
**Author:** [@jukeboxhero85](https://discuss.elastic.co/u/jukeboxhero85)\ **Replies:** 1\ **Last updated:** [July 25, 2017, 12:07am UTC](https://discuss.elastic.co/t/finding-patterns-logstash-5-5/94394 "2017-07-25T00:07:58Z")
Hi, I'm trying to send data from some logs I have of type FILE, this are created as a big string separated by "|" I checked another post with a similar issue but could not figure out what it's happening. Test configura… --- ## [Logstash-filter-verifier vs grok debugger differences](https://discuss.elastic.co/t/logstash-filter-verifier-vs-grok-debugger-differences/94389)
**Author:** [@csobchuk](https://discuss.elastic.co/u/csobchuk)\ **Replies:** 1\ **Last updated:** [July 24, 2017, 11:04pm UTC](https://discuss.elastic.co/t/logstash-filter-verifier-vs-grok-debugger-differences/94389 "2017-07-24T23:04:13Z")
Hi I am using logstash 5.2 and the latest logstash-filter-verifier and am noticing a difference between the output of the logstash-filter-verifier and what I submitted to the grok debugger. Grok Debugger input: "2017-0… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=938) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=940)