# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=842 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 843 --- ## [Logstash-filter-virustotal](https://discuss.elastic.co/t/logstash-filter-virustotal/115270)
**Author:** [@anji](https://discuss.elastic.co/u/anji)\ **Replies:** 0\ **Last updated:** [January 12, 2018, 10:26am UTC](https://discuss.elastic.co/t/logstash-filter-virustotal/115270 "2018-01-12T10:26:06Z")
Hello Everyone, I am trying to install Logstash-filter-virustotal plugin but i am getting some errors i am using windows OS with ELK stake- 6.1.1 can anyone help? \[filter error\] Thanks, --- ## [Logstash\_parsing](https://discuss.elastic.co/t/logstash-parsing/115249)
**Author:** [@cilzzz](https://discuss.elastic.co/u/cilzzz)\ **Replies:** 0\ **Last updated:** [January 12, 2018, 9:19am UTC](https://discuss.elastic.co/t/logstash-parsing/115249 "2018-01-12T09:19:46Z")
Hello everyone, i have an export of a script below that i need to parse through logstash. Job Name : save\_start\_config-sftp Job Status: Success (0) Schedule Name : daily\_save\_start\_config Us… --- ## [Problem with writing to created index](https://discuss.elastic.co/t/problem-with-writing-to-created-index/115092)
**Author:** [@antimion](https://discuss.elastic.co/u/antimion)\ **Replies:** 2\ **Last updated:** [January 12, 2018, 7:29am UTC](https://discuss.elastic.co/t/problem-with-writing-to-created-index/115092 "2018-01-12T07:29:51Z")
Hello i have used lates ES with latest logstash, and i have this kind of problem. When i try to write my logs to ES i receive this error. i have old ES5, and don't have any problem with configuration files. This is ful… --- ## [ConfigurationError Expected one of #, {, } at line 13, column 121](https://discuss.elastic.co/t/configurationerror-expected-one-of-at-line-13-column-121/114658)
**Author:** [@yuganshu4042](https://discuss.elastic.co/u/yuganshu4042)\ **Replies:** 8\ **Last updated:** [January 12, 2018, 6:26am UTC](https://discuss.elastic.co/t/configurationerror-expected-one-of-at-line-13-column-121/114658 "2018-01-12T06:26:43Z")
/\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*Filtered Data need to send on elasticSearch \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*/ 15:33:43.214 \[tomcat-http--1\] DEBUG com.txxixxd.cc.gatx.rePoinx - POSRequest {"enterpriseId":"5a4dd83a6c085324048fbbd1","storeId":"4042","… --- ## [Sourceloader - No configuration found in the configured sources error in logstash](https://discuss.elastic.co/t/sourceloader-no-configuration-found-in-the-configured-sources-error-in-logstash/112789)
**Author:** [@vaidehi9039](https://discuss.elastic.co/u/vaidehi9039)\ **Replies:** 3\ **Last updated:** [January 12, 2018, 6:14am UTC](https://discuss.elastic.co/t/sourceloader-no-configuration-found-in-the-configured-sources-error-in-logstash/112789 "2018-01-12T06:14:24Z")
Continuing the discussion from Logstash 6.0.0 Command Line error: --- ## [Seperating fields in a string](https://discuss.elastic.co/t/seperating-fields-in-a-string/115193)
**Author:** [@Sjaak01](https://discuss.elastic.co/u/Sjaak01)\ **Replies:** 0\ **Last updated:** [January 12, 2018, 12:27am UTC](https://discuss.elastic.co/t/seperating-fields-in-a-string/115193 "2018-01-12T00:27:59Z")
Hello, I will attempt this once more, hopefully somebody can help me out because I'm breaking my head over this one. I'm using the throttle filter and I need to combine a string of fields I want to throttle. throttle … --- ## [Resume reading Logstash](https://discuss.elastic.co/t/resume-reading-logstash/115021)
**Author:** [@Andualem\_Tilahun](https://discuss.elastic.co/u/Andualem_Tilahun)\ **Replies:** 3\ **Last updated:** [January 11, 2018, 11:40pm UTC](https://discuss.elastic.co/t/resume-reading-logstash/115021 "2018-01-11T23:40:40Z")
i am using a logstash server to read a data from my log file but suddenly my computer restart. i want the logstash server to start up Automatic when the computer start's and to resume reading where it stopped (not from t… --- ## [How to convert "/" separated string to time data](https://discuss.elastic.co/t/how-to-convert-separated-string-to-time-data/115109)
**Author:** [@Sava](https://discuss.elastic.co/u/Sava)\ **Replies:** 0\ **Last updated:** [January 11, 2018, 2:29pm UTC](https://discuss.elastic.co/t/how-to-convert-separated-string-to-time-data/115109 "2018-01-11T14:29:00Z")
I want to convert string to timestamp(ISO8601) data My message looks like: /2018/01/11/08/39/02 How to convert /2018/01/11/08/39/02 to 2018-01-11 08:39:02? Thank you. --- ## [Logstash-input-imap plugin error](https://discuss.elastic.co/t/logstash-input-imap-plugin-error/115166)
**Author:** [@sliu](https://discuss.elastic.co/u/sliu)\ **Replies:** 0\ **Last updated:** [January 11, 2018, 7:52pm UTC](https://discuss.elastic.co/t/logstash-input-imap-plugin-error/115166 "2018-01-11T19:52:07Z")
We use logstash 5.4.1. Following the example here: https://qbox.io/blog/indexing-emails-to-elasticsearch-logstash-imap, I got this error: \[2018-01-11T14:22:38,062\]\[ERROR\]\[logstash.pipeline \] A plugin had an unre… --- ## [Logstash Email Output Plugin not working](https://discuss.elastic.co/t/logstash-email-output-plugin-not-working/114964)
**Author:** [@yinzhi7](https://discuss.elastic.co/u/yinzhi7)\ **Replies:** 3\ **Last updated:** [January 11, 2018, 7:42pm UTC](https://discuss.elastic.co/t/logstash-email-output-plugin-not-working/114964 "2018-01-11T19:42:26Z")
Here is my email setting email { to =\> "lovework@happywork.com" from =\> "lovework@happywork.com" subject =\> "Alert" body =\> "Content:\\n%{message}" domain =\> "outlook.corp3.happywork.com" port =\> 25 } Domain is d… --- ## [Filter:drop performance varies by 200x](https://discuss.elastic.co/t/filter-drop-performance-varies-by-200x/115154)
**Author:** [@Supermathie](https://discuss.elastic.co/u/Supermathie)\ **Replies:** 0\ **Last updated:** [January 11, 2018, 6:32pm UTC](https://discuss.elastic.co/t/filter-drop-performance-varies-by-200x/115154 "2018-01-11T18:32:26Z")
TL;DR: for no apparent reason, logstash will take as low as 0.24ms to drop events, or up to 56ms (a 200x difference!). I've been trying to track down a performance issue with our logstash setup. We're getting huge backl… --- ## [Remove json array elements that are objects](https://discuss.elastic.co/t/remove-json-array-elements-that-are-objects/115022)
**Author:** [@Jason\_Brooks](https://discuss.elastic.co/u/Jason_Brooks)\ **Replies:** 5\ **Last updated:** [January 11, 2018, 6:15pm UTC](https://discuss.elastic.co/t/remove-json-array-elements-that-are-objects/115022 "2018-01-11T18:15:06Z")
Hello, I have the following (shortened) json array: "identifiers": \[{ "name": "ENCODED\_RECIPIENT\_ID", "value": "LTMxNjcxMTc4NjQS1", "isOriginal": false }, { "name": "ENCODED\_RECIPIENT\_ID", "value": "LTMxNjcxODA5NzQS1",… --- ## [Fresh install - high CPU load](https://discuss.elastic.co/t/fresh-install-high-cpu-load/114552)
**Author:** [@dmnc](https://discuss.elastic.co/u/dmnc)\ **Replies:** 2\ **Last updated:** [January 11, 2018, 5:01pm UTC](https://discuss.elastic.co/t/fresh-install-high-cpu-load/114552 "2018-01-11T17:01:45Z")
Hi, I'm completely new in Elastic Search. I've installed E+L+K (logstash 6.0.1) on the officially supported Debian 9.3 following the official documentation. After installation I let it be and did nothing with it, today I… --- ## [How to use grok patterns coming from event data](https://discuss.elastic.co/t/how-to-use-grok-patterns-coming-from-event-data/115137)
**Author:** [@whites11](https://discuss.elastic.co/u/whites11)\ **Replies:** 0\ **Last updated:** [January 11, 2018, 4:36pm UTC](https://discuss.elastic.co/t/how-to-use-grok-patterns-coming-from-event-data/115137 "2018-01-11T16:36:55Z")
I have an ELK stack deployed on kubernetes used to collect containers' data. Among all the rest, it is using a grok filter to parse the actual log line based on a pattern. My wish is to be able to setup this pattern by … --- ## [Logstash Invalid Character for UTF-16/Unicode encoding](https://discuss.elastic.co/t/logstash-invalid-character-for-utf-16-unicode-encoding/56702)
**Author:** [@sreeram](https://discuss.elastic.co/u/sreeram)\ **Replies:** 6\ **Last updated:** [January 11, 2018, 4:35pm UTC](https://discuss.elastic.co/t/logstash-invalid-character-for-utf-16-unicode-encoding/56702 "2018-01-11T16:35:35Z")
Hi, Below is the logstash Input plugin configured to read logs generated by my Windows Application, input { file { path =\> \["D:/ELK/LoggerTestApp/Server/Logger/GetAssetPointerById/\*\*/\*.txt"\] codec =\> plain { charset… --- ## [Send file logs to logstash](https://discuss.elastic.co/t/send-file-logs-to-logstash/115135)
**Author:** [@dinaWork](https://discuss.elastic.co/u/dinaWork)\ **Replies:** 0\ **Last updated:** [January 11, 2018, 4:27pm UTC](https://discuss.elastic.co/t/send-file-logs-to-logstash/115135 "2018-01-11T16:27:48Z")
I have files in GCS, each file contains json lines file example: {"msg": "hello world", "level": "info", "timestamp": "2017-09-01T00:00"} {"msg": "some error", "level": "error", "timestamp": "2017-09-02T00:00"} {"msg"… --- ## [Odd behavior when logstash finished pipeline](https://discuss.elastic.co/t/odd-behavior-when-logstash-finished-pipeline/114612)
**Author:** [@arisbanach](https://discuss.elastic.co/u/arisbanach)\ **Replies:** 3\ **Last updated:** [January 11, 2018, 2:32pm UTC](https://discuss.elastic.co/t/odd-behavior-when-logstash-finished-pipeline/114612 "2018-01-11T14:32:04Z")
I have two pipelines defined in pipelines.yml: one for my main pipeline which imports data from jdbc input, and another for the dead\_letter\_queue input. When I run Logstash when it gets to the end of the database, it se… --- ## [How to replace existing index after reindexing using Logstash](https://discuss.elastic.co/t/how-to-replace-existing-index-after-reindexing-using-logstash/115100)
**Author:** [@barlask](https://discuss.elastic.co/u/barlask)\ **Replies:** 0\ **Last updated:** [January 11, 2018, 1:50pm UTC](https://discuss.elastic.co/t/how-to-replace-existing-index-after-reindexing-using-logstash/115100 "2018-01-11T13:50:49Z")
Hi, We are using Logstash mysql\_jdbc connector to fetch data from Database and push them to Elasticsearch index. We scheduled a job working every minute for delta imports where Logstash queries DB for changes in the las… --- ## [Jar dependencies - editing bundler.gemspec file](https://discuss.elastic.co/t/jar-dependencies-editing-bundler-gemspec-file/115088)
**Author:** [@Milad\_Hamid](https://discuss.elastic.co/u/Milad_Hamid)\ **Replies:** 0\ **Last updated:** [January 11, 2018, 12:34pm UTC](https://discuss.elastic.co/t/jar-dependencies-editing-bundler-gemspec-file/115088 "2018-01-11T12:34:13Z")
Hi, I am getting this error when running logstash 6.1.1: Gem::LoadError: You have already activated jar-dependencies 0.3.12, but your Gemfile requires jar-dependencies 0.3.11. Prepending \`bundle exec\` to your c… --- ## [Udp input with multiline codec - Exception in inputworker](https://discuss.elastic.co/t/udp-input-with-multiline-codec-exception-in-inputworker/115079)
**Author:** [@breezzz](https://discuss.elastic.co/u/breezzz)\ **Replies:** 2\ **Last updated:** [January 11, 2018, 12:32pm UTC](https://discuss.elastic.co/t/udp-input-with-multiline-codec-exception-in-inputworker/115079 "2018-01-11T12:32:02Z")
Logstash falls every 2-3 days. Is there anything i can do about this? "@timestamp"=\>2018-01-10T16:30:00.523Z, "message"=\>"", "tags"=\>\["\_dissectfailure"\]}} \[2018-01-10T16:30:02,636\]\[WARN \]\[org.logstash.dissect.Dissector\]… --- ## [Use term "id“ in input plugin, but "id" is not output and not shown in Kibana](https://discuss.elastic.co/t/use-term-id-in-input-plugin-but-id-is-not-output-and-not-shown-in-kibana/115038)
**Author:** [@fan-chao](https://discuss.elastic.co/u/fan-chao)\ **Replies:** 2\ **Last updated:** [January 11, 2018, 11:57am UTC](https://discuss.elastic.co/t/use-term-id-in-input-plugin-but-id-is-not-output-and-not-shown-in-kibana/115038 "2018-01-11T11:57:20Z")
Hi, We added common option "id" in input plugin. But this option is not output and it is not shown up in Kibana. The config is as below. input { udp { id =\> "my\_plugin\_id" port =\> 5040 codec =\> "line" type =\> "MA… --- ## [Same config file does produce result](https://discuss.elastic.co/t/same-config-file-does-produce-result/115041)
**Author:** [@alpha\_gamma](https://discuss.elastic.co/u/alpha_gamma)\ **Replies:** 1\ **Last updated:** [January 11, 2018, 11:50am UTC](https://discuss.elastic.co/t/same-config-file-does-produce-result/115041 "2018-01-11T11:50:22Z")
The config file which was working earlier not working now,i haven't altered anything ,why it happens.what need to be done --- ## [Saving a string from an array of strings as a field](https://discuss.elastic.co/t/saving-a-string-from-an-array-of-strings-as-a-field/114915)
**Author:** [@rjm61](https://discuss.elastic.co/u/rjm61)\ **Replies:** 1\ **Last updated:** [January 11, 2018, 11:47am UTC](https://discuss.elastic.co/t/saving-a-string-from-an-array-of-strings-as-a-field/114915 "2018-01-11T11:47:39Z")
I am using a kv filter to split an event message of comma separated values on the coma character to create a string array fa and saving a specific array element as a field as follows filter { kv { # Split comma separa… --- ## [Make optional character part of a defined regex (HOSTNAME)?](https://discuss.elastic.co/t/make-optional-character-part-of-a-defined-regex-hostname/114787)
**Author:** [@sdndude](https://discuss.elastic.co/u/sdndude)\ **Replies:** 1\ **Last updated:** [January 11, 2018, 11:16am UTC](https://discuss.elastic.co/t/make-optional-character-part-of-a-defined-regex-hostname/114787 "2018-01-11T11:16:35Z")
I couldn't really figure out a good way to explain this in the topic heading. I have been using the following grok filter, and up until a couple days ago, it was working fine. grok { match =\> {"message\_id" =\> "%… --- ## [Logstash Windows events pattern](https://discuss.elastic.co/t/logstash-windows-events-pattern/115057)
**Author:** [@jonviedma](https://discuss.elastic.co/u/jonviedma)\ **Replies:** 1\ **Last updated:** [January 11, 2018, 10:47am UTC](https://discuss.elastic.co/t/logstash-windows-events-pattern/115057 "2018-01-11T10:47:24Z")
Hi everyone, I'm pretty new in ELK Stack and I would like to know if is there any grok pattern for windows events. I'm shipping events to ELK stack with Winlogbeat by sending Application, security and system events. --- ## [How to remove double quotes from Field](https://discuss.elastic.co/t/how-to-remove-double-quotes-from-field/115046)
**Author:** [@Saket\_Kumar](https://discuss.elastic.co/u/Saket_Kumar)\ **Replies:** 1\ **Last updated:** [January 11, 2018, 10:31am UTC](https://discuss.elastic.co/t/how-to-remove-double-quotes-from-field/115046 "2018-01-11T10:31:06Z")
I am parsing a CSV which has a column field name and values within double quotes. I have to rename the field such as: mutate {rename =\> { '"ttfb\_ms"' =\> "Time\_to\_First\_Byte\_(ms)" }} And also have to remove double quot… --- ## [Using http\_proxy in the S3 output plugin?](https://discuss.elastic.co/t/using-http-proxy-in-the-s3-output-plugin/112180)
**Author:** [@Patrik\_Iselind](https://discuss.elastic.co/u/Patrik_Iselind)\ **Replies:** 2\ **Last updated:** [January 11, 2018, 9:18am UTC](https://discuss.elastic.co/t/using-http-proxy-in-the-s3-output-plugin/112180 "2018-01-11T09:18:21Z")
I am having issues using the S3 output plugin logstash\_1 | \[2017-12-18T08:37:47,589\]\[ERROR\]\[logstash.outputs.s3 \] Error validating bucket write permissions! {:message=\>"Network is unreachable", :class=\>"S… --- ## [Xml filter: Unable to get parent and child attributes in one row](https://discuss.elastic.co/t/xml-filter-unable-to-get-parent-and-child-attributes-in-one-row/114643)
**Author:** [@robert2](https://discuss.elastic.co/u/robert2)\ **Replies:** 7\ **Last updated:** [January 11, 2018, 8:38am UTC](https://discuss.elastic.co/t/xml-filter-unable-to-get-parent-and-child-attributes-in-one-row/114643 "2018-01-11T08:38:03Z")
Hi Team I am new to Kibana, logstash. I am trying push xml (generated as output of nunit) in logstash. I want to have the xml elements attributes, its Parents attributes and child attributes in the same row in Kibana. … --- ## [Gem::LoadError activated jar-dependencies](https://discuss.elastic.co/t/gem-loaderror-activated-jar-dependencies/114873)
**Author:** [@Milad\_Hamid](https://discuss.elastic.co/u/Milad_Hamid)\ **Replies:** 1\ **Last updated:** [January 11, 2018, 8:03am UTC](https://discuss.elastic.co/t/gem-loaderror-activated-jar-dependencies/114873 "2018-01-11T08:03:04Z")
Hi, I installed logstash 6.1 and when running logstash and pipeline, getting following error message: Gem::LoadError: You have already activated jar-dependencies 0.3.12, but your Gemfile requires jar-dependencies 0.3.1… --- ## [Multiple pipelines : failed to read pipelines yaml file](https://discuss.elastic.co/t/multiple-pipelines-failed-to-read-pipelines-yaml-file/113603)
**Author:** [@Nonow53](https://discuss.elastic.co/u/Nonow53)\ **Replies:** 2\ **Last updated:** [January 11, 2018, 7:22am UTC](https://discuss.elastic.co/t/multiple-pipelines-failed-to-read-pipelines-yaml-file/113603 "2018-01-11T07:22:17Z")
Hello, I try to use multiple piplelines but I always encounter this error without any log to help me to resolve the problem. PS C:\\Users\\agiffard\\logstash-6.1.1\> bin/logstash Sending Logstash's logs to C:/Users/agiffa… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=841) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=843)