# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=703 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 704 --- ## [Logstash error with Kinesis plugin only on an account](https://discuss.elastic.co/t/logstash-error-with-kinesis-plugin-only-on-an-account/143903)
**Author:** [@thomasdang](https://discuss.elastic.co/u/thomasdang)\ **Replies:** 0\ **Last updated:** [August 10, 2018, 4:31pm UTC](https://discuss.elastic.co/t/logstash-error-with-kinesis-plugin-only-on-an-account/143903 "2018-08-10T16:31:06Z")
Hi Experts, I'm running Logstash 6.3.2 on Windows server, and have successfully installed the Kinesis plugin. The installation directory is C:\\Logstash... I can see the Kinesis plugin (logstash-input-kinesis (2.0.7)) w… --- ## [Create separate documents from xml file](https://discuss.elastic.co/t/create-separate-documents-from-xml-file/144147)
**Author:** [@safaa](https://discuss.elastic.co/u/safaa)\ **Replies:** 2\ **Last updated:** [August 13, 2018, 11:15am UTC](https://discuss.elastic.co/t/create-separate-documents-from-xml-file/144147 "2018-08-13T11:15:36Z")
Hi, I have an xml file , and want to parse it to logstash . is it possible to create a separate documents for each tag ? here is my xml file: \ \ \Value1\ \ Value2 \ **Author:** [@Marco\_Strambelli](https://discuss.elastic.co/u/Marco_Strambelli)\ **Replies:** 2\ **Last updated:** [August 13, 2018, 10:32am UTC](https://discuss.elastic.co/t/unexpected-behaviour-for-exec-input-plugin/143664 "2018-08-13T10:32:22Z") The curl command returns empty message. input { exec { command =\> "curl -s -I \" interval =\> 30 } } --- ## [Cidr plugin just generate warnings](https://discuss.elastic.co/t/cidr-plugin-just-generate-warnings/141776)
**Author:** [@maxf](https://discuss.elastic.co/u/maxf)\ **Replies:** 5\ **Last updated:** [August 13, 2018, 10:22am UTC](https://discuss.elastic.co/t/cidr-plugin-just-generate-warnings/141776 "2018-08-13T10:22:15Z")
Hello. I've got a trouble with cidr plugin. Im trying to use it this way: if (\[maliciousIP\] == 'false' and \[target\_host\]=~ /^(\[0-9\]{1,3}.){3}(\[0-9\]{1,3})/) { cidr { address =\> \["%{target\_host}"\] network\_path =\> '… --- ## [Logstash plugin interface](https://discuss.elastic.co/t/logstash-plugin-interface/144137)
**Author:** [@Ayden](https://discuss.elastic.co/u/Ayden)\ **Replies:** 1\ **Last updated:** [August 13, 2018, 9:51am UTC](https://discuss.elastic.co/t/logstash-plugin-interface/144137 "2018-08-13T09:51:34Z")
Hello. I'm documenting my elasticsearch + logstash solution and I was wondering how the connection between logstash and its plugins work. Do the plugins implement any interface? Which? Or is this via HTTP? To clarify … --- ## [Location](https://discuss.elastic.co/t/location/141391)
**Author:** [@Arun\_Murugappan](https://discuss.elastic.co/u/Arun_Murugappan)\ **Replies:** 8\ **Last updated:** [August 13, 2018, 9:04am UTC](https://discuss.elastic.co/t/location/141391 "2018-08-13T09:04:07Z")
What is the location of logstash.yml --- ## [Logstash s3 backup directory structure](https://discuss.elastic.co/t/logstash-s3-backup-directory-structure/144112)
**Author:** [@bradmunz](https://discuss.elastic.co/u/bradmunz)\ **Replies:** 0\ **Last updated:** [August 13, 2018, 9:00am UTC](https://discuss.elastic.co/t/logstash-s3-backup-directory-structure/144112 "2018-08-13T09:00:08Z")
It looks like the s3 input plugin just dumps processed files into the root of the backup\_to\_dir specified. Is there anyway the plugin can keep the original file structure from S3 when backing up the file? What if there … --- ## [Monitoring User for Logstash](https://discuss.elastic.co/t/monitoring-user-for-logstash/142290)
**Author:** [@toni](https://discuss.elastic.co/u/toni)\ **Replies:** 6\ **Last updated:** [August 13, 2018, 8:36am UTC](https://discuss.elastic.co/t/monitoring-user-for-logstash/142290 "2018-08-13T08:36:33Z")
Hi guys, I took over an elastic cluster with 3 nodes. All of the nodes have x-pack installed. What I'm trying to do is enable pipelines and monitoring for logstash through kibana. The pipeline configuration in logstash… --- ## [Cloned Logstash installation identity overlapping in Monitoring](https://discuss.elastic.co/t/cloned-logstash-installation-identity-overlapping-in-monitoring/144105)
**Author:** [@hpniemitube](https://discuss.elastic.co/u/hpniemitube)\ **Replies:** 0\ **Last updated:** [August 13, 2018, 8:13am UTC](https://discuss.elastic.co/t/cloned-logstash-installation-identity-overlapping-in-monitoring/144105 "2018-08-13T08:13:31Z")
Just installed 2 new logstash-nodes, cloned them in vmware. Hostnames, ip's and nodenames in logstash.yml are ok, but in Kibana monitoring, these new nodes are overlapping oddly in logstash monitoring. There is somethim… --- ## [Problem with replace some char using RegExp. under mutate/gsub](https://discuss.elastic.co/t/problem-with-replace-some-char-using-regexp-under-mutate-gsub/143273)
**Author:** [@richard20427176](https://discuss.elastic.co/u/richard20427176)\ **Replies:** 4\ **Last updated:** [August 13, 2018, 7:23am UTC](https://discuss.elastic.co/t/problem-with-replace-some-char-using-regexp-under-mutate-gsub/143273 "2018-08-13T07:23:28Z")
Hi Guys, I meet some problem when I use logstash to parse some data from the nginx logs. I have the log content as below: {"reqtime":"06/Aug/2018:09:02:17 +0800", "reqbody":"------WebKitFormBoundaryM2dvVjK2xtgaOP6a\\r\\… --- ## [System hangs : Sending Logstash's logs to /var/log/logstash which is now configured via log4j2.properties](https://discuss.elastic.co/t/system-hangs-sending-logstashs-logs-to-var-log-logstash-which-is-now-configured-via-log4j2-properties/144087)
**Author:** [@Sainu](https://discuss.elastic.co/u/Sainu)\ **Replies:** 0\ **Last updated:** [August 13, 2018, 7:10am UTC](https://discuss.elastic.co/t/system-hangs-sending-logstashs-logs-to-var-log-logstash-which-is-now-configured-via-log4j2-properties/144087 "2018-08-13T07:10:02Z")
I have used the following command , bin/logstash -f "/etc/logstash/conf.d/first-pipeline.conf" --path.settings=/etc/logstash However the system just shows the following message and then nothing happens Sending Logstas… --- ## [Problem with elastic template in Logstash when indexing](https://discuss.elastic.co/t/problem-with-elastic-template-in-logstash-when-indexing/143751)
**Author:** [@varun1992](https://discuss.elastic.co/u/varun1992)\ **Replies:** 12\ **Last updated:** [August 13, 2018, 6:15am UTC](https://discuss.elastic.co/t/problem-with-elastic-template-in-logstash-when-indexing/143751 "2018-08-13T06:15:32Z")
i have a working index in Logstash version 5.x but its not working in version 6.3.2 date fields are not reading as date. it's reading as keyword please help { "template" : "index\_i", "settings": { … --- ## [XML parsing logstash](https://discuss.elastic.co/t/xml-parsing-logstash/143798)
**Author:** [@Josh\_Vj](https://discuss.elastic.co/u/Josh_Vj)\ **Replies:** 3\ **Last updated:** [August 13, 2018, 6:11am UTC](https://discuss.elastic.co/t/xml-parsing-logstash/143798 "2018-08-13T06:11:12Z")
Hi all, I am currently using file beat to export all the below messages from application logs (XML) to ECS. But i like to parse the xml so only few required details are viewed in ECS. I like to get the value of below t… --- ## [Error=\>"translation missing:](https://discuss.elastic.co/t/error-translation-missing/144010)
**Author:** [@Gosforth](https://discuss.elastic.co/u/Gosforth)\ **Replies:** 3\ **Last updated:** [August 12, 2018, 8:57pm UTC](https://discuss.elastic.co/t/error-translation-missing/144010 "2018-08-12T20:57:51Z")
logstash config: input { file { path =\> "C:/Pentaho/Workspace/ElasticSearch/logstash/in/sprzedaz.csv" start\_position =\> "beginning" sincedb\_path =\> "nul" } } filter{ csv{ separator =\> ";" columns =\> \["SF\_type"… --- ## [How can we know the event sent per second](https://discuss.elastic.co/t/how-can-we-know-the-event-sent-per-second/144006)
**Author:** [@frank\_rib](https://discuss.elastic.co/u/frank_rib)\ **Replies:** 1\ **Last updated:** [August 11, 2018, 10:38pm UTC](https://discuss.elastic.co/t/how-can-we-know-the-event-sent-per-second/144006 "2018-08-11T22:38:40Z")
hello evry body i want to know if there is a method to know the eps (event sent per second) of elastic stack --- ## [Calculate difference from row values instead columns](https://discuss.elastic.co/t/calculate-difference-from-row-values-instead-columns/143978)
**Author:** [@rkhapre](https://discuss.elastic.co/u/rkhapre)\ **Replies:** 1\ **Last updated:** [August 11, 2018, 3:47pm UTC](https://discuss.elastic.co/t/calculate-difference-from-row-values-instead-columns/143978 "2018-08-11T15:47:19Z")
Hi All How can i Calculate difference from row values instead columns. Below is the example Below Data is available in csv file Month Amount Type Value Jan Type A 150 Jan Type B 100 Jan Type C 20 Feb Ty… --- ## [Logstash kafka output plugin](https://discuss.elastic.co/t/logstash-kafka-output-plugin/143817)
**Author:** [@jogoinar10](https://discuss.elastic.co/u/jogoinar10)\ **Replies:** 4\ **Last updated:** [August 11, 2018, 9:38am UTC](https://discuss.elastic.co/t/logstash-kafka-output-plugin/143817 "2018-08-11T09:38:41Z")
Is there anyone here tried to use logstash to push data from elasticserach to kafka? --- ## [Split a csv fied into nested array](https://discuss.elastic.co/t/split-a-csv-fied-into-nested-array/143893)
**Author:** [@Orange](https://discuss.elastic.co/u/Orange)\ **Replies:** 2\ **Last updated:** [August 11, 2018, 2:13am UTC](https://discuss.elastic.co/t/split-a-csv-fied-into-nested-array/143893 "2018-08-11T02:13:39Z")
hello,I use logstash to import csv to elasticsearch, my question is that one of the fields contains the string formatted as \[{"level1":aa,"level2":bb},{"level1":aaa,"level2",bbb},...\],The field, is array.and i need to p… --- ## [Logstash wait input Redis clients](https://discuss.elastic.co/t/logstash-wait-input-redis-clients/143943)
**Author:** [@vld53](https://discuss.elastic.co/u/vld53)\ **Replies:** 0\ **Last updated:** [August 10, 2018, 9:03pm UTC](https://discuss.elastic.co/t/logstash-wait-input-redis-clients/143943 "2018-08-10T21:03:58Z")
Hello! I have the follow issue. Logstash takes logs from Redis only when Redis has incoming clients. Redis is used like input plugin from Logstash. Other words if Redis doesn't have any clients then Logstash doesn't take… --- ## [Logstash is Unable to Emit Events after Setting Up X-Pack Security](https://discuss.elastic.co/t/logstash-is-unable-to-emit-events-after-setting-up-x-pack-security/142974)
**Author:** [@tvoll](https://discuss.elastic.co/u/tvoll)\ **Replies:** 4\ **Last updated:** [August 10, 2018, 9:00pm UTC](https://discuss.elastic.co/t/logstash-is-unable-to-emit-events-after-setting-up-x-pack-security/142974 "2018-08-10T21:00:28Z")
Running 6.3.2 and everything seems to be working with security, except that I am no longer receiving events from Logstash. I verified from my machines running Filebeat that they are able to send logs out to Logstash, but… --- ## [Logstash Geoip filter with Packetbeat](https://discuss.elastic.co/t/logstash-geoip-filter-with-packetbeat/142985)
**Author:** [@jkevan](https://discuss.elastic.co/u/jkevan)\ **Replies:** 4\ **Last updated:** [August 10, 2018, 6:39pm UTC](https://discuss.elastic.co/t/logstash-geoip-filter-with-packetbeat/142985 "2018-08-10T18:39:07Z")
Hi everyone, to start, I'm running Logstash, elasticsearch and kibana 6.3.1 and multiple beats that are all on 6.3.2. I have loaded the various default beats index templates as outlined in https://www.elastic.co/guide/e… --- ## [Logstash parse out \\r and \\t entries](https://discuss.elastic.co/t/logstash-parse-out-r-and-t-entries/143888)
**Author:** [@motts](https://discuss.elastic.co/u/motts)\ **Replies:** 10\ **Last updated:** [August 10, 2018, 5:54pm UTC](https://discuss.elastic.co/t/logstash-parse-out-r-and-t-entries/143888 "2018-08-10T17:54:33Z")
Hello there, I am having some issues with parsing out some logstash data. In my data, I have \\r's and \\t's that are floating around that I need to get rid of. \\tLogon ID:\\t\\t(0x2,0x50722C31)\\r I started trying to remo… --- ## [How to sent application/xml request body in a http\_poller endpoint](https://discuss.elastic.co/t/how-to-sent-application-xml-request-body-in-a-http-poller-endpoint/143824)
**Author:** [@Jishnu\_Prathap](https://discuss.elastic.co/u/Jishnu_Prathap)\ **Replies:** 2\ **Last updated:** [August 10, 2018, 5:37pm UTC](https://discuss.elastic.co/t/how-to-sent-application-xml-request-body-in-a-http-poller-endpoint/143824 "2018-08-10T17:37:21Z")
I am having difficulties setting XML in request body to a http\_poller endpoint. When I set the request body in http\_poller endpoint is showing error My conf file input { http\_poller { schedule =\> { cron =\> "\* \* … --- ## [LS 6.3.2 Updating document with full content from multiple log files](https://discuss.elastic.co/t/ls-6-3-2-updating-document-with-full-content-from-multiple-log-files/143530)
**Author:** [@ld\_pvl](https://discuss.elastic.co/u/ld_pvl)\ **Replies:** 5\ **Last updated:** [August 10, 2018, 5:36pm UTC](https://discuss.elastic.co/t/ls-6-3-2-updating-document-with-full-content-from-multiple-log-files/143530 "2018-08-10T17:36:01Z")
Hi Team, I have an application that produces lines in a log file with a unique UUID in its file name. When the log file grows to a certain size it gets rolled. The files are not updated after they are rolled. Eg: UUID\_… --- ## [LS 6.3.2 Appending data to a field of an existing document](https://discuss.elastic.co/t/ls-6-3-2-appending-data-to-a-field-of-an-existing-document/143878)
**Author:** [@ld\_pvl](https://discuss.elastic.co/u/ld_pvl)\ **Replies:** 5\ **Last updated:** [August 10, 2018, 4:58pm UTC](https://discuss.elastic.co/t/ls-6-3-2-appending-data-to-a-field-of-an-existing-document/143878 "2018-08-10T16:58:17Z")
Hi Team, Does Logstash support appending to / extending a field's data of an existing event given a deterministic document ID? Cheers, --- ## [Grok Pattern Help Please](https://discuss.elastic.co/t/grok-pattern-help-please/143905)
**Author:** [@jenyphur](https://discuss.elastic.co/u/jenyphur)\ **Replies:** 3\ **Last updated:** [August 10, 2018, 4:57pm UTC](https://discuss.elastic.co/t/grok-pattern-help-please/143905 "2018-08-10T16:57:43Z")
Hi there, pretty new to Logstash and Grok patterns. I am trying to parse out a custom log file and here is what I'm trying to do. Log Sample; \<15\>1 2018-08-09T07:37:48.306-05:00 MacBook-Pro.local ReaccomTask - Audit \[… --- ## [Not in tags condition](https://discuss.elastic.co/t/not-in-tags-condition/143885)
**Author:** [@chandra0651](https://discuss.elastic.co/u/chandra0651)\ **Replies:** 3\ **Last updated:** [August 10, 2018, 4:54pm UTC](https://discuss.elastic.co/t/not-in-tags-condition/143885 "2018-08-10T16:54:41Z")
Hello I am trying to ingest into Elasticsearch if the documents dont have couple of tags, but some how still the documents are getting persisted if the tags present. below is my code .The tags with "\_split\_type\_failure" … --- ## [Try to parse different log formats in one file?](https://discuss.elastic.co/t/try-to-parse-different-log-formats-in-one-file/143385)
**Author:** [@javadevmtl](https://discuss.elastic.co/u/javadevmtl)\ **Replies:** 3\ **Last updated:** [August 10, 2018, 3:57pm UTC](https://discuss.elastic.co/t/try-to-parse-different-log-formats-in-one-file/143385 "2018-08-10T15:57:03Z")
Hi, I have situation where two different formats end up in the same log file. I have the standard log4j format and that parses fine. But I also get google glog format. The glog looks like: I0807 19:07:40.477815 13465… --- ## [Logstrash is not connection to Elasticsearch](https://discuss.elastic.co/t/logstrash-is-not-connection-to-elasticsearch/143882)
**Author:** [@Yolanda\_Goncalves](https://discuss.elastic.co/u/Yolanda_Goncalves)\ **Replies:** 1\ **Last updated:** [August 10, 2018, 2:44pm UTC](https://discuss.elastic.co/t/logstrash-is-not-connection-to-elasticsearch/143882 "2018-08-10T14:44:37Z")
I'm getting the follow error while trying to create a new index at elasticsearch. \[root@wazuh-server logstash\]# bin/logstash -f /usr/share/logstash/logstashnessus.config WARNING: Could not find logstash.yml which is typ… --- ## [X-Pack monitoring shows one of all of the configured Logstash nodes](https://discuss.elastic.co/t/x-pack-monitoring-shows-one-of-all-of-the-configured-logstash-nodes/142760)
**Author:** [@aaber](https://discuss.elastic.co/u/aaber)\ **Replies:** 1\ **Last updated:** [August 10, 2018, 2:42pm UTC](https://discuss.elastic.co/t/x-pack-monitoring-shows-one-of-all-of-the-configured-logstash-nodes/142760 "2018-08-10T14:42:39Z")
When you configure X-Pack to monitor multiple Logstash instances make sure each instance has a unique uuid. We copied AMIs in AWS to create multiple Logstash instances and all of the instances had the same uuid which co… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=702) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=704)