# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=865 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 866 --- ## [High pressure pipeline causes orphaned TCP/SSL Connections](https://discuss.elastic.co/t/high-pressure-pipeline-causes-orphaned-tcp-ssl-connections/109828)
**Author:** [@Troy\_Heanssgen](https://discuss.elastic.co/u/Troy_Heanssgen)\ **Replies:** 0\ **Last updated:** [November 30, 2017, 7:38pm UTC](https://discuss.elastic.co/t/high-pressure-pipeline-causes-orphaned-tcp-ssl-connections/109828 "2017-11-30T19:38:47Z")
It appears that Logstash is not adhering to the TLS specification of allowing half-closed connections and is instead orphaning TCP sockets when it cannot send its TCP FIN packet to the client. This can happen when there … --- ## [MS Windows Logstash 5.x bootstrap window unable to terminate](https://discuss.elastic.co/t/ms-windows-logstash-5-x-bootstrap-window-unable-to-terminate/109820)
**Author:** [@cchamberlain](https://discuss.elastic.co/u/cchamberlain)\ **Replies:** 0\ **Last updated:** [November 30, 2017, 7:06pm UTC](https://discuss.elastic.co/t/ms-windows-logstash-5-x-bootstrap-window-unable-to-terminate/109820 "2017-11-30T19:06:12Z")
Hi, In WIndows Powershell I run a file with elasticsearch input and output plugins \*Set-Location E:\* E:\\logstash-5.4.0\\bin\\logstash -f "E:\\mylogstash\\ConfigFiles\\FILE.config" -l "E:\\mylogstash\\logs\\prodcopy" --path.da… --- ## [Logstash conf file unable to read file for csv import](https://discuss.elastic.co/t/logstash-conf-file-unable-to-read-file-for-csv-import/108514)
**Author:** [@shivakv](https://discuss.elastic.co/u/shivakv)\ **Replies:** 7\ **Last updated:** [November 30, 2017, 3:09pm UTC](https://discuss.elastic.co/t/logstash-conf-file-unable-to-read-file-for-csv-import/108514 "2017-11-30T15:09:23Z")
I am trying to get log stash to import my csv data load to elastic search cluster and view in Kibana. Logstash/ES/Kibana instances are all up and running but LS does not create the index. Below is my Logstash.conf file … --- ## [Elasticsearch hosts from configuration file in logstash JMS plugin](https://discuss.elastic.co/t/elasticsearch-hosts-from-configuration-file-in-logstash-jms-plugin/108608)
**Author:** [@roblopes](https://discuss.elastic.co/u/roblopes)\ **Replies:** 2\ **Last updated:** [November 30, 2017, 2:51pm UTC](https://discuss.elastic.co/t/elasticsearch-hosts-from-configuration-file-in-logstash-jms-plugin/108608 "2017-11-30T14:51:01Z")
Hi, How can I configure the logstash configuration to retrieve the hosts (and maybe other parameters) from a configuration file (or at least from a single variable)? The problem is that we want to make automated update… --- ## [Nested XML with Logstash- Recommendation needed](https://discuss.elastic.co/t/nested-xml-with-logstash-recommendation-needed/106333)
**Author:** [@moriol](https://discuss.elastic.co/u/moriol)\ **Replies:** 3\ **Last updated:** [November 30, 2017, 2:38pm UTC](https://discuss.elastic.co/t/nested-xml-with-logstash-recommendation-needed/106333 "2017-11-30T14:38:34Z")
Hello, I am looking for some recommendations, where I have an XML file which is nested and needs to be an input in Logstash. My xml has the following structure: \< subsystem\> \< component\> \< Parameter\> \< /Parameter\> … --- ## [Logstash show sending filter but nothing on elstic](https://discuss.elastic.co/t/logstash-show-sending-filter-but-nothing-on-elstic/109768)
**Author:** [@Yehuda\_Alkrif](https://discuss.elastic.co/u/Yehuda_Alkrif)\ **Replies:** 0\ **Last updated:** [November 30, 2017, 2:19pm UTC](https://discuss.elastic.co/t/logstash-show-sending-filter-but-nothing-on-elstic/109768 "2017-11-30T14:19:21Z")
hi we have system with Logstatsh that read from shared repository once a day (30GB).on the logstash we see that filter is working and logs are proccesd but the info is shown only after few hours on the Elastic??any thou… --- ## [Logstash drop() combinations](https://discuss.elastic.co/t/logstash-drop-combinations/109423)
**Author:** [@mathurin68](https://discuss.elastic.co/u/mathurin68)\ **Replies:** 6\ **Last updated:** [November 30, 2017, 2:04pm UTC](https://discuss.elastic.co/t/logstash-drop-combinations/109423 "2017-11-30T14:04:34Z")
Working with windows event logs, ML 'rare' function and trying to eliminate the 'fake rare' i.e. windows event logs that pop up and it's the same thing over and over again but with different random 'identifiers' in each… --- ## [Issue with pipelines stopping to work](https://discuss.elastic.co/t/issue-with-pipelines-stopping-to-work/109763)
**Author:** [@Enrico\_Maria\_Fusi](https://discuss.elastic.co/u/Enrico_Maria_Fusi)\ **Replies:** 0\ **Last updated:** [November 30, 2017, 1:57pm UTC](https://discuss.elastic.co/t/issue-with-pipelines-stopping-to-work/109763 "2017-11-30T13:57:39Z")
Hello All. I am experiencing an issue on logstash , but I cannot find a solution. Basically I have a TCP listener in input, which sends me line like: \[some garbage\] Info:\[json message\] So what I do is: dissect { … --- ## [Audit Log Logstash Conf not working](https://discuss.elastic.co/t/audit-log-logstash-conf-not-working/109686)
**Author:** [@sedawk](https://discuss.elastic.co/u/sedawk)\ **Replies:** 9\ **Last updated:** [November 30, 2017, 11:26am UTC](https://discuss.elastic.co/t/audit-log-logstash-conf-not-working/109686 "2017-11-30T11:26:20Z")
Hi, I am trying to parse the audit log but I am not able to. This is my conf. input { file { path =\> "/var/log/audit/audit.log" start\_position =\> "beginning" } } filter { grok { match =\> { "messa… --- ## [SIGPIPE is not honoured consistently](https://discuss.elastic.co/t/sigpipe-is-not-honoured-consistently/109719)
**Author:** [@amitavmohanty01](https://discuss.elastic.co/u/amitavmohanty01)\ **Replies:** 0\ **Last updated:** [November 30, 2017, 9:25am UTC](https://discuss.elastic.co/t/sigpipe-is-not-honoured-consistently/109719 "2017-11-30T09:25:46Z")
While running logstash on a remote server from the command line not as a service, I found that when there is a broken pipe, sometimes logstash process is killed and in other cases, the process keeps running on the server… --- ## [Logstash basic grok field types description](https://discuss.elastic.co/t/logstash-basic-grok-field-types-description/109690)
**Author:** [@elasticheart](https://discuss.elastic.co/u/elasticheart)\ **Replies:** 1\ **Last updated:** [November 30, 2017, 8:40am UTC](https://discuss.elastic.co/t/logstash-basic-grok-field-types-description/109690 "2017-11-30T08:40:39Z")
Hi, From this link, we can have a reference at the different types supported in Logstash grok pattern. I am listing down some basic types in it; USERNAME USER INT BASE10NUM NUMBER BASE16NUM BASE16FLOAT POSINT NONNEGIN… --- ## [Persistent queue viewing tool and issue with parsing the records in queue](https://discuss.elastic.co/t/persistent-queue-viewing-tool-and-issue-with-parsing-the-records-in-queue/109699)
**Author:** [@Viachaslau\_Kabak](https://discuss.elastic.co/u/Viachaslau_Kabak)\ **Replies:** 0\ **Last updated:** [November 30, 2017, 7:49am UTC](https://discuss.elastic.co/t/persistent-queue-viewing-tool-and-issue-with-parsing-the-records-in-queue/109699 "2017-11-30T07:49:38Z")
Sometimes seems the logs are changed somewhere and grok freezing ang logstash uses 100% CPU permanently and does not filtering logs. After restart logstash is trying to parse the records in persistent queue but in some … --- ## [Best practices for exclusions in logstash](https://discuss.elastic.co/t/best-practices-for-exclusions-in-logstash/109625)
**Author:** [@sfchrisgleason](https://discuss.elastic.co/u/sfchrisgleason)\ **Replies:** 1\ **Last updated:** [November 30, 2017, 6:26am UTC](https://discuss.elastic.co/t/best-practices-for-exclusions-in-logstash/109625 "2017-11-30T06:26:12Z")
Good afternoon all, I just had a quick question regarding exclusions or drops in logstash. I know you can drop specific messages on the input or output in logstash. I'm wondering what the best practice is for large li… --- ## [What is the meaning of this log message](https://discuss.elastic.co/t/what-is-the-meaning-of-this-log-message/109565)
**Author:** [@Kasun\_Siyambalapitiy](https://discuss.elastic.co/u/Kasun_Siyambalapitiy)\ **Replies:** 7\ **Last updated:** [November 30, 2017, 6:11am UTC](https://discuss.elastic.co/t/what-is-the-meaning-of-this-log-message/109565 "2017-11-30T06:11:21Z")
In my /var/log/logstash/logstash.log.1 has only the following and nothing is getting published from logstash. Please help me to fix this, {:timestamp=\>"2017-11-29T06:07:47.853000+0000", :message=\>"SIGTERM received. Shu… --- ## [Now that support for type has been removed in filebeat 6, how to add filters in logstash config?](https://discuss.elastic.co/t/now-that-support-for-type-has-been-removed-in-filebeat-6-how-to-add-filters-in-logstash-config/109671)
**Author:** [@Vivek\_Menon](https://discuss.elastic.co/u/Vivek_Menon)\ **Replies:** 1\ **Last updated:** [November 30, 2017, 6:20am UTC](https://discuss.elastic.co/t/now-that-support-for-type-has-been-removed-in-filebeat-6-how-to-add-filters-in-logstash-config/109671 "2017-11-30T06:20:21Z")
From what I read, filebeat does not honor document\_type anymore (from version 6 onwards) and all \_type gets hardcoded to doc. I read some people getting around this by adding a field called document\_type. Something like… --- ## [Need help with BIND DNS Pattern](https://discuss.elastic.co/t/need-help-with-bind-dns-pattern/109231)
**Author:** [@Blason](https://discuss.elastic.co/u/Blason)\ **Replies:** 4\ **Last updated:** [November 30, 2017, 3:17am UTC](https://discuss.elastic.co/t/need-help-with-bind-dns-pattern/109231 "2017-11-30T03:17:25Z")
Hi Guys, I am bit stuck up in parsing BIND DNS logs and need help since something is wrong and patterns are not being matched. Here is the log 27-Nov-2017 18:57:19.747 client 10.1.1.10#53338 (ericweb.co.za): query: er… --- ## [Uploading GeoIP data into kibana from logstash](https://discuss.elastic.co/t/uploading-geoip-data-into-kibana-from-logstash/109621)
**Author:** [@I\_like\_dogs](https://discuss.elastic.co/u/I_like_dogs)\ **Replies:** 1\ **Last updated:** [November 29, 2017, 7:00pm UTC](https://discuss.elastic.co/t/uploading-geoip-data-into-kibana-from-logstash/109621 "2017-11-29T19:00:30Z")
I uploaded some honeypot data into kibana which works but now I want to add the GeoIP. I've created the following config file but its not working :frowning: input { file { path =\> "/vagrant/headdionaea.csv" start\_po… --- ## [Retrieving partitionkey via logstash-input-kinesis](https://discuss.elastic.co/t/retrieving-partitionkey-via-logstash-input-kinesis/109618)
**Author:** [@keithzhang](https://discuss.elastic.co/u/keithzhang)\ **Replies:** 0\ **Last updated:** [November 29, 2017, 3:32pm UTC](https://discuss.elastic.co/t/retrieving-partitionkey-via-logstash-input-kinesis/109618 "2017-11-29T15:32:08Z")
We are trying to use logstash-input-kinesis to ingest data from AWS Kinesis into Elasticsearch. Here is our logstash pipeline: input { kinesis{ kinesis\_stream\_name=\>"xxx" application\_name=\>"yyy" codec=\> json region… --- ## [Logstash XML file headers](https://discuss.elastic.co/t/logstash-xml-file-headers/109613)
**Author:** [@GGA](https://discuss.elastic.co/u/GGA)\ **Replies:** 0\ **Last updated:** [November 29, 2017, 2:41pm UTC](https://discuss.elastic.co/t/logstash-xml-file-headers/109613 "2017-11-29T14:41:01Z")
Hi, I'm parsing xml files coming from filebeat, with logstash. I have a configuration that works fine, but it's not parsing the header and with the end of the files One of the headers of a file: \ **Author:** [@Kasun\_Siyambalapitiy](https://discuss.elastic.co/u/Kasun_Siyambalapitiy)\ **Replies:** 2\ **Last updated:** [November 29, 2017, 2:13pm UTC](https://discuss.elastic.co/t/once-a-configuration-is-changed-is-sudo-service-logstash-restart-is-enough/109588 "2017-11-29T14:13:08Z") I changed existing configurations under /opt/logstash/config.d/ and ran a sudo service logstash restart but it is not still not publishing data, do I need to run sudo update-rc.d logstash defaults 96 9 to make effect of… --- ## [Mapper\_parsing\_exception", "reason"=\>"\[include\_in\_all - is not allowed for indices created on or after version 6.0.0](https://discuss.elastic.co/t/mapper-parsing-exception-reason-include-in-all-is-not-allowed-for-indices-created-on-or-after-version-6-0-0/109586)
**Author:** [@hispeed](https://discuss.elastic.co/u/hispeed)\ **Replies:** 6\ **Last updated:** [November 29, 2017, 1:45pm UTC](https://discuss.elastic.co/t/mapper-parsing-exception-reason-include-in-all-is-not-allowed-for-indices-created-on-or-after-version-6-0-0/109586 "2017-11-29T13:45:29Z")
Hi, I get the following error: \[2017-11-29T12:43:23,143\]\[WARN \]\[logstash.outputs.elasticsearch\] Could not index eve… --- ## [Json logs with a field named "time", won´t be shown in kibana](https://discuss.elastic.co/t/json-logs-with-a-field-named-time-won-t-be-shown-in-kibana/109601)
**Author:** [@Janick\_Brischle](https://discuss.elastic.co/u/Janick_Brischle)\ **Replies:** 0\ **Last updated:** [November 29, 2017, 1:16pm UTC](https://discuss.elastic.co/t/json-logs-with-a-field-named-time-won-t-be-shown-in-kibana/109601 "2017-11-29T13:16:59Z")
Example json: {"time":"2017-11-29 13:39:43"} Json which works: {"timstamp":"2017-11-29 13:39:43"} --- ## [How to configure the output section in logstash conf make copy of logs on my local disk](https://discuss.elastic.co/t/how-to-configure-the-output-section-in-logstash-conf-make-copy-of-logs-on-my-local-disk/109383)
**Author:** [@Milad\_Hamid](https://discuss.elastic.co/u/Milad_Hamid)\ **Replies:** 6\ **Last updated:** [November 29, 2017, 1:00pm UTC](https://discuss.elastic.co/t/how-to-configure-the-output-section-in-logstash-conf-make-copy-of-logs-on-my-local-disk/109383 "2017-11-29T13:00:41Z")
Hi, How can i get copy or output of parsed logs in logstash on my local disk? Our Case: We are thinking not just send logs to Elasticsearch by logstash also to have output from logs on our disk, then we have backup/ar… --- ## [Rejecting mapping update for new index](https://discuss.elastic.co/t/rejecting-mapping-update-for-new-index/109576)
**Author:** [@elasticheart](https://discuss.elastic.co/u/elasticheart)\ **Replies:** 4\ **Last updated:** [November 29, 2017, 12:43pm UTC](https://discuss.elastic.co/t/rejecting-mapping-update-for-new-index/109576 "2017-11-29T12:43:27Z")
Hi, I am using ELK GA 6, and I am trying to create a brand new index. Before pushing into the topic using Logstash, I have created a template like below; PUT \_template/my\_template { "index\_patterns": \["mylogs\*"\], "… --- ## [How I can drop anyone fields if a value in this field contains "null"?](https://discuss.elastic.co/t/how-i-can-drop-anyone-fields-if-a-value-in-this-field-contains-null/109543)
**Author:** [@srolskyi](https://discuss.elastic.co/u/srolskyi)\ **Replies:** 4\ **Last updated:** [November 29, 2017, 11:28am UTC](https://discuss.elastic.co/t/how-i-can-drop-anyone-fields-if-a-value-in-this-field-contains-null/109543 "2017-11-29T11:28:16Z")
How I can drop anyone fields if a value in this field contains "null"? I use plugins kv, xml, and json sometimes value in the field = null. I need a flexible solution, that so as not to describe each field separately. … --- ## [How to output to multiple http endpoints](https://discuss.elastic.co/t/how-to-output-to-multiple-http-endpoints/109549)
**Author:** [@Kasun\_Siyambalapitiy](https://discuss.elastic.co/u/Kasun_Siyambalapitiy)\ **Replies:** 4\ **Last updated:** [November 29, 2017, 11:00am UTC](https://discuss.elastic.co/t/how-to-output-to-multiple-http-endpoints/109549 "2017-11-29T11:00:27Z")
Hi, Folllowing is the output section of my config file, output { if \[type\] == "product\_log" { http { url =\> "https://xxxxxxxxxx.com/endpoints/nginx\_product\_receiver" http\_method =\> "post" format =\> "json" header… --- ## [Grok filter not working (but ok in herokuapp). It even prints nothing in the output](https://discuss.elastic.co/t/grok-filter-not-working-but-ok-in-herokuapp-it-even-prints-nothing-in-the-output/109412)
**Author:** [@JeremyLG](https://discuss.elastic.co/u/JeremyLG)\ **Replies:** 0\ **Last updated:** [November 28, 2017, 2:17pm UTC](https://discuss.elastic.co/t/grok-filter-not-working-but-ok-in-herokuapp-it-even-prints-nothing-in-the-output/109412 "2017-11-28T14:17:43Z")
Hey, I have seen that multiple people had my problem, but I can't seem to find why did I do wrong. Can someone explain me why this doesn't work. Log example : Jun 2 00:00:00 192.168.14.4 date=2016-06-01 time=23:56:0… --- ## [Logstash (elastic stack 5.6) grok regex](https://discuss.elastic.co/t/logstash-elastic-stack-5-6-grok-regex/109553)
**Author:** [@yishain11](https://discuss.elastic.co/u/yishain11)\ **Replies:** 3\ **Last updated:** [November 29, 2017, 10:24am UTC](https://discuss.elastic.co/t/logstash-elastic-stack-5-6-grok-regex/109553 "2017-11-29T10:24:23Z")
Hi everyone! There is a problem I struggle with for a while: I have data in a json format: "documentsList": \[{ "commandScn": "108058599", "commandCommitScn": "108058600", "commandSequence": "0", "commandType":… --- ## [Logstash Elasticsearch output does not add the first document of a collection](https://discuss.elastic.co/t/logstash-elasticsearch-output-does-not-add-the-first-document-of-a-collection/106675)
**Author:** [@MariSpirandelli](https://discuss.elastic.co/u/MariSpirandelli)\ **Replies:** 4\ **Last updated:** [November 29, 2017, 10:11am UTC](https://discuss.elastic.co/t/logstash-elasticsearch-output-does-not-add-the-first-document-of-a-collection/106675 "2017-11-29T10:11:04Z")
I'm trying to use Logstash to get data from Mongodb and insert into Elasticsearch. I'm using Logstash input plugin for Mongodb from here My logstash.conf is as follow: input { mongodb { uri =\> 'mongodb\_host… --- ## [Combining two inputs how to access the fields](https://discuss.elastic.co/t/combining-two-inputs-how-to-access-the-fields/109415)
**Author:** [@ericohtake](https://discuss.elastic.co/u/ericohtake)\ **Replies:** 4\ **Last updated:** [November 29, 2017, 9:39am UTC](https://discuss.elastic.co/t/combining-two-inputs-how-to-access-the-fields/109415 "2017-11-29T09:39:57Z")
Hi, I have 2 inputs in a config file where I need to send a field from one to another if a condition is met. However I'm failing to find documentation on how to reference the fields living in different input events. Be… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=864) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=866)