# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=363 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 364 --- ## [HTTP output Plugin](https://discuss.elastic.co/t/http-output-plugin/230729)
**Author:** [@InfiniteLoops](https://discuss.elastic.co/u/InfiniteLoops)\ **Replies:** 1\ **Last updated:** [May 5, 2020, 3:42pm UTC](https://discuss.elastic.co/t/http-output-plugin/230729 "2020-05-05T15:42:45Z")
One of my pipelines has a very simple format of a HTTP input plugin where the data is sent straight to a HTTP output plugin. input { pipeline { address =\> "pipeline-name" } } output { http { url =\> "addres… --- ## [Can I point to multiple folders in pipelines.yml? and use wildcard on it](https://discuss.elastic.co/t/can-i-point-to-multiple-folders-in-pipelines-yml-and-use-wildcard-on-it/231050)
**Author:** [@ElasticLiver](https://discuss.elastic.co/u/ElasticLiver)\ **Replies:** 1\ **Last updated:** [May 5, 2020, 3:30pm UTC](https://discuss.elastic.co/t/can-i-point-to-multiple-folders-in-pipelines-yml-and-use-wildcard-on-it/231050 "2020-05-05T15:30:01Z")
Hi, in my pipeline.yml I currently have this: - pipeline.id: main path.config: "/usr/share/logstash/pipeline/\*.conf" but I need to point to another folder an use wildcard again and use something like: - pipeline.id:… --- ## [Should Elastic Search be reindexed if I make any configuration changes to Logstash](https://discuss.elastic.co/t/should-elastic-search-be-reindexed-if-i-make-any-configuration-changes-to-logstash/231132)
**Author:** [@harish411](https://discuss.elastic.co/u/harish411)\ **Replies:** 1\ **Last updated:** [May 5, 2020, 3:20pm UTC](https://discuss.elastic.co/t/should-elastic-search-be-reindexed-if-i-make-any-configuration-changes-to-logstash/231132 "2020-05-05T15:20:22Z")
Hi If I make any configuration changes to Logstash. Will I see the chnages applied to Elastic Search For Example If I change the grok pattern and added new Fields, Will I be able to see the chnages effected on alrea… --- ## [Logstash Shipping Performance (degrading with newer version)](https://discuss.elastic.co/t/logstash-shipping-performance-degrading-with-newer-version/231143)
**Author:** [@USP\_rol](https://discuss.elastic.co/u/USP_rol)\ **Replies:** 0\ **Last updated:** [May 5, 2020, 12:05pm UTC](https://discuss.elastic.co/t/logstash-shipping-performance-degrading-with-newer-version/231143 "2020-05-05T12:05:42Z")
Hi community! We are using ELK-Stack for a couple of years having a Logstash Shipper / Filter architecture (currently 2 Shipper nodes with 6 Filter nodes). With the latest updates from version 6.2 to 6.8 we've discovere… --- ## [Logstash won't execute unless i delete hsperfdata\_root in tmp folder](https://discuss.elastic.co/t/logstash-wont-execute-unless-i-delete-hsperfdata-root-in-tmp-folder/231096)
**Author:** [@haythem](https://discuss.elastic.co/u/haythem)\ **Replies:** 5\ **Last updated:** [May 5, 2020, 11:46am UTC](https://discuss.elastic.co/t/logstash-wont-execute-unless-i-delete-hsperfdata-root-in-tmp-folder/231096 "2020-05-05T11:46:43Z")
Hello guys , lately i've had a problem with logstash not executing correctly , so the first time it happend i've found that the cause of the problem was that i did not have enough space in my disk so i freed um some spac… --- ## [Grok parsefailure](https://discuss.elastic.co/t/grok-parsefailure/230398)
**Author:** [@GedeoN](https://discuss.elastic.co/u/GedeoN)\ **Replies:** 7\ **Last updated:** [May 5, 2020, 9:06am UTC](https://discuss.elastic.co/t/grok-parsefailure/230398 "2020-05-05T09:06:06Z")
Hi experts, I have a new problem, and this time, i don't think i can resolve it myself. I'm trying to parse a little message like that: 2020-02-12 13:52:04.15 spid4s Execution of SSIS\_HOTFIX\_INSTALL.SQL completed… --- ## [Some way to temporarily pause pipelines](https://discuss.elastic.co/t/some-way-to-temporarily-pause-pipelines/231090)
**Author:** [@willemdh](https://discuss.elastic.co/u/willemdh)\ **Replies:** 0\ **Last updated:** [May 5, 2020, 7:20am UTC](https://discuss.elastic.co/t/some-way-to-temporarily-pause-pipelines/231090 "2020-05-05T07:20:34Z")
Hello, When debugging pipelines of during unexpected or expected event spikes, it can be useful to be able to temporarily pause ingestion for 1 centralized pipeline management pipeline. During this pauze, we can make ch… --- ## [Duplicate documents in elaticsearch](https://discuss.elastic.co/t/duplicate-documents-in-elaticsearch/231057)
**Author:** [@hiba](https://discuss.elastic.co/u/hiba)\ **Replies:** 1\ **Last updated:** [May 5, 2020, 5:56am UTC](https://discuss.elastic.co/t/duplicate-documents-in-elaticsearch/231057 "2020-05-05T05:56:56Z")
Hi, i have below configuration of logstash. input { http\_poller { urls =\> { url =\> "XXXX" } request\_timeout =\> 60 schedule =\> { every =\> "60s" } codec =\> "json" } } filter { split { field =\> "\[bookings\]" } … --- ## [Why to use Logstash if FileBeat can communicate with Elasticsearch directly](https://discuss.elastic.co/t/why-to-use-logstash-if-filebeat-can-communicate-with-elasticsearch-directly/230290)
**Author:** [@Ashish\_Chaudhari](https://discuss.elastic.co/u/Ashish_Chaudhari)\ **Replies:** 2\ **Last updated:** [May 5, 2020, 3:13am UTC](https://discuss.elastic.co/t/why-to-use-logstash-if-filebeat-can-communicate-with-elasticsearch-directly/230290 "2020-05-05T03:13:02Z")
Hi All, I have started learning about ELK. The initial tutorial I came across , uses NGINX logs and NGINX Filebeat module for the demo. I see that we can configure Filebeat to send the logs directly to elasticsearch. If… --- ## [Logstash ingestion slows rapidly after 1 hour](https://discuss.elastic.co/t/logstash-ingestion-slows-rapidly-after-1-hour/230741)
**Author:** [@davignbd](https://discuss.elastic.co/u/davignbd)\ **Replies:** 3\ **Last updated:** [May 5, 2020, 12:58am UTC](https://discuss.elastic.co/t/logstash-ingestion-slows-rapidly-after-1-hour/230741 "2020-05-05T00:58:07Z")
I have a logstash 6.7 on kubernetes which ingests .gz log files from s3 input at a rate of about 20k/minute for roughly 1 hour, and then it suddenly drops to 1400/minute. It will remain like that until I kill the pod. Wh… --- ## [Is possible send logs from specific kubernetes namespaces to diferent elasticsearch using logstash?](https://discuss.elastic.co/t/is-possible-send-logs-from-specific-kubernetes-namespaces-to-diferent-elasticsearch-using-logstash/231030)
**Author:** [@Adonai\_Costa](https://discuss.elastic.co/u/Adonai_Costa)\ **Replies:** 1\ **Last updated:** [May 4, 2020, 8:24pm UTC](https://discuss.elastic.co/t/is-possible-send-logs-from-specific-kubernetes-namespaces-to-diferent-elasticsearch-using-logstash/231030 "2020-05-04T20:24:29Z")
I have 1 kubernetes cluster send all logs to logstash, this logstash send log to elasticsearch. It's possible send logs from diferent pods of namespaces to diferent elasticsearchs using logstash? how can I do that? --- ## [Aggregate\_maps\_path file not visible](https://discuss.elastic.co/t/aggregate-maps-path-file-not-visible/230657)
**Author:** [@xyz2](https://discuss.elastic.co/u/xyz2)\ **Replies:** 4\ **Last updated:** [May 4, 2020, 6:37pm UTC](https://discuss.elastic.co/t/aggregate-maps-path-file-not-visible/230657 "2020-05-04T18:37:36Z")
HI, I have set aggregate\_maps\_path in logstash aggregate filter. However i do not see it. I somewhere read that logstash deletes the file and loads it into memory. Is that right ? What i want to do is, during my deplo… --- ## [DNS filter couldn't perform reverse lookup](https://discuss.elastic.co/t/dns-filter-couldnt-perform-reverse-lookup/230856)
**Author:** [@ladakalina](https://discuss.elastic.co/u/ladakalina)\ **Replies:** 3\ **Last updated:** [May 4, 2020, 6:00pm UTC](https://discuss.elastic.co/t/dns-filter-couldnt-perform-reverse-lookup/230856 "2020-05-04T18:00:42Z")
Hello everyone! I have a problem with DNS filter in Logstash. Source data: I use logstash to parse ulogd logs from my router (based on Open-wrt). The last problem I faced it's problem with perform reverse lookup IP-add… --- ## [Logstash GeoIP database support](https://discuss.elastic.co/t/logstash-geoip-database-support/230976)
**Author:** [@Mohammad\_Mousavi](https://discuss.elastic.co/u/Mohammad_Mousavi)\ **Replies:** 2\ **Last updated:** [May 4, 2020, 2:43pm UTC](https://discuss.elastic.co/t/logstash-geoip-database-support/230976 "2020-05-04T14:43:44Z")
Hi, I wanna buy a commercial IP2location database from IP2Location . I know I can use .bin file in logstash in GeoIP as a source database. In my scenario, I need to import this .bin or .csv database into mysql or post… --- ## [Logstash single or multi pipelining?](https://discuss.elastic.co/t/logstash-single-or-multi-pipelining/230977)
**Author:** [@pup\_seba](https://discuss.elastic.co/u/pup_seba)\ **Replies:** 0\ **Last updated:** [May 4, 2020, 1:28pm UTC](https://discuss.elastic.co/t/logstash-single-or-multi-pipelining/230977 "2020-05-04T13:28:33Z")
Hi, What would be the recommended approach to using multiple pipelines vs using a single pipeline with conditionals about wether to apply or not the filter. I'm trying to understand when to use each one, but I'm unsure … --- ## [Database socket error](https://discuss.elastic.co/t/database-socket-error/230967)
**Author:** [@jagan](https://discuss.elastic.co/u/jagan)\ **Replies:** 0\ **Last updated:** [May 4, 2020, 12:25pm UTC](https://discuss.elastic.co/t/database-socket-error/230967 "2020-05-04T12:25:40Z")
Hi mates, I have to ingest events from kafka & databases into ES via Logstash.when i run the logstashjob the database connection gets failed.Please help me to solve the issue. The following error: \[WARN \]\[logstash.fil… --- ## [Ruby exception occurred: Invalid FieldReference: \`eventData\[logReferences\]\`](https://discuss.elastic.co/t/ruby-exception-occurred-invalid-fieldreference-eventdata-logreferences/230558)
**Author:** [@Abinash](https://discuss.elastic.co/u/Abinash)\ **Replies:** 2\ **Last updated:** [May 4, 2020, 12:17pm UTC](https://discuss.elastic.co/t/ruby-exception-occurred-invalid-fieldreference-eventdata-logreferences/230558 "2020-05-04T12:17:09Z")
There was issue with lagstash version 7.X version on words. But the my logstash script worked with 6.2.4 with out this kind of this error. Input { "Versions": { "0.1": { "eventData": { … --- ## [Failed to parse field \[http.content\_range\]](https://discuss.elastic.co/t/failed-to-parse-field-http-content-range/230959)
**Author:** [@Chris\_Anagnostou](https://discuss.elastic.co/u/Chris_Anagnostou)\ **Replies:** 0\ **Last updated:** [May 4, 2020, 11:44am UTC](https://discuss.elastic.co/t/failed-to-parse-field-http-content-range/230959 "2020-05-04T11:44:15Z")
Hello, i have the following error failed to parse field \[http.content\_range\] of type \[keyword\] in document with id 'Mcp433EB7yM1scj0azAg'. Preview of field's value: '{size=12693598, start=7340032, raw=bytes 7340032-8388… --- ## [Duplicate index with script python](https://discuss.elastic.co/t/duplicate-index-with-script-python/230932)
**Author:** [@Youssef\_SBAI](https://discuss.elastic.co/u/Youssef_SBAI)\ **Replies:** 3\ **Last updated:** [May 4, 2020, 9:08am UTC](https://discuss.elastic.co/t/duplicate-index-with-script-python/230932 "2020-05-04T09:08:11Z")
I created a python script to remove duplicates from my index, in order to compare #from elasticsearch import Elasticsearch # Settings es = Elasticsearch('http://localhost:9000/') def search\_datas(): # Corp de la r… --- ## [Extract path in logstash](https://discuss.elastic.co/t/extract-path-in-logstash/230801)
**Author:** [@Mohammad.ali](https://discuss.elastic.co/u/Mohammad.ali)\ **Replies:** 2\ **Last updated:** [May 4, 2020, 8:27am UTC](https://discuss.elastic.co/t/extract-path-in-logstash/230801 "2020-05-04T08:27:16Z")
Hello every one I need to access to my filename date. this is the filter i wrote in my logstash filter { grok { match =\> { "path" =\> \[ "^%{GREEDYDATA}(?\\[0-9\]{4})(?\\[0-9\]{2})(?\\[0-9\]{2})(?\ **Author:** [@gpolini](https://discuss.elastic.co/u/gpolini)\ **Replies:** 3\ **Last updated:** [May 4, 2020, 6:59am UTC](https://discuss.elastic.co/t/logstash-elasticsearch-input-doesnt-start/227924 "2020-05-04T06:59:10Z") Hi! I'm trying to add new fields in my index documents which are calculated from data already present in the documents. For example, I have two timestamps and I have to calculate the difference in seconds. I wrote a lo… --- ## [Logstash connection error](https://discuss.elastic.co/t/logstash-connection-error/230912)
**Author:** [@Guilherme\_Almeida](https://discuss.elastic.co/u/Guilherme_Almeida)\ **Replies:** 0\ **Last updated:** [May 4, 2020, 2:55am UTC](https://discuss.elastic.co/t/logstash-connection-error/230912 "2020-05-04T02:55:59Z")
Hello guys Every time I try to check the logstash data I get an alert that the data cannot be received in Kibana "Module status Check that data is received from the Filebeat logstash module No data has been received … --- ## [Logstash not starting up - keeps on restarting](https://discuss.elastic.co/t/logstash-not-starting-up-keeps-on-restarting/230650)
**Author:** [@snehid](https://discuss.elastic.co/u/snehid)\ **Replies:** 2\ **Last updated:** [May 3, 2020, 11:30pm UTC](https://discuss.elastic.co/t/logstash-not-starting-up-keeps-on-restarting/230650 "2020-05-03T23:30:10Z")
Not able to get the logstash up and running \[2020-05-01T10:51:44,146\]\[ERROR\]\[org.logstash.Logstash \] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit \[2020-05-01T10:… --- ## [Unable to start logstash after SSL config](https://discuss.elastic.co/t/unable-to-start-logstash-after-ssl-config/230903)
**Author:** [@srv95](https://discuss.elastic.co/u/srv95)\ **Replies:** 0\ **Last updated:** [May 3, 2020, 10:53pm UTC](https://discuss.elastic.co/t/unable-to-start-logstash-after-ssl-config/230903 "2020-05-03T22:53:11Z")
\-- The start-up result is RESULT. May 03 22:50:44 ip-172-26-2-38 kibana\[18148\]: {"type":"response","@timestamp":"2020-05-03T22:50:44Z","tags":,"pid":1814 May 03 22:50:51 ip-172-26-2-38 logstash\[31241\]: Sending Logstash… --- ## [Calculate time between different events with same ID](https://discuss.elastic.co/t/calculate-time-between-different-events-with-same-id/230900)
**Author:** [@John\_Guzman](https://discuss.elastic.co/u/John_Guzman)\ **Replies:** 0\ **Last updated:** [May 3, 2020, 9:47pm UTC](https://discuss.elastic.co/t/calculate-time-between-different-events-with-same-id/230900 "2020-05-03T21:47:24Z")
Hi! I been struggling with this some few months and i haven't found a good way to do it. I'm passing events to logstash correspondent to transactions, each transaction is composed of n events and i have to calculate the… --- ## [Winlogbeat not able to send log to the logstash](https://discuss.elastic.co/t/winlogbeat-not-able-to-send-log-to-the-logstash/230833)
**Author:** [@srv95](https://discuss.elastic.co/u/srv95)\ **Replies:** 7\ **Last updated:** [May 3, 2020, 7:35pm UTC](https://discuss.elastic.co/t/winlogbeat-not-able-to-send-log-to-the-logstash/230833 "2020-05-03T19:35:27Z")
emstats":{"gc\_next":57446288,"memory\_alloc":28758904,"memory\_total":636265600},"runtime":{"goroutines":28}},"libbeat":{"config":{"module":{"running":0}},"output":{"read":{"errors":1},"write":{"bytes":124}},"pipeline":{"c… --- ## [Creating metrics from a filebeat input](https://discuss.elastic.co/t/creating-metrics-from-a-filebeat-input/230894)
**Author:** [@WimDH](https://discuss.elastic.co/u/WimDH)\ **Replies:** 0\ **Last updated:** [May 3, 2020, 6:58pm UTC](https://discuss.elastic.co/t/creating-metrics-from-a-filebeat-input/230894 "2020-05-03T18:58:58Z")
Hello everybody, Can someone explain me how to create metrics from values retrieved fro a logfile (or point me to some good docs)? I have a logfile on a server that I parse with filebeat and which are forwarded to aa l… --- ## [Pipeline Workers and CPU count?](https://discuss.elastic.co/t/pipeline-workers-and-cpu-count/230893)
**Author:** [@AtyNewman](https://discuss.elastic.co/u/AtyNewman)\ **Replies:** 0\ **Last updated:** [May 3, 2020, 6:47pm UTC](https://discuss.elastic.co/t/pipeline-workers-and-cpu-count/230893 "2020-05-03T18:47:37Z")
I have a Logstash instance sending data into an Elasticsearch cluster what take place on the same machine. 2 Elasticsearch and 1 Logstash. Should i separate CPU Cores between them like this? in case of 10 CPU: logsta… --- ## [I cannot see every column or see wrong output for the exisiting ones](https://discuss.elastic.co/t/i-cannot-see-every-column-or-see-wrong-output-for-the-exisiting-ones/230343)
**Author:** [@Burak\_Sakallioglu](https://discuss.elastic.co/u/Burak_Sakallioglu)\ **Replies:** 9\ **Last updated:** [May 3, 2020, 1:30pm UTC](https://discuss.elastic.co/t/i-cannot-see-every-column-or-see-wrong-output-for-the-exisiting-ones/230343 "2020-05-03T13:30:20Z")
Hi everyone, I have been trying to import my csv file with logstash, however, the output is not seen as I expected. Firstly, I cannot see every column in the output, then I cannot see the correct value for some existin… --- ## [java.lang.IllegalArgumentException: File does not contain valid private key](https://discuss.elastic.co/t/java-lang-illegalargumentexception-file-does-not-contain-valid-private-key/230874)
**Author:** [@Nassos](https://discuss.elastic.co/u/Nassos)\ **Replies:** 0\ **Last updated:** [May 3, 2020, 12:50pm UTC](https://discuss.elastic.co/t/java-lang-illegalargumentexception-file-does-not-contain-valid-private-key/230874 "2020-05-03T12:50:54Z")
Hello, I've been trying to setup the elastic stack based on the following article. I have managed to configure elasticsearch and kibana but logstash won't start. Basically stuck on step 6-5 and the pipeline won't st… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=362) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=364)