# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=107 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 108 --- ## [Increase of memory in logstash when certificate reload time is kept as 5min](https://discuss.elastic.co/t/increase-of-memory-in-logstash-when-certificate-reload-time-is-kept-as-5min/317870)
**Author:** [@teja\_tata](https://discuss.elastic.co/u/teja_tata)\ **Replies:** 6\ **Last updated:** [November 3, 2022, 11:36am UTC](https://discuss.elastic.co/t/increase-of-memory-in-logstash-when-certificate-reload-time-is-kept-as-5min/317870 "2022-11-03T11:36:07Z")
When certificate rotations time is kept as 5min will there be any increase in memory in logstash? --- ## [AutoRun multiple config file from nssm in windows](https://discuss.elastic.co/t/autorun-multiple-config-file-from-nssm-in-windows/318099)
**Author:** [@Rakhshunda\_Noorein\_J](https://discuss.elastic.co/u/Rakhshunda_Noorein_J)\ **Replies:** 0\ **Last updated:** [November 3, 2022, 10:51am UTC](https://discuss.elastic.co/t/autorun-multiple-config-file-from-nssm-in-windows/318099 "2022-11-03T10:51:36Z")
Hello, I am using nssm to autostart my logstash configs when the system starts. what i have done is, download nssm -\> nssm install logstash -\> added logstash application paths and arguments -\> start the service. It is … --- ## [How to regex urls with Grok](https://discuss.elastic.co/t/how-to-regex-urls-with-grok/318011)
**Author:** [@lamdba](https://discuss.elastic.co/u/lamdba)\ **Replies:** 2\ **Last updated:** [November 3, 2022, 9:46am UTC](https://discuss.elastic.co/t/how-to-regex-urls-with-grok/318011 "2022-11-03T09:46:21Z")
Hello there, Sorry to bother but I'm slowly working my way to fully use and understand the stack and right now I'm blocked with a (I guess) simple problem with Logstash. I parse multiple events through Logstash and eve… --- ## [Multiline codec not expected bulk](https://discuss.elastic.co/t/multiline-codec-not-expected-bulk/318052)
**Author:** [@INS](https://discuss.elastic.co/u/INS)\ **Replies:** 1\ **Last updated:** [November 3, 2022, 6:55am UTC](https://discuss.elastic.co/t/multiline-codec-not-expected-bulk/318052 "2022-11-03T06:55:11Z")
My case is that I have a separate pipelines for each file. Here for "Numbers\*" files, it doesn't work properly for me to put data into elastic. On input side I have files divided into 50\_000 lines and with a name that is… --- ## [Not able to connect Kafka server in output kafka plugin using SASL\_SSL](https://discuss.elastic.co/t/not-able-to-connect-kafka-server-in-output-kafka-plugin-using-sasl-ssl/318062)
**Author:** [@danishbit09](https://discuss.elastic.co/u/danishbit09)\ **Replies:** 0\ **Last updated:** [November 3, 2022, 3:53am UTC](https://discuss.elastic.co/t/not-able-to-connect-kafka-server-in-output-kafka-plugin-using-sasl-ssl/318062 "2022-11-03T03:53:23Z")
I am not able to publish log into Kafka topic. Below error is showing. Kindly suggest for the below error log. be removed in future version. Please use 'use\_all\_dns\_ips' or another non-deprecated value. \[2022-11-03T11:… --- ## [Pulsar connectivity](https://discuss.elastic.co/t/pulsar-connectivity/317939)
**Author:** [@omicronian8](https://discuss.elastic.co/u/omicronian8)\ **Replies:** 2\ **Last updated:** [November 2, 2022, 11:40pm UTC](https://discuss.elastic.co/t/pulsar-connectivity/317939 "2022-11-02T23:40:43Z")
I need to store the output of logstash onto a pulsar topic. Looking at the documentation and other threads here, I couldn't find any leads on it. I could only get a git repo where we can use pulsar topic as an input to l… --- ## [Does logstash 8.4 supports Elastic Search 7.8 later?](https://discuss.elastic.co/t/does-logstash-8-4-supports-elastic-search-7-8-later/317960)
**Author:** [@djrshn2346](https://discuss.elastic.co/u/djrshn2346)\ **Replies:** 1\ **Last updated:** [November 2, 2022, 9:47pm UTC](https://discuss.elastic.co/t/does-logstash-8-4-supports-elastic-search-7-8-later/317960 "2022-11-02T21:47:13Z")
I am using logstash 8.4.3, along with Elasticsearch versions 7.8.1 and 7.10.2. Does logstash will support this version of ES? --- ## [Logstash does not take all the logs from eventhub](https://discuss.elastic.co/t/logstash-does-not-take-all-the-logs-from-eventhub/318024)
**Author:** [@Pavelm](https://discuss.elastic.co/u/Pavelm)\ **Replies:** 0\ **Last updated:** [November 2, 2022, 4:19pm UTC](https://discuss.elastic.co/t/logstash-does-not-take-all-the-logs-from-eventhub/318024 "2022-11-02T16:19:22Z")
Hi, I am using Logstash to connect to Eventhub in Azure and then push logs to ELK. For some reason, the Outgoing queue is much smaller than the Incoming as you may see in the screenshot: What can be the reason? --- ## [Parsing big xml files in logstash](https://discuss.elastic.co/t/parsing-big-xml-files-in-logstash/318018)
**Author:** [@mhoro](https://discuss.elastic.co/u/mhoro)\ **Replies:** 0\ **Last updated:** [November 2, 2022, 3:57pm UTC](https://discuss.elastic.co/t/parsing-big-xml-files-in-logstash/318018 "2022-11-02T15:57:31Z")
I would like to load data from multiple big xml files to elasticsearch using logstash. Files are up to 3gb with many lines, all with the same structure. They contain a list of elements and I would like one element to bec… --- ## [Logstash Arcsight Module Implementation Issues](https://discuss.elastic.co/t/logstash-arcsight-module-implementation-issues/317977)
**Author:** [@jakinmayowa](https://discuss.elastic.co/u/jakinmayowa)\ **Replies:** 2\ **Last updated:** [November 2, 2022, 2:44pm UTC](https://discuss.elastic.co/t/logstash-arcsight-module-implementation-issues/317977 "2022-11-02T14:44:42Z")
Completed installation of the ELK stack version 8.4 - single node installation Used a test pipeline to confirm comms between Elasticsearch and Logstash on local machine on the commandline interface. So, I tried impleme… --- ## [Elasticserach cannot show indices](https://discuss.elastic.co/t/elasticserach-cannot-show-indices/318004)
**Author:** [@toughLuck](https://discuss.elastic.co/u/toughLuck)\ **Replies:** 0\ **Last updated:** [November 2, 2022, 2:43pm UTC](https://discuss.elastic.co/t/elasticserach-cannot-show-indices/318004 "2022-11-02T14:43:51Z")
I am new to ELK, trying to set up a basic ELK following a youtube video, but cannot load the index to elasticsearch/ The logstash log \[2022-11-02T10:38:08,489\]\[ERROR\]\[logstash.outputs.elasticsearch\]\[main\]\[bef0a8230c929… --- ## [Logstash webhdfs output plugin AlreadyBeingCreatedException](https://discuss.elastic.co/t/logstash-webhdfs-output-plugin-alreadybeingcreatedexception/317887)
**Author:** [@Rakhshunda\_Noorein\_J](https://discuss.elastic.co/u/Rakhshunda_Noorein_J)\ **Replies:** 10\ **Last updated:** [November 2, 2022, 1:20pm UTC](https://discuss.elastic.co/t/logstash-webhdfs-output-plugin-alreadybeingcreatedexception/317887 "2022-11-02T13:20:54Z")
Hello, I have created a webhdfc logstash configuaration file which will take file input and insert the data in hdfs through webhdfs output. sharing you the config file below. input { file { path =\> "C:/Users/user/… --- ## [Logstash jdbc config throwing error](https://discuss.elastic.co/t/logstash-jdbc-config-throwing-error/317974)
**Author:** [@ashiqab](https://discuss.elastic.co/u/ashiqab)\ **Replies:** 0\ **Last updated:** [November 2, 2022, 10:57am UTC](https://discuss.elastic.co/t/logstash-jdbc-config-throwing-error/317974 "2022-11-02T10:57:54Z")
Hello, I am facing the error below when trying to execute logstash jdbc config file: \[ERROR\] 2022-11-02 11:52:26.435 \[Converge PipelineAction::Create\] agent - Failed to execute action {:action=\>LogStash::PipelineAction… --- ## [Failed to fetch X-pack information from ES - Failure to reach live ES Cluster](https://discuss.elastic.co/t/failed-to-fetch-x-pack-information-from-es-failure-to-reach-live-es-cluster/317306)
**Author:** [@jakinmayowa](https://discuss.elastic.co/u/jakinmayowa)\ **Replies:** 2\ **Last updated:** [November 2, 2022, 10:49am UTC](https://discuss.elastic.co/t/failed-to-fetch-x-pack-information-from-es-failure-to-reach-live-es-cluster/317306 "2022-11-02T10:49:12Z")
I'm currently configuring Logstash ArcSight Module, however I've reached a road-block, error message below: Sending Logstash logs to /usr/share/logstash/logs which is now configured via log4j2.properties \[2022-10-24T10:… --- ## [Aggregation of fields](https://discuss.elastic.co/t/aggregation-of-fields/317969)
**Author:** [@arphillips](https://discuss.elastic.co/u/arphillips)\ **Replies:** 0\ **Last updated:** [November 2, 2022, 10:48am UTC](https://discuss.elastic.co/t/aggregation-of-fields/317969 "2022-11-02T10:48:39Z")
How do I aggregate fields with vega in a sankey graph? I want to build a graph that shows AS paths using bytes. I have most of the graph in place. However, when I use bytes I get a 'flow' for each different number, can … --- ## [Logstash profile](https://discuss.elastic.co/t/logstash-profile/317934)
**Author:** [@GEHsu](https://discuss.elastic.co/u/GEHsu)\ **Replies:** 1\ **Last updated:** [November 2, 2022, 8:59am UTC](https://discuss.elastic.co/t/logstash-profile/317934 "2022-11-02T08:59:46Z")
System is Ubuntu Logstash version 7.17 The current idea is to collect logs (via syslog-514Port) from different servers or network devices Logstash will then send the log cut to Elasticsearch I have successfully recei… --- ## [Logstash stuck when lumberjack server is down](https://discuss.elastic.co/t/logstash-stuck-when-lumberjack-server-is-down/317955)
**Author:** [@ferdose\_shaik](https://discuss.elastic.co/u/ferdose_shaik)\ **Replies:** 0\ **Last updated:** [November 2, 2022, 8:44am UTC](https://discuss.elastic.co/t/logstash-stuck-when-lumberjack-server-is-down/317955 "2022-11-02T08:44:07Z")
We are using Logstash 7.15.2 with following lumberjack output configured. output { lumberjack { id =\> "dcae5gcprod" hosts =\> \["dcae-5gc-prod.ecomp.idns.cci.att.com"\] codec =\> json port =\> 31135 ssl\_cert… --- ## [Steps to configure and use memcached in logstash. Can you please share with me the steps](https://discuss.elastic.co/t/steps-to-configure-and-use-memcached-in-logstash-can-you-please-share-with-me-the-steps/317727)
**Author:** [@danishbit09](https://discuss.elastic.co/u/danishbit09)\ **Replies:** 4\ **Last updated:** [November 2, 2022, 8:32am UTC](https://discuss.elastic.co/t/steps-to-configure-and-use-memcached-in-logstash-can-you-please-share-with-me-the-steps/317727 "2022-11-02T08:32:42Z")
Can you please share with me the steps to configure and run memcached in filter plugin. Is memcached server configured in logstash internally or I have to install and start memcached externally? Please guide me. --- ## [Grok pattern](https://discuss.elastic.co/t/grok-pattern/317891)
**Author:** [@abkonred1](https://discuss.elastic.co/u/abkonred1)\ **Replies:** 4\ **Last updated:** [November 2, 2022, 7:25am UTC](https://discuss.elastic.co/t/grok-pattern/317891 "2022-11-02T07:25:06Z")
Hi Team, I need to ingest the below log file, and i required all the fields without pipe. In order to create visualization and dashboards for the same. kindly any one please help, how to split the fields with this log e… --- ## [Logstash - Adding http\_poller meta data into document](https://discuss.elastic.co/t/logstash-adding-http-poller-meta-data-into-document/317863)
**Author:** [@zx8086](https://discuss.elastic.co/u/zx8086)\ **Replies:** 2\ **Last updated:** [November 1, 2022, 4:48pm UTC](https://discuss.elastic.co/t/logstash-adding-http-poller-meta-data-into-document/317863 "2022-11-01T16:48:36Z")
The value in this field is what i require -\> http\_poller\_metadata.input.http\_poller.request.name mutate { add\_field =\> { "metric" =\> "\[http\_poller\_metadata\]\[request\]\[name\]" } } Not sure the notation when the … --- ## [How to use environment variable for logstash output elasticsearch plugin for multiple elasticsearch hosts?](https://discuss.elastic.co/t/how-to-use-environment-variable-for-logstash-output-elasticsearch-plugin-for-multiple-elasticsearch-hosts/317169)
**Author:** [@asp](https://discuss.elastic.co/u/asp)\ **Replies:** 4\ **Last updated:** [November 1, 2022, 3:07pm UTC](https://discuss.elastic.co/t/how-to-use-environment-variable-for-logstash-output-elasticsearch-plugin-for-multiple-elasticsearch-hosts/317169 "2022-11-01T15:07:54Z")
Hi, I am deploying logstash and its pipelines via ansible. I want to use the same pipeline on different installations. So I don't want to modify the elasticsearch output plugin in each pipeline. I want to use an environ… --- ## [Curling ElasticSearch API using logstash keystore credentials](https://discuss.elastic.co/t/curling-elasticsearch-api-using-logstash-keystore-credentials/317647)
**Author:** [@Broken08](https://discuss.elastic.co/u/Broken08)\ **Replies:** 2\ **Last updated:** [November 1, 2022, 12:49pm UTC](https://discuss.elastic.co/t/curling-elasticsearch-api-using-logstash-keystore-credentials/317647 "2022-11-01T12:49:16Z")
Is it possible to call Elasticsearch API requests with the Logstash Keystore credentials that are stored (If they obviously are the right username/password/etc...). I know API/curl calls can be done utilizing Username p… --- ## [Help with conditionals in logstash](https://discuss.elastic.co/t/help-with-conditionals-in-logstash/317836)
**Author:** [@RaonyO](https://discuss.elastic.co/u/RaonyO)\ **Replies:** 2\ **Last updated:** [October 31, 2022, 5:31pm UTC](https://discuss.elastic.co/t/help-with-conditionals-in-logstash/317836 "2022-10-31T17:31:34Z")
hello, I'm trying to perform a filter with some conditions, but when performing as follows, it doesn't work. if "dns\_server\_process\_query\_send" or "Not authoritative" in \[message\] { drop {} } but if I use the followin… --- ## [Whitelist nested field](https://discuss.elastic.co/t/whitelist-nested-field/317773)
**Author:** [@yuswanul](https://discuss.elastic.co/u/yuswanul)\ **Replies:** 5\ **Last updated:** [October 31, 2022, 4:53pm UTC](https://discuss.elastic.co/t/whitelist-nested-field/317773 "2022-10-31T16:53:07Z")
Hi everyone, i plan to whitelist some fields, but they are nested field. i already tried prune to whitelist them and it didn't work. for example: Request Body : {"tipe":"FF","nama":"ABC","nik":"","handphone":"00012","… --- ## [Parsed Multi Object in Json](https://discuss.elastic.co/t/parsed-multi-object-in-json/317705)
**Author:** [@yuswanul](https://discuss.elastic.co/u/yuswanul)\ **Replies:** 7\ **Last updated:** [October 31, 2022, 4:42pm UTC](https://discuss.elastic.co/t/parsed-multi-object-in-json/317705 "2022-10-31T16:42:38Z")
Hi there, so i have a log look like this: 2022-10-27 08:39:02 \[https-jsse-nio-9078-exec-7\] INFO i.c.p.va.security.LoggerFilter - Response Body : {"responseCode":"00","responseDesc":"Approved","data":"{"vaNumber":"111… --- ## [Error uploading data from PostgresQL to Elasticsearch via logstash](https://discuss.elastic.co/t/error-uploading-data-from-postgresql-to-elasticsearch-via-logstash/317832)
**Author:** [@Evgen\_G](https://discuss.elastic.co/u/Evgen_G)\ **Replies:** 0\ **Last updated:** [October 31, 2022, 4:21pm UTC](https://discuss.elastic.co/t/error-uploading-data-from-postgresql-to-elasticsearch-via-logstash/317832 "2022-10-31T16:21:34Z")
Tell me, please, can anyone come across such a problem: when unloading data from the database in elasticsearch, an error occurs \[ERROR\] 2022-10-31 16:02:14.628 \[\[main\]\>worker1\] elasticsearch - Encountered a retryable er… --- ## [How to grok a certain fields from a log file](https://discuss.elastic.co/t/how-to-grok-a-certain-fields-from-a-log-file/315996)
**Author:** [@anupvtr](https://discuss.elastic.co/u/anupvtr)\ **Replies:** 1\ **Last updated:** [October 31, 2022, 6:43am UTC](https://discuss.elastic.co/t/how-to-grok-a-certain-fields-from-a-log-file/315996 "2022-10-31T06:43:42Z")
Preformatted textHi All, I am quite new to the magic world of Grok. Any help will be thankful. I need to apply filter for the following file. 2022-08-22 22:18:59 , 666 INFO @ (blockurcolumn-11) \[rbbit\_MQ\_Versa.appa… --- ## [Fresh ELK install 8.4.3. Logstash problem](https://discuss.elastic.co/t/fresh-elk-install-8-4-3-logstash-problem/317670)
**Author:** [@Xgraver1](https://discuss.elastic.co/u/Xgraver1)\ **Replies:** 2\ **Last updated:** [October 31, 2022, 6:06am UTC](https://discuss.elastic.co/t/fresh-elk-install-8-4-3-logstash-problem/317670 "2022-10-31T06:06:50Z")
Hello Could use some help with fresh ELK installation on premises. I installed Elasticstack and Kibana with mostly default configuration. In Elasticstack conf i changed Data path and uncommented host and port settings. … --- ## [Grokparse failure seen with tcp input plugin in logstash pipeline](https://discuss.elastic.co/t/grokparse-failure-seen-with-tcp-input-plugin-in-logstash-pipeline/317447)
**Author:** [@Arinjay\_Jain](https://discuss.elastic.co/u/Arinjay_Jain)\ **Replies:** 5\ **Last updated:** [October 30, 2022, 11:52am UTC](https://discuss.elastic.co/t/grokparse-failure-seen-with-tcp-input-plugin-in-logstash-pipeline/317447 "2022-10-30T11:52:14Z")
Hi, I have the following logstash pipeline configuration. input { tcp { port =\> 5102 codec =\> plain } } filter { grok { match =\> {"message" =\> "%{SYSLOGTIMESTAMP:time} %{DATA:trace\_n… --- ## [Output Batch Sizes](https://discuss.elastic.co/t/output-batch-sizes/317486)
**Author:** [@dawiro](https://discuss.elastic.co/u/dawiro)\ **Replies:** 3\ **Last updated:** [October 30, 2022, 9:39am UTC](https://discuss.elastic.co/t/output-batch-sizes/317486 "2022-10-30T09:39:55Z")
Hi, How can we see the size of the batches logstash is pushing to elasticsearch? Thx D [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=106) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=108)