# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=140 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 141 --- ## [Http\_poller input and excluding parent json properties for output to elastic](https://discuss.elastic.co/t/http-poller-input-and-excluding-parent-json-properties-for-output-to-elastic/305098)
**Author:** [@dwittner](https://discuss.elastic.co/u/dwittner)\ **Replies:** 1\ **Last updated:** [May 18, 2022, 5:12pm UTC](https://discuss.elastic.co/t/http-poller-input-and-excluding-parent-json-properties-for-output-to-elastic/305098 "2022-05-18T17:12:08Z")
afternoon My poller reads from a source where the array that I wish to be the source of the Elasticsearch output plugin in not the parent and not the only property. How can I either filter appropriately OR instruct the… --- ## [Best way to parse multiple message patterns](https://discuss.elastic.co/t/best-way-to-parse-multiple-message-patterns/304836)
**Author:** [@Thuunder7](https://discuss.elastic.co/u/Thuunder7)\ **Replies:** 7\ **Last updated:** [May 18, 2022, 3:59pm UTC](https://discuss.elastic.co/t/best-way-to-parse-multiple-message-patterns/304836 "2022-05-18T15:59:02Z")
Hello, I want to ask if it is possible to have multiple dissect patterns? I know i can create conditionals based on the "\_dissectfailure" and create another dissect to parse other patterns, but this doesn't prevent the… --- ## [Using field value as value in message\[x\]](https://discuss.elastic.co/t/using-field-value-as-value-in-message-x/305035)
**Author:** [@alex\_vermex](https://discuss.elastic.co/u/alex_vermex)\ **Replies:** 4\ **Last updated:** [May 18, 2022, 2:43pm UTC](https://discuss.elastic.co/t/using-field-value-as-value-in-message-x/305035 "2022-05-18T14:43:47Z")
Hi, I would like to ask that i have field x =\> 0 it was an integer field value and i have a message\[\] array i want to consume the value of x inside the message for example: message\[x\] . here is an example of my code: f… --- ## [How do I connect logstash with ES?](https://discuss.elastic.co/t/how-do-i-connect-logstash-with-es/304797)
**Author:** [@McLaine](https://discuss.elastic.co/u/McLaine)\ **Replies:** 2\ **Last updated:** [May 18, 2022, 12:00pm UTC](https://discuss.elastic.co/t/how-do-i-connect-logstash-with-es/304797 "2022-05-18T12:00:43Z")
Hi! What do I have to do? Something like Elasticsearch-users useradd ? And add some roles? Best Andy --- ## [Logstash to influxdb version2.x is it possible?](https://discuss.elastic.co/t/logstash-to-influxdb-version2-x-is-it-possible/304913)
**Author:** [@surinkim](https://discuss.elastic.co/u/surinkim)\ **Replies:** 2\ **Last updated:** [May 18, 2022, 2:14am UTC](https://discuss.elastic.co/t/logstash-to-influxdb-version2-x-is-it-possible/304913 "2022-05-18T02:14:24Z")
HI. I designed a structure to put data from Logstash to influxdb. So, I succeeded in putting data into influx db version 1.8 in the window environment. However, influxdb version 2 gives this error. Connection Error w… --- ## [Logstash Kafka input and output with oauth](https://discuss.elastic.co/t/logstash-kafka-input-and-output-with-oauth/304886)
**Author:** [@Jockj](https://discuss.elastic.co/u/Jockj)\ **Replies:** 3\ **Last updated:** [May 17, 2022, 10:02pm UTC](https://discuss.elastic.co/t/logstash-kafka-input-and-output-with-oauth/304886 "2022-05-17T22:02:08Z")
Does anyone know how to use the logstash Kafka input/output plugin with oauth2? I couldn't find any settings with the current plugins. and I'm trying to develop one with the Kafka Java client. It failed with the followin… --- ## [Es document count != wc -l output of file after sending data with logstash?](https://discuss.elastic.co/t/es-document-count-wc-l-output-of-file-after-sending-data-with-logstash/304293)
**Author:** [@elk-user-99](https://discuss.elastic.co/u/elk-user-99)\ **Replies:** 20\ **Last updated:** [May 17, 2022, 9:19pm UTC](https://discuss.elastic.co/t/es-document-count-wc-l-output-of-file-after-sending-data-with-logstash/304293 "2022-05-17T21:19:47Z")
Hi friends! I've got what I thought was a simple configuration -- the input is a file that passes through 3 filters -- csv to parse the file, mutate to do a character replace that makes the timestamp automatically detec… --- ## [Initialize counter in logstash and replace the '%{\[message\]\[0\]}'?](https://discuss.elastic.co/t/initialize-counter-in-logstash-and-replace-the-message-0/304912)
**Author:** [@alex\_vermex](https://discuss.elastic.co/u/alex_vermex)\ **Replies:** 9\ **Last updated:** [May 17, 2022, 4:28pm UTC](https://discuss.elastic.co/t/initialize-counter-in-logstash-and-replace-the-message-0/304912 "2022-05-17T16:28:12Z")
Hi, after doing the split filter, so my message was an array ok, so %{\[message\]\[0\]} %{\[message\]\[1\]} ... that gives me their value fine. now I have an array of length 30 for example and I start the condition if {} examp… --- ## [Cluster health turns to red](https://discuss.elastic.co/t/cluster-health-turns-to-red/304870)
**Author:** [@rachelyang](https://discuss.elastic.co/u/rachelyang)\ **Replies:** 2\ **Last updated:** [May 17, 2022, 4:21pm UTC](https://discuss.elastic.co/t/cluster-health-turns-to-red/304870 "2022-05-17T16:21:52Z")
Please have a look at the output of \_cluster/health?pretty { "cluster\_name" : "Elasticsearch", "status" : "red", "timed\_out" : false, "number\_of\_nodes" : 2, "number\_of\_data\_nodes" : 2, "active\_primary\_shards… --- ## [Use file values as variable or iterate over values](https://discuss.elastic.co/t/use-file-values-as-variable-or-iterate-over-values/304947)
**Author:** [@S-elk](https://discuss.elastic.co/u/S-elk)\ **Replies:** 1\ **Last updated:** [May 17, 2022, 3:42pm UTC](https://discuss.elastic.co/t/use-file-values-as-variable-or-iterate-over-values/304947 "2022-05-17T15:42:22Z")
Hello! Right now I have a pipeline that creates a field using a ruby scanner. When this scanner detects one of the values entered in the regular expression, it creates a field with that value. For example: if \[x\] { … --- ## [Get initial log from Logstash warning](https://discuss.elastic.co/t/get-initial-log-from-logstash-warning/304796)
**Author:** [@arazdolski](https://discuss.elastic.co/u/arazdolski)\ **Replies:** 9\ **Last updated:** [May 17, 2022, 3:37pm UTC](https://discuss.elastic.co/t/get-initial-log-from-logstash-warning/304796 "2022-05-17T15:37:05Z")
Hi, I have a rsyslog -\> logstash -\> Elasticsearch setup and noticed a lot of warnings with tried to parse field \[v\] as an object, but found a concrete value in Logstash logs, but I cannot find the initial syslog which L… --- ## [Logstash monitoring help not seeing in Kibana](https://discuss.elastic.co/t/logstash-monitoring-help-not-seeing-in-kibana/303689)
**Author:** [@Mary2022](https://discuss.elastic.co/u/Mary2022)\ **Replies:** 14\ **Last updated:** [May 17, 2022, 2:54pm UTC](https://discuss.elastic.co/t/logstash-monitoring-help-not-seeing-in-kibana/303689 "2022-05-17T14:54:40Z")
Hi, I am new to ELK and I have been trying to follow the guides and so far I have been abled to install version 7.17 in our 2 kibana nodes and our 3 elasticsearch nodes. If I go to Kibana and look under Cluster Over… --- ## [Logstash bulk request to ElasticSearch error - may be due to old indexes](https://discuss.elastic.co/t/logstash-bulk-request-to-elasticsearch-error-may-be-due-to-old-indexes/304943)
**Author:** [@DenSeb](https://discuss.elastic.co/u/DenSeb)\ **Replies:** 0\ **Last updated:** [May 17, 2022, 12:26pm UTC](https://discuss.elastic.co/t/logstash-bulk-request-to-elasticsearch-error-may-be-due-to-old-indexes/304943 "2022-05-17T12:26:23Z")
If you get these errors in LogStash : Attempted to send a bulk request to elasticsearch' but Elasticsearch appears to be unreachable or down! {:error\_message=\>"Elasticsearch Unreachable: \[http://localhost:9200/\]\[Mantico… --- ## [Grok pattern Syslog auth](https://discuss.elastic.co/t/grok-pattern-syslog-auth/304911)
**Author:** [@CemG](https://discuss.elastic.co/u/CemG)\ **Replies:** 2\ **Last updated:** [May 17, 2022, 11:47am UTC](https://discuss.elastic.co/t/grok-pattern-syslog-auth/304911 "2022-05-17T11:47:51Z")
Hello, I have to grok my syslog auth logs from Linux, I have a pattern, but it still nonmatch with the logs, can someone help me to see if there is an error in my gros pattern ? My filter filter { json { … --- ## [Map MySQL columns into nested fields Elastic using Logstash](https://discuss.elastic.co/t/map-mysql-columns-into-nested-fields-elastic-using-logstash/304820)
**Author:** [@Alrote](https://discuss.elastic.co/u/Alrote)\ **Replies:** 4\ **Last updated:** [May 17, 2022, 11:20am UTC](https://discuss.elastic.co/t/map-mysql-columns-into-nested-fields-elastic-using-logstash/304820 "2022-05-17T11:20:03Z")
Hello, I am struggling getting done some filtering to get in Elastic the information in the format I want. First, I have a MySQL table that I process in the input (no problem in that part). I want some of the columns t… --- ## [Logstash Docker Container Syslog input](https://discuss.elastic.co/t/logstash-docker-container-syslog-input/304874)
**Author:** [@doublejz](https://discuss.elastic.co/u/doublejz)\ **Replies:** 2\ **Last updated:** [May 16, 2022, 10:30pm UTC](https://discuss.elastic.co/t/logstash-docker-container-syslog-input/304874 "2022-05-16T22:30:18Z")
I've been fighting this the last couple days and well, I give up and need help. So I have an ELK stack docker setup and I'm simply trying to setup logstash to accept syslog directly to it. I pictured random port 5514 and… --- ## [Rename logstash.yml](https://discuss.elastic.co/t/rename-logstash-yml/304850)
**Author:** [@Marc\_Jacques](https://discuss.elastic.co/u/Marc_Jacques)\ **Replies:** 2\ **Last updated:** [May 16, 2022, 7:26pm UTC](https://discuss.elastic.co/t/rename-logstash-yml/304850 "2022-05-16T19:26:54Z")
Hello Elastic, is this possible to rename logstash.yml? I would like to have something like logstash-name-name.yml I haven't found anything on the doc or web. There is a setting directory in startup.options, but no v… --- ## [Ruby filter logstash (.length)](https://discuss.elastic.co/t/ruby-filter-logstash-length/304832)
**Author:** [@alex\_vermex](https://discuss.elastic.co/u/alex_vermex)\ **Replies:** 1\ **Last updated:** [May 16, 2022, 2:32pm UTC](https://discuss.elastic.co/t/ruby-filter-logstash-length/304832 "2022-05-16T14:32:12Z")
Hi, I would like to fix the length of message logstash.conf: filter { mutate { split =\> {"message" =\> "|"} ruby { code =\> "event.set('number\_of\_elements', event.get('message').length) " } … --- ## [Logstash how to keep only last elements of an array?](https://discuss.elastic.co/t/logstash-how-to-keep-only-last-elements-of-an-array/304366)
**Author:** [@zid57](https://discuss.elastic.co/u/zid57)\ **Replies:** 2\ **Last updated:** [May 16, 2022, 2:01pm UTC](https://discuss.elastic.co/t/logstash-how-to-keep-only-last-elements-of-an-array/304366 "2022-05-16T14:01:28Z")
Hello all, I have a field which is an array and I would like to keep only the last element and put it into a new field. "data": { "path": \[ 1111, 2222, 3333 \] I tried to copy da… --- ## [Generic solution for masking PII data in JSON payload](https://discuss.elastic.co/t/generic-solution-for-masking-pii-data-in-json-payload/303464)
**Author:** [@Hammad\_Ali](https://discuss.elastic.co/u/Hammad_Ali)\ **Replies:** 5\ **Last updated:** [May 16, 2022, 8:05am UTC](https://discuss.elastic.co/t/generic-solution-for-masking-pii-data-in-json-payload/303464 "2022-05-16T08:05:55Z")
Hello! Is there any generic solution to mask PII data in JSON payload, I know about mutate gsub but that doesn't seem like a generic solution. I am looking for something that filters out all of the fields and masks any … --- ## [Getting error in logstash with no other informations](https://discuss.elastic.co/t/getting-error-in-logstash-with-no-other-informations/304782)
**Author:** [@perez.koh](https://discuss.elastic.co/u/perez.koh)\ **Replies:** 0\ **Last updated:** [May 16, 2022, 6:59am UTC](https://discuss.elastic.co/t/getting-error-in-logstash-with-no-other-informations/304782 "2022-05-16T06:59:59Z")
Hello Can anyone advice on the below error, I am trying to connect the logstash to Elasticsearch, but show the below error, there is no other information, and hard to troubleshoot sudo tail /var/log/logstash/logstash-p… --- ## [Elasticsearch&Logstash Log4j Vulnerabilities](https://discuss.elastic.co/t/elasticsearch-logstash-log4j-vulnerabilities/304640)
**Author:** [@kyarali](https://discuss.elastic.co/u/kyarali)\ **Replies:** 5\ **Last updated:** [May 16, 2022, 5:08am UTC](https://discuss.elastic.co/t/elasticsearch-logstash-log4j-vulnerabilities/304640 "2022-05-16T05:08:26Z")
Hello, We have log4j vulnerabilities for Elasticsearch and Logstash in the following paths: Path : /usr/share/Elasticsearch/lib/log4j-core-2.11.1.jar Path : /usr/share/logstash/logstash-core/lib/jars/log4j-core-2.14.0… --- ## [OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release](https://discuss.elastic.co/t/openjdk-64-bit-server-vm-warning-option-useconcmarksweepgc-was-deprecated-in-version-9-0-and-will-likely-be-removed-in-a-future-release/304686)
**Author:** [@Onsrm](https://discuss.elastic.co/u/Onsrm)\ **Replies:** 2\ **Last updated:** [May 16, 2022, 2:33am UTC](https://discuss.elastic.co/t/openjdk-64-bit-server-vm-warning-option-useconcmarksweepgc-was-deprecated-in-version-9-0-and-will-likely-be-removed-in-a-future-release/304686 "2022-05-16T02:33:23Z")
hello when i use this command bin/logstash -f /etc/logstash/conf.d/apache.conf i get this error : OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed i… --- ## [How to uninstall logstash service in windows?](https://discuss.elastic.co/t/how-to-uninstall-logstash-service-in-windows/304740)
**Author:** [@jisha](https://discuss.elastic.co/u/jisha)\ **Replies:** 2\ **Last updated:** [May 15, 2022, 6:31am UTC](https://discuss.elastic.co/t/how-to-uninstall-logstash-service-in-windows/304740 "2022-05-15T06:31:42Z")
I want to know how to uninstall Logstash in the windows server? someone please help me with the appropriate command. --- ## [Getting error on logstash server](https://discuss.elastic.co/t/getting-error-on-logstash-server/304742)
**Author:** [@mangeshmj1992](https://discuss.elastic.co/u/mangeshmj1992)\ **Replies:** 0\ **Last updated:** [May 15, 2022, 5:57am UTC](https://discuss.elastic.co/t/getting-error-on-logstash-server/304742 "2022-05-15T05:57:32Z")
Hello team, I am getting below error on logstash. Can you please help me on this. How to handle this in logstash "log"=\>"{\\"timestamp\\":\\"2022-05-15 09:50:00,012+0400\\",\\"nodeId\\":\\"E9EBC60D-F4BD50A1-E285711E-EFEABBB0-… --- ## [Insufficient memory to run logstash-keystore](https://discuss.elastic.co/t/insufficient-memory-to-run-logstash-keystore/304734)
**Author:** [@mcosta](https://discuss.elastic.co/u/mcosta)\ **Replies:** 2\ **Last updated:** [May 14, 2022, 10:16pm UTC](https://discuss.elastic.co/t/insufficient-memory-to-run-logstash-keystore/304734 "2022-05-14T22:16:17Z")
Hi all, Running logstash 7.17.3 on a 8 GB RHEL 8.5 with -Xms4g -Xmx4g. Until now I was able to run "bin/logstash-keystore --path.settings ${LOGSTASH\_CONF} list" command but now I get the following error: OpenJDK 64-Bi… --- ## [Limit logstash 7.5 memory usage](https://discuss.elastic.co/t/limit-logstash-7-5-memory-usage/304556)
**Author:** [@smm](https://discuss.elastic.co/u/smm)\ **Replies:** 10\ **Last updated:** [May 14, 2022, 12:26pm UTC](https://discuss.elastic.co/t/limit-logstash-7-5-memory-usage/304556 "2022-05-14T12:26:38Z")
Hi there, is there a way to limit memory consumption of logstash 7.5? In our case logstash 'eats up' all the memory so that Elasticsearch itself gets killed on the same host (OOM killer). From 7.8 onwards a possibility… --- ## [Running logstash(pipeline.conf) through terminal by passing environment variables](https://discuss.elastic.co/t/running-logstash-pipeline-conf-through-terminal-by-passing-environment-variables/304649)
**Author:** [@vaseemQA](https://discuss.elastic.co/u/vaseemQA)\ **Replies:** 5\ **Last updated:** [May 13, 2022, 8:56pm UTC](https://discuss.elastic.co/t/running-logstash-pipeline-conf-through-terminal-by-passing-environment-variables/304649 "2022-05-13T20:56:01Z")
Hi Team, I need some help in running my Logstash project files. I've a list of .conf files in a project called pilotProject, currently I test them from my local by running following command ./logstash -f ~/pilotProjec… --- ## [Substring logstash](https://discuss.elastic.co/t/substring-logstash/304710)
**Author:** [@mvasqueznr](https://discuss.elastic.co/u/mvasqueznr)\ **Replies:** 1\ **Last updated:** [May 13, 2022, 6:53pm UTC](https://discuss.elastic.co/t/substring-logstash/304710 "2022-05-13T18:53:52Z")
Hi. Im use grok patterns to extract info from Palo Alto log. But im try to extract a substring on description field like this authenticated for user 'newUser'. auth profile 'D', server profile 'LTY', server address … --- ## [Logstash parse PDF from S3](https://discuss.elastic.co/t/logstash-parse-pdf-from-s3/304700)
**Author:** [@vsangam](https://discuss.elastic.co/u/vsangam)\ **Replies:** 0\ **Last updated:** [May 13, 2022, 4:56pm UTC](https://discuss.elastic.co/t/logstash-parse-pdf-from-s3/304700 "2022-05-13T16:56:06Z")
Hi, I have few files in s3 bucket. They are all PDF. Can I use S3 plugin to extract content from those PDF files. If not, is there any way to parse the PDF files from S3.... Please help. Thanks, Vsangam [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=139) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=141)