# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=48 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 49 --- ## [Could not add the UUID fingerprint in producer on logstash](https://discuss.elastic.co/t/could-not-add-the-uuid-fingerprint-in-producer-on-logstash/349227)
**Author:** [@pradeep.kumar](https://discuss.elastic.co/u/pradeep.kumar)\ **Replies:** 1\ **Last updated:** [December 13, 2023, 5:40pm UTC](https://discuss.elastic.co/t/could-not-add-the-uuid-fingerprint-in-producer-on-logstash/349227 "2023-12-13T17:40:34Z")
Hi Team, Im adding the fingerprint of method UUID to avoid the duplication of the data. Im using kafka as producer and Elasticsearch as consumer. Example pipeline conf looks like this input { kafka {… --- ## [Elastic SNMP - Best Practices?](https://discuss.elastic.co/t/elastic-snmp-best-practices/349285)
**Author:** [@erikg](https://discuss.elastic.co/u/erikg)\ **Replies:** 0\ **Last updated:** [December 13, 2023, 5:07pm UTC](https://discuss.elastic.co/t/elastic-snmp-best-practices/349285 "2023-12-13T17:07:47Z")
Continuing the discussion from Possability to use ELK-Stack for SNMP Monitoring like PRTG, CheckMK: Hello, It would be nice to hear how more people are using Elastic's SNMP (Logstash Input plugin) with a large number o… --- ## [Logstash is unable to connect to Elasticsearch where entire ELK is setup in docker](https://discuss.elastic.co/t/logstash-is-unable-to-connect-to-elasticsearch-where-entire-elk-is-setup-in-docker/349224)
**Author:** [@Nurarao](https://discuss.elastic.co/u/Nurarao)\ **Replies:** 0\ **Last updated:** [December 13, 2023, 7:04am UTC](https://discuss.elastic.co/t/logstash-is-unable-to-connect-to-elasticsearch-where-entire-elk-is-setup-in-docker/349224 "2023-12-13T07:04:15Z")
Hi, I was setting up the ELK setup using docker and using certs too. The Elastic has 3 nodes (es01, es02, es03) which is running separate containers and even Kibana , logstash also running in individual containers. The … --- ## [How to add day of month field but with certain timezone](https://discuss.elastic.co/t/how-to-add-day-of-month-field-but-with-certain-timezone/349207)
**Author:** [@yuswanul](https://discuss.elastic.co/u/yuswanul)\ **Replies:** 5\ **Last updated:** [December 13, 2023, 4:42am UTC](https://discuss.elastic.co/t/how-to-add-day-of-month-field-but-with-certain-timezone/349207 "2023-12-13T04:42:18Z")
i want to create a "day of month" field for my visualization. I already did this using a scripted field before. but since I chose Grafana to visualize my data, I can't use that scripted field there. so I want to generate… --- ## [Need help to create a grok patter for my syslog pattern](https://discuss.elastic.co/t/need-help-to-create-a-grok-patter-for-my-syslog-pattern/349103)
**Author:** [@ameeto17](https://discuss.elastic.co/u/ameeto17)\ **Replies:** 2\ **Last updated:** [December 12, 2023, 10:17pm UTC](https://discuss.elastic.co/t/need-help-to-create-a-grok-patter-for-my-syslog-pattern/349103 "2023-12-12T22:17:28Z")
my log message looks like this message Dec 12 12:01:27 ppdtest302 test-checker: Context SHA of TEST Software Version 3.0.1\_RC5 0b1f71223180bf0df9330b13e17f8d7c62dfdaad16b97a80b8a25c99409c1109 How do i use a grok patte… --- ## [Need help about starting logstash-8.11.2](https://discuss.elastic.co/t/need-help-about-starting-logstash-8-11-2/349125)
**Author:** [@AlexLWei](https://discuss.elastic.co/u/AlexLWei)\ **Replies:** 13\ **Last updated:** [December 12, 2023, 8:12pm UTC](https://discuss.elastic.co/t/need-help-about-starting-logstash-8-11-2/349125 "2023-12-12T20:12:08Z")
I installed Logstash by downloading and unzipping the zip file from the official website and it occurs an error about JDK , Using bundled JDK: /opt/logstash-8.11.2/jdk Unrecognized VM option 'UseConcMarkSweepGC' Erro… --- ## [Palo Alto Next-Gen Firewall compatibility with Global Protect VPN Client](https://discuss.elastic.co/t/palo-alto-next-gen-firewall-compatibility-with-global-protect-vpn-client/349084)
**Author:** [@CodeMonky](https://discuss.elastic.co/u/CodeMonky)\ **Replies:** 5\ **Last updated:** [December 12, 2023, 2:52pm UTC](https://discuss.elastic.co/t/palo-alto-next-gen-firewall-compatibility-with-global-protect-vpn-client/349084 "2023-12-12T14:52:15Z")
Good day all! I'm looking for confirmation on the features of the Palo Alto Next-Gen Firewall integration with elastic. On the overview page of the integration, it details support of the Global Protect type of message. … --- ## [Splitting an array of objects using Logstash](https://discuss.elastic.co/t/splitting-an-array-of-objects-using-logstash/349066)
**Author:** [@M0hsen](https://discuss.elastic.co/u/M0hsen)\ **Replies:** 2\ **Last updated:** [December 11, 2023, 4:21pm UTC](https://discuss.elastic.co/t/splitting-an-array-of-objects-using-logstash/349066 "2023-12-11T16:21:38Z")
Hello everyone, I'm trying to split the following array of objects into multiple log events: \[ { "time": "\*", "twkMessageId": "\*", "environmentName": "\*", "virtualhostName": "default", "apiproxyNa… --- ## [Grok with custom pattern works in debugger but not in pipline](https://discuss.elastic.co/t/grok-with-custom-pattern-works-in-debugger-but-not-in-pipline/348957)
**Author:** [@helldunkel](https://discuss.elastic.co/u/helldunkel)\ **Replies:** 5\ **Last updated:** [December 11, 2023, 3:48pm UTC](https://discuss.elastic.co/t/grok-with-custom-pattern-works-in-debugger-but-not-in-pipline/348957 "2023-12-11T15:48:22Z")
Hi, I´m have a lot of problems to get a dataset in elastic. In Debugger it works. Log \<30\>2023:12:08-12:59:39 fw-swr-2 ulogd\[32373\]: grock .\*\>%{SOPHOS\_TIMESTAMP:\_tmp.timestamp} %{TEST:firewall.name} custom pattern … --- ## [Scripted upsert is failing in Elasticsearch output](https://discuss.elastic.co/t/scripted-upsert-is-failing-in-elasticsearch-output/349057)
**Author:** [@gshankar-elastic](https://discuss.elastic.co/u/gshankar-elastic)\ **Replies:** 0\ **Last updated:** [December 11, 2023, 3:02pm UTC](https://discuss.elastic.co/t/scripted-upsert-is-failing-in-elasticsearch-output/349057 "2023-12-11T15:02:26Z")
I am using an indexed script in the output to transform the event data like this: input { kafka { bootstrap\_servers =\> "kafka.localhost.com:9092" topics =\> \["enriched"\] } } filter { json { … --- ## [Is there any way to update the \`last\_run\_metadata\_file\` in the output plugin?](https://discuss.elastic.co/t/is-there-any-way-to-update-the-last-run-metadata-file-in-the-output-plugin/348919)
**Author:** [@gayatri\_SN](https://discuss.elastic.co/u/gayatri_SN)\ **Replies:** 2\ **Last updated:** [December 11, 2023, 10:02am UTC](https://discuss.elastic.co/t/is-there-any-way-to-update-the-last-run-metadata-file-in-the-output-plugin/348919 "2023-12-11T10:02:30Z")
My scenario is as follows: I am using the JDBC input plugin with a tracking column and last\_run\_metadata\_file, and it is working as expected. However, when the ETL host is down or unreachable, the filter API throws an er… --- ## [Beats\_input\_raw\_event](https://discuss.elastic.co/t/beats-input-raw-event/348575)
**Author:** [@Yuval\_Algresi](https://discuss.elastic.co/u/Yuval_Algresi)\ **Replies:** 3\ **Last updated:** [December 10, 2023, 10:28pm UTC](https://discuss.elastic.co/t/beats-input-raw-event/348575 "2023-12-10T22:28:42Z")
Hello, I use winlogbeat to ship event viewer logs to my elastic stack. It first goes to logstash and from there to elastic - I use beats input plugin. Usually there is an event.original field that contains the raw eve… --- ## [Kafka integration plug v11.3.2 with AWS MSK 2.8.1](https://discuss.elastic.co/t/kafka-integration-plug-v11-3-2-with-aws-msk-2-8-1/348963)
**Author:** [@bbenne821](https://discuss.elastic.co/u/bbenne821)\ **Replies:** 2\ **Last updated:** [December 9, 2023, 6:38pm UTC](https://discuss.elastic.co/t/kafka-integration-plug-v11-3-2-with-aws-msk-2-8-1/348963 "2023-12-09T18:38:41Z")
We have a TLS-enabled AWS MSK (Managed Streaming Kafka) 2.8.1 cluster and using logstash kafka integration plug v11.3.2 to read from topics. Our input logstash pipeline: input { kafka { id =\> "sentinel\_one-… --- ## [Logstash sflow plugin install by default](https://discuss.elastic.co/t/logstash-sflow-plugin-install-by-default/346670)
**Author:** [@VamPikmin](https://discuss.elastic.co/u/VamPikmin)\ **Replies:** 4\ **Last updated:** [December 9, 2023, 12:25am UTC](https://discuss.elastic.co/t/logstash-sflow-plugin-install-by-default/346670 "2023-12-09T00:25:59Z")
Hi all, Is it possible to turn on a setting so that with each new Elastic Stack upgrade this logstash plugin with install automatically. Currently I have to stop logstash, upgrade the stack and run /usr/share/logstash… --- ## [Ruby script with dynamic variables in logstash](https://discuss.elastic.co/t/ruby-script-with-dynamic-variables-in-logstash/348870)
**Author:** [@Dor-Alter](https://discuss.elastic.co/u/Dor-Alter)\ **Replies:** 3\ **Last updated:** [December 8, 2023, 3:24pm UTC](https://discuss.elastic.co/t/ruby-script-with-dynamic-variables-in-logstash/348870 "2023-12-08T15:24:34Z")
Is it possible to have a dynamic variables in ruby that changes based on the input values I am getting from the inputfile? For example I have 5 input values which the attribute linked, if the value of link is 'connected… --- ## [Exec output plugin](https://discuss.elastic.co/t/exec-output-plugin/348869)
**Author:** [@Dor-Alter](https://discuss.elastic.co/u/Dor-Alter)\ **Replies:** 2\ **Last updated:** [December 8, 2023, 1:43pm UTC](https://discuss.elastic.co/t/exec-output-plugin/348869 "2023-12-08T13:43:32Z")
I am trying to create a pipeline that takes a csv file as input write it out to a different file and then run some bash script on the output file and create a new output file. Input\_file.csv -\> logstash -\> temp\_file -\> … --- ## [Field is of the wrong type](https://discuss.elastic.co/t/field-is-of-the-wrong-type/348762)
**Author:** [@soad20000](https://discuss.elastic.co/u/soad20000)\ **Replies:** 10\ **Last updated:** [December 7, 2023, 10:57pm UTC](https://discuss.elastic.co/t/field-is-of-the-wrong-type/348762 "2023-12-07T22:57:50Z")
Hello, I am getting an error when trying to use some fields that apparently aren't being mapped correctly. The fields are source.ip, source.port, destination.ip, and destination.port I have checked the mapping of th… --- ## [SNMP Trap - Encoding Issue](https://discuss.elastic.co/t/snmp-trap-encoding-issue/347034)
**Author:** [@erikg](https://discuss.elastic.co/u/erikg)\ **Replies:** 1\ **Last updated:** [December 7, 2023, 6:57pm UTC](https://discuss.elastic.co/t/snmp-trap-encoding-issue/347034 "2023-12-07T18:57:49Z")
Hello, I am using snmp trap input plugin to receive snmp traps. I am receiving SNMP v1 traps but on some fields there seems to be hex binary encoding. Here's a sample of my messages: #\ **Author:** [@musk\_elon](https://discuss.elastic.co/u/musk_elon)\ **Replies:** 0\ **Last updated:** [December 7, 2023, 5:15pm UTC](https://discuss.elastic.co/t/how-to-input-the-evtx-file-in-logstash/348834 "2023-12-07T17:15:14Z") Hello, everyone. I want to know how to input the evtx file in logstash. output is json. help me. thanks --- ## [I got this error in using logstash](https://discuss.elastic.co/t/i-got-this-error-in-using-logstash/348665)
**Author:** [@musk\_elon](https://discuss.elastic.co/u/musk_elon)\ **Replies:** 13\ **Last updated:** [December 7, 2023, 5:11pm UTC](https://discuss.elastic.co/t/i-got-this-error-in-using-logstash/348665 "2023-12-07T17:11:08Z")
Hello. I installed the logstash and set the configuration. # Sample Logstash configuration for creating a simple # Beats -\> Logstash -\> Elasticsearch pipeline. input { file{ type =\>"csv" path =\> "Z:/5/upwork/… --- ## [\[ERROR\] Logstash: \_dateparsefailure for xml output using the date plugin](https://discuss.elastic.co/t/error-logstash-dateparsefailure-for-xml-output-using-the-date-plugin/348742)
**Author:** [@Jospaul](https://discuss.elastic.co/u/Jospaul)\ **Replies:** 1\ **Last updated:** [December 7, 2023, 5:06pm UTC](https://discuss.elastic.co/t/error-logstash-dateparsefailure-for-xml-output-using-the-date-plugin/348742 "2023-12-07T17:06:12Z")
I am getting a dateparsefailure when trying to match the @timestamp with the UNIX\_MS date. I tried it as a separate field and that fails too - Below is the filter - filter { xml { source =\> "message" … --- ## [Running Logstah in Windows](https://discuss.elastic.co/t/running-logstah-in-windows/348749)
**Author:** [@Alberto\_Jimenez1](https://discuss.elastic.co/u/Alberto_Jimenez1)\ **Replies:** 1\ **Last updated:** [December 7, 2023, 10:52am UTC](https://discuss.elastic.co/t/running-logstah-in-windows/348749 "2023-12-07T10:52:37Z")
Im running in Windows Logstah the basic Test Official website recommends: logstash.bat -e "input { stdin { } } output { stdout {} }" but I receive following error in the cmd: \`\`\` "\[FATAL\] 2023-12-06 14:24:21.428 \[ma… --- ## [We are getting two different offset values for same message](https://discuss.elastic.co/t/we-are-getting-two-different-offset-values-for-same-message/348779)
**Author:** [@prashant1](https://discuss.elastic.co/u/prashant1)\ **Replies:** 0\ **Last updated:** [December 7, 2023, 5:44am UTC](https://discuss.elastic.co/t/we-are-getting-two-different-offset-values-for-same-message/348779 "2023-12-07T05:44:39Z")
Hi, We have two logstash pods which are reading the data from elasticsearch from one index for last 24 hr data and then sending data to Kafka server. We can see two different offset are created for similar log message. … --- ## [Logstash service is active, enabled but netstat output shows port not listening](https://discuss.elastic.co/t/logstash-service-is-active-enabled-but-netstat-output-shows-port-not-listening/348695)
**Author:** [@jayadevp](https://discuss.elastic.co/u/jayadevp)\ **Replies:** 17\ **Last updated:** [December 6, 2023, 7:54pm UTC](https://discuss.elastic.co/t/logstash-service-is-active-enabled-but-netstat-output-shows-port-not-listening/348695 "2023-12-06T19:54:58Z")
If i run the command to manually run logstash " sudo /usr/share/logstash/bin/logstash -f "/etc/logstash/conf.d/fortigate.conf" --config.reload.automatic" im able to see the output and netstat also shows port listening … --- ## [Use Logstash for access REST APIs and do complex queries or better Connector Clients](https://discuss.elastic.co/t/use-logstash-for-access-rest-apis-and-do-complex-queries-or-better-connector-clients/348725)
**Author:** [@sebastianboelling](https://discuss.elastic.co/u/sebastianboelling)\ **Replies:** 1\ **Last updated:** [December 6, 2023, 3:47pm UTC](https://discuss.elastic.co/t/use-logstash-for-access-rest-apis-and-do-complex-queries-or-better-connector-clients/348725 "2023-12-06T15:47:55Z")
Hi anybody, has anybody experiences in using Logstash to gather data from a complex REST/JSON API. The API delivers user specific data similar to OneDrive or SharePoint. That means I have to access the (1) users list a… --- ## [Use logstash to connect VMware vCenter API?](https://discuss.elastic.co/t/use-logstash-to-connect-vmware-vcenter-api/348517)
**Author:** [@pyk346](https://discuss.elastic.co/u/pyk346)\ **Replies:** 7\ **Last updated:** [December 6, 2023, 2:33pm UTC](https://discuss.elastic.co/t/use-logstash-to-connect-vmware-vcenter-api/348517 "2023-12-06T14:33:35Z")
I'm trying to utilize the elastic logstash to obtain VMware vcenter datacenter metrics via API but failed to connect them. The vCenter version is 8.0.1. I had successfully configured "syslog" as input and recieved logs… --- ## [Missing metrics in Logstash node stats](https://discuss.elastic.co/t/missing-metrics-in-logstash-node-stats/348710)
**Author:** [@ofekinger](https://discuss.elastic.co/u/ofekinger)\ **Replies:** 3\ **Last updated:** [December 6, 2023, 2:20pm UTC](https://discuss.elastic.co/t/missing-metrics-in-logstash-node-stats/348710 "2023-12-06T14:20:08Z")
Hello everyone, I went over the code for a few Logstash plugins and noticed they had metrics that I can't see when running: curl http://localhost:9600/\_node/stats I'm talking about metrics like: And a few other plac… --- ## [Logstash stdout output text as in file](https://discuss.elastic.co/t/logstash-stdout-output-text-as-in-file/348675)
**Author:** [@carter.kovrov](https://discuss.elastic.co/u/carter.kovrov)\ **Replies:** 6\ **Last updated:** [December 6, 2023, 11:48am UTC](https://discuss.elastic.co/t/logstash-stdout-output-text-as-in-file/348675 "2023-12-06T11:48:31Z")
Hi all Tell me how to display information as in a file without additional fields? For example, there is a file app.log with the contents 12-15-2023 app running... 12-15-2023 app login user test necessary information … --- ## [JDBC Static Filter Plugin - Error handling, how to skip enrichment when Database is down](https://discuss.elastic.co/t/jdbc-static-filter-plugin-error-handling-how-to-skip-enrichment-when-database-is-down/347204)
**Author:** [@tori](https://discuss.elastic.co/u/tori)\ **Replies:** 2\ **Last updated:** [December 6, 2023, 10:40am UTC](https://discuss.elastic.co/t/jdbc-static-filter-plugin-error-handling-how-to-skip-enrichment-when-database-is-down/347204 "2023-12-06T10:40:58Z")
Hi, We've got logstash fetching some information from a MySQL database for log enrichment via JDBC Static Filter Plugin. The settings work just fine when things are working as expected: ... jdbc\_static { l… --- ## [Fields are not populating from logstash to elastic](https://discuss.elastic.co/t/fields-are-not-populating-from-logstash-to-elastic/348593)
**Author:** [@mmercaldi](https://discuss.elastic.co/u/mmercaldi)\ **Replies:** 10\ **Last updated:** [December 5, 2023, 10:44pm UTC](https://discuss.elastic.co/t/fields-are-not-populating-from-logstash-to-elastic/348593 "2023-12-05T22:44:34Z")
I am using logstash to populate elastic I have it set so this filter: filter { json { source =\> "message" target =\> "jsoncontent" remove\_field =\> \["message"\] } } and jsoncontent: {"switchname": "swi… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=47) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=49)