# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=90 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 91 --- ## [Implement User interface buttons for Logstash](https://discuss.elastic.co/t/implement-user-interface-buttons-for-logstash/323641)
**Author:** [@Sandeep\_Raju](https://discuss.elastic.co/u/Sandeep_Raju)\ **Replies:** 1\ **Last updated:** [February 14, 2023, 3:42pm UTC](https://discuss.elastic.co/t/implement-user-interface-buttons-for-logstash/323641 "2023-02-14T15:42:39Z")
Hi all, I am currently pushing data to indices in my elasticsearch 8.4 using logstash in my terminal. However , One of my friends does not know logstash and wants to work with logstash in User interface and push data i… --- ## [Logstash fetched data not available in elastic search](https://discuss.elastic.co/t/logstash-fetched-data-not-available-in-elastic-search/325216)
**Author:** [@ekambaram\_varathan](https://discuss.elastic.co/u/ekambaram_varathan)\ **Replies:** 1\ **Last updated:** [February 14, 2023, 3:26am UTC](https://discuss.elastic.co/t/logstash-fetched-data-not-available-in-elastic-search/325216 "2023-02-14T03:26:58Z")
Hi Team, I am using ELK version: 6.8.23. Though Logstash uploaded csv file data are not present in elasticsearch. my logstash conf file content as follows, input { file { path =\> "/home/data/reports/\*.csv" … --- ## [Remove json object from nested log](https://discuss.elastic.co/t/remove-json-object-from-nested-log/325468)
**Author:** [@Sharoze\_Meraj](https://discuss.elastic.co/u/Sharoze_Meraj)\ **Replies:** 0\ **Last updated:** [February 14, 2023, 12:03pm UTC](https://discuss.elastic.co/t/remove-json-object-from-nested-log/325468 "2023-02-14T12:03:08Z")
How can I use ruby code or some other filter plugin to detect and remove json object fields from my nested json logs. This is required because the fields can either be json objects or strings. If I remove the json objec… --- ## [Logstash issue](https://discuss.elastic.co/t/logstash-issue/325414)
**Author:** [@namdev](https://discuss.elastic.co/u/namdev)\ **Replies:** 1\ **Last updated:** [February 14, 2023, 10:00am UTC](https://discuss.elastic.co/t/logstash-issue/325414 "2023-02-14T10:00:44Z")
Hi , I am new to elk stack,I want to create a new field which is the difference between two dates field. I want to do it in logstash. The two dates field data is given. I am using filter like this but not getting the … --- ## [Ruby exception occurred: undefined method \`\*' for nil:NilClass](https://discuss.elastic.co/t/ruby-exception-occurred-undefined-method-for-nil-nilclass/325411)
**Author:** [@mc96](https://discuss.elastic.co/u/mc96)\ **Replies:** 2\ **Last updated:** [February 14, 2023, 9:25am UTC](https://discuss.elastic.co/t/ruby-exception-occurred-undefined-method-for-nil-nilclass/325411 "2023-02-14T09:25:03Z")
I have a filter that is as follows: event.set('\[json\]\[event\]\[packetloss1\]', event.get('\[packets-received\]') / event.get('\[packets-sent\]') \* 100) event.set('\[json\]\[event\]\[packetlosspercentage\]', 100 - event.get('\[json\]… --- ## [Logstash error: Failed to publish events](https://discuss.elastic.co/t/logstash-error-failed-to-publish-events/325400)
**Author:** [@Zack\_09](https://discuss.elastic.co/u/Zack_09)\ **Replies:** 3\ **Last updated:** [February 13, 2023, 4:11pm UTC](https://discuss.elastic.co/t/logstash-error-failed-to-publish-events/325400 "2023-02-13T16:11:57Z")
Hello all ,im new in elastic when I run logstash i get the following error Error: Cannot assign requested address desktop-logstash-1 | Exception: Java::JavaNet::BindException desktop-logstash-1 | Stack: sun… --- ## [Logstash to Elastic Multiple Connections](https://discuss.elastic.co/t/logstash-to-elastic-multiple-connections/325382)
**Author:** [@yago82](https://discuss.elastic.co/u/yago82)\ **Replies:** 1\ **Last updated:** [February 13, 2023, 3:01pm UTC](https://discuss.elastic.co/t/logstash-to-elastic-multiple-connections/325382 "2023-02-13T15:01:36Z")
Hello everyone, I was wandering if you can help in this particular matter: the actual situation is that we maintain a large Elastic Stack (15+ nodes) with an ingestion workflow composed by two Logstash server on VMs, wi… --- ## [Logstash JMS Input version 3.1.2](https://discuss.elastic.co/t/logstash-jms-input-version-3-1-2/325355)
**Author:** [@m3bgwad](https://discuss.elastic.co/u/m3bgwad)\ **Replies:** 1\ **Last updated:** [February 13, 2023, 2:11pm UTC](https://discuss.elastic.co/t/logstash-jms-input-version-3-1-2/325355 "2023-02-13T14:11:38Z")
Hello, I want to know what JMS version is compatible with Logstash JMS Input version 3.1.2 and can you share with me some references? Thank you, --- ## [Change @Timestamp to date from API response](https://discuss.elastic.co/t/change-timestamp-to-date-from-api-response/325369)
**Author:** [@Renat](https://discuss.elastic.co/u/Renat)\ **Replies:** 0\ **Last updated:** [February 13, 2023, 9:46am UTC](https://discuss.elastic.co/t/change-timestamp-to-date-from-api-response/325369 "2023-02-13T09:46:14Z")
Hello everyone, first of all i'm sorry if this common issue, but i really tried to solve it by myself. but searching in web didn't help me, may be because i never used Logstash. So i got request to receive "slowQuer… --- ## [Failed to parse date field](https://discuss.elastic.co/t/failed-to-parse-date-field/325324)
**Author:** [@juancamiloll](https://discuss.elastic.co/u/juancamiloll)\ **Replies:** 5\ **Last updated:** [February 12, 2023, 3:31am UTC](https://discuss.elastic.co/t/failed-to-parse-date-field/325324 "2023-02-12T03:31:45Z")
I currently have a problem with an error message that is appearing that refers to a date field but is not detecting it as such. \[2023-02-11T15:16:39,111\]\[WARN \]\[logstash.outputs.elasticsearch\]\[main\] Could not index even… --- ## [Kafka input - resync missing items from topic](https://discuss.elastic.co/t/kafka-input-resync-missing-items-from-topic/325294)
**Author:** [@Chris\_Denneen](https://discuss.elastic.co/u/Chris_Denneen)\ **Replies:** 7\ **Last updated:** [February 12, 2023, 2:04am UTC](https://discuss.elastic.co/t/kafka-input-resync-missing-items-from-topic/325294 "2023-02-12T02:04:14Z")
Ran into issue where we have logstash input reading kafka topics for log events. Last night the ES index rolled over and the write alias was lost (not on the new index... so nothing with is\_write\_index = true) therefore… --- ## [Cache mechanism and log when transfer fails](https://discuss.elastic.co/t/cache-mechanism-and-log-when-transfer-fails/325230)
**Author:** [@YasuhiroOkumura](https://discuss.elastic.co/u/YasuhiroOkumura)\ **Replies:** 1\ **Last updated:** [February 12, 2023, 1:26am UTC](https://discuss.elastic.co/t/cache-mechanism-and-log-when-transfer-fails/325230 "2023-02-12T01:26:05Z")
1.When transferring messages from logstash(8.6) to pagerduty using pagerduty output plugin, if the transfer fails, is it possible to cache and resend? 2.When using the pagerduty output plugin to transfer messages from l… --- ## [Help. My filebeat stops working after I installed my wazuh server after 30 minutes](https://discuss.elastic.co/t/help-my-filebeat-stops-working-after-i-installed-my-wazuh-server-after-30-minutes/324536)
**Author:** [@Joshua\_John\_Consulta](https://discuss.elastic.co/u/Joshua_John_Consulta)\ **Replies:** 1\ **Last updated:** [February 12, 2023, 1:20am UTC](https://discuss.elastic.co/t/help-my-filebeat-stops-working-after-i-installed-my-wazuh-server-after-30-minutes/324536 "2023-02-12T01:20:01Z")
Here's the error: × filebeat.service - Filebeat sends log files to Logstash or directly to Elasti\> Loaded: loaded (/lib/systemd/system/filebeat.service; enabled; preset: ena\> Active: failed (Result: exit-code) since T… --- ## [Logstash heartbeat input error](https://discuss.elastic.co/t/logstash-heartbeat-input-error/325315)
**Author:** [@mostafaelsayed](https://discuss.elastic.co/u/mostafaelsayed)\ **Replies:** 2\ **Last updated:** [February 11, 2023, 8:34pm UTC](https://discuss.elastic.co/t/logstash-heartbeat-input-error/325315 "2023-02-11T20:34:40Z")
Hello All, I am using heartbeat input to perform a specific job periodically. However, on my local windows machine, I occasionally keep getting this error message \[2023-02-09T02:35:37,209\]\[ERROR\]\[logstash.javapipeline… --- ## [Remove plain text message in Logstash file input](https://discuss.elastic.co/t/remove-plain-text-message-in-logstash-file-input/325269)
**Author:** [@djrshn2346](https://discuss.elastic.co/u/djrshn2346)\ **Replies:** 1\ **Last updated:** [February 10, 2023, 1:24pm UTC](https://discuss.elastic.co/t/remove-plain-text-message-in-logstash-file-input/325269 "2023-02-10T13:24:17Z")
I am adding filter to remove a plain text as it causes error while JSON parsing. file { id =\> "my\_lt\_log" path =\> "/logs/logtransformer.log" type =\> "log" start\_position =\> "beginning" … --- ## [Gelf Input plugin dropping messages](https://discuss.elastic.co/t/gelf-input-plugin-dropping-messages/325260)
**Author:** [@karlo95](https://discuss.elastic.co/u/karlo95)\ **Replies:** 0\ **Last updated:** [February 10, 2023, 12:08pm UTC](https://discuss.elastic.co/t/gelf-input-plugin-dropping-messages/325260 "2023-02-10T12:08:36Z")
Hello, I'm having problems with Gelf Input plugin dropping messages when listening on UDP. When a lot of messagess comes in same time, it seems like logstash plugin is dropping them randomly. E.g. when I restart quark… --- ## [Grok filter request for json/custom pattern](https://discuss.elastic.co/t/grok-filter-request-for-json-custom-pattern/325135)
**Author:** [@a.emrekaraman](https://discuss.elastic.co/u/a.emrekaraman)\ **Replies:** 0\ **Last updated:** [February 9, 2023, 11:39am UTC](https://discuss.elastic.co/t/grok-filter-request-for-json-custom-pattern/325135 "2023-02-09T11:39:16Z")
Hi Team, I installed logstash and try to create right grok filter for my logs to parse it. How can I parse below logs ? ( timestamp seems as custom.%{TIMESTAMP\_ISO8601:timestamp}" doesnt work) 09-Feb-2023 09:52:49 tmp… --- ## [GROK Pattern creation](https://discuss.elastic.co/t/grok-pattern-creation/324605)
**Author:** [@anushka1203](https://discuss.elastic.co/u/anushka1203)\ **Replies:** 7\ **Last updated:** [February 10, 2023, 6:25am UTC](https://discuss.elastic.co/t/grok-pattern-creation/324605 "2023-02-10T06:25:33Z")
Hi all, Need help in creating grok pattern that works for both the following type of logs 01/25-05:17:51.314622 192.168.1.1:138 -\> 192.168.1.255:138 UDP TTL:64 TOS:0x0 ID:50222 IpLen:20 DgmLen:229 DF Len: 201 =+=+=+=+=… --- ## [Logstash to Elasticsearch connection](https://discuss.elastic.co/t/logstash-to-elasticsearch-connection/325198)
**Author:** [@vassiliy.vins](https://discuss.elastic.co/u/vassiliy.vins)\ **Replies:** 0\ **Last updated:** [February 9, 2023, 9:39pm UTC](https://discuss.elastic.co/t/logstash-to-elasticsearch-connection/325198 "2023-02-09T21:39:17Z")
Hello! I have filebeat running on 192.168.035 and ELK on 192.168.0.36 (name of the remote server is not good - " logstash" which can be confusing) I'm runnnig filebeat -e -c filebeat.yml -d "publish" using logstash… --- ## [Changing date format through Logstash](https://discuss.elastic.co/t/changing-date-format-through-logstash/325061)
**Author:** [@zaeemmasood](https://discuss.elastic.co/u/zaeemmasood)\ **Replies:** 4\ **Last updated:** [February 9, 2023, 9:39pm UTC](https://discuss.elastic.co/t/changing-date-format-through-logstash/325061 "2023-02-09T21:39:16Z")
Hello. We are getting feed from Kafka topic. One of the fields rdt appears in the following format: I need to change the date format to YYYY-MM-dd This was added in the logstash conf file: if "KafkCollect" in … --- ## [Http filter in Logstash](https://discuss.elastic.co/t/http-filter-in-logstash/325146)
**Author:** [@manikanta](https://discuss.elastic.co/u/manikanta)\ **Replies:** 2\ **Last updated:** [February 9, 2023, 8:28pm UTC](https://discuss.elastic.co/t/http-filter-in-logstash/325146 "2023-02-09T20:28:42Z")
hey I want to know how to pass a filters output as request body to http filter I am trying to pass output of mutate filter as body to http filter. mutate { remove\_field =\> \[ "@timestamp", "@version"\] } … --- ## [Need to know ways of controlling the write to storage account for event hub access](https://discuss.elastic.co/t/need-to-know-ways-of-controlling-the-write-to-storage-account-for-event-hub-access/325179)
**Author:** [@karthik\_Ravichandran](https://discuss.elastic.co/u/karthik_Ravichandran)\ **Replies:** 2\ **Last updated:** [February 9, 2023, 6:27pm UTC](https://discuss.elastic.co/t/need-to-know-ways-of-controlling-the-write-to-storage-account-for-event-hub-access/325179 "2023-02-09T18:27:56Z")
since its writing every 5 seconds from azure event hub to storage , its causing more cost utilization , instead we want to make it delay so that we can reduce the cost of ingesting the events , we are not having any iss… --- ## [Unable to connect logstash 8.6.1 to elastic 6.8.23 with SSL](https://discuss.elastic.co/t/unable-to-connect-logstash-8-6-1-to-elastic-6-8-23-with-ssl/324932)
**Author:** [@Patrick\_Lacson](https://discuss.elastic.co/u/Patrick_Lacson)\ **Replies:** 7\ **Last updated:** [February 9, 2023, 6:21pm UTC](https://discuss.elastic.co/t/unable-to-connect-logstash-8-6-1-to-elastic-6-8-23-with-ssl/324932 "2023-02-09T18:21:06Z")
I'm able to connect to elasticsearch 6.8.23 with logstash 8.6.1 but when I connect to an SSL enabled elasticsearch (using signed CERTS), I get the 503 error unable to connect. My output config looks like this: It works… --- ## [Update by query with Logstash http output message](https://discuss.elastic.co/t/update-by-query-with-logstash-http-output-message/325180)
**Author:** [@Carlitoz](https://discuss.elastic.co/u/Carlitoz)\ **Replies:** 0\ **Last updated:** [February 9, 2023, 4:56pm UTC](https://discuss.elastic.co/t/update-by-query-with-logstash-http-output-message/325180 "2023-02-09T16:56:04Z")
I am using Logstash to update by query existing Elasticsearch documents with an additional field that contains aggregate values extracted from Potgresql table. I use elastichsearch output to load one index using document… --- ## [Logstash syslog fields not removed in index](https://discuss.elastic.co/t/logstash-syslog-fields-not-removed-in-index/325060)
**Author:** [@gt2847c](https://discuss.elastic.co/u/gt2847c)\ **Replies:** 3\ **Last updated:** [February 9, 2023, 1:25pm UTC](https://discuss.elastic.co/t/logstash-syslog-fields-not-removed-in-index/325060 "2023-02-09T13:25:39Z")
I created a config file to ingest Cisco syslog output. When I run the config via command line (/usr/share/logstash/bin/logstash -f cisco.conf -r) everything works as expected. The fields I want show up properly in both… --- ## [Fetching, ingesting and merging data from REST API in 2023](https://discuss.elastic.co/t/fetching-ingesting-and-merging-data-from-rest-api-in-2023/325143)
**Author:** [@NiklasHBB](https://discuss.elastic.co/u/NiklasHBB)\ **Replies:** 0\ **Last updated:** [February 9, 2023, 1:07pm UTC](https://discuss.elastic.co/t/fetching-ingesting-and-merging-data-from-rest-api-in-2023/325143 "2023-02-09T13:07:18Z")
I want to fetch and ingest data from several REST APIs. In previous discussions in 2019 and 2020 it was recommended to use logstash http\_poller input plugin. Is that still the best way to go with fleet and elastic agents… --- ## [How to use PagerDuty plugin to insert data into pagerduty payload](https://discuss.elastic.co/t/how-to-use-pagerduty-plugin-to-insert-data-into-pagerduty-payload/324524)
**Author:** [@YasuhiroOkumura](https://discuss.elastic.co/u/YasuhiroOkumura)\ **Replies:** 3\ **Last updated:** [February 9, 2023, 5:15am UTC](https://discuss.elastic.co/t/how-to-use-pagerduty-plugin-to-insert-data-into-pagerduty-payload/324524 "2023-02-09T05:15:48Z")
I am trying to send data to PageDuty side using PagerDuty plugin in Logstash output plugin. I want to insert data into the "payload" field of PagerDuty, but when I use the PagerDuty plugin, the data goes into the "detai… --- ## [I am getting Mapping\_parsing error in logtsah](https://discuss.elastic.co/t/i-am-getting-mapping-parsing-error-in-logtsah/325083)
**Author:** [@upreddy](https://discuss.elastic.co/u/upreddy)\ **Replies:** 0\ **Last updated:** [February 9, 2023, 4:52am UTC](https://discuss.elastic.co/t/i-am-getting-mapping-parsing-error-in-logtsah/325083 "2023-02-09T04:52:35Z")
Hi, We can see there are different values for "partition" for the "XYZ" operation logs (logtype also same). loglines are: (1). 2023-01-05T20:46:21.36348538Z stdout F 2023-01-05 20:46:21.363 \[INFO\] - {"logtype":"ABC"… --- ## [Changed field in filebeat but not working in logstash](https://discuss.elastic.co/t/changed-field-in-filebeat-but-not-working-in-logstash/324925)
**Author:** [@yc99](https://discuss.elastic.co/u/yc99)\ **Replies:** 1\ **Last updated:** [February 8, 2023, 9:16pm UTC](https://discuss.elastic.co/t/changed-field-in-filebeat-but-not-working-in-logstash/324925 "2023-02-08T21:16:42Z")
I changed the source from access to admin and restarted the filebeat service, when I use grok to filter only the admin source, it is empty, but if using grok to filter only the access, it working. It should be source adm… --- ## [JDBC Pipeline Log data question](https://discuss.elastic.co/t/jdbc-pipeline-log-data-question/325058)
**Author:** [@nbrenke](https://discuss.elastic.co/u/nbrenke)\ **Replies:** 0\ **Last updated:** [February 8, 2023, 7:34pm UTC](https://discuss.elastic.co/t/jdbc-pipeline-log-data-question/325058 "2023-02-08T19:34:56Z")
I have several pipelines that run on a timed basis. They are all JDBC pipelines that ingest data from a SQL database. I am looking to see if it's possible to have the output to the logs state the time the ingest starte… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=89) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=91)