# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=194 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 195 --- ## [Bearer token in HTTP filter](https://discuss.elastic.co/t/bearer-token-in-http-filter/284641)
**Author:** [@Juan\_Montoya](https://discuss.elastic.co/u/Juan_Montoya)\ **Replies:** 0\ **Last updated:** [September 20, 2021, 5:03pm UTC](https://discuss.elastic.co/t/bearer-token-in-http-filter/284641 "2021-09-20T17:03:53Z")
Hi, I'm trying to get data from an HTTP API, the main API needs a bearer token to authenticate, a secondary API gives me the token that allows the authentication in the main API. I Solved using http\_poller input to get … --- ## [Logstash ruby length does not count correctly](https://discuss.elastic.co/t/logstash-ruby-length-does-not-count-correctly/284323)
**Author:** [@Rauly\_Koto](https://discuss.elastic.co/u/Rauly_Koto)\ **Replies:** 2\ **Last updated:** [September 20, 2021, 1:19pm UTC](https://discuss.elastic.co/t/logstash-ruby-length-does-not-count-correctly/284323 "2021-09-20T13:19:19Z")
The execution\_length\_output count string correctly, but execution\_length\_input string is to low. ruby { code =\> "event.set('execution\_length\_output', event.get('\[execution\]\[output\]').length) event.set('execution\_lengt… --- ## [Check if event field does not exist when 'nil' should be considered as "exists"](https://discuss.elastic.co/t/check-if-event-field-does-not-exist-when-nil-should-be-considered-as-exists/284605)
**Author:** [@e.sharshenaliev](https://discuss.elastic.co/u/e.sharshenaliev)\ **Replies:** 1\ **Last updated:** [September 20, 2021, 1:09pm UTC](https://discuss.elastic.co/t/check-if-event-field-does-not-exist-when-nil-should-be-considered-as-exists/284605 "2021-09-20T13:09:29Z")
We are indexing metrics that arrive to logstash in json format There is a value "MeasurementDate" that can either: be missing has null value has date value (iso8601) When field has valid value or is missing - we need… --- ## [Logstash service getting restarted after trying to set keystore on \`v7.14\`](https://discuss.elastic.co/t/logstash-service-getting-restarted-after-trying-to-set-keystore-on-v7-14/284582)
**Author:** [@prat](https://discuss.elastic.co/u/prat)\ **Replies:** 15\ **Last updated:** [September 20, 2021, 2:28am UTC](https://discuss.elastic.co/t/logstash-service-getting-restarted-after-trying-to-set-keystore-on-v7-14/284582 "2021-09-20T02:28:40Z")
Hi Team, I am trying to set logstash keystore but having hard time. I have refer below link but still its not working. (not creating keystore password from below) I am creating keystore as, echo y | /usr/share/log… --- ## [Failed to Parse Date Field](https://discuss.elastic.co/t/failed-to-parse-date-field/284576)
**Author:** [@Tim\_Mobley](https://discuss.elastic.co/u/Tim_Mobley)\ **Replies:** 2\ **Last updated:** [September 19, 2021, 10:34pm UTC](https://discuss.elastic.co/t/failed-to-parse-date-field/284576 "2021-09-19T22:34:18Z")
I have some CSV files I'm trying to ingest using Logstash's CVS filter plugin. I'm needing the "Date" column to represent the event time field in Kibana (rather than @timestamp, which just tells me when I imported the CS… --- ## [Logstash bug while removing field](https://discuss.elastic.co/t/logstash-bug-while-removing-field/284562)
**Author:** [@zeoiioze](https://discuss.elastic.co/u/zeoiioze)\ **Replies:** 0\ **Last updated:** [September 19, 2021, 12:13pm UTC](https://discuss.elastic.co/t/logstash-bug-while-removing-field/284562 "2021-09-19T12:13:57Z")
Hey, I realized that the following line : remove\_field =\> \[ "\[doc\]\[vegetables\]\[%{\[tmp\_vegetable\_data\]\[name\]}\]" \] Was not only removing \[doc\]\[vegetables\]\[%{\[tmp\_vegetable\_data\]\[name\]}\], but was also like removing \[tmp\_… --- ## [Can't send winlogbeat data to logstash](https://discuss.elastic.co/t/cant-send-winlogbeat-data-to-logstash/284489)
**Author:** [@Minh\_Ti\_n\_Tr\_n](https://discuss.elastic.co/u/Minh_Ti_n_Tr_n)\ **Replies:** 2\ **Last updated:** [September 19, 2021, 10:51am UTC](https://discuss.elastic.co/t/cant-send-winlogbeat-data-to-logstash/284489 "2021-09-19T10:51:23Z")
Hi all, I now do the ELK lab with docker The repo: https://github.com/deviantony/docker-elk/ ELK stack host IP: 192.168.87.52 My logstash/pipeline/logstash.conf input { beats { port =\> 5044 … --- ## [How can I combine some mysql query events with logstash aggregate filter?](https://discuss.elastic.co/t/how-can-i-combine-some-mysql-query-events-with-logstash-aggregate-filter/284533)
**Author:** [@e-fo](https://discuss.elastic.co/u/e-fo)\ **Replies:** 2\ **Last updated:** [September 19, 2021, 6:13am UTC](https://discuss.elastic.co/t/how-can-i-combine-some-mysql-query-events-with-logstash-aggregate-filter/284533 "2021-09-19T06:13:09Z")
Hello, I implemented pipeline with JDBC input plugin, my table in mysql follow the EAV (Entity-Attribute-Value) structure, therefore I can't get all attributes that related to one user (Entity) in one row with mysql que… --- ## [How to match 2 key value pairs in unstructured string that looks like column](https://discuss.elastic.co/t/how-to-match-2-key-value-pairs-in-unstructured-string-that-looks-like-column/284541)
**Author:** [@ansamHox](https://discuss.elastic.co/u/ansamHox)\ **Replies:** 10\ **Last updated:** [September 18, 2021, 7:15pm UTC](https://discuss.elastic.co/t/how-to-match-2-key-value-pairs-in-unstructured-string-that-looks-like-column/284541 "2021-09-18T19:15:51Z")
I have a document with some lines like: Example requested: 24:00:00 Example Used: 01:14:11 What I want is to have is: { "example\_requested": "24:00:00", "example\_used": 01:14:11 } What I tried (t… --- ## [Check if a field exists](https://discuss.elastic.co/t/check-if-a-field-exists/284545)
**Author:** [@zeoiioze](https://discuss.elastic.co/u/zeoiioze)\ **Replies:** 1\ **Last updated:** [September 18, 2021, 4:25pm UTC](https://discuss.elastic.co/t/check-if-a-field-exists/284545 "2021-09-18T16:25:50Z")
I'm using nested fields and a field name in the nested field name, as you can see below (example) : \[doc\]\[vegetables\]\[%{\[tmp\_data\]\[vegetable\_name\]}\]\[number\] The field above is getting created, but now, I want in anot… --- ## [Match complete line after some regex pattern](https://discuss.elastic.co/t/match-complete-line-after-some-regex-pattern/284385)
**Author:** [@ansamHox](https://discuss.elastic.co/u/ansamHox)\ **Replies:** 9\ **Last updated:** [September 17, 2021, 9:57pm UTC](https://discuss.elastic.co/t/match-complete-line-after-some-regex-pattern/284385 "2021-09-17T21:57:55Z")
I have lines in document like this. \*\*\* Begin time: Sat Jun 26 21:11:14 AEST 2019 Want to assign new field begin\_time = Sat Jun 26 21:11:14 AEST 2019 This is what I tried: if x =~ /(\\bBegin time:\\s+(.\*))/ … --- ## [Pipeline.workers configuration and aggregation filter](https://discuss.elastic.co/t/pipeline-workers-configuration-and-aggregation-filter/284461)
**Author:** [@Roberto\_B](https://discuss.elastic.co/u/Roberto_B)\ **Replies:** 8\ **Last updated:** [September 17, 2021, 8:04pm UTC](https://discuss.elastic.co/t/pipeline-workers-configuration-and-aggregation-filter/284461 "2021-09-17T20:04:38Z")
Hi all, I'm going to use a brand new server with 4 vCPU and 16GB RAM, I've some pipelines (+60) and I'll run multiple pipeline (for eg: 1 pipeline for 10 "easy pipelines" 8 for "medium pipelines" and so on). Some of th… --- ## [Parse Hostname from offline logs and then apply to all events](https://discuss.elastic.co/t/parse-hostname-from-offline-logs-and-then-apply-to-all-events/284467)
**Author:** [@sirReeall](https://discuss.elastic.co/u/sirReeall)\ **Replies:** 6\ **Last updated:** [September 17, 2021, 3:16pm UTC](https://discuss.elastic.co/t/parse-hostname-from-offline-logs-and-then-apply-to-all-events/284467 "2021-09-17T15:16:23Z")
Hi, I'm building an offline log parser using logstash, ES and Kibana. In this scenario the logs are provided in a zip from servers. They are placed in a volume mounted into the logstash container, then pushed into ES an… --- ## [Split field into multiple fields](https://discuss.elastic.co/t/split-field-into-multiple-fields/284096)
**Author:** [@mavericknd](https://discuss.elastic.co/u/mavericknd)\ **Replies:** 3\ **Last updated:** [September 17, 2021, 2:14pm UTC](https://discuss.elastic.co/t/split-field-into-multiple-fields/284096 "2021-09-17T14:14:29Z")
Hi, i have the following log and want the key value pairs inside message (ipAddress=1.1.1.1 realmId=some\_realm) to be separate fields. I tried with grok, kv, mutate, nothing works, no change in kibana. If somebody can … --- ## [Elapsed Plugin 'expired' events create new index with incorrect field mapping](https://discuss.elastic.co/t/elapsed-plugin-expired-events-create-new-index-with-incorrect-field-mapping/284486)
**Author:** [@stevedearl](https://discuss.elastic.co/u/stevedearl)\ **Replies:** 0\ **Last updated:** [September 17, 2021, 1:29pm UTC](https://discuss.elastic.co/t/elapsed-plugin-expired-events-create-new-index-with-incorrect-field-mapping/284486 "2021-09-17T13:29:35Z")
Hi All, I had an issue a few months ago where a float field in my documents began to be displayed as a numeric (no decimal values). The field was 'elapsedtime' which is measured seconds/millis. I raised this here in an… --- ## [Less than or equal not working logstash filter and crashing](https://discuss.elastic.co/t/less-than-or-equal-not-working-logstash-filter-and-crashing/283729)
**Author:** [@Shaiju\_Sam](https://discuss.elastic.co/u/Shaiju_Sam)\ **Replies:** 21\ **Last updated:** [September 17, 2021, 10:02am UTC](https://discuss.elastic.co/t/less-than-or-equal-not-working-logstash-filter-and-crashing/283729 "2021-09-17T10:02:19Z")
Hi Team, I'm shipping logs from windows servers using filebeat. I have created a field which shows the reponse time in milliseconds. I have created a field "Create\_Indexing\_response\_time\_ms" using grok and it works per… --- ## [Unable to connect with GCP pubsub through logstash](https://discuss.elastic.co/t/unable-to-connect-with-gcp-pubsub-through-logstash/284470)
**Author:** [@anandsaini014](https://discuss.elastic.co/u/anandsaini014)\ **Replies:** 0\ **Last updated:** [September 17, 2021, 9:50am UTC](https://discuss.elastic.co/t/unable-to-connect-with-gcp-pubsub-through-logstash/284470 "2021-09-17T09:50:22Z")
Hi Team, I am trying to use Google pubsub input plugin through logstash. Followed this link: I am getting the following error: com.google.cloud.pubsub.v1.StreamingSubscriberConnection$1 onFailure SEVERE: terminate… --- ## [Ingest fixed length data from internet](https://discuss.elastic.co/t/ingest-fixed-length-data-from-internet/284142)
**Author:** [@sparmar2000](https://discuss.elastic.co/u/sparmar2000)\ **Replies:** 2\ **Last updated:** [September 17, 2021, 6:30am UTC](https://discuss.elastic.co/t/ingest-fixed-length-data-from-internet/284142 "2021-09-17T06:30:20Z")
I have internet based web site, fixed length data, that I need to ingest. Question: Can beats be used - assuming have to use Logtash The input has about 100 fixed length. Is there a 'easy' way to define the fields - r… --- ## [Logstash with snmp does not run for multiple 'GET' requests](https://discuss.elastic.co/t/logstash-with-snmp-does-not-run-for-multiple-get-requests/284300)
**Author:** [@Juan\_David\_Jaramillo](https://discuss.elastic.co/u/Juan_David_Jaramillo)\ **Replies:** 10\ **Last updated:** [September 16, 2021, 7:42pm UTC](https://discuss.elastic.co/t/logstash-with-snmp-does-not-run-for-multiple-get-requests/284300 "2021-09-16T19:42:53Z")
Good morning everyone, I have a problem when running my configuration file in logstash, it does not show me the metrics that I am requesting with the GET method, I've been doing some tests and I realized that when I try… --- ## [Outputting to Kafka is failing with obscure error](https://discuss.elastic.co/t/outputting-to-kafka-is-failing-with-obscure-error/284429)
**Author:** [@bgingras](https://discuss.elastic.co/u/bgingras)\ **Replies:** 0\ **Last updated:** [September 16, 2021, 7:18pm UTC](https://discuss.elastic.co/t/outputting-to-kafka-is-failing-with-obscure-error/284429 "2021-09-16T19:18:41Z")
I am using a Kafka output plugin to send syslogs to a topic but am encountering errors: \[org.apache.kafka.common.protocol.Errors\]\[main\] Unexpected error code: 87. As well as the following: KafkaProducer.send() failed… --- ## [SQSSNSS3 plugin error queue not valid for endpoint](https://discuss.elastic.co/t/sqssnss3-plugin-error-queue-not-valid-for-endpoint/282685)
**Author:** [@afoster](https://discuss.elastic.co/u/afoster)\ **Replies:** 39\ **Last updated:** [September 16, 2021, 6:14pm UTC](https://discuss.elastic.co/t/sqssnss3-plugin-error-queue-not-valid-for-endpoint/282685 "2021-09-16T18:14:13Z")
cannot establish connection to amazon SQS the address https://sqs.us-gov-east-1.amazonaws is not valid for this endpoint. is there any options or suggestions to troubleshoot this further --- ## [Exclusão de Index no elasticsearch via logstash](https://discuss.elastic.co/t/exclusao-de-index-no-elasticsearch-via-logstash/283302)
**Author:** [@lucianojr](https://discuss.elastic.co/u/lucianojr)\ **Replies:** 1\ **Last updated:** [September 3, 2021, 3:57pm UTC](https://discuss.elastic.co/t/exclusao-de-index-no-elasticsearch-via-logstash/283302 "2021-09-03T15:57:07Z")
Olá a todos. Estamos iniciando o uso da elastic. Estou precisando saber como faço para fazer a exclusão de um index no elasticsearch, porém, via logstash, e após fazer a nova carga das informações. Caso não seja possível… --- ## [SQL Server data to Elasticsearch using LogStash](https://discuss.elastic.co/t/sql-server-data-to-elasticsearch-using-logstash/283879)
**Author:** [@Navlesh](https://discuss.elastic.co/u/Navlesh)\ **Replies:** 3\ **Last updated:** [September 16, 2021, 5:49am UTC](https://discuss.elastic.co/t/sql-server-data-to-elasticsearch-using-logstash/283879 "2021-09-16T05:49:40Z")
Hi input { jdbc { # SqlServer jdbc connection string to your database, productdb # "jdbc:sqlserver://HostName\\instanceName;database=DBName;user=UserName;password=Password" jdbc\_connection\_string =\> "jdbc:sqlserver:… --- ## [How to Autoreload kafka broker details in Logstash if any new broker added to the kafka cluster](https://discuss.elastic.co/t/how-to-autoreload-kafka-broker-details-in-logstash-if-any-new-broker-added-to-the-kafka-cluster/284350)
**Author:** [@suresh\_choudhary](https://discuss.elastic.co/u/suresh_choudhary)\ **Replies:** 0\ **Last updated:** [September 16, 2021, 3:42am UTC](https://discuss.elastic.co/t/how-to-autoreload-kafka-broker-details-in-logstash-if-any-new-broker-added-to-the-kafka-cluster/284350 "2021-09-16T03:42:46Z")
I am publishing data to Kafka using Logstash output. If the load increases, My Kafka cluster will add a new broker to the cluster. How to feed these changes(new broker IP and port) to Logstash without restarting it. Will… --- ## [Logstash tags \_dateparsefailure - How to separate pasted fields or fields together?](https://discuss.elastic.co/t/logstash-tags-dateparsefailure-how-to-separate-pasted-fields-or-fields-together/284318)
**Author:** [@Cristiane\_Marcarini](https://discuss.elastic.co/u/Cristiane_Marcarini)\ **Replies:** 3\ **Last updated:** [September 15, 2021, 9:15pm UTC](https://discuss.elastic.co/t/logstash-tags-dateparsefailure-how-to-separate-pasted-fields-or-fields-together/284318 "2021-09-15T21:15:05Z")
Hi! I have logs that have fields together or fields pasted together, and I don't know how to separate them. Here's a screenshot of how it looks in Kibana and another how it looks in Logstash. I need help with this as I… --- ## [Logstash filter plugin development](https://discuss.elastic.co/t/logstash-filter-plugin-development/284306)
**Author:** [@wssnetwork](https://discuss.elastic.co/u/wssnetwork)\ **Replies:** 0\ **Last updated:** [September 15, 2021, 3:44pm UTC](https://discuss.elastic.co/t/logstash-filter-plugin-development/284306 "2021-09-15T15:44:30Z")
Hello team. I would like to develop Logstash filter plugin. During build the gem file I got error as below: FAILURE: Build failed with an exception. \* Where: Script '/home/kali/github/logstash/rubyUtils.gradle' line: 2… --- ## [GROK pattern for syslogs](https://discuss.elastic.co/t/grok-pattern-for-syslogs/284174)
**Author:** [@Christer\_Palmen](https://discuss.elastic.co/u/Christer_Palmen)\ **Replies:** 3\ **Last updated:** [September 15, 2021, 12:02pm UTC](https://discuss.elastic.co/t/grok-pattern-for-syslogs/284174 "2021-09-15T12:02:36Z")
"Hello. I have several sources of syslogs that I want to filter with logstash grok, have some issue and questions about this syslog event and how to use grok. {"@version":"1","message":"\<44\> Sep 14 09:01:09 172.24.4.202… --- ## [Pattern matching issues in Logstash](https://discuss.elastic.co/t/pattern-matching-issues-in-logstash/284255)
**Author:** [@Vajb12](https://discuss.elastic.co/u/Vajb12)\ **Replies:** 1\ **Last updated:** [September 15, 2021, 10:27am UTC](https://discuss.elastic.co/t/pattern-matching-issues-in-logstash/284255 "2021-09-15T10:27:13Z")
I'm having issues with Pattern matching with Logstash. Sample log pattern \[DEBUG\] 2021-09-13T23:58:24.361 \[http-nio-8080-exec-1\] \[FB-3D\] localhost - \[i.i.i.a.f.AuthFilter\] :: doFilter :: formName B-3D Grok Pattern th… --- ## [How to add all field values in a single field?](https://discuss.elastic.co/t/how-to-add-all-field-values-in-a-single-field/284068)
**Author:** [@Husnain](https://discuss.elastic.co/u/Husnain)\ **Replies:** 2\ **Last updated:** [September 15, 2021, 10:16am UTC](https://discuss.elastic.co/t/how-to-add-all-field-values-in-a-single-field/284068 "2021-09-15T10:16:36Z")
Hii.I am trying to create a new\_field called "log\_data" and I want to add each value in "log\_data" field present in the message field.I want only one field with multiple values.How can I achieve this? This are the docu… --- ## [I want to set up sending notifications about errors](https://discuss.elastic.co/t/i-want-to-set-up-sending-notifications-about-errors/283287)
**Author:** [@sasha198407](https://discuss.elastic.co/u/sasha198407)\ **Replies:** 8\ **Last updated:** [September 15, 2021, 6:13am UTC](https://discuss.elastic.co/t/i-want-to-set-up-sending-notifications-about-errors/283287 "2021-09-15T06:13:27Z")
Hello. I want to set up sending notifications about errors (errors in the operation of Apache, MySQL, Nginx, as well as if the server is not available) from logstesh to e-mail. Wrote at ./logstash/pipeline/logstash.conf … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=193) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=195)