# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=31 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 32 --- ## [Logstash mutate can not get variable](https://discuss.elastic.co/t/logstash-mutate-can-not-get-variable/359516)
**Author:** [@Fodesu](https://discuss.elastic.co/u/Fodesu)\ **Replies:** 3\ **Last updated:** [May 15, 2024, 1:58pm UTC](https://discuss.elastic.co/t/logstash-mutate-can-not-get-variable/359516 "2024-05-15T13:58:54Z")
Hi Term, i am trying to make a grok parser for every two lines of log, i config filebeat.yml to parsing every two lines in filebeat. but it can't parser variable successful. I use grokdebug Configuration is follow , … --- ## [Logstash nested array with special key value pair](https://discuss.elastic.co/t/logstash-nested-array-with-special-key-value-pair/357726)
**Author:** [@PieterF](https://discuss.elastic.co/u/PieterF)\ **Replies:** 4\ **Last updated:** [May 15, 2024, 1:35pm UTC](https://discuss.elastic.co/t/logstash-nested-array-with-special-key-value-pair/357726 "2024-05-15T13:35:01Z")
Hi, We ingest Firebase logs with format json. Now there are some properties that have an array of objects with the following structure: the key is always a string but the value is saved depending on type. event\_params… --- ## [I'm trying to use the jdbc\_static plugin to join two tables from MySql and index it but I'm receiving this error (warning: already initialized constant Manticore::Client::ByteArrayEntity)](https://discuss.elastic.co/t/im-trying-to-use-the-jdbc-static-plugin-to-join-two-tables-from-mysql-and-index-it-but-im-receiving-this-error-warning-already-initialized-constant-manticore-bytearrayentity/359532)
**Author:** [@Danyal\_Danish](https://discuss.elastic.co/u/Danyal_Danish)\ **Replies:** 0\ **Last updated:** [May 15, 2024, 10:42am UTC](https://discuss.elastic.co/t/im-trying-to-use-the-jdbc-static-plugin-to-join-two-tables-from-mysql-and-index-it-but-im-receiving-this-error-warning-already-initialized-constant-manticore-bytearrayentity/359532 "2024-05-15T10:42:41Z")
I'm trying to use the jdbc\_static plugin to join two tables from MySql and index it but I'm receiving this error (warning: already initialized constant Manticore::Client::ByteArrayEntity) This is my pipeline: input { … --- ## [Logstash pipelines and multiple outputs](https://discuss.elastic.co/t/logstash-pipelines-and-multiple-outputs/359451)
**Author:** [@ISR\_FY](https://discuss.elastic.co/u/ISR_FY)\ **Replies:** 3\ **Last updated:** [May 15, 2024, 10:11am UTC](https://discuss.elastic.co/t/logstash-pipelines-and-multiple-outputs/359451 "2024-05-15T10:11:25Z")
Morning. I have an issue with the last version of elastic stack. When I try to use pipelines to send logs to 2 different outputs one it´s receiving but the other doesn´t have anything. There is no errors in logstash(debu… --- ## [Winlogbeat data enrich with CSV lookup adding weird field](https://discuss.elastic.co/t/winlogbeat-data-enrich-with-csv-lookup-adding-weird-field/359475)
**Author:** [@s0p4L1n3](https://discuss.elastic.co/u/s0p4L1n3)\ **Replies:** 4\ **Last updated:** [May 15, 2024, 10:01am UTC](https://discuss.elastic.co/t/winlogbeat-data-enrich-with-csv-lookup-adding-weird-field/359475 "2024-05-15T10:01:48Z")
Hello, I'm currently using Winlogbeat to send logs to ELK with logstash output. I would like to create a new field based on the winlog.event\_id field. I have a CSV field for the lookup. This is to enrich data visuali… --- ## [How to drop/create index in Logstash config?](https://discuss.elastic.co/t/how-to-drop-create-index-in-logstash-config/359496)
**Author:** [@dev8100](https://discuss.elastic.co/u/dev8100)\ **Replies:** 2\ **Last updated:** [May 15, 2024, 4:10am UTC](https://discuss.elastic.co/t/how-to-drop-create-index-in-logstash-config/359496 "2024-05-15T04:10:10Z")
We have a scenario where documents are still in the index that are no longer in the source data being fed into the index. Is there a way to delete the index and just re-create it with the latest data? Or is there a way… --- ## [SNMP input not working running Logstash as Service / detached](https://discuss.elastic.co/t/snmp-input-not-working-running-logstash-as-service-detached/359483)
**Author:** [@gergog](https://discuss.elastic.co/u/gergog)\ **Replies:** 0\ **Last updated:** [May 14, 2024, 6:07pm UTC](https://discuss.elastic.co/t/snmp-input-not-working-running-logstash-as-service-detached/359483 "2024-05-14T18:07:33Z")
Hello there! I try to use snmp input plugin. I tried it with on simple VM installation run as Service. Without any luck. Logs shows that the pipeline starts. Debug messages show that the pipeline work, but the part, wh… --- ## [Syslog load balancing in front of Logstash?](https://discuss.elastic.co/t/syslog-load-balancing-in-front-of-logstash/359407)
**Author:** [@Craig\_Sharp](https://discuss.elastic.co/u/Craig_Sharp)\ **Replies:** 4\ **Last updated:** [May 14, 2024, 5:53pm UTC](https://discuss.elastic.co/t/syslog-load-balancing-in-front-of-logstash/359407 "2024-05-14T17:53:59Z")
I have a lot of syslogs flowing from multiple sources into four logstash nodes and then to Elastic. At this time I am just using DNS round robin to balance the Logstash nodes for the inbound syslog traffic. I want to us… --- ## [Unable to run pipeline on logstash 8.13.4 version](https://discuss.elastic.co/t/unable-to-run-pipeline-on-logstash-8-13-4-version/359426)
**Author:** [@gadde36256](https://discuss.elastic.co/u/gadde36256)\ **Replies:** 6\ **Last updated:** [May 14, 2024, 1:10pm UTC](https://discuss.elastic.co/t/unable-to-run-pipeline-on-logstash-8-13-4-version/359426 "2024-05-14T13:10:07Z")
Hi, Getting below error while running the logstash pipe line with ing-bank cassandra jdbc driver. \[ERROR\] 2024-05-14 05:16:54.776 \[Converge PipelineAction::Create\\] agent - Failed to execute action {:id=\>:main, :a… --- ## [Logstash Pipeline not eligible for data streams](https://discuss.elastic.co/t/logstash-pipeline-not-eligible-for-data-streams/359458)
**Author:** [@s0p4L1n3](https://discuss.elastic.co/u/s0p4L1n3)\ **Replies:** 2\ **Last updated:** [May 14, 2024, 12:58pm UTC](https://discuss.elastic.co/t/logstash-pipeline-not-eligible-for-data-streams/359458 "2024-05-14T12:58:31Z")
Hello, I'm using Winlogbeat and filebeat to ingest logs into ELK, the beats agents output is logstash. I've setup according this process order: Point winlogbeat to Elasticsearch run setup winlogbeat.exe setup -e Star… --- ## [Http\_poller](https://discuss.elastic.co/t/http-poller/358763)
**Author:** [@ACHQUE](https://discuss.elastic.co/u/ACHQUE)\ **Replies:** 6\ **Last updated:** [May 13, 2024, 1:33pm UTC](https://discuss.elastic.co/t/http-poller/358763 "2024-05-13T13:33:22Z")
Hi there. Http\_poller found here: Http\_poller input plugin | Logstash Reference \[8.13\] | Elastic I have tried using keystore and truststore, converting .pem to .p12 and importing them. Reference: IBM Documentation an… --- ## [How to install File input plugin on Fedora core-os?](https://discuss.elastic.co/t/how-to-install-file-input-plugin-on-fedora-core-os/359347)
**Author:** [@yogeshk04](https://discuss.elastic.co/u/yogeshk04)\ **Replies:** 0\ **Last updated:** [May 13, 2024, 7:22am UTC](https://discuss.elastic.co/t/how-to-install-file-input-plugin-on-fedora-core-os/359347 "2024-05-13T07:22:09Z")
How to install File input plugin on Fedora core-os? --- ## [Problems indexing events into Elasticsearch](https://discuss.elastic.co/t/problems-indexing-events-into-elasticsearch/359148)
**Author:** [@Compte\_Personnel](https://discuss.elastic.co/u/Compte_Personnel)\ **Replies:** 6\ **Last updated:** [May 12, 2024, 4:16am UTC](https://discuss.elastic.co/t/problems-indexing-events-into-elasticsearch/359148 "2024-05-12T04:16:53Z")
Hello everyone, It seems there are issues with indexing events into Elasticsearch ● logstash.service - logstash Loaded: loaded (/lib/systemd/system/logstash.service; enabled; vendor preset: enabled) Active: a… --- ## [Google pubsub plugin(input-output) not found](https://discuss.elastic.co/t/google-pubsub-plugin-input-output-not-found/359250)
**Author:** [@ISR\_FY](https://discuss.elastic.co/u/ISR_FY)\ **Replies:** 2\ **Last updated:** [May 10, 2024, 12:56pm UTC](https://discuss.elastic.co/t/google-pubsub-plugin-input-output-not-found/359250 "2024-05-10T12:56:55Z")
Good morning. Recently we migrated the instances from Centos to Rocky Linux. We are suffering this problem: When we install logstash and before start it We try to add the plugins for pubsub(input and output) but the ser… --- ## [What does this logstash filter do?](https://discuss.elastic.co/t/what-does-this-logstash-filter-do/359229)
**Author:** [@moomoo21](https://discuss.elastic.co/u/moomoo21)\ **Replies:** 5\ **Last updated:** [May 10, 2024, 12:30pm UTC](https://discuss.elastic.co/t/what-does-this-logstash-filter-do/359229 "2024-05-10T12:30:10Z")
Im having trouble understanding what does this logstash filter do: if ("" in \[somefield1\]\[somefield2\]){ drop{} } what does "" in the field mean? --- ## [Issue with @timestamp field when importing data from SQL Server to Elasticsearch using Logstash](https://discuss.elastic.co/t/issue-with-timestamp-field-when-importing-data-from-sql-server-to-elasticsearch-using-logstash/359209)
**Author:** [@hcherry](https://discuss.elastic.co/u/hcherry)\ **Replies:** 6\ **Last updated:** [May 9, 2024, 11:04pm UTC](https://discuss.elastic.co/t/issue-with-timestamp-field-when-importing-data-from-sql-server-to-elasticsearch-using-logstash/359209 "2024-05-09T23:04:06Z")
Hello, I’m currently trying to import log data from a SQL Server database table into Elasticsearch using Logstash. The SQL table includes a field named “logDateTime” for the timestamp of the log entries. I want to use t… --- ## [Sequence Number Generation](https://discuss.elastic.co/t/sequence-number-generation/358563)
**Author:** [@dawiro](https://discuss.elastic.co/u/dawiro)\ **Replies:** 3\ **Last updated:** [May 9, 2024, 2:39pm UTC](https://discuss.elastic.co/t/sequence-number-generation/358563 "2024-05-09T14:39:13Z")
We would like to annotate our inbound log messages with sequence numbers which we can then use to detect and measure message loss. What is the best way to go about this? Use in-line ruby code to generate sequence number… --- ## [How to create columns instead rows in Logstash with aggregate or other plugin](https://discuss.elastic.co/t/how-to-create-columns-instead-rows-in-logstash-with-aggregate-or-other-plugin/359124)
**Author:** [@epreciado](https://discuss.elastic.co/u/epreciado)\ **Replies:** 0\ **Last updated:** [May 9, 2024, 4:26am UTC](https://discuss.elastic.co/t/how-to-create-columns-instead-rows-in-logstash-with-aggregate-or-other-plugin/359124 "2024-05-09T04:26:56Z")
I have a log similar to this and I need to get these lines and upload in kibana: \[03/07/2024 00:00:31.306\]\[123456\]\[newRequest\]\[Request\]\[Company|signature|L|id|123456789|0987654321\] \[03/07/2024 00:00:31.634\]\[123456\]\[Comp… --- ## [How can free up /var on logstash server](https://discuss.elastic.co/t/how-can-free-up-var-on-logstash-server/359136)
**Author:** [@baber1223](https://discuss.elastic.co/u/baber1223)\ **Replies:** 1\ **Last updated:** [May 9, 2024, 10:45am UTC](https://discuss.elastic.co/t/how-can-free-up-var-on-logstash-server/359136 "2024-05-09T10:45:09Z")
My logstash path on /var is full so logstash service cannot start . Now want to know How can free up /var on log server ? Do I have to remove file manually from /var/log or have to do free up by elastic GUI ? --- ## [MIB conversion of received traps using logstash snmptrap plugin](https://discuss.elastic.co/t/mib-conversion-of-received-traps-using-logstash-snmptrap-plugin/359137)
**Author:** [@miya-dit](https://discuss.elastic.co/u/miya-dit)\ **Replies:** 0\ **Last updated:** [May 9, 2024, 6:53am UTC](https://discuss.elastic.co/t/mib-conversion-of-received-traps-using-logstash-snmptrap-plugin/359137 "2024-05-09T06:53:04Z")
I want to convert MIBs for traps received using the logstash snmptrap plugin. Currently, Logstash inputs SNMPTRAP as follows and outputs it to AWS SNS. SNMPTRAP -\> Logstash -\> AWS SNS (E-mail) SNS notifications are se… --- ## [Avoid duplicate in nested table via logstagh](https://discuss.elastic.co/t/avoid-duplicate-in-nested-table-via-logstagh/359123)
**Author:** [@karthikeyanc2003](https://discuss.elastic.co/u/karthikeyanc2003)\ **Replies:** 1\ **Last updated:** [May 9, 2024, 4:26am UTC](https://discuss.elastic.co/t/avoid-duplicate-in-nested-table-via-logstagh/359123 "2024-05-09T04:26:09Z")
Can someone please help with mapping when there are more than 1 nested type properties in the aggregate mapping I am seeing a duplicate of data getting created in the document for the nested type properties, when in the… --- ## [Http\_poller time out error](https://discuss.elastic.co/t/http-poller-time-out-error/359099)
**Author:** [@kishorkumar](https://discuss.elastic.co/u/kishorkumar)\ **Replies:** 1\ **Last updated:** [May 8, 2024, 7:45pm UTC](https://discuss.elastic.co/t/http-poller-time-out-error/359099 "2024-05-08T19:45:43Z")
we are recieving the timeout error in http\_poller by connecting it to guardium, our database is guardium . my http pollar settings as follow input { http\_poller { urls = \> { "d1" … --- ## [Http\_poller timezone is not being recognized](https://discuss.elastic.co/t/http-poller-timezone-is-not-being-recognized/358980)
**Author:** [@kishorkumar](https://discuss.elastic.co/u/kishorkumar)\ **Replies:** 3\ **Last updated:** [May 8, 2024, 1:31pm UTC](https://discuss.elastic.co/t/http-poller-timezone-is-not-being-recognized/358980 "2024-05-08T13:31:21Z")
I am trying to run http\_poller form logstash local machine input { http\_poller { urls =\> { test1 =\> "https://randomuser.me/api" } request\_timeout =\> 60 schedule =\> { cron =\> "\* \* \* \* \* UTC"} cod… --- ## [Custom Time field doesnot contain any data after create in grok pattern](https://discuss.elastic.co/t/custom-time-field-doesnot-contain-any-data-after-create-in-grok-pattern/358995)
**Author:** [@baber1223](https://discuss.elastic.co/u/baber1223)\ **Replies:** 1\ **Last updated:** [May 8, 2024, 4:00am UTC](https://discuss.elastic.co/t/custom-time-field-doesnot-contain-any-data-after-create-in-grok-pattern/358995 "2024-05-08T04:00:59Z")
I have written follow grok pattern in logstash filter{ grok{ match =\> { "message" =\> '%{IPORHOST:clientip} %{HTTPDUSER:ident} %{USER:auth} \\\[%{HTTPDATE:logtimestamp}\\\] "(?:%{WORD:verb} %{NOTSPACE:request}(?: HT… --- ## [How to bypass ssl in http\_poller](https://discuss.elastic.co/t/how-to-bypass-ssl-in-http-poller/358948)
**Author:** [@kishorkumar](https://discuss.elastic.co/u/kishorkumar)\ **Replies:** 2\ **Last updated:** [May 7, 2024, 6:24pm UTC](https://discuss.elastic.co/t/how-to-bypass-ssl-in-http-poller/358948 "2024-05-07T18:24:41Z")
How do i by pass sssl in the logstash input http\_poller logstash Logstash --- ## [Json parse error on elastic output](https://discuss.elastic.co/t/json-parse-error-on-elastic-output/358910)
**Author:** [@matan21m](https://discuss.elastic.co/u/matan21m)\ **Replies:** 0\ **Last updated:** [May 7, 2024, 9:14am UTC](https://discuss.elastic.co/t/json-parse-error-on-elastic-output/358910 "2024-05-07T09:14:52Z")
I am trying to create a logging service using the elastic stack. I have some very large logs that also require filtering to adjust to JSON format, and one of them gives me the following json parse error: JSON parse err… --- ## [Start logstash on docker -\> Could not find or load main class org](https://discuss.elastic.co/t/start-logstash-on-docker-could-not-find-or-load-main-class-org/358906)
**Author:** [@INS](https://discuss.elastic.co/u/INS)\ **Replies:** 0\ **Last updated:** [May 7, 2024, 8:43am UTC](https://discuss.elastic.co/t/start-logstash-on-docker-could-not-find-or-load-main-class-org/358906 "2024-05-07T08:43:06Z")
Hi I'm struggling the case after upgrade from logstash 8.2.1 to 8.13.3 with Using bundled JDK: /usr/share/logstash/jdk Error: Could not find or load main class org.logstash.launchers.JvmOptionsParser Caused by: java.l… --- ## [When using logstash to consume binary data, some bytes are replaced with ef bf bd](https://discuss.elastic.co/t/when-using-logstash-to-consume-binary-data-some-bytes-are-replaced-with-ef-bf-bd/358797)
**Author:** [@imdong](https://discuss.elastic.co/u/imdong)\ **Replies:** 3\ **Last updated:** [May 7, 2024, 3:12am UTC](https://discuss.elastic.co/t/when-using-logstash-to-consume-binary-data-some-bytes-are-replaced-with-ef-bf-bd/358797 "2024-05-07T03:12:56Z")
The original data is avro data, but only plain =\> { charset =\> "BINARY" } is used for debugging to reduce interference. avro data original (hexadecimal) 00000000: 00 80 40 00 cc e1 85 98 0e 00 3a 32 30 32 34 2d ..@...… --- ## [Plugin installation](https://discuss.elastic.co/t/plugin-installation/358679)
**Author:** [@Honestabe](https://discuss.elastic.co/u/Honestabe)\ **Replies:** 4\ **Last updated:** [May 6, 2024, 3:47pm UTC](https://discuss.elastic.co/t/plugin-installation/358679 "2024-05-06T15:47:02Z")
Is there a way to specify the directory to install the plugin to? Our home directory of logstash is not in /usr/share/logstash or the /etc/logstash. I have ran the install command but i am not sure if it was installed i… --- ## [Painless script errors](https://discuss.elastic.co/t/painless-script-errors/358700)
**Author:** [@sravan\_kaheti](https://discuss.elastic.co/u/sravan_kaheti)\ **Replies:** 3\ **Last updated:** [May 6, 2024, 11:40am UTC](https://discuss.elastic.co/t/painless-script-errors/358700 "2024-05-06T11:40:29Z")
Hi There , I need a quick help for painless script implementation for one logic The Environment : I am ingesting huge number of documents in following manner Filebeat \> logstash \> elasticsearch Filebeat will just co… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=30) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=32)