# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=111 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 112 --- ## [Getting the same values multiple times while integrating Mysql with Logstash](https://discuss.elastic.co/t/getting-the-same-values-multiple-times-while-integrating-mysql-with-logstash/316277)
**Author:** [@Ritikapawar](https://discuss.elastic.co/u/Ritikapawar)\ **Replies:** 1\ **Last updated:** [October 11, 2022, 2:37pm UTC](https://discuss.elastic.co/t/getting-the-same-values-multiple-times-while-integrating-mysql-with-logstash/316277 "2022-10-11T14:37:22Z")
I've created a pipeline to get data directly from Mysql to kibana using logstash and Jdbc input plugin But when I do the actual execution on .conf file I'm getting the same values multiple time in the elastic index. he… --- ## [Deploy Logstash pipeline through Gitlab CI/CD](https://discuss.elastic.co/t/deploy-logstash-pipeline-through-gitlab-ci-cd/316257)
**Author:** [@kmz161](https://discuss.elastic.co/u/kmz161)\ **Replies:** 3\ **Last updated:** [October 11, 2022, 12:56pm UTC](https://discuss.elastic.co/t/deploy-logstash-pipeline-through-gitlab-ci-cd/316257 "2022-10-11T12:56:22Z")
Hi! I use Centralized Pipeline Management for creation and management pipeline. But I can’t saved changing of pipeline settings . Because of it I started use Gitlab to store pipelines config. And now I need perform tw… --- ## [Issue in parsing JSON data](https://discuss.elastic.co/t/issue-in-parsing-json-data/316210)
**Author:** [@Aniket\_Pant](https://discuss.elastic.co/u/Aniket_Pant)\ **Replies:** 1\ **Last updated:** [October 11, 2022, 12:06pm UTC](https://discuss.elastic.co/t/issue-in-parsing-json-data/316210 "2022-10-11T12:06:10Z")
Sample Logs \[{ "Name":"abc", "Phone\_no":"9877231", "Gender":"M" } \] I am getting data from filebeat - paths: - /root/test.log fields: {log\_type: json-multiline} multiline.type: pattern multiline.pattern: '^\\… --- ## [Salesforce integration with ELK](https://discuss.elastic.co/t/salesforce-integration-with-elk/316317)
**Author:** [@Ayaan\_Shaik](https://discuss.elastic.co/u/Ayaan_Shaik)\ **Replies:** 0\ **Last updated:** [October 11, 2022, 10:40am UTC](https://discuss.elastic.co/t/salesforce-integration-with-elk/316317 "2022-10-11T10:40:45Z")
Hi team, I'm Using ELK version : 8.3.3 I am trying to ingest Salesforce data with the help of file beat module Salesforce through logstash , But I'm unable to get the data into kibana and elasticsearch . Suppose if I … --- ## [Logstash input configuration](https://discuss.elastic.co/t/logstash-input-configuration/316296)
**Author:** [@ttyser](https://discuss.elastic.co/u/ttyser)\ **Replies:** 2\ **Last updated:** [October 11, 2022, 10:09am UTC](https://discuss.elastic.co/t/logstash-input-configuration/316296 "2022-10-11T10:09:00Z")
Hello, I have a question how should Logstash input config look like in case I have Fleet server, multiple agents/endpoints, checkpoint firewall etc logging to Logstash. What I am trying to ask is, should every agent have… --- ## [Logstash Pipeline getting Terminated due to avro format of kakfka topic](https://discuss.elastic.co/t/logstash-pipeline-getting-terminated-due-to-avro-format-of-kakfka-topic/316276)
**Author:** [@Akumar22](https://discuss.elastic.co/u/Akumar22)\ **Replies:** 0\ **Last updated:** [October 11, 2022, 6:45am UTC](https://discuss.elastic.co/t/logstash-pipeline-getting-terminated-due-to-avro-format-of-kakfka-topic/316276 "2022-10-11T06:45:24Z")
Hi, I have a topic in kafka which has messages in avro format, pasting a part of message below .. We are using this avro schema format, pasting a part of schema { "type" : "record", "name" : "ProductDetailsSc… --- ## [Logstash was having pipeline error: event executor terminated. What would possibly be the problem?](https://discuss.elastic.co/t/logstash-was-having-pipeline-error-event-executor-terminated-what-would-possibly-be-the-problem/316212)
**Author:** [@prashanthk](https://discuss.elastic.co/u/prashanthk)\ **Replies:** 5\ **Last updated:** [October 11, 2022, 3:52am UTC](https://discuss.elastic.co/t/logstash-was-having-pipeline-error-event-executor-terminated-what-would-possibly-be-the-problem/316212 "2022-10-11T03:52:28Z")
input { http { host =\> "0.0.0.0" port =\> 8443 user =\> provider password =\> "c6N65uxZ$M5@" # ssl =\> true # keystore =\> "/etc/logstash/keystore.jks" # keystore\_password =\> "password" } } filter { mutate { add\_f… --- ## [Logstash auto reload the pipeline?](https://discuss.elastic.co/t/logstash-auto-reload-the-pipeline/316152)
**Author:** [@Nikolas1306](https://discuss.elastic.co/u/Nikolas1306)\ **Replies:** 1\ **Last updated:** [October 11, 2022, 1:38am UTC](https://discuss.elastic.co/t/logstash-auto-reload-the-pipeline/316152 "2022-10-11T01:38:02Z")
hello i've installed elk on cloud but the image have the original configuration pipeline if change the config and reload the image all file is reset with originals exist a method with modify pipeline and reload logstas… --- ## [Logstash not starting](https://discuss.elastic.co/t/logstash-not-starting/316188)
**Author:** [@sandeep\_singh3](https://discuss.elastic.co/u/sandeep_singh3)\ **Replies:** 2\ **Last updated:** [October 10, 2022, 5:44pm UTC](https://discuss.elastic.co/t/logstash-not-starting/316188 "2022-10-10T17:44:57Z")
I am trying to run Logstash on my window machine but unable to do so due below error \\Eclipse was unexpected at this time. Command used to run the logstash logstash.bat -f logstash.conf Version of Logstash - 7.16.3 l… --- ## [Set variable in configuration file](https://discuss.elastic.co/t/set-variable-in-configuration-file/316096)
**Author:** [@RickT](https://discuss.elastic.co/u/RickT)\ **Replies:** 5\ **Last updated:** [October 10, 2022, 3:41pm UTC](https://discuss.elastic.co/t/set-variable-in-configuration-file/316096 "2022-10-10T15:41:30Z")
Hi all ! Simply, is there some method in Logstash news versions to code a variable which can be used in input, filter and output sections ? if so, how to code it ? Thanks. --- ## [Combine Metricbeat events into one with Logstash](https://discuss.elastic.co/t/combine-metricbeat-events-into-one-with-logstash/315896)
**Author:** [@bg4erem](https://discuss.elastic.co/u/bg4erem)\ **Replies:** 5\ **Last updated:** [October 10, 2022, 3:21am UTC](https://discuss.elastic.co/t/combine-metricbeat-events-into-one-with-logstash/315896 "2022-10-10T03:21:15Z")
Dear community I have spent several days trying to combine Metricbeat events into one with Logstash. I understand it should be done with the aggregate filter of Logstash. Still, I haven't figured it out. I use Logstas… --- ## [Dead letter queue not working with logtash output mongo](https://discuss.elastic.co/t/dead-letter-queue-not-working-with-logtash-output-mongo/316030)
**Author:** [@van\_le1](https://discuss.elastic.co/u/van_le1)\ **Replies:** 2\ **Last updated:** [October 10, 2022, 1:04am UTC](https://discuss.elastic.co/t/dead-letter-queue-not-working-with-logtash-output-mongo/316030 "2022-10-10T01:04:20Z")
Hi everyone, I intend to use dead letter queue on log event failure from sending log into mongodb use logtash. My config includes: logtash.yml http.host: "0.0.0.0" pipeline.batch.delay: 10 pipeline.batch.size: 10… --- ## [Force index rollover with a new index name](https://discuss.elastic.co/t/force-index-rollover-with-a-new-index-name/315983)
**Author:** [@frank\_rib](https://discuss.elastic.co/u/frank_rib)\ **Replies:** 2\ **Last updated:** [October 9, 2022, 8:35pm UTC](https://discuss.elastic.co/t/force-index-rollover-with-a-new-index-name/315983 "2022-10-09T20:35:05Z")
Hello, I was configured filebeat agent to send data directly to elasticsearch from multiples agent. afterwards I added the index to a have a problem with the rollover I added the index to the retention policy test0. T… --- ## [JSON with comma separated string to multiple tags](https://discuss.elastic.co/t/json-with-comma-separated-string-to-multiple-tags/315111)
**Author:** [@Ranger\_Rick](https://discuss.elastic.co/u/Ranger_Rick)\ **Replies:** 2\ **Last updated:** [October 9, 2022, 1:38am UTC](https://discuss.elastic.co/t/json-with-comma-separated-string-to-multiple-tags/315111 "2022-10-09T01:38:22Z")
Good evening! I really think this should be easy and straightforward but I am having a heck of a time with it so decided to ask the community. My pipeline accepts daily json files and splits it out and further enriches … --- ## [Unable to Parse AVRO using Kafka Input and Avro Codec](https://discuss.elastic.co/t/unable-to-parse-avro-using-kafka-input-and-avro-codec/316133)
**Author:** [@mastersmit](https://discuss.elastic.co/u/mastersmit)\ **Replies:** 0\ **Last updated:** [October 8, 2022, 4:23pm UTC](https://discuss.elastic.co/t/unable-to-parse-avro-using-kafka-input-and-avro-codec/316133 "2022-10-08T16:23:09Z")
Hi, I am trying to use Kafka Input and Codec Avro, to read the messages but unable to do so. In Kafka, { "id": 5839355690199358000, "name": "VOH\*a|\[RF6y?" } In Logstash Output: "@timestamp" =\> 2022-10-08T14:… --- ## [Remove / using logstash filter](https://discuss.elastic.co/t/remove-using-logstash-filter/316106)
**Author:** [@Giridharan\_C](https://discuss.elastic.co/u/Giridharan_C)\ **Replies:** 3\ **Last updated:** [October 8, 2022, 1:52pm UTC](https://discuss.elastic.co/t/remove-using-logstash-filter/316106 "2022-10-08T13:52:32Z")
I used multiple mutate filters to remove the / in the message field, which is not working. Also, I want to parse these message fields into valid JSON. Please suggest and help. { "host" =\> "mylocalhost.mydomain.com", … --- ## [Store multiple values using ruby in multivalue field](https://discuss.elastic.co/t/store-multiple-values-using-ruby-in-multivalue-field/316059)
**Author:** [@bertr](https://discuss.elastic.co/u/bertr)\ **Replies:** 2\ **Last updated:** [October 8, 2022, 1:40pm UTC](https://discuss.elastic.co/t/store-multiple-values-using-ruby-in-multivalue-field/316059 "2022-10-08T13:40:57Z")
By nature all fields in elasticsearch are multivalue-enabled, i.e. they can have multiple values in one document. Using mutate filter it is easy to add multiple values, but I need to use ruby. I am trying to add multiple… --- ## [Problem with geoip plugin and IP whitelist filtering](https://discuss.elastic.co/t/problem-with-geoip-plugin-and-ip-whitelist-filtering/316051)
**Author:** [@Lynow](https://discuss.elastic.co/u/Lynow)\ **Replies:** 2\ **Last updated:** [October 7, 2022, 4:46pm UTC](https://discuss.elastic.co/t/problem-with-geoip-plugin-and-ip-whitelist-filtering/316051 "2022-10-07T16:46:45Z")
Good morning ! I am using Logstash to fetch data by Wazuh (HIDS) and Suricata (NIDS). So I use the Geoip plugin to geolocate the IPs that connect to my server. My first problem is this: When I do a test by connecting … --- ## [Logstash pipeline.workers](https://discuss.elastic.co/t/logstash-pipeline-workers/316053)
**Author:** [@S-elk](https://discuss.elastic.co/u/S-elk)\ **Replies:** 1\ **Last updated:** [October 7, 2022, 4:31pm UTC](https://discuss.elastic.co/t/logstash-pipeline-workers/316053 "2022-10-07T16:31:47Z")
Hello! Currently we have our pipeline.yml without defined workers. We understand that we take by default the number of cpu cores of the machine. Should we define all in pipeline.worker: 1 ? How do we decide which work… --- ## [Remote\_addr showing 127.0.01](https://discuss.elastic.co/t/remote-addr-showing-127-0-01/315738)
**Author:** [@AkankshaSS](https://discuss.elastic.co/u/AkankshaSS)\ **Replies:** 21\ **Last updated:** [October 7, 2022, 4:20pm UTC](https://discuss.elastic.co/t/remote-addr-showing-127-0-01/315738 "2022-10-07T16:20:46Z")
Hi,I am able to see only 127.0.01 in http-x\_forwarded\_for.Can someone please help me what's the changes required. --- ## [Grok pattern for suricata/barnyard alert in COMPLETE mode](https://discuss.elastic.co/t/grok-pattern-for-suricata-barnyard-alert-in-complete-mode/316009)
**Author:** [@mvrk](https://discuss.elastic.co/u/mvrk)\ **Replies:** 2\ **Last updated:** [October 6, 2022, 9:12pm UTC](https://discuss.elastic.co/t/grok-pattern-for-suricata-barnyard-alert-in-complete-mode/316009 "2022-10-06T21:12:37Z")
Hi, I have my suricata in pfsense sending the alerts using barnyard in COMPLETE mode to my greylog server. Example message: | \[SNORTIDS\[ALERT\]: \[pfsense.local\] \] || 2022-10-06 19:13:55.186+001 2 \[1:2403344:77870\] ET C… --- ## [Make Single output call with multiple events](https://discuss.elastic.co/t/make-single-output-call-with-multiple-events/315808)
**Author:** [@LJ\_LongWing](https://discuss.elastic.co/u/LJ_LongWing)\ **Replies:** 7\ **Last updated:** [October 6, 2022, 9:11pm UTC](https://discuss.elastic.co/t/make-single-output-call-with-multiple-events/315808 "2022-10-06T21:11:42Z")
I've done a few days of searching but can't find what I'm looking for....the closest I've found is this topic (How to aggregate multiple events into single output) I'll state that I'm starting with a functional pipeline… --- ## [How to grok first line from an XML format](https://discuss.elastic.co/t/how-to-grok-first-line-from-an-xml-format/315877)
**Author:** [@anupvtr](https://discuss.elastic.co/u/anupvtr)\ **Replies:** 1\ **Last updated:** [October 6, 2022, 4:19pm UTC](https://discuss.elastic.co/t/how-to-grok-first-line-from-an-xml-format/315877 "2022-10-06T16:19:04Z")
Hi All, I am quite new to the magic world of Grok. Any help will be thankful. I need to apply filter for the following file. 2022-08-22 22:18:59 , 666 INFO @ (blockurcolumn-11) \[rbbit\_MQ\_Versa.appache 75\] start col… --- ## [Remove backslash from xml log](https://discuss.elastic.co/t/remove-backslash-from-xml-log/315891)
**Author:** [@Jathurshan\_Sumandira](https://discuss.elastic.co/u/Jathurshan_Sumandira)\ **Replies:** 4\ **Last updated:** [October 6, 2022, 12:32pm UTC](https://discuss.elastic.co/t/remove-backslash-from-xml-log/315891 "2022-10-06T12:32:39Z")
\ I wanted to remove the backslashs ("\\") in above xml log-line and I have tried ruby { code =\> ' d= event.… --- ## [Need to disable idle indices in logstash config](https://discuss.elastic.co/t/need-to-disable-idle-indices-in-logstash-config/315942)
**Author:** [@yasar](https://discuss.elastic.co/u/yasar)\ **Replies:** 0\ **Last updated:** [October 6, 2022, 8:53am UTC](https://discuss.elastic.co/t/need-to-disable-idle-indices-in-logstash-config/315942 "2022-10-06T08:53:33Z")
Hi team, In our TEST Environment, N number of indices are idle in position. Those are unwantedly roll back over itself. Could you giude us, what are the procedure to disable the unwanted indices in logstash config or s… --- ## [Facing issue while starting logstash which connects with Maria Database](https://discuss.elastic.co/t/facing-issue-while-starting-logstash-which-connects-with-maria-database/315233)
**Author:** [@nawaz462](https://discuss.elastic.co/u/nawaz462)\ **Replies:** 2\ **Last updated:** [October 6, 2022, 8:25am UTC](https://discuss.elastic.co/t/facing-issue-while-starting-logstash-which-connects-with-maria-database/315233 "2022-10-06T08:25:23Z")
On linux server, i wanted to have ELK setup with MariaDB through Docker. I want to setup the Elastic search data with indexes for the maria DB tables from scratch and as well as for incremental changes sync. These are d… --- ## [Logstasg ERROR filewatch.tailmode.handlers.grow](https://discuss.elastic.co/t/logstasg-error-filewatch-tailmode-handlers-grow/315550)
**Author:** [@alex\_96](https://discuss.elastic.co/u/alex_96)\ **Replies:** 4\ **Last updated:** [October 6, 2022, 8:12am UTC](https://discuss.elastic.co/t/logstasg-error-filewatch-tailmode-handlers-grow/315550 "2022-10-06T08:12:57Z")
Hi, set up reading mysql logs from nfs using multiline codec and grok filter The first problem is that it doesn’t read the file after rotation, restarting doesn’t help, it still doesn’t seem to see it, there was an erro… --- ## [Logstash health check failing for TG](https://discuss.elastic.co/t/logstash-health-check-failing-for-tg/315541)
**Author:** [@sandeepbisht](https://discuss.elastic.co/u/sandeepbisht)\ **Replies:** 1\ **Last updated:** [October 6, 2022, 7:04am UTC](https://discuss.elastic.co/t/logstash-health-check-failing-for-tg/315541 "2022-10-06T07:04:02Z")
Hi All, I had one logstash ec2 ubuntu machine, Due to frequent load issue i have added one more logstash machine and put both machines behind AWS ALB. Now both my instances are unhealthy under TG and i am not sure what… --- ## [Logstash service is not started in linux](https://discuss.elastic.co/t/logstash-service-is-not-started-in-linux/315802)
**Author:** [@Jathurshan\_Sumandira](https://discuss.elastic.co/u/Jathurshan_Sumandira)\ **Replies:** 3\ **Last updated:** [October 6, 2022, 5:11am UTC](https://discuss.elastic.co/t/logstash-service-is-not-started-in-linux/315802 "2022-10-06T05:11:49Z")
Hi, I am beginner to ELK stack and I am working on Linux server, I could try to start the logstash service but the service does not start and I could not see any logs in the command line(empty). Even though the permissio… --- ## [Logstash fails to fetch Configuration](https://discuss.elastic.co/t/logstash-fails-to-fetch-configuration/315915)
**Author:** [@Chma](https://discuss.elastic.co/u/Chma)\ **Replies:** 6\ **Last updated:** [October 6, 2022, 1:32am UTC](https://discuss.elastic.co/t/logstash-fails-to-fetch-configuration/315915 "2022-10-06T01:32:40Z")
My logstash isn't starting. When I check the logs, I see these: Could not fetch all the sources {:exception=\>Errno::EACCES, :message=\>"Permission denied - /etc/logstash/conf.d/log.conf", :backtrace=\>\["org/jruby/RubyIO.j… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=110) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=112)