# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=50 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 51 --- ## [Logstash elasticsearch input plugin](https://discuss.elastic.co/t/logstash-elasticsearch-input-plugin/347207)
**Author:** [@Haytham\_Shammout](https://discuss.elastic.co/u/Haytham_Shammout)\ **Replies:** 3\ **Last updated:** [November 28, 2023, 12:39pm UTC](https://discuss.elastic.co/t/logstash-elasticsearch-input-plugin/347207 "2023-11-28T12:39:22Z")
Hello dears, i am trying to create logstash job that have input from elasticsearch index pattern and to take a specific logs then to save them in a historical index so the configuration as below, input{ elasticsearch … --- ## [Update an event fields based on another event](https://discuss.elastic.co/t/update-an-event-fields-based-on-another-event/347975)
**Author:** [@marcowiskhy](https://discuss.elastic.co/u/marcowiskhy)\ **Replies:** 5\ **Last updated:** [November 27, 2023, 12:25pm UTC](https://discuss.elastic.co/t/update-an-event-fields-based-on-another-event/347975 "2023-11-27T12:25:34Z")
Hi, i have some logs indexed in elasticsearch by logstash that provides two types of events: { "@timestamp": "Nov 23, 2023 @ 15:24:33.064", "Detection ID": "ldt:87654321", "logSource": "CS De… --- ## [Trouble connecting logstash to a secure elastic search cluster](https://discuss.elastic.co/t/trouble-connecting-logstash-to-a-secure-elastic-search-cluster/348045)
**Author:** [@debo9912](https://discuss.elastic.co/u/debo9912)\ **Replies:** 0\ **Last updated:** [November 27, 2023, 12:22pm UTC](https://discuss.elastic.co/t/trouble-connecting-logstash-to-a-secure-elastic-search-cluster/348045 "2023-11-27T12:22:17Z")
Hi, I'm facing issues connecting logstash to elasticsearch cluster over https. I'm using the official elastic helm charts to setup my cluster. I'm pretty new to setting up such clusters so might be missing out something. … --- ## [Logstash filter to create a subfield based on specific text in a log message](https://discuss.elastic.co/t/logstash-filter-to-create-a-subfield-based-on-specific-text-in-a-log-message/347978)
**Author:** [@Dokh\_Ahmed](https://discuss.elastic.co/u/Dokh_Ahmed)\ **Replies:** 1\ **Last updated:** [November 26, 2023, 7:39pm UTC](https://discuss.elastic.co/t/logstash-filter-to-create-a-subfield-based-on-specific-text-in-a-log-message/347978 "2023-11-26T19:39:20Z")
I've been working on a Logstash configuration where I'm trying to create a subfield within the 'message1' field based on a specific text pattern ('Started'). Here's a snippet of my current Logstash filter: filter { gr… --- ## [/etc/default/logstash](https://discuss.elastic.co/t/etc-default-logstash/347994)
**Author:** [@Tal\_Blat](https://discuss.elastic.co/u/Tal_Blat)\ **Replies:** 1\ **Last updated:** [November 26, 2023, 1:57pm UTC](https://discuss.elastic.co/t/etc-default-logstash/347994 "2023-11-26T13:57:46Z")
Hi How do i add to logstash env file "/etc/default/logstash" a line with the following format: ELK\_SERVERS="host1:9200","host2:9200","host3:9200","host4:9200" Thanks --- ## [Hostname not extracted when i run logstash as a service on rhel](https://discuss.elastic.co/t/hostname-not-extracted-when-i-run-logstash-as-a-service-on-rhel/347977)
**Author:** [@Indeed2000](https://discuss.elastic.co/u/Indeed2000)\ **Replies:** 6\ **Last updated:** [November 26, 2023, 2:30am UTC](https://discuss.elastic.co/t/hostname-not-extracted-when-i-run-logstash-as-a-service-on-rhel/347977 "2023-11-26T02:30:37Z")
Hi When i run logstash normally like this: ./logstash -f logstash.cfg It extract hostname. But when i run as service not extract hostname. Any idea? Thanks --- ## [No verify ssl input elasticsearch](https://discuss.elastic.co/t/no-verify-ssl-input-elasticsearch/347860)
**Author:** [@lstoneir](https://discuss.elastic.co/u/lstoneir)\ **Replies:** 3\ **Last updated:** [November 25, 2023, 1:11pm UTC](https://discuss.elastic.co/t/no-verify-ssl-input-elasticsearch/347860 "2023-11-25T13:11:46Z")
Hi Dears Is there any way to not verify ssl in input elasticsearch plugin? logstash 7.17 I can not do this! please help --- ## [Logstash 8.1 multiple patterns](https://discuss.elastic.co/t/logstash-8-1-multiple-patterns/347943)
**Author:** [@Dokh\_Ahmed](https://discuss.elastic.co/u/Dokh_Ahmed)\ **Replies:** 1\ **Last updated:** [November 24, 2023, 4:33pm UTC](https://discuss.elastic.co/t/logstash-8-1-multiple-patterns/347943 "2023-11-24T16:33:36Z")
According to the doc of logstash " \`\`\` filter { grok { match =\> \[ "message", "PATTERN1", "PATTERN2" \] } } I wrote my filter as : filter { grok { match =\> { "message" =\> \[ "%{TIMESTAMP\_ISO860… --- ## [Add value to a previously indexed field with logstash](https://discuss.elastic.co/t/add-value-to-a-previously-indexed-field-with-logstash/347940)
**Author:** [@elk-user-0001](https://discuss.elastic.co/u/elk-user-0001)\ **Replies:** 0\ **Last updated:** [November 24, 2023, 3:25pm UTC](https://discuss.elastic.co/t/add-value-to-a-previously-indexed-field-with-logstash/347940 "2023-11-24T15:25:21Z")
Hello! I have a pipeline that has many inputs ( 19 ) and I use the update on the output using a document\_id to avoid duplicates and the elasticsearch filter and update the values. Is it possible to add the value of a f… --- ## [logstash-output-elasticsearch fails with Permission denied](https://discuss.elastic.co/t/logstash-output-elasticsearch-fails-with-permission-denied/347787)
**Author:** [@mirceastoian](https://discuss.elastic.co/u/mirceastoian)\ **Replies:** 18\ **Last updated:** [November 24, 2023, 2:46pm UTC](https://discuss.elastic.co/t/logstash-output-elasticsearch-fails-with-permission-denied/347787 "2023-11-24T14:46:15Z")
Logstash information: Logstash version: 7.17.9 Logstash installation source: deb How is Logstash being run: systemd How was the Logstash Plugin installed: sudo /usr/share/logstash/bin/logstash-plugin install logstash-o… --- ## [Logstash is processing old documents](https://discuss.elastic.co/t/logstash-is-processing-old-documents/347678)
**Author:** [@Cruz](https://discuss.elastic.co/u/Cruz)\ **Replies:** 3\ **Last updated:** [November 24, 2023, 9:12am UTC](https://discuss.elastic.co/t/logstash-is-processing-old-documents/347678 "2023-11-24T09:12:38Z")
When I restart the logstash service, the old documents are coming out. I tried to stopping the filebeat service where the logs are coming from and I deleted the old documents. But when I restart the logstash service the… --- ## [What does "\_ignored" tag mean in hits](https://discuss.elastic.co/t/what-does-ignored-tag-mean-in-hits/344300)
**Author:** [@Aiswarya\_S](https://discuss.elastic.co/u/Aiswarya_S)\ **Replies:** 2\ **Last updated:** [November 24, 2023, 7:19am UTC](https://discuss.elastic.co/t/what-does-ignored-tag-mean-in-hits/344300 "2023-11-24T07:19:49Z")
In my Elastic search pulled data, I am getting an ignored tag in the hits but yet the data is coming correctly... so what does that ignored tag mean? { "took": 9, "timed\_out": false, "\_shards": { "total": 1, … --- ## [Setting Up Logstash In Docker-Compose For Bulk Ingest Of CSV Files In Local Machine](https://discuss.elastic.co/t/setting-up-logstash-in-docker-compose-for-bulk-ingest-of-csv-files-in-local-machine/346916)
**Author:** [@Ethan777100](https://discuss.elastic.co/u/Ethan777100)\ **Replies:** 112\ **Last updated:** [November 23, 2023, 5:55pm UTC](https://discuss.elastic.co/t/setting-up-logstash-in-docker-compose-for-bulk-ingest-of-csv-files-in-local-machine/346916 "2023-11-23T17:55:14Z")
CONTINUATION FROM kibana-8-11-0-failed-to-start-exit-code-1 My use case is to bulk ingest csv files into Elasticsearch. Understand i need Logstash to do it. Not sure how to start. Should I be using a default or cus… --- ## [Duplicate logs in Logstash](https://discuss.elastic.co/t/duplicate-logs-in-logstash/347630)
**Author:** [@marcowiskhy](https://discuss.elastic.co/u/marcowiskhy)\ **Replies:** 8\ **Last updated:** [November 23, 2023, 6:15pm UTC](https://discuss.elastic.co/t/duplicate-logs-in-logstash/347630 "2023-11-23T18:15:47Z")
I collect VPN logs through Logstash and index them in Elasticsearch, but I'm having the following problem: For each unique VPN connection (represented by TunnelID), there should be only one tunnel-up event and one tunne… --- ## [The Output Isolator Pattern: Inquiry regarding downstream pipeline failures](https://discuss.elastic.co/t/the-output-isolator-pattern-inquiry-regarding-downstream-pipeline-failures/347878)
**Author:** [@Kihyun\_Hwang](https://discuss.elastic.co/u/Kihyun_Hwang)\ **Replies:** 2\ **Last updated:** [November 23, 2023, 5:14pm UTC](https://discuss.elastic.co/t/the-output-isolator-pattern-inquiry-regarding-downstream-pipeline-failures/347878 "2023-11-23T17:14:12Z")
I have applied the Output Isolator pattern to send logs to two ES clusters. However, as mentioned in the reference: "If any of the persistent queues of the downstream pipelines (in the example above, buffered-es and bu… --- ## [Sync 2 indices diffrenet remote clusters](https://discuss.elastic.co/t/sync-2-indices-diffrenet-remote-clusters/347851)
**Author:** [@lstoneir](https://discuss.elastic.co/u/lstoneir)\ **Replies:** 0\ **Last updated:** [November 23, 2023, 11:58am UTC](https://discuss.elastic.co/t/sync-2-indices-diffrenet-remote-clusters/347851 "2023-11-23T11:58:50Z")
Hi I have cluser A with index e.g. User\_info I have another cluster named B I want to sync User\_info (B) with User\_info (A) all time!! Can i do this with logstash? how? --- ## [CSV::MalformedCSVError: Missing or stray quote in line 1](https://discuss.elastic.co/t/csv-missing-or-stray-quote-in-line-1/346726)
**Author:** [@parosio](https://discuss.elastic.co/u/parosio)\ **Replies:** 1\ **Last updated:** [November 23, 2023, 11:05am UTC](https://discuss.elastic.co/t/csv-missing-or-stray-quote-in-line-1/346726 "2023-11-23T11:05:25Z")
Hello, I've read the previuos posts on this topic (and related), but still have problems with csv files containing windows command lines... For example: 98792634295,https://falcon.eu-1.crowdstrike.com/activity/detecti… --- ## [Snowflake to Elasticsearch](https://discuss.elastic.co/t/snowflake-to-elasticsearch/347543)
**Author:** [@Shalinicts](https://discuss.elastic.co/u/Shalinicts)\ **Replies:** 8\ **Last updated:** [November 23, 2023, 8:14am UTC](https://discuss.elastic.co/t/snowflake-to-elasticsearch/347543 "2023-11-23T08:14:57Z")
Hi Team , We are trying to pull data from Snowflake database to Elasticsaerch via Logstash JDBC plugin Input Config: input { jdbc { jdbc\_driver\_library =\> "/usr/share/logstash/logstash-core/lib/jars/snowflake-jd… --- ## [Send output socket tcp or udp in line protocol format](https://discuss.elastic.co/t/send-output-socket-tcp-or-udp-in-line-protocol-format/347810)
**Author:** [@Indeed2000](https://discuss.elastic.co/u/Indeed2000)\ **Replies:** 0\ **Last updated:** [November 23, 2023, 4:37am UTC](https://discuss.elastic.co/t/send-output-socket-tcp-or-udp-in-line-protocol-format/347810 "2023-11-23T04:37:15Z")
Hi need to send data with tcp or udp in line protocol format instead on influx or http output plugin. Is it possible to create message format like http output plugin? Any idea? Thank --- ## [Logstash to influxdb2 aggregate datapoints issue](https://discuss.elastic.co/t/logstash-to-influxdb2-aggregate-datapoints-issue/347809)
**Author:** [@Indeed2000](https://discuss.elastic.co/u/Indeed2000)\ **Replies:** 0\ **Last updated:** [November 23, 2023, 4:14am UTC](https://discuss.elastic.co/t/logstash-to-influxdb2-aggregate-datapoints-issue/347809 "2023-11-23T04:14:32Z")
Hi I have lots of log lines like this in exact same time, when i try to use logstash to pars and send to influxdb2, influx or ligstash aggregates some lines! e.g here is the sample lines that aggregate is I\[847676\] 20… --- ## [Reason: Setting "monitoring.enabled" doesn't exist](https://discuss.elastic.co/t/reason-setting-monitoring-enabled-doesnt-exist/347731)
**Author:** [@Vivi\_Allen](https://discuss.elastic.co/u/Vivi_Allen)\ **Replies:** 4\ **Last updated:** [November 23, 2023, 3:48am UTC](https://discuss.elastic.co/t/reason-setting-monitoring-enabled-doesnt-exist/347731 "2023-11-23T03:48:33Z")
I want to enable monitor for logstash with metricbeat. Following this guide Collect Logstash monitoring data with Metricbeat | Logstash Reference \[8.11\] | Elastic, I add monitoring.enabled: false to the logstash.yml. T… --- ## [Logstash / problem with windows index](https://discuss.elastic.co/t/logstash-problem-with-windows-index/347545)
**Author:** [@secsec](https://discuss.elastic.co/u/secsec)\ **Replies:** 4\ **Last updated:** [November 23, 2023, 3:35am UTC](https://discuss.elastic.co/t/logstash-problem-with-windows-index/347545 "2023-11-23T03:35:21Z")
Hello, could you please help me? Im using Elastic version 8.11.1 Im trying to create new 2 indexes for windows and linux. This code below is working for linux (it is automaticaly creating indexes every day), but it i… --- ## [Logstash unable to receive data from MQTT](https://discuss.elastic.co/t/logstash-unable-to-receive-data-from-mqtt/347712)
**Author:** [@Shah\_Zain](https://discuss.elastic.co/u/Shah_Zain)\ **Replies:** 6\ **Last updated:** [November 22, 2023, 6:08pm UTC](https://discuss.elastic.co/t/logstash-unable-to-receive-data-from-mqtt/347712 "2023-11-22T18:08:10Z")
Logstash unable to receive data from MQTT. Getting this from logs: //Stack: C:/Users/Shah Zain/Downloads/logstash-8.11.1-windows-x86\_64/logstash-8.11.1/vendor/bundle/jruby/3.1.0/gems/logstash-input-mqtt-0.0.2/lib/logst… --- ## [Send logstash output to questdb](https://discuss.elastic.co/t/send-logstash-output-to-questdb/347717)
**Author:** [@Indeed2000](https://discuss.elastic.co/u/Indeed2000)\ **Replies:** 4\ **Last updated:** [November 22, 2023, 5:57pm UTC](https://discuss.elastic.co/t/send-logstash-output-to-questdb/347717 "2023-11-22T17:57:01Z")
Hi, how can i send logstash output to questdb? which plugin suitable for this aim and compatible with questdb? which port suitable on questdb for this aim? I have if condition on output if tag = send then write to tab… --- ## [Logstash Permission Issue](https://discuss.elastic.co/t/logstash-permission-issue/347771)
**Author:** [@ranjini](https://discuss.elastic.co/u/ranjini)\ **Replies:** 7\ **Last updated:** [November 22, 2023, 5:40pm UTC](https://discuss.elastic.co/t/logstash-permission-issue/347771 "2023-11-22T17:40:54Z")
Error Message \[2023-11-22T13:15:51,460\]\[WARN \]\[filewatch.sincedbcollection\]\[main\]\[fd97ffae0e8f2b3b8d71c9b308ee7a3feac45d9133bd3968160c580a4d2e603e\] sincedb\_write: unable to write atomically due to permissions error, fal… --- ## [How to write to the same datastream from MetricBeat and Logstash](https://discuss.elastic.co/t/how-to-write-to-the-same-datastream-from-metricbeat-and-logstash/347778)
**Author:** [@Igal\_Hanoch](https://discuss.elastic.co/u/Igal_Hanoch)\ **Replies:** 0\ **Last updated:** [November 22, 2023, 5:26pm UTC](https://discuss.elastic.co/t/how-to-write-to-the-same-datastream-from-metricbeat-and-logstash/347778 "2023-11-22T17:26:18Z")
I'm writing metricbeat data from several computes to my elasticsearch. Some metricbeats are writing directly to ElasicSearch and some through logstash. The data from the metricbeat is written to a datastream named .ds-… --- ## [Grok isn't getting parsed (grok debugger is parsing it fine but it's throwing error in logstash)](https://discuss.elastic.co/t/grok-isnt-getting-parsed-grok-debugger-is-parsing-it-fine-but-its-throwing-error-in-logstash/347706)
**Author:** [@sudhir\_singh](https://discuss.elastic.co/u/sudhir_singh)\ **Replies:** 10\ **Last updated:** [November 22, 2023, 3:06pm UTC](https://discuss.elastic.co/t/grok-isnt-getting-parsed-grok-debugger-is-parsing-it-fine-but-its-throwing-error-in-logstash/347706 "2023-11-22T15:06:46Z")
\- "22/Nov/2023:12:21:04 +0530" 196.24.23.101 GET "GET /api/status HTTP/1.1" 191 200 439 83 - "nginx/1.23.4 (health check server\_103.225.61.177\_Pool-1\_http\_ok)" 127.0.0.1:8080 200 0.002 0.002 0.000 0.002 - pauth.mumbcms.… --- ## [How to link input contain to output contain](https://discuss.elastic.co/t/how-to-link-input-contain-to-output-contain/347635)
**Author:** [@Christian\_1974](https://discuss.elastic.co/u/Christian_1974)\ **Replies:** 9\ **Last updated:** [November 22, 2023, 8:36am UTC](https://discuss.elastic.co/t/how-to-link-input-contain-to-output-contain/347635 "2023-11-22T08:36:42Z")
Hi, I have a question. I have this in my input file : root@Big-Monster:/etc/logstash/conf.d# cat 00\_input.conf input { file { id =\> "TEST-Syslog" path =\> \[ "/var/log/syslog" \] } file { id =\> "TEST-C… --- ## [Rotating File Input Logstash](https://discuss.elastic.co/t/rotating-file-input-logstash/347662)
**Author:** [@hjsroldan](https://discuss.elastic.co/u/hjsroldan)\ **Replies:** 0\ **Last updated:** [November 21, 2023, 6:34pm UTC](https://discuss.elastic.co/t/rotating-file-input-logstash/347662 "2023-11-21T18:34:45Z")
Hi, Good day, I have this scenario where I have a rotating files picked up by Logstash. The .log file is the active file. If the file reaches a certain size it will rollover and create a new file with the format .log.\<+… --- ## [Advantage of logstash input/output plugin over http input/output plugin](https://discuss.elastic.co/t/advantage-of-logstash-input-output-plugin-over-http-input-output-plugin/347659)
**Author:** [@Thiruvikraman](https://discuss.elastic.co/u/Thiruvikraman)\ **Replies:** 1\ **Last updated:** [November 21, 2023, 6:09pm UTC](https://discuss.elastic.co/t/advantage-of-logstash-input-output-plugin-over-http-input-output-plugin/347659 "2023-11-21T18:09:45Z")
Logstash to Logstash considerations This is the preferred method to implement Logstash-to-Logstash. It replaces Logstash-to-Logstash: HTTP output to HTTP input and has these considerations: It relies on HTTP as the co… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=49) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=51)