# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=27 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 28 --- ## [Snmp poller - Logstash - Query](https://discuss.elastic.co/t/snmp-poller-logstash-query/362095)
**Author:** [@vijjigani](https://discuss.elastic.co/u/vijjigani)\ **Replies:** 0\ **Last updated:** [June 26, 2024, 1:17pm UTC](https://discuss.elastic.co/t/snmp-poller-logstash-query/362095 "2024-06-26T13:17:16Z")
I have a query about the snmp plugin - (Logstash). I want to fetch the cpu and memory usage value from the network device through snmp OIDs . and i need to create the two different events(one is for CPU and another one i… --- ## [Logstash with CVE-2023-26604 - systemd blocked for production](https://discuss.elastic.co/t/logstash-with-cve-2023-26604-systemd-blocked-for-production/362042)
**Author:** [@Madhab\_Chandra\_Pal](https://discuss.elastic.co/u/Madhab_Chandra_Pal)\ **Replies:** 1\ **Last updated:** [June 26, 2024, 12:42pm UTC](https://discuss.elastic.co/t/logstash-with-cve-2023-26604-systemd-blocked-for-production/362042 "2024-06-26T12:42:41Z")
Hi, I am trying to use logstash 8.12.2 for our production but I am struck due to High vulnerability from CVE-2023-26604 - systemd and it does not have a remediation as of today being an OS package. Is there any alterna… --- ## [Logstash ElasticSearch Filter doesn't work with dissect fields or IP ranges](https://discuss.elastic.co/t/logstash-elasticsearch-filter-doesnt-work-with-dissect-fields-or-ip-ranges/362068)
**Author:** [@Laverio](https://discuss.elastic.co/u/Laverio)\ **Replies:** 0\ **Last updated:** [June 26, 2024, 6:30am UTC](https://discuss.elastic.co/t/logstash-elasticsearch-filter-doesnt-work-with-dissect-fields-or-ip-ranges/362068 "2024-06-26T06:30:10Z")
Hi guys, I still rely on an old minemeld installation that exports its IOC to and Elasticsearch index. Exported ip ranges are similar to this one: 104.156.155.0-104.156.155.255, that is not a CIDR notation nor and elast… --- ## [Versions of components used in logstash 8.12.2](https://discuss.elastic.co/t/versions-of-components-used-in-logstash-8-12-2/361768)
**Author:** [@inkumari](https://discuss.elastic.co/u/inkumari)\ **Replies:** 2\ **Last updated:** [June 26, 2024, 4:16am UTC](https://discuss.elastic.co/t/versions-of-components-used-in-logstash-8-12-2/361768 "2024-06-26T04:16:44Z")
Hi We are using logstash 8.12.2 in our setup in docker. Due to security reasons, we need to know what are the versions of the following components used in this version- giflib lcms libnss-nis libnss-nisplus logstash-e… --- ## [I am not able to load csv data into elasticsearch & kibana](https://discuss.elastic.co/t/i-am-not-able-to-load-csv-data-into-elasticsearch-kibana/361988)
**Author:** [@Hellboy1](https://discuss.elastic.co/u/Hellboy1)\ **Replies:** 3\ **Last updated:** [June 25, 2024, 8:42pm UTC](https://discuss.elastic.co/t/i-am-not-able-to-load-csv-data-into-elasticsearch-kibana/361988 "2024-06-25T20:42:40Z")
I am not able to load data to elasticsearch using logstash. myconfig file input { file{ path=\>"/Users/hellboy/Documents/vscode/notebooks/files/new.csv" start\_position =\> "beginning" sincedb\_… --- ## [Extract data from nested json output and assign to new felids](https://discuss.elastic.co/t/extract-data-from-nested-json-output-and-assign-to-new-felids/361901)
**Author:** [@vijjigani](https://discuss.elastic.co/u/vijjigani)\ **Replies:** 1\ **Last updated:** [June 25, 2024, 2:22pm UTC](https://discuss.elastic.co/t/extract-data-from-nested-json-output-and-assign-to-new-felids/361901 "2024-06-25T14:22:04Z")
Dear team , i want to extract the details ,description,sourceDescription and map to other fields . here i am using the http\_poller plug in .i need the filter section , can anyone help here ?i have tried in multiple ways … --- ## [Convert the default ES UTC time to local timezone with logstash](https://discuss.elastic.co/t/convert-the-default-es-utc-time-to-local-timezone-with-logstash/361843)
**Author:** [@gbL2k](https://discuss.elastic.co/u/gbL2k)\ **Replies:** 11\ **Last updated:** [June 25, 2024, 2:11pm UTC](https://discuss.elastic.co/t/convert-the-default-es-utc-time-to-local-timezone-with-logstash/361843 "2024-06-25T14:11:10Z")
Hello everyone, My goal is export alerts from elasticsearch and I have 2 problems left to solve: My current configuration: input { elasticsearch { hosts =\> "https://ip:9200" index =\> ".internal.alerts-securi… --- ## [Badly formatted index](https://discuss.elastic.co/t/badly-formatted-index/361709)
**Author:** [@Khaled\_Saidi](https://discuss.elastic.co/u/Khaled_Saidi)\ **Replies:** 4\ **Last updated:** [June 24, 2024, 2:56pm UTC](https://discuss.elastic.co/t/badly-formatted-index/361709 "2024-06-24T14:56:12Z")
Hi the team, I have a logstash pipeline which worked well sometimes for 2 or 3 weeks, but this time it worked for 2 days, and this morning it return this error : Badly formatted index, after interpolation still contain… --- ## [How to ingest large files in ELK stack](https://discuss.elastic.co/t/how-to-ingest-large-files-in-elk-stack/361772)
**Author:** [@mradulag](https://discuss.elastic.co/u/mradulag)\ **Replies:** 1\ **Last updated:** [June 24, 2024, 9:15am UTC](https://discuss.elastic.co/t/how-to-ingest-large-files-in-elk-stack/361772 "2024-06-24T09:15:50Z")
I have line separated JSON files in the order of 1 TB, I can split the files as per the requirement, I just want to efficiently ingest all the data in my ELK stack. I am trying to use Logstash for this purpose, and this … --- ## [How do I get Internal original IP addresses of the devices from router logs](https://discuss.elastic.co/t/how-do-i-get-internal-original-ip-addresses-of-the-devices-from-router-logs/361934)
**Author:** [@Md\_Wahid\_Sadiq\_Ratul](https://discuss.elastic.co/u/Md_Wahid_Sadiq_Ratul)\ **Replies:** 0\ **Last updated:** [June 24, 2024, 8:19am UTC](https://discuss.elastic.co/t/how-do-i-get-internal-original-ip-addresses-of-the-devices-from-router-logs/361934 "2024-06-24T08:19:09Z")
I am a new learner of the ELK stack. I am taking logs from the router using SNMP and UDP protocol. But I am only getting Post Nat IPV4 addresses. I am including the config code here. input { snmptrap { port =\> 162… --- ## [Logstash not ingesting fast moving log](https://discuss.elastic.co/t/logstash-not-ingesting-fast-moving-log/361807)
**Author:** [@zaeemmasood](https://discuss.elastic.co/u/zaeemmasood)\ **Replies:** 10\ **Last updated:** [June 24, 2024, 2:36am UTC](https://discuss.elastic.co/t/logstash-not-ingesting-fast-moving-log/361807 "2024-06-24T02:36:22Z")
Hi All, We have a very fast moving/ rolling log file on our Linux server. This log moves very quickly and a job gzips it every hour. I enabled filebeat output to console using the following in filebeat.yml: output.co… --- ## [Logstash - Invalid version of beats error](https://discuss.elastic.co/t/logstash-invalid-version-of-beats-error/361698)
**Author:** [@Viktor\_Movita](https://discuss.elastic.co/u/Viktor_Movita)\ **Replies:** 7\ **Last updated:** [June 23, 2024, 7:41am UTC](https://discuss.elastic.co/t/logstash-invalid-version-of-beats-error/361698 "2024-06-23T07:41:44Z")
Hello everyone, I have a Logstash server that receives data from Elastic agents and writes it to Elasticsearch. For testing purposes, I have one standalone agent running on a Linux server and one agent managed by a flee… --- ## [How to ingest very large json file into elastic search quickly](https://discuss.elastic.co/t/how-to-ingest-very-large-json-file-into-elastic-search-quickly/361816)
**Author:** [@mradulag](https://discuss.elastic.co/u/mradulag)\ **Replies:** 3\ **Last updated:** [June 21, 2024, 4:37am UTC](https://discuss.elastic.co/t/how-to-ingest-very-large-json-file-into-elastic-search-quickly/361816 "2024-06-21T04:37:16Z")
My setup: I have a very large line separated JSON file in the order of TBs. I want to ingest this file as quickly as possible into my Elasticsearch server, I have enough hardware requirements and a cluster setup using K… --- ## [Not able to use templates with Log Stash](https://discuss.elastic.co/t/not-able-to-use-templates-with-log-stash/361777)
**Author:** [@mradulag](https://discuss.elastic.co/u/mradulag)\ **Replies:** 2\ **Last updated:** [June 21, 2024, 3:20am UTC](https://discuss.elastic.co/t/not-able-to-use-templates-with-log-stash/361777 "2024-06-21T03:20:22Z")
I am trying to configure the number of shards in Logstash using templates, this is my template: { "template": "logstash-\*", "settings": { "number\_of\_shards": 32, "number\_of\_replicas" : 0 } } … --- ## [GCS Input not working as expected](https://discuss.elastic.co/t/gcs-input-not-working-as-expected/361713)
**Author:** [@Random\_BB](https://discuss.elastic.co/u/Random_BB)\ **Replies:** 5\ **Last updated:** [June 20, 2024, 7:17am UTC](https://discuss.elastic.co/t/gcs-input-not-working-as-expected/361713 "2024-06-20T07:17:24Z")
I have an AWS Domain to which I am trying to Ingest Json codec files from GCS bucket using Logstash. I can see a log stating that my logstash has connected to domain and a few other logs stating it is trying to fetch log… --- ## [Logstash stopped processing because of an error](https://discuss.elastic.co/t/logstash-stopped-processing-because-of-an-error/361716)
**Author:** [@SIM\_Vik](https://discuss.elastic.co/u/SIM_Vik)\ **Replies:** 4\ **Last updated:** [June 20, 2024, 7:02am UTC](https://discuss.elastic.co/t/logstash-stopped-processing-because-of-an-error/361716 "2024-06-20T07:02:56Z")
I tried many solutions from other topics, but they didn’t help me. I installed logstash, wrote conf and now logstash can’t start, what am I doing wrong. I run logstash via systemctl logstash-plain.log \[2024-06-19T11… --- ## [Index Permission blocking ingestion of the whole pipeline](https://discuss.elastic.co/t/index-permission-blocking-ingestion-of-the-whole-pipeline/361189)
**Author:** [@Perry\_Lam](https://discuss.elastic.co/u/Perry_Lam)\ **Replies:** 7\ **Last updated:** [June 19, 2024, 3:30pm UTC](https://discuss.elastic.co/t/index-permission-blocking-ingestion-of-the-whole-pipeline/361189 "2024-06-19T15:30:15Z")
Hi, My Logstash containers stopped their ingestion when one index pattern is not permitted in ESS. Code (simplified): output { opensearch { id =\> "oss\_output\_cluster-unknown" hosts =\> \[… --- ## [Index not showing up in Kibana while using Logstash](https://discuss.elastic.co/t/index-not-showing-up-in-kibana-while-using-logstash/361717)
**Author:** [@dou07](https://discuss.elastic.co/u/dou07)\ **Replies:** 6\ **Last updated:** [June 19, 2024, 10:46pm UTC](https://discuss.elastic.co/t/index-not-showing-up-in-kibana-while-using-logstash/361717 "2024-06-19T22:46:07Z")
Hello everyone, I want to load a json file into elasticsearch using Logstash. I am following all the steps but the index is still not showing up in Kibana. For reference, this is my Logstash configuration file : input… --- ## [Get user.name from user.id in Entity Analytics Azure Entra ID dataset](https://discuss.elastic.co/t/get-user-name-from-user-id-in-entity-analytics-azure-entra-id-dataset/359019)
**Author:** [@willemdh](https://discuss.elastic.co/u/willemdh)\ **Replies:** 5\ **Last updated:** [June 19, 2024, 7:25pm UTC](https://discuss.elastic.co/t/get-user-name-from-user-id-in-entity-analytics-azure-entra-id-dataset/359019 "2024-06-19T19:25:20Z")
Hello, So now we have the entity analytics, is it on Elastic's to do to enrich certain datasets with the real user.name from the user.id ? For example in the Azure Graph API logs only a user.id field is known. The user… --- ## [Parse date from @timestamp](https://discuss.elastic.co/t/parse-date-from-timestamp/361721)
**Author:** [@TasK](https://discuss.elastic.co/u/TasK)\ **Replies:** 3\ **Last updated:** [June 19, 2024, 11:27am UTC](https://discuss.elastic.co/t/parse-date-from-timestamp/361721 "2024-06-19T11:27:07Z")
Hello, I'm trying to replace the date in kibana with the date from the logfile, which gets pre-parsed with the json parser from filebeat and looks like this: { "@timestamp": "2024-06-19T10:00:00.469Z", "log.lev… --- ## [Pipeline error - special characters are not allowed in reader](https://discuss.elastic.co/t/pipeline-error-special-characters-are-not-allowed-in-reader/360926)
**Author:** [@Bela\_Kovacs](https://discuss.elastic.co/u/Bela_Kovacs)\ **Replies:** 2\ **Last updated:** [June 7, 2024, 11:57am UTC](https://discuss.elastic.co/t/pipeline-error-special-characters-are-not-allowed-in-reader/360926 "2024-06-07T11:57:46Z")
Hello Logstash 7.11.1 starting with errors and not creating indices. It writes pipeline error - special characters are not allowed in reader, but I don't know which file and it's worked from 2021 but now it's stopped wi… --- ## [Logstash XML Filter Plugin - XML Parsing General Question](https://discuss.elastic.co/t/logstash-xml-filter-plugin-xml-parsing-general-question/360954)
**Author:** [@domino](https://discuss.elastic.co/u/domino)\ **Replies:** 6\ **Last updated:** [June 19, 2024, 11:04am UTC](https://discuss.elastic.co/t/logstash-xml-filter-plugin-xml-parsing-general-question/360954 "2024-06-19T11:04:02Z")
Hi all I need to parse all xml-elements on all hierarchies in millions of xml-files with logstash as pasted below and store them in separate json fields in Elasticsearch. One xml file shall end up in one json document i… --- ## [Dynamically set connection string on jdbc\_streaming filter](https://discuss.elastic.co/t/dynamically-set-connection-string-on-jdbc-streaming-filter/361675)
**Author:** [@DMcP89](https://discuss.elastic.co/u/DMcP89)\ **Replies:** 1\ **Last updated:** [June 18, 2024, 11:56pm UTC](https://discuss.elastic.co/t/dynamically-set-connection-string-on-jdbc-streaming-filter/361675 "2024-06-18T23:56:07Z")
Is it possible to dynamically set the connection string on a jdbc\_streaming filter? Something like this for example: mutate { add\_field =\> { "connection\_string" =\> "jdbc:sqlserver://test.mydb.co… --- ## [Make aggregate filter to take only unique (first) message](https://discuss.elastic.co/t/make-aggregate-filter-to-take-only-unique-first-message/361143)
**Author:** [@Delvin](https://discuss.elastic.co/u/Delvin)\ **Replies:** 6\ **Last updated:** [June 18, 2024, 8:12am UTC](https://discuss.elastic.co/t/make-aggregate-filter-to-take-only-unique-first-message/361143 "2024-06-18T08:12:22Z")
Hi, everyone! The problem is that sometimes we get logs with same task\_id (instock.rf.screenId in our case) and message fields. It look like this: "\_source": { "@timestamp": "2024-05-28T05:43:34.769Z", "input": { "… --- ## [Issue with unintentionally becoming non\_running in Logstash management](https://discuss.elastic.co/t/issue-with-unintentionally-becoming-non-running-in-logstash-management/361298)
**Author:** [@shibadog](https://discuss.elastic.co/u/shibadog)\ **Replies:** 7\ **Last updated:** [June 18, 2024, 7:29am UTC](https://discuss.elastic.co/t/issue-with-unintentionally-becoming-non-running-in-logstash-management/361298 "2024-06-18T07:29:10Z")
Using Logstash management, I was adding pipeline settings from Kibana. Although it was working fine before, after adding a pipeline, the newly added pipeline unintentionally became non-running. The environment is as fol… --- ## [Fatal level logs are not processing in Logstash](https://discuss.elastic.co/t/fatal-level-logs-are-not-processing-in-logstash/358308)
**Author:** [@ErGeek](https://discuss.elastic.co/u/ErGeek)\ **Replies:** 1\ **Last updated:** [June 17, 2024, 12:46pm UTC](https://discuss.elastic.co/t/fatal-level-logs-are-not-processing-in-logstash/358308 "2024-06-17T12:46:57Z")
Hi All, We have an architecture where the log-flow process starts from Beats -\> AWS MSK (Kafka) -\> Logstash (Self hosted)-\> Elasticsearch -\> Kibana. But we are facing a challenge where the FATAL level logs are coming t… --- ## [A plugin had an unrecoverable error. Will restart this plugin](https://discuss.elastic.co/t/a-plugin-had-an-unrecoverable-error-will-restart-this-plugin/361261)
**Author:** [@thami](https://discuss.elastic.co/u/thami)\ **Replies:** 3\ **Last updated:** [June 17, 2024, 9:52am UTC](https://discuss.elastic.co/t/a-plugin-had-an-unrecoverable-error-will-restart-this-plugin/361261 "2024-06-17T09:52:36Z")
\[2024-06-11T15:18:52,306\]\[ERROR\]\[logstash.javapipeline \]\[main\]\[426367acac36cefc65edfdc25912c990f3c7b8ca6c411d526967d63232a012e3\] A plugin had an unrecoverable error. Will restart this plugin. ", "i-"\]}, id=\>"426367ac… --- ## [Logstash geopif time out](https://discuss.elastic.co/t/logstash-geopif-time-out/361557)
**Author:** [@ranjini](https://discuss.elastic.co/u/ranjini)\ **Replies:** 0\ **Last updated:** [June 17, 2024, 5:49am UTC](https://discuss.elastic.co/t/logstash-geopif-time-out/361557 "2024-06-17T05:49:49Z")
\[2024-06-13T11:02:33,820\]\[INFO \]\[logstash.filters.geoip.downloadmanager\] new database version detected? true \[2024-06-13T11:02:36,516\]\[INFO \]\[logstash.filters.geoip.databasemanager\] geoip plugin will use database /usr/sh… --- ## [Filter by multiple source ports](https://discuss.elastic.co/t/filter-by-multiple-source-ports/361500)
**Author:** [@mmercaldi](https://discuss.elastic.co/u/mmercaldi)\ **Replies:** 3\ **Last updated:** [June 14, 2024, 2:26pm UTC](https://discuss.elastic.co/t/filter-by-multiple-source-ports/361500 "2024-06-14T14:26:52Z")
I have 2 different streams going into the same logstash instance. I want it so if the sourceport is 5510 output to connection A and if sourceport is 5511 output to connection B. But I do not know how to reference the s… --- ## [Drop filter erros](https://discuss.elastic.co/t/drop-filter-erros/361364)
**Author:** [@Honestabe](https://discuss.elastic.co/u/Honestabe)\ **Replies:** 5\ **Last updated:** [June 14, 2024, 11:24am UTC](https://discuss.elastic.co/t/drop-filter-erros/361364 "2024-06-14T11:24:49Z")
I am using the dissect filter to ingest csv files into elastic I used the drop filter to eliminate the headers. filter { if \[Email\] == "Email" { drop{} } } but when I look at the index there where still docume… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=26) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=28)