# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=118 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 119 --- ## [Parsing JSON data](https://discuss.elastic.co/t/parsing-json-data/313217)
**Author:** [@Pukar](https://discuss.elastic.co/u/Pukar)\ **Replies:** 4\ **Last updated:** [September 1, 2022, 12:54pm UTC](https://discuss.elastic.co/t/parsing-json-data/313217 "2022-09-01T12:54:58Z")
Hi Team, New to ELK environment so please bear with me. I'm trying to ingest data from json file and visualise in kibana. The ingest itself is working but the message is coming as: "message" =\> " "totalCount": … --- ## [Logstash don't reach elasticsearch](https://discuss.elastic.co/t/logstash-dont-reach-elasticsearch/312311)
**Author:** [@SilasMuniz1](https://discuss.elastic.co/u/SilasMuniz1)\ **Replies:** 16\ **Last updated:** [September 1, 2022, 11:36am UTC](https://discuss.elastic.co/t/logstash-dont-reach-elasticsearch/312311 "2022-09-01T11:36:39Z")
Hi all, I set up an elasticsearch's cluster with 3 nodes. I deploy the first one e installed after kibana in the same VM(RHEL). It's work ok. With Xpack-security enable. elasticsearch.yml: Added other 2 nodes usin… --- ## [Logstash Error : S3 input plugins](https://discuss.elastic.co/t/logstash-error-s3-input-plugins/313432)
**Author:** [@priyankamondalhcl](https://discuss.elastic.co/u/priyankamondalhcl)\ **Replies:** 0\ **Last updated:** [September 1, 2022, 10:48am UTC](https://discuss.elastic.co/t/logstash-error-s3-input-plugins/313432 "2022-09-01T10:48:40Z")
Hi Team, I am using us3 input plugins which was connecting with S3 bucket and provided data to elasticsearch. but suddenly, It stopped working. My configuration input { s3 { "access\_key\_id" =\> "" "secret\_access\_key… --- ## [Logstash unable to decode Cisco protobuf](https://discuss.elastic.co/t/logstash-unable-to-decode-cisco-protobuf/313424)
**Author:** [@Siddarajgj](https://discuss.elastic.co/u/Siddarajgj)\ **Replies:** 0\ **Last updated:** [September 1, 2022, 9:13am UTC](https://discuss.elastic.co/t/logstash-unable-to-decode-cisco-protobuf/313424 "2022-09-01T09:13:53Z")
Hoping someone has configured logstash to receive Cisco telemetry before, I'm able to receive the events but can't decode the telemetry. I'm getting the folowing error from logstash: \[2022-09-01T09:01:23,341\]\[WARN \]\[lo… --- ## [Autodetect\_column\_name with 2 different CSV](https://discuss.elastic.co/t/autodetect-column-name-with-2-different-csv/313310)
**Author:** [@EliottB](https://discuss.elastic.co/u/EliottB)\ **Replies:** 4\ **Last updated:** [September 1, 2022, 8:37am UTC](https://discuss.elastic.co/t/autodetect-column-name-with-2-different-csv/313310 "2022-09-01T08:37:16Z")
Hello all, I have an issue with the autodetect\_column\_name of my pipeline below: filter { csv { separator =\>"," autodetect\_column\_names =\> true } When I process the first CSV message country,city,name… --- ## [Fetch values from excel sheet in Logstash](https://discuss.elastic.co/t/fetch-values-from-excel-sheet-in-logstash/313406)
**Author:** [@ImranArif](https://discuss.elastic.co/u/ImranArif)\ **Replies:** 0\ **Last updated:** [September 1, 2022, 3:46am UTC](https://discuss.elastic.co/t/fetch-values-from-excel-sheet-in-logstash/313406 "2022-09-01T03:46:05Z")
Hi all, I am parsing CDN logs, which contains a web url that serves images to users, and push these logs to Elasticsearch using Logstash. I want to add a field 'site region' in the logs which contains the region that we… --- ## [Parse Nested Airflow Logs with Logstash](https://discuss.elastic.co/t/parse-nested-airflow-logs-with-logstash/313401)
**Author:** [@Chma](https://discuss.elastic.co/u/Chma)\ **Replies:** 0\ **Last updated:** [August 31, 2022, 10:58pm UTC](https://discuss.elastic.co/t/parse-nested-airflow-logs-with-logstash/313401 "2022-08-31T22:58:08Z")
I am new to Logstash and ELK as a whole. I am trying to send my airflow logs to Logstash. I am confused on how to configure my configuration file, especially because I have several (nested) log files. My airflow is depl… --- ## [Can I push logs to logstash via an API endpoint?](https://discuss.elastic.co/t/can-i-push-logs-to-logstash-via-an-api-endpoint/313399)
**Author:** [@clarkmcc](https://discuss.elastic.co/u/clarkmcc)\ **Replies:** 1\ **Last updated:** [August 31, 2022, 10:43pm UTC](https://discuss.elastic.co/t/can-i-push-logs-to-logstash-via-an-api-endpoint/313399 "2022-08-31T22:43:33Z")
We have ~30 services that run on customer-premise (not our premise) and we'd like to push logs from our service on their machine to an Elasticsearch instance for debugging and analysis. We'd like to avoid having to insta… --- ## [Empty array getting dropped from output, how do I keep it?](https://discuss.elastic.co/t/empty-array-getting-dropped-from-output-how-do-i-keep-it/313392)
**Author:** [@Ben-G](https://discuss.elastic.co/u/Ben-G)\ **Replies:** 2\ **Last updated:** [August 31, 2022, 10:02pm UTC](https://discuss.elastic.co/t/empty-array-getting-dropped-from-output-how-do-i-keep-it/313392 "2022-08-31T22:02:12Z")
The input to my logstash is is populating \[Current\]\[Alarms\] from the input ...\\"Alarms\\":null... when there are no alarms, and when there are alarms then it is a list. When there are no alarms, I still want to see "Ala… --- ## [=\>"Expected one of \[ \\\\t\\\\r\\\\n\], \\"#\\", \\"{\\", \\"}\\"](https://discuss.elastic.co/t/expected-one-of-t-r-n/313258)
**Author:** [@syedabdullah](https://discuss.elastic.co/u/syedabdullah)\ **Replies:** 2\ **Last updated:** [August 31, 2022, 1:59pm UTC](https://discuss.elastic.co/t/expected-one-of-t-r-n/313258 "2022-08-31T13:59:17Z")
I am getting the following error and is resulting in a ingestion issue and cant seem to figure this out: \[2022-08-29T14:23:39,615\]\[ERROR\]\[logstash.agent \] Failed to execute action {:action=\>LogStash::PipelineA… --- ## [Multiple Labels for same field](https://discuss.elastic.co/t/multiple-labels-for-same-field/313358)
**Author:** [@saif.khan](https://discuss.elastic.co/u/saif.khan)\ **Replies:** 0\ **Last updated:** [August 31, 2022, 12:31pm UTC](https://discuss.elastic.co/t/multiple-labels-for-same-field/313358 "2022-08-31T12:31:45Z")
Hi, I'm using SNMP-Logstash plugin to get data from SNMP enabled Network devices(PDUs), It works fine to an extent, The issues comes in for a field that has multiple values, example "IP Address " as I have 3 endpoint d… --- ## [Logstash date filter error](https://discuss.elastic.co/t/logstash-date-filter-error/313253)
**Author:** [@Aniket\_Pant](https://discuss.elastic.co/u/Aniket_Pant)\ **Replies:** 10\ **Last updated:** [August 31, 2022, 12:07pm UTC](https://discuss.elastic.co/t/logstash-date-filter-error/313253 "2022-08-31T12:07:58Z")
I have this field in my logs event\_time=2022-08-30 17:30:42.000 my logstash pipeline file is filter { if "average\_weight" in \[tags\] { kv { source=\>"message" target =\> "test" field\_split=\>"," } mutate { rename =\> { "\[t… --- ## [How to calculate first pass logic and reren for testcase using logstash grok](https://discuss.elastic.co/t/how-to-calculate-first-pass-logic-and-reren-for-testcase-using-logstash-grok/313089)
**Author:** [@aish794726](https://discuss.elastic.co/u/aish794726)\ **Replies:** 4\ **Last updated:** [August 31, 2022, 11:23am UTC](https://discuss.elastic.co/t/how-to-calculate-first-pass-logic-and-reren-for-testcase-using-logstash-grok/313089 "2022-08-31T11:23:50Z")
am working on grok, have very few knowledge about it, we are preparing regression dashboard there we need first pass and reren so that system should understand, these script got pass infirst these are fail n might requir… --- ## [Plugin imap: get attachment](https://discuss.elastic.co/t/plugin-imap-get-attachment/313342)
**Author:** [@Bepsi](https://discuss.elastic.co/u/Bepsi)\ **Replies:** 0\ **Last updated:** [August 31, 2022, 10:17am UTC](https://discuss.elastic.co/t/plugin-imap-get-attachment/313342 "2022-08-31T10:17:17Z")
Hi all, With logstash-oss v7.17.6, i want to read a .zip attachment in a mail. I can read the mail itself, but the attachment can not be read. Logstash conf file: input { imap { id =\> "dmarc" host =\> "xxx" … --- ## [Logstash CSV filter plugins](https://discuss.elastic.co/t/logstash-csv-filter-plugins/313327)
**Author:** [@priyankamondalhcl](https://discuss.elastic.co/u/priyankamondalhcl)\ **Replies:** 1\ **Last updated:** [August 31, 2022, 8:19am UTC](https://discuss.elastic.co/t/logstash-csv-filter-plugins/313327 "2022-08-31T08:19:18Z")
Hi Team, I am trying to read csv file and storing it in elasticsearch. Is there any way to read column from file itself (header of csv file ) in logstash "csv filter plugins". currently I am manually setting up the co… --- ## [Logstash import XML/ HTTP fails](https://discuss.elastic.co/t/logstash-import-xml-http-fails/313305)
**Author:** [@phamquenhu95](https://discuss.elastic.co/u/phamquenhu95)\ **Replies:** 0\ **Last updated:** [August 31, 2022, 4:02am UTC](https://discuss.elastic.co/t/logstash-import-xml-http-fails/313305 "2022-08-31T04:02:53Z")
I am new in this field. I have the following XML file, I want to separate the fields eg "so2Grade", "coFlag",... into rows one by one. My conf file is as follows: input { http\_poller { urls =\> { urlname =\> … --- ## [S3 input plugin point to today's date or skip old data](https://discuss.elastic.co/t/s3-input-plugin-point-to-todays-date-or-skip-old-data/313304)
**Author:** [@sw0701](https://discuss.elastic.co/u/sw0701)\ **Replies:** 0\ **Last updated:** [August 31, 2022, 2:33am UTC](https://discuss.elastic.co/t/s3-input-plugin-point-to-todays-date-or-skip-old-data/313304 "2022-08-31T02:33:18Z")
Hi, I want to ignore the old data in the bucket and point to start from today's date for example: 2022-08-31T01:00:\* Please let me know how i can point to that timestamp or latest timestamp. bash-4.2$ cat sincedb\_3109… --- ## [Any suggestions for creating logstash pipeline by REST APIs](https://discuss.elastic.co/t/any-suggestions-for-creating-logstash-pipeline-by-rest-apis/313219)
**Author:** [@jskuo](https://discuss.elastic.co/u/jskuo)\ **Replies:** 0\ **Last updated:** [August 30, 2022, 7:02am UTC](https://discuss.elastic.co/t/any-suggestions-for-creating-logstash-pipeline-by-rest-apis/313219 "2022-08-30T07:02:17Z")
platform: es/logstash 7.17.5 on centos 7 I intend to create a logstash pipeline, named test, by REST APIs curl -XPUT localhost:9600/\_logstash/pipeline/test?pretty -H "content-type:application/json" -d '{"pipeline": "in… --- ## [Executing bash script in logstash's filter and geting data back to field](https://discuss.elastic.co/t/executing-bash-script-in-logstashs-filter-and-geting-data-back-to-field/313297)
**Author:** [@rayg00n](https://discuss.elastic.co/u/rayg00n)\ **Replies:** 3\ **Last updated:** [August 30, 2022, 10:17pm UTC](https://discuss.elastic.co/t/executing-bash-script-in-logstashs-filter-and-geting-data-back-to-field/313297 "2022-08-30T22:17:47Z")
Hello friends! I need some help with my logstash config. I want to get event field "winlog.event\_data.ObjectName" and take it to a bash script. Then I want to get the result back and put it in the "winlog.event\_data.O… --- ## [Gzip file input failing again in Logstash 8.4](https://discuss.elastic.co/t/gzip-file-input-failing-again-in-logstash-8-4/313255)
**Author:** [@freddyh](https://discuss.elastic.co/u/freddyh)\ **Replies:** 1\ **Last updated:** [August 30, 2022, 4:17pm UTC](https://discuss.elastic.co/t/gzip-file-input-failing-again-in-logstash-8-4/313255 "2022-08-30T16:17:08Z")
There's a few discussion posts (1, 2) and a github issue about a similar issue in the past, but previously it was reported to happen with unsupported Java versions, but I'm now getting this with version 8.4 of the offici… --- ## [Unable to create Kafka consumer from given configuration after upgrade to 8.4.0](https://discuss.elastic.co/t/unable-to-create-kafka-consumer-from-given-configuration-after-upgrade-to-8-4-0/313254)
**Author:** [@mcosta](https://discuss.elastic.co/u/mcosta)\ **Replies:** 0\ **Last updated:** [August 30, 2022, 12:48pm UTC](https://discuss.elastic.co/t/unable-to-create-kafka-consumer-from-given-configuration-after-upgrade-to-8-4-0/313254 "2022-08-30T12:48:02Z")
Hi all, Today Elastic and Kibana was upgraded from 7.17.3 to 8.4.0 without problems. This is an DEV/QA environment. After that, I upgrade logstash also from 7.17.3 to 8.4.0. Since logstash upgrade it can't connect to K… --- ## [Oracle JDBC connection - sql\_last\_value TIMESTAMP comparison isn't working as expected](https://discuss.elastic.co/t/oracle-jdbc-connection-sql-last-value-timestamp-comparison-isnt-working-as-expected/313193)
**Author:** [@LChand](https://discuss.elastic.co/u/LChand)\ **Replies:** 3\ **Last updated:** [August 30, 2022, 12:37pm UTC](https://discuss.elastic.co/t/oracle-jdbc-connection-sql-last-value-timestamp-comparison-isnt-working-as-expected/313193 "2022-08-30T12:37:54Z")
Hello, Looking for some help asap please... I've an issue where same set of data i.e. all records for the current day gets picked with every scheduled run during the day even though the sql\_last\_value has the latest dat… --- ## [Logstash syncing from mongodb to elastic search null vaule](https://discuss.elastic.co/t/logstash-syncing-from-mongodb-to-elastic-search-null-vaule/313192)
**Author:** [@bindu1](https://discuss.elastic.co/u/bindu1)\ **Replies:** 0\ **Last updated:** [August 29, 2022, 7:18pm UTC](https://discuss.elastic.co/t/logstash-syncing-from-mongodb-to-elastic-search-null-vaule/313192 "2022-08-29T19:18:15Z")
Continuing the discussion from Logstash syncing from Mongodb to Elasticsearch - Null values in array data objects: --- ## [Grok csv+json](https://discuss.elastic.co/t/grok-csv-json/313010)
**Author:** [@exesition](https://discuss.elastic.co/u/exesition)\ **Replies:** 2\ **Last updated:** [August 29, 2022, 5:46pm UTC](https://discuss.elastic.co/t/grok-csv-json/313010 "2022-08-29T17:46:23Z")
Colleagues, good afternoon. Tell me I'm new to ELK. I have a log with the following file structure 2022-08-04T12:04:02,410|DEBUG|apg-b2p-keepr|instance\_3|http-8343-exec-5345|gtwRequestOut.com.B2pGateway|POST /trnproces… --- ## [SNMP Trap listener died {:exception=\>#\ **Author:** [@bijender\_kr](https://discuss.elastic.co/u/bijender_kr)\ **Replies:** 0\ **Last updated:** [August 29, 2022, 2:22pm UTC](https://discuss.elastic.co/t/snmp-trap-listener-died-exception-errno-eaddrinuse/313175 "2022-08-29T14:22:51Z") Hi, I am getting below error while using snmptrap plugin. \[2022-08-29T10:20:54,371\]\[INFO \]\[logstash.inputs.snmptrap \]\[snmptrap\]\[aef6e9ad69b62db067a46e53d1b0e0b15b5d216cd954b6f321afcb5a16300adc\] It's a Trap! {:Port=\>106… --- ## [Can’t send winlogbeat data to ELK/logtash](https://discuss.elastic.co/t/can-t-send-winlogbeat-data-to-elk-logtash/313172)
**Author:** [@DanyGuiZH](https://discuss.elastic.co/u/DanyGuiZH)\ **Replies:** 0\ **Last updated:** [August 29, 2022, 1:38pm UTC](https://discuss.elastic.co/t/can-t-send-winlogbeat-data-to-elk-logtash/313172 "2022-08-29T13:38:19Z")
Hello I am asking for help in setting up a Winlogbeat configuration. The final goal is: "Winlogbeat sends events to a specific ELK index \* The specific ELK index has 1 month of retention\* There is a notificati… --- ## [Is there a way to dump data in a file if logstash fails to dump the data in elasticsearch for a particular run](https://discuss.elastic.co/t/is-there-a-way-to-dump-data-in-a-file-if-logstash-fails-to-dump-the-data-in-elasticsearch-for-a-particular-run/313134)
**Author:** [@anand\_tripathi](https://discuss.elastic.co/u/anand_tripathi)\ **Replies:** 2\ **Last updated:** [August 29, 2022, 9:42am UTC](https://discuss.elastic.co/t/is-there-a-way-to-dump-data-in-a-file-if-logstash-fails-to-dump-the-data-in-elasticsearch-for-a-particular-run/313134 "2022-08-29T09:42:10Z")
I am using Logstash to dump my data from Postgres to Elasticsearch. So the input is JDBC that is fetching data from Postgres using tracking\_column and let's say for one run it is taking 1000 records from Postgres Output… --- ## [Closing (Connection reset by peer) keeps being logged on logstash logs](https://discuss.elastic.co/t/closing-connection-reset-by-peer-keeps-being-logged-on-logstash-logs/311992)
**Author:** [@Thuunder7](https://discuss.elastic.co/u/Thuunder7)\ **Replies:** 9\ **Last updated:** [August 29, 2022, 9:23am UTC](https://discuss.elastic.co/t/closing-connection-reset-by-peer-keeps-being-logged-on-logstash-logs/311992 "2022-08-29T09:23:37Z")
Hello, Since i upgraded my two logstash instances from 7.16.3 to 8.3.1 i am seeing the appearence of the above logs: Does someone know why this could be happening? I didnt had these INFO logs before on the previous … --- ## [Trying to understand how Logstash Redis input plugin works](https://discuss.elastic.co/t/trying-to-understand-how-logstash-redis-input-plugin-works/313042)
**Author:** [@ImranArif](https://discuss.elastic.co/u/ImranArif)\ **Replies:** 3\ **Last updated:** [August 26, 2022, 8:54pm UTC](https://discuss.elastic.co/t/trying-to-understand-how-logstash-redis-input-plugin-works/313042 "2022-08-26T20:54:56Z")
Hi, Please consider the following input plugin configuration of Logstash: input { redis { host =\> "es1" port =\> 6380 batch\_count =\> 2000 data\_type =\> "list" key =\> "log\_list" codec =\> plain … --- ## [.logstash\_jdbc\_last\_run not created when running logstash as a service](https://discuss.elastic.co/t/logstash-jdbc-last-run-not-created-when-running-logstash-as-a-service/313008)
**Author:** [@Kowsalya\_Mouttou](https://discuss.elastic.co/u/Kowsalya_Mouttou)\ **Replies:** 6\ **Last updated:** [August 26, 2022, 4:08pm UTC](https://discuss.elastic.co/t/logstash-jdbc-last-run-not-created-when-running-logstash-as-a-service/313008 "2022-08-26T16:08:05Z")
Hi, My problem is when I run logstash as a service. I'm working with centos6 and ELK 7.9.2 For a same config file with logstash-input-jdbc, evrything works fine when running logstash with the command line : /usr/shar… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=117) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=119)