# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=66 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 67 --- ## [NOOB, Elastic Stack (Search/Logstash) Netflow Cisco SD-WAN](https://discuss.elastic.co/t/noob-elastic-stack-search-logstash-netflow-cisco-sd-wan/338687)
**Author:** [@mhollingsworth1](https://discuss.elastic.co/u/mhollingsworth1)\ **Replies:** 0\ **Last updated:** [July 18, 2023, 2:01pm UTC](https://discuss.elastic.co/t/noob-elastic-stack-search-logstash-netflow-cisco-sd-wan/338687 "2023-07-18T14:01:36Z")
I've been using an ELK stack for Netflow collection for roughly 1.5yr with my Cisco environment. Recently I've migrated onto SD-WAN and am sending my netflow data to the collector but for whatever reason all I'm seeing i… --- ## [How to configure multi-pipeline configuration in logstash 8.6.1?](https://discuss.elastic.co/t/how-to-configure-multi-pipeline-configuration-in-logstash-8-6-1/339071)
**Author:** [@Koele](https://discuss.elastic.co/u/Koele)\ **Replies:** 4\ **Last updated:** [July 25, 2023, 3:26am UTC](https://discuss.elastic.co/t/how-to-configure-multi-pipeline-configuration-in-logstash-8-6-1/339071 "2023-07-25T03:26:59Z")
Basic Information logstash version: 8.6.1 Logstash installation method: tar.gz logstash installation directory: /opt OS: CentOS 7 ogstash.yml configuration file node.name: logstash01 path.data: /data/logstash01 pipe… --- ## [Logstash Create pipeline API: \[pipeline\] failed to parse object / json\_parse\_exception](https://discuss.elastic.co/t/logstash-create-pipeline-api-pipeline-failed-to-parse-object-json-parse-exception/339055)
**Author:** [@alexus](https://discuss.elastic.co/u/alexus)\ **Replies:** 1\ **Last updated:** [July 24, 2023, 4:46pm UTC](https://discuss.elastic.co/t/logstash-create-pipeline-api-pipeline-failed-to-parse-object-json-parse-exception/339055 "2023-07-24T16:46:21Z")
Hello World! I'm trying to follow this: to duplicate an existing Logstash pipeline, however I'm running into following error: % export PIPELINE=$(curl --silent --request GET --insecure "https://elastic:$es\_password@… --- ## [Logstash is not getting Kubernetes Metadata for logs coming from S3](https://discuss.elastic.co/t/logstash-is-not-getting-kubernetes-metadata-for-logs-coming-from-s3/339102)
**Author:** [@akshayw](https://discuss.elastic.co/u/akshayw)\ **Replies:** 0\ **Last updated:** [July 24, 2023, 1:18pm UTC](https://discuss.elastic.co/t/logstash-is-not-getting-kubernetes-metadata-for-logs-coming-from-s3/339102 "2023-07-24T13:18:20Z")
Hi, I have setup ELK-FluentBit stack in Kubernetes cluster. I'm pushing my application logs in cluster to S3 using FluentBit and adding those logs to ELK using S3-input plugin of Logstash. \[App Logs --\> FluentBit --\> S3… --- ## [Pipeline Worker Loop Initialization Error](https://discuss.elastic.co/t/pipeline-worker-loop-initialization-error/338972)
**Author:** [@dro](https://discuss.elastic.co/u/dro)\ **Replies:** 2\ **Last updated:** [July 22, 2023, 5:15am UTC](https://discuss.elastic.co/t/pipeline-worker-loop-initialization-error/338972 "2023-07-22T05:15:29Z")
Been running into an issue with Logstash failing to initialize due to pipeline worker error. Containerized version being used: Logstash 8.8.2 Error: Using bundled JDK: /usr/share/logstash/jdk Sending Logstash logs to … --- ## [Logstash container crashing on AWS Fargate](https://discuss.elastic.co/t/logstash-container-crashing-on-aws-fargate/338940)
**Author:** [@Amine1979](https://discuss.elastic.co/u/Amine1979)\ **Replies:** 0\ **Last updated:** [July 21, 2023, 10:27am UTC](https://discuss.elastic.co/t/logstash-container-crashing-on-aws-fargate/338940 "2023-07-21T10:27:30Z")
Hello, Im using Below DockerFile and taskdefinition to deploy Logstash container on AWS Fargate. Issue : Locally it works fine, logstash server start listening on 5044 On Fargate, container crashs with this message / … --- ## [Badly formatted index, after interpolation still contains placeholder: \[%{\[@metadat a\]\[target\_index\]}\]](https://discuss.elastic.co/t/badly-formatted-index-after-interpolation-still-contains-placeholder-metadat-a-target-index/338905)
**Author:** [@dsv](https://discuss.elastic.co/u/dsv)\ **Replies:** 2\ **Last updated:** [July 21, 2023, 11:00am UTC](https://discuss.elastic.co/t/badly-formatted-index-after-interpolation-still-contains-placeholder-metadat-a-target-index/338905 "2023-07-21T11:00:59Z")
Hey guys, trying to aggregate audit events from a linux with logstash-8.8.1-1.x86\_64 and got the error: Badly formatted index, after interpolation still contains placeholder: \[%{\[@metadat a\]\[target\_index\]}\] The stdo… --- ## [Calculate the difference between two fields with time](https://discuss.elastic.co/t/calculate-the-difference-between-two-fields-with-time/338859)
**Author:** [@akeelow](https://discuss.elastic.co/u/akeelow)\ **Replies:** 2\ **Last updated:** [July 21, 2023, 5:06am UTC](https://discuss.elastic.co/t/calculate-the-difference-between-two-fields-with-time/338859 "2023-07-21T05:06:26Z")
Hello! One event has two fields ConnectTime and DisconnectTime. We need to calculate how long the event lasted. To do this, we need to subtract ConnectTime from DisconnectTime. It seems to be very close to the solution,… --- ## [Help with aggregation code](https://discuss.elastic.co/t/help-with-aggregation-code/338794)
**Author:** [@vymk](https://discuss.elastic.co/u/vymk)\ **Replies:** 3\ **Last updated:** [July 20, 2023, 4:22pm UTC](https://discuss.elastic.co/t/help-with-aggregation-code/338794 "2023-07-20T16:22:21Z")
We process mails through multiple modules resulting in logs with the same mail\_id, kinda like this: mail\_id\_X module1: key1 key2 mail\_id\_X module2: key3 key4 mail\_id\_X module3: key5 key6 key7 What I would like to do … --- ## [Combining 3 logs into one](https://discuss.elastic.co/t/combining-3-logs-into-one/338832)
**Author:** [@Cruz](https://discuss.elastic.co/u/Cruz)\ **Replies:** 1\ **Last updated:** [July 20, 2023, 2:18am UTC](https://discuss.elastic.co/t/combining-3-logs-into-one/338832 "2023-07-20T02:18:15Z")
I have this 3 events logs that are almost the same. I want to combine the 3 events logs into one log but somehow it won't work. this is my filter aggregate { task\_id =\> "%{\_id}" code =\> " … --- ## [Output set by field](https://discuss.elastic.co/t/output-set-by-field/338748)
**Author:** [@tbs575](https://discuss.elastic.co/u/tbs575)\ **Replies:** 4\ **Last updated:** [July 20, 2023, 1:29am UTC](https://discuss.elastic.co/t/output-set-by-field/338748 "2023-07-20T01:29:02Z")
Hi Guys, set logstash output influxdb, as title can out by field. output part like output { influxdb\_v2 { host =\> "10.200.101.18" port =\> "8086" org =\> "icep" token =\> "SIRq-QX3d7ddOI33Z9XfmZETHHGAFj… --- ## ["The connection is broken and recovery is not possible" after MSSQL reboot](https://discuss.elastic.co/t/the-connection-is-broken-and-recovery-is-not-possible-after-mssql-reboot/338793)
**Author:** [@mr18](https://discuss.elastic.co/u/mr18)\ **Replies:** 1\ **Last updated:** [July 19, 2023, 4:02pm UTC](https://discuss.elastic.co/t/the-connection-is-broken-and-recovery-is-not-possible-after-mssql-reboot/338793 "2023-07-19T16:02:50Z")
Hi all ! I use a jdbc input plugin to get MS SQL data. Everything works fine, but when my SQL server reboots for maintenance, the queries don't work anymore, and I have the following error: \[ERROR\]\[logstash.inputs.jdb… --- ## [Logstash stops treatment when database goes down](https://discuss.elastic.co/t/logstash-stops-treatment-when-database-goes-down/338806)
**Author:** [@thibaut\_a](https://discuss.elastic.co/u/thibaut_a)\ **Replies:** 0\ **Last updated:** [July 19, 2023, 3:30pm UTC](https://discuss.elastic.co/t/logstash-stops-treatment-when-database-goes-down/338806 "2023-07-19T15:30:35Z")
Hi, I have multiple JDBC inputs waiting for PostgreSQL database lines each minute. The database has a problem and is down. Due to this mistake, I don't receive any log in Kibana. I tried to find some fields in the docum… --- ## [Example Inputs logtash configuration](https://discuss.elastic.co/t/example-inputs-logtash-configuration/338792)
**Author:** [@willsy](https://discuss.elastic.co/u/willsy)\ **Replies:** 0\ **Last updated:** [July 19, 2023, 1:39pm UTC](https://discuss.elastic.co/t/example-inputs-logtash-configuration/338792 "2023-07-19T13:39:45Z")
Hello, Can i get help with an input configuration for logstash please? I currently am using this as a curl to get my information, but i need this in yaml if possible \ \curl -X GET 'https://192.168.3.21:9200/\_cat… --- ## [Not able to index array in logstash](https://discuss.elastic.co/t/not-able-to-index-array-in-logstash/338774)
**Author:** [@Mohit\_Gupta2](https://discuss.elastic.co/u/Mohit_Gupta2)\ **Replies:** 0\ **Last updated:** [July 19, 2023, 11:11am UTC](https://discuss.elastic.co/t/not-able-to-index-array-in-logstash/338774 "2023-07-19T11:11:40Z")
As it says, Logstash is not able to index array of strings or any kind of string for that matter. eg. - country :\["some\_name"\] is not mapped while country: '\["some\_name"\]' is indexed although it is showing the unmapped … --- ## [How to create container name specific indexes](https://discuss.elastic.co/t/how-to-create-container-name-specific-indexes/338766)
**Author:** [@Affan\_Mir](https://discuss.elastic.co/u/Affan_Mir)\ **Replies:** 0\ **Last updated:** [July 19, 2023, 10:42am UTC](https://discuss.elastic.co/t/how-to-create-container-name-specific-indexes/338766 "2023-07-19T10:42:51Z")
I've deployed filebeat as a daemonset now for every container I need to access to their respective container names and read them from logstash so I can have indexes like 'container-name-YYYY-MM-DD' in my Elasticsearch. T… --- ## [The connection between Logstash and Elasticsearch is not working](https://discuss.elastic.co/t/the-connection-between-logstash-and-elasticsearch-is-not-working/338750)
**Author:** [@chldnjs8899](https://discuss.elastic.co/u/chldnjs8899)\ **Replies:** 1\ **Last updated:** [July 19, 2023, 8:14am UTC](https://discuss.elastic.co/t/the-connection-between-logstash-and-elasticsearch-is-not-working/338750 "2023-07-19T08:14:42Z")
The following content has been translated using ChatGPT. Thank you for your understanding. Hello, I'm currently learning Elasticsearch. I'm using Elasticsearch version 8.8.2. I have written the following pipeline in ord… --- ## [Optimize logstash tcp input plugin](https://discuss.elastic.co/t/optimize-logstash-tcp-input-plugin/337847)
**Author:** [@true64gurus](https://discuss.elastic.co/u/true64gurus)\ **Replies:** 7\ **Last updated:** [July 18, 2023, 1:33pm UTC](https://discuss.elastic.co/t/optimize-logstash-tcp-input-plugin/337847 "2023-07-18T13:33:26Z")
Hello, I have 10 Kubernetes clusters forward their logs to logstash VM (k8s fluentd ---\> logstash port 7000) . Logstash gets to a point where logs are being missed and source pods doing retries to get logs through . (… --- ## [Logstash not showing field with null values](https://discuss.elastic.co/t/logstash-not-showing-field-with-null-values/338648)
**Author:** [@Mohit\_Gupta2](https://discuss.elastic.co/u/Mohit_Gupta2)\ **Replies:** 5\ **Last updated:** [July 18, 2023, 12:44pm UTC](https://discuss.elastic.co/t/logstash-not-showing-field-with-null-values/338648 "2023-07-18T12:44:26Z")
I am indexing elasticsearch via logstash but it is showing only document's fields with not null values. Earlier I used to do this through transporter and it returns the null values as well. Also the mapping in both case… --- ## [In data table visualization i have 5 columns but in one column, only one data is coming](https://discuss.elastic.co/t/in-data-table-visualization-i-have-5-columns-but-in-one-column-only-one-data-is-coming/338421)
**Author:** [@Malikmamta](https://discuss.elastic.co/u/Malikmamta)\ **Replies:** 4\ **Last updated:** [July 18, 2023, 10:11am UTC](https://discuss.elastic.co/t/in-data-table-visualization-i-have-5-columns-but-in-one-column-only-one-data-is-coming/338421 "2023-07-18T10:11:35Z")
in data table visualization, I have 5 columns but in one column, only one row is missing instead of 10 rows. in that column multiline are there. In discover, i can see complete data but for that one column, only single … --- ## [Running multiple pipelines and a single pipeline triggered with shell script simultaneously](https://discuss.elastic.co/t/running-multiple-pipelines-and-a-single-pipeline-triggered-with-shell-script-simultaneously/338441)
**Author:** [@PodarcisMuralis](https://discuss.elastic.co/u/PodarcisMuralis)\ **Replies:** 1\ **Last updated:** [July 18, 2023, 7:48am UTC](https://discuss.elastic.co/t/running-multiple-pipelines-and-a-single-pipeline-triggered-with-shell-script-simultaneously/338441 "2023-07-18T07:48:22Z")
Hi. I have 5 pipelines already running on linux server. The conf files are listed in a pipeline.yml and it has already been started with sudo systemctl start logstash, up and running. I created another single pipeline… --- ## [java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit](https://discuss.elastic.co/t/java-lang-illegalstateexception-logstash-stopped-processing-because-of-an-error-systemexit-exit/338620)
**Author:** [@Nghia\_D\_ng](https://discuss.elastic.co/u/Nghia_D_ng)\ **Replies:** 1\ **Last updated:** [July 18, 2023, 3:18am UTC](https://discuss.elastic.co/t/java-lang-illegalstateexception-logstash-stopped-processing-because-of-an-error-systemexit-exit/338620 "2023-07-18T03:18:52Z")
Please help me! --- ## [Help ingesting Data](https://discuss.elastic.co/t/help-ingesting-data/338580)
**Author:** [@Tom\_Dixon](https://discuss.elastic.co/u/Tom_Dixon)\ **Replies:** 2\ **Last updated:** [July 17, 2023, 4:14pm UTC](https://discuss.elastic.co/t/help-ingesting-data/338580 "2023-07-17T16:14:42Z")
Hi all, I'm new to Elastic and Logstash. I have a source of event data which I'm having problems ingesting. I think it is because the data itself, but being new to Logstash it could also be me, so I'm not sure where the … --- ## [Kafka input plugin cannot parse key or value due to message keys](https://discuss.elastic.co/t/kafka-input-plugin-cannot-parse-key-or-value-due-to-message-keys/338560)
**Author:** [@kohlbecker](https://discuss.elastic.co/u/kohlbecker)\ **Replies:** 2\ **Last updated:** [July 17, 2023, 3:38pm UTC](https://discuss.elastic.co/t/kafka-input-plugin-cannot-parse-key-or-value-due-to-message-keys/338560 "2023-07-17T15:38:03Z")
Key and value of the topic messages consumed by the Kafka input plugin are prefixed with the message ids, which causes the json parser to fail: Here an example from the logstash log with decorate\_events =\> "extended" pr… --- ## [Implement proxy-protocol support for beats inputs](https://discuss.elastic.co/t/implement-proxy-protocol-support-for-beats-inputs/338561)
**Author:** [@bilel\_meddeb](https://discuss.elastic.co/u/bilel_meddeb)\ **Replies:** 0\ **Last updated:** [July 17, 2023, 1:14pm UTC](https://discuss.elastic.co/t/implement-proxy-protocol-support-for-beats-inputs/338561 "2023-07-17T13:14:41Z")
Hello :wave:t4: Would it be possible to support proxy-protocol for beats inputs ? I send logs from winlogbeat to logstash and i have Haproxy between them. without proxy and with this configuration of logstash, i got … --- ## [Ruby into file](https://discuss.elastic.co/t/ruby-into-file/338102)
**Author:** [@hofrichterovak](https://discuss.elastic.co/u/hofrichterovak)\ **Replies:** 4\ **Last updated:** [July 17, 2023, 9:27am UTC](https://discuss.elastic.co/t/ruby-into-file/338102 "2023-07-17T09:27:23Z")
Hello, I read the documentation about the ruby script and I did not correctly understand the conversion of the ruby script into a file. If I have a converted ruby script into a file, do I have to rewrite the script int… --- ## [Persistent data support for logstash in ECK 2.8?](https://discuss.elastic.co/t/persistent-data-support-for-logstash-in-eck-2-8/338514)
**Author:** [@Claudio\_Tassini](https://discuss.elastic.co/u/Claudio_Tassini)\ **Replies:** 0\ **Last updated:** [July 17, 2023, 8:30am UTC](https://discuss.elastic.co/t/persistent-data-support-for-logstash-in-eck-2-8/338514 "2023-07-17T08:30:44Z")
Hi all! I'm trying to deploy an ECK cluster composed of elasticsearch, kibana, beats and a logstash instance. The only problem I'm facing is that the logstash CRD does not seem to support the definition of a volumeclaim… --- ## [Filebeat module ingest pipeline not working in logstash](https://discuss.elastic.co/t/filebeat-module-ingest-pipeline-not-working-in-logstash/338500)
**Author:** [@nbindal](https://discuss.elastic.co/u/nbindal)\ **Replies:** 4\ **Last updated:** [July 17, 2023, 5:55am UTC](https://discuss.elastic.co/t/filebeat-module-ingest-pipeline-not-working-in-logstash/338500 "2023-07-17T05:55:43Z")
Hi Team, I am using apache module and fileset in Beats+ELK stack where Filebeat is sending logs to logstash, logstash is using Ingest pipeline(we get module ingest pipeline - filebeat-8.7.1-apache-access-pipeline) , but… --- ## [How to sort my data in elasticsearch](https://discuss.elastic.co/t/how-to-sort-my-data-in-elasticsearch/338072)
**Author:** [@lz840408](https://discuss.elastic.co/u/lz840408)\ **Replies:** 8\ **Last updated:** [July 17, 2023, 5:13am UTC](https://discuss.elastic.co/t/how-to-sort-my-data-in-elasticsearch/338072 "2023-07-17T05:13:38Z")
how to sort by asc in logstash? i want new add field,it's self increment column,and insert dest index,how make it? --- ## [Getting logstasg error in rhel 8 and not running in logstash in rhel 8](https://discuss.elastic.co/t/getting-logstasg-error-in-rhel-8-and-not-running-in-logstash-in-rhel-8/338480)
**Author:** [@talbehat](https://discuss.elastic.co/u/talbehat)\ **Replies:** 3\ **Last updated:** [July 17, 2023, 4:28am UTC](https://discuss.elastic.co/t/getting-logstasg-error-in-rhel-8-and-not-running-in-logstash-in-rhel-8/338480 "2023-07-17T04:28:47Z")
Logstash is not running in rhel 8 and getting error while start logstash. logstash version :- 7.4.3 \[ERROR\] 2023-07-15 18:52:56.228 \[main\] Logstash - java.lang.IllegalStateException: Logstash stopped processing because… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=65) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=67)