# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=103 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 104 --- ## [Mixed line types JSON and Plain Text in the same file with multiline](https://discuss.elastic.co/t/mixed-line-types-json-and-plain-text-in-the-same-file-with-multiline/319648)
**Author:** [@sirReeall](https://discuss.elastic.co/u/sirReeall)\ **Replies:** 0\ **Last updated:** [November 23, 2022, 10:53am UTC](https://discuss.elastic.co/t/mixed-line-types-json-and-plain-text-in-the-same-file-with-multiline/319648 "2022-11-23T10:53:45Z")
Hello, I've got log files that can contain plain text lines from a Java application and JSON lines. To complicate matters the plain text lines can be multiline , for example a Java stack trace, so I'm using the multili… --- ## [Logstash Not sending data to Elasticsearch](https://discuss.elastic.co/t/logstash-not-sending-data-to-elasticsearch/319643)
**Author:** [@abkonred](https://discuss.elastic.co/u/abkonred)\ **Replies:** 0\ **Last updated:** [November 23, 2022, 10:37am UTC](https://discuss.elastic.co/t/logstash-not-sending-data-to-elasticsearch/319643 "2022-11-23T10:37:41Z")
Hi Team, I have created a pipeline and sending data from filebeat to logstash and to Elasticsearch. and I have created one custom field from filebeat to filter the data. Not sue why it is not sending data to "apa" index… --- ## [\[logstash.outputs.elasticsearch\] Attempted to resurrect connection to dead ES instance - AWS Govcloud](https://discuss.elastic.co/t/logstash-outputs-elasticsearch-attempted-to-resurrect-connection-to-dead-es-instance-aws-govcloud/319572)
**Author:** [@Vtech](https://discuss.elastic.co/u/Vtech)\ **Replies:** 1\ **Last updated:** [November 23, 2022, 7:01am UTC](https://discuss.elastic.co/t/logstash-outputs-elasticsearch-attempted-to-resurrect-connection-to-dead-es-instance-aws-govcloud/319572 "2022-11-23T07:01:27Z")
Getting the following error when attempting to send on-prem logstash to elasticsearch in the cloud The version running in the cloud is 8.2.2 Log file \[2022-11-22T08:24:43,581\]\[INFO \]\[logstash.runner \] Log4j … --- ## [Logstash loki standalone setup with input file form /var/log/ dir](https://discuss.elastic.co/t/logstash-loki-standalone-setup-with-input-file-form-var-log-dir/319614)
**Author:** [@jijesh\_vu](https://discuss.elastic.co/u/jijesh_vu)\ **Replies:** 0\ **Last updated:** [November 23, 2022, 6:11am UTC](https://discuss.elastic.co/t/logstash-loki-standalone-setup-with-input-file-form-var-log-dir/319614 "2022-11-23T06:11:36Z")
Im using logstash client with input file plugin to read the kubernetes logs from /var/log/\*\* directory. I have also a log generator is running on a node, and grafana is unable to detect the log generator logs .Grafana is… --- ## [Logstash.inputs.jdbc connect to Azure SQL Server throw exception](https://discuss.elastic.co/t/logstash-inputs-jdbc-connect-to-azure-sql-server-throw-exception/319603)
**Author:** [@liuxiao617111](https://discuss.elastic.co/u/liuxiao617111)\ **Replies:** 0\ **Last updated:** [November 23, 2022, 1:59am UTC](https://discuss.elastic.co/t/logstash-inputs-jdbc-connect-to-azure-sql-server-throw-exception/319603 "2022-11-23T01:59:15Z")
when I using "authentication=ActiveDirectoryPassword" to connect Azure SQL server. It throw an exception: "Failed to load MSAL4J Java library for performing ActiveDirectoryPassword authentication.". Logstash Version : … --- ## [How to Reference field value(s) in Logstash](https://discuss.elastic.co/t/how-to-reference-field-value-s-in-logstash/319593)
**Author:** [@RCooper56](https://discuss.elastic.co/u/RCooper56)\ **Replies:** 2\ **Last updated:** [November 22, 2022, 10:41pm UTC](https://discuss.elastic.co/t/how-to-reference-field-value-s-in-logstash/319593 "2022-11-22T22:41:06Z")
Hey Elastic Team, First time post here so please let me know if you need more detail, I'll try to provide as much context as possible. I am trying to create a visualisation based on snmp data coming from our firewall. … --- ## [Email - Something happen while delivering an email {:exception=\>#\}](https://discuss.elastic.co/t/email-something-happen-while-delivering-an-email-exception-net-execution-expired/319590)
**Author:** [@David\_Graciano](https://discuss.elastic.co/u/David_Graciano)\ **Replies:** 0\ **Last updated:** [November 22, 2022, 9:49pm UTC](https://discuss.elastic.co/t/email-something-happen-while-delivering-an-email-exception-net-execution-expired/319590 "2022-11-22T21:49:03Z")
Attempting to send an email from the logstash output produces the following error in the output. email - Something happen while delivering an email {:exception=\>#\} this is my curren… --- ## [Filebeat.yml and logstash.conf are not getting loaded automatically](https://discuss.elastic.co/t/filebeat-yml-and-logstash-conf-are-not-getting-loaded-automatically/319573)
**Author:** [@Manjiri](https://discuss.elastic.co/u/Manjiri)\ **Replies:** 1\ **Last updated:** [November 22, 2022, 6:53pm UTC](https://discuss.elastic.co/t/filebeat-yml-and-logstash-conf-are-not-getting-loaded-automatically/319573 "2022-11-22T18:53:18Z")
When we starting the filebeat and logstash services through services.msc , filebeat.yml and logstash.conf is not getting loaded automatically. Everytime we need to load 2 files manually through command prompt using below… --- ## [Output isolator pattern, not working as expected when one pipeline is down](https://discuss.elastic.co/t/output-isolator-pattern-not-working-as-expected-when-one-pipeline-is-down/319329)
**Author:** [@rravitech](https://discuss.elastic.co/u/rravitech)\ **Replies:** 9\ **Last updated:** [November 22, 2022, 5:28pm UTC](https://discuss.elastic.co/t/output-isolator-pattern-not-working-as-expected-when-one-pipeline-is-down/319329 "2022-11-22T17:28:16Z")
All the outputs stop when one goes down here, i am trying to implement the output isolator pattern. In the below configuration when hosts =\> \["ext-se:9200"\] goes down the other host "es-host1" { hosts =\> \["eric-data-se… --- ## [Logstash connection with Oracle wallet](https://discuss.elastic.co/t/logstash-connection-with-oracle-wallet/319300)
**Author:** [@Num](https://discuss.elastic.co/u/Num)\ **Replies:** 3\ **Last updated:** [November 22, 2022, 2:31pm UTC](https://discuss.elastic.co/t/logstash-connection-with-oracle-wallet/319300 "2022-11-22T14:31:17Z")
Hello, I'm having issues connecting Logstash using Oracle wallet. I am getting an invalid login/password even though I am trying to connect through the wallet, without using said login/password. \[ERROR\]\[logstash.input… --- ## [Not breaking down Embeded json field and instead keep it in a single field](https://discuss.elastic.co/t/not-breaking-down-embeded-json-field-and-instead-keep-it-in-a-single-field/319505)
**Author:** [@stillfreem](https://discuss.elastic.co/u/stillfreem)\ **Replies:** 2\ **Last updated:** [November 22, 2022, 1:42pm UTC](https://discuss.elastic.co/t/not-breaking-down-embeded-json-field-and-instead-keep-it-in-a-single-field/319505 "2022-11-22T13:42:34Z")
Hi All, I use json filter plugin to parse log entries in json format such as the below {"actor\_ip":"xxx","from":"Api::ActionsRunnerRegistration#POST","actor":"xxx","actor\_id":2480,"org":"xxx","org\_id":13,"action":"org.… --- ## [The content length (938946807) is bigger than the maximum allowed string (536870888) sending logstash-plain.log to elasticsearch](https://discuss.elastic.co/t/the-content-length-938946807-is-bigger-than-the-maximum-allowed-string-536870888-sending-logstash-plain-log-to-elasticsearch/319533)
**Author:** [@Neelam\_Zanvar](https://discuss.elastic.co/u/Neelam_Zanvar)\ **Replies:** 8\ **Last updated:** [November 22, 2022, 1:00pm UTC](https://discuss.elastic.co/t/the-content-length-938946807-is-bigger-than-the-maximum-allowed-string-536870888-sending-logstash-plain-log-to-elasticsearch/319533 "2022-11-22T13:00:00Z")
Hi I have setup filebeat to send logstash logs to logstah then elasticsearch. Getting this error when trying to access it on discover page Search Error The content length (938946807) is bigger than the maximum allowed s… --- ## [Encountered a retryable error. Will Retry with exponential backoff](https://discuss.elastic.co/t/encountered-a-retryable-error-will-retry-with-exponential-backoff/319547)
**Author:** [@pubudu.a](https://discuss.elastic.co/u/pubudu.a)\ **Replies:** 1\ **Last updated:** [November 22, 2022, 11:53am UTC](https://discuss.elastic.co/t/encountered-a-retryable-error-will-retry-with-exponential-backoff/319547 "2022-11-22T11:53:20Z")
Hi All, I encountered the following error in the logstash server after updating Opensearh 1.3 to OpenSearch 2.3 in the AWS Kibana dashboard. The current configuration worked fine with 1.3 but after updating it to 2.3 it… --- ## [Is it possible to add condition in elasticsearch logstash jdbc input plugin](https://discuss.elastic.co/t/is-it-possible-to-add-condition-in-elasticsearch-logstash-jdbc-input-plugin/319195)
**Author:** [@niveditakathal](https://discuss.elastic.co/u/niveditakathal)\ **Replies:** 7\ **Last updated:** [November 22, 2022, 9:07am UTC](https://discuss.elastic.co/t/is-it-possible-to-add-condition-in-elasticsearch-logstash-jdbc-input-plugin/319195 "2022-11-22T09:07:50Z")
Hi Experts, Is it possible to add condition in elasticsearch logstash jdbc input plugin for the use case mentioned below - Usecase - Select data from table1 If datetime (refer elapseTime in code mentioned below) fetc… --- ## [Should I need raw data to create a Grok Filter?](https://discuss.elastic.co/t/should-i-need-raw-data-to-create-a-grok-filter/319386)
**Author:** [@aaron.chang](https://discuss.elastic.co/u/aaron.chang)\ **Replies:** 1\ **Last updated:** [November 22, 2022, 8:33am UTC](https://discuss.elastic.co/t/should-i-need-raw-data-to-create-a-grok-filter/319386 "2022-11-22T08:33:08Z")
Hi , I am a new user to use ELK. I want to analyze Aruba Controller & MM's syslog. But ELK don't have Aruba's template to show clearly. So I need to grok it by myself. From some article ,I know that ELK will not save… --- ## [Logstash server label shipped with logs](https://discuss.elastic.co/t/logstash-server-label-shipped-with-logs/319478)
**Author:** [@lubimamp](https://discuss.elastic.co/u/lubimamp)\ **Replies:** 2\ **Last updated:** [November 22, 2022, 2:46am UTC](https://discuss.elastic.co/t/logstash-server-label-shipped-with-logs/319478 "2022-11-22T02:46:37Z")
Hi Everyone, i have 2+ logstash servers in cluster and i would like to have some tag/flag/whatever which to tell me from which logstash server exactly the message was shipped. I have cases in which one of the logstash s… --- ## [Sometimes events write issues from Logstash to Elasticsearch](https://discuss.elastic.co/t/sometimes-events-write-issues-from-logstash-to-elasticsearch/319417)
**Author:** [@Jonathan\_G](https://discuss.elastic.co/u/Jonathan_G)\ **Replies:** 0\ **Last updated:** [November 21, 2022, 10:03am UTC](https://discuss.elastic.co/t/sometimes-events-write-issues-from-logstash-to-elasticsearch/319417 "2022-11-21T10:03:30Z")
Hello! Issue: Usually everything works correctly. Logstash succsessfuly writes bulk requests with events to Elasticsearch. But sometimes we have issue with connectivity between Logstash && Elasticsearch. Issue relates … --- ## [Logstash s3 input .gz/folder/file patter possible?](https://discuss.elastic.co/t/logstash-s3-input-gz-folder-file-patter-possible/319493)
**Author:** [@acavalier](https://discuss.elastic.co/u/acavalier)\ **Replies:** 3\ **Last updated:** [November 21, 2022, 9:14pm UTC](https://discuss.elastic.co/t/logstash-s3-input-gz-folder-file-patter-possible/319493 "2022-11-21T21:14:29Z")
I want the s3 input to read the .gz/folder/file is this possible or will i need to use somthing else i.e. lambda to extract the files first. Currently the s3 input is reading the folder as the file. Any help is appreciat… --- ## [Logstash input google pubsub - clarification](https://discuss.elastic.co/t/logstash-input-google-pubsub-clarification/319489)
**Author:** [@metalshanked](https://discuss.elastic.co/u/metalshanked)\ **Replies:** 0\ **Last updated:** [November 21, 2022, 6:48pm UTC](https://discuss.elastic.co/t/logstash-input-google-pubsub-clarification/319489 "2022-11-21T18:48:28Z")
Hi, I wanted clarification on the below statement in the docs for the logstash Google pubsub input plugin All messages received from Pub/Sub will be converted to a logstash event and added to the processing pipeline qu… --- ## [Logstash dissect change datatype to timestamp](https://discuss.elastic.co/t/logstash-dissect-change-datatype-to-timestamp/319470)
**Author:** [@edim2525](https://discuss.elastic.co/u/edim2525)\ **Replies:** 1\ **Last updated:** [November 21, 2022, 3:04pm UTC](https://discuss.elastic.co/t/logstash-dissect-change-datatype-to-timestamp/319470 "2022-11-21T15:04:40Z")
Hi , Is there a way to change the mapping field to timestamp ? This is my input "Time :2022-11-21 12:01:30.85" I'm using "dissect \> mapping" to map this field, but the output data type is "text" and I want it as a tim… --- ## [Logstash Filter output in different files according to incomming IP address](https://discuss.elastic.co/t/logstash-filter-output-in-different-files-according-to-incomming-ip-address/319305)
**Author:** [@Flo\_h](https://discuss.elastic.co/u/Flo_h)\ **Replies:** 4\ **Last updated:** [November 21, 2022, 1:10pm UTC](https://discuss.elastic.co/t/logstash-filter-output-in-different-files-according-to-incomming-ip-address/319305 "2022-11-21T13:10:17Z")
Hello, I am using the last version of Logstash 8.5.1. And I am trying to output log in files according to the source IP in the logstash input. I DO know the incomming IPs. I am trying to filter them out in the filter pa… --- ## [Setup Logstash with Loki in Kubernetes](https://discuss.elastic.co/t/setup-logstash-with-loki-in-kubernetes/319431)
**Author:** [@jijesh\_vu](https://discuss.elastic.co/u/jijesh_vu)\ **Replies:** 0\ **Last updated:** [November 21, 2022, 11:13am UTC](https://discuss.elastic.co/t/setup-logstash-with-loki-in-kubernetes/319431 "2022-11-21T11:13:44Z")
Hi any documentations for configuring Logstash with Loki in kubernetes without filebeat --- ## [Replace special character in stirng to other special character](https://discuss.elastic.co/t/replace-special-character-in-stirng-to-other-special-character/319382)
**Author:** [@lusynda](https://discuss.elastic.co/u/lusynda)\ **Replies:** 2\ **Last updated:** [November 21, 2022, 9:54am UTC](https://discuss.elastic.co/t/replace-special-character-in-stirng-to-other-special-character/319382 "2022-11-21T09:54:08Z")
Hi all I have a case that i've been trying to do for a very long time. I have a string that contain #0D#0A character that i want to replace them with the string \\r\\n I've been trying to use gsub but it doesn't seem to… --- ## [Logstash not stopping if elasticsearch down - although persistent pipeline queue is enabled](https://discuss.elastic.co/t/logstash-not-stopping-if-elasticsearch-down-although-persistent-pipeline-queue-is-enabled/318403)
**Author:** [@asp](https://discuss.elastic.co/u/asp)\ **Replies:** 1\ **Last updated:** [November 21, 2022, 9:32am UTC](https://discuss.elastic.co/t/logstash-not-stopping-if-elasticsearch-down-although-persistent-pipeline-queue-is-enabled/318403 "2022-11-21T09:32:12Z")
Hi, I am using multi-pipelining in logstash. I encountered issues before when I want to stop logstash and elasticsearch is unavailable. Thought the issue was the following: logstash has read events from redis and pro… --- ## [Master-Slave architecture Log Collection and Mapping](https://discuss.elastic.co/t/master-slave-architecture-log-collection-and-mapping/319410)
**Author:** [@Poongkuyil\_Muse](https://discuss.elastic.co/u/Poongkuyil_Muse)\ **Replies:** 0\ **Last updated:** [November 21, 2022, 9:28am UTC](https://discuss.elastic.co/t/master-slave-architecture-log-collection-and-mapping/319410 "2022-11-21T09:28:55Z")
I am able to retrieve apache web logs from one system and visualize them in kibana. However, we have more than one system that work as master-slave. My Scenario: If there are 3 systems A,B,C A - Master (IP1: 192.165.… --- ## [Failed to ingest Data to an Index](https://discuss.elastic.co/t/failed-to-ingest-data-to-an-index/319405)
**Author:** [@Jecks\_Speed](https://discuss.elastic.co/u/Jecks_Speed)\ **Replies:** 0\ **Last updated:** [November 21, 2022, 8:58am UTC](https://discuss.elastic.co/t/failed-to-ingest-data-to-an-index/319405 "2022-11-21T08:58:53Z")
Hi everyone, Good day! I am newbie to ELK. currently i am working a return data of an API. upon checking my target body it has the return data from API call, However after processing these data using mutate and some if… --- ## [How to get size of each event/record in logstash /elasticsearch](https://discuss.elastic.co/t/how-to-get-size-of-each-event-record-in-logstash-elasticsearch/319235)
**Author:** [@priti](https://discuss.elastic.co/u/priti)\ **Replies:** 1\ **Last updated:** [November 20, 2022, 5:57am UTC](https://discuss.elastic.co/t/how-to-get-size-of-each-event-record-in-logstash-elasticsearch/319235 "2022-11-20T05:57:28Z")
How to get size of each event/record in logstash /elasticsearch. Based on that size i am trying to calculate size of each application in that index. --- ## [Logstash split the child json to the parent layer](https://discuss.elastic.co/t/logstash-split-the-child-json-to-the-parent-layer/319339)
**Author:** [@lnsane](https://discuss.elastic.co/u/lnsane)\ **Replies:** 1\ **Last updated:** [November 19, 2022, 3:32pm UTC](https://discuss.elastic.co/t/logstash-split-the-child-json-to-the-parent-layer/319339 "2022-11-19T15:32:14Z")
this is json { "@timestamp" =\> 2022-11-19T06:19:08.249Z, "database" =\> "12", "data": { "a": 1, "b": 2 } } how resolve to { "@timestamp" =\> 2022-11-19T06:19:08.249Z, "database" =\> "12… --- ## [Inconsistent results with http\_poller & XML parsing](https://discuss.elastic.co/t/inconsistent-results-with-http-poller-xml-parsing/319310)
**Author:** [@jdswifty](https://discuss.elastic.co/u/jdswifty)\ **Replies:** 1\ **Last updated:** [November 18, 2022, 3:53pm UTC](https://discuss.elastic.co/t/inconsistent-results-with-http-poller-xml-parsing/319310 "2022-11-18T15:53:09Z")
Looking for some advice here on an issue i'm facing with http\_poller & xml parsing. I'm polling 2 urls every 60 secs that return XML & getting some very inconsistent results. Sometimes i'm getting perfect output (3 dis… --- ## [Logstash ConfigurationError](https://discuss.elastic.co/t/logstash-configurationerror/319218)
**Author:** [@diegz](https://discuss.elastic.co/u/diegz)\ **Replies:** 4\ **Last updated:** [November 18, 2022, 3:33pm UTC](https://discuss.elastic.co/t/logstash-configurationerror/319218 "2022-11-18T15:33:05Z")
Hello, When I run logstash with this configuration file I get these errors. Do you have a solution? Best regards, /etc/logstash/conf.d/logstash-syslog.conf input { tcp { port =\> 5000 type =\> syslog } u… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=102) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=104)