# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=99 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 100 --- ## [Logstash filter code that sends a query to related indices by wild card \[indexname-\*\]](https://discuss.elastic.co/t/logstash-filter-code-that-sends-a-query-to-related-indices-by-wild-card-indexname/321718)
**Author:** [@PRASHANT\_MEHTA](https://discuss.elastic.co/u/PRASHANT_MEHTA)\ **Replies:** 0\ **Last updated:** [December 21, 2022, 5:54am UTC](https://discuss.elastic.co/t/logstash-filter-code-that-sends-a-query-to-related-indices-by-wild-card-indexname/321718 "2022-12-21T05:54:50Z")
Hello All, Can someone please help me out how can I write custom filter in logstash to achieve the following: Making a custom logstash filter that sends a query to related indices by wild card (indexname-\*) for records… --- ## [Azure App Service Logs to Elastic Cloud (without Filebeat)](https://discuss.elastic.co/t/azure-app-service-logs-to-elastic-cloud-without-filebeat/321661)
**Author:** [@CrystalDesignDR](https://discuss.elastic.co/u/CrystalDesignDR)\ **Replies:** 0\ **Last updated:** [December 20, 2022, 2:30pm UTC](https://discuss.elastic.co/t/azure-app-service-logs-to-elastic-cloud-without-filebeat/321661 "2022-12-20T14:30:41Z")
Hi, we send various types of data into our Elastic Cloud deployments, whether it be custom indexes or APM (RUM, node.js, ASP.NET, .NET Core ecc.) data, which is all very straight forward. But when it comes to applicati… --- ## [Regex Pattern to fetch field from message](https://discuss.elastic.co/t/regex-pattern-to-fetch-field-from-message/320563)
**Author:** [@Ganesh2303](https://discuss.elastic.co/u/Ganesh2303)\ **Replies:** 1\ **Last updated:** [December 20, 2022, 9:14pm UTC](https://discuss.elastic.co/t/regex-pattern-to-fetch-field-from-message/320563 "2022-12-20T21:14:55Z")
HI Team, I'm trying to fetch some key field from my message and i need pattern to match exact keyword. i need UserID and User agent matching pattern \[INFO \] 2022-12-05 17:20:45:409 Logger - 365 Service Request: \U… --- ## [Logstash split one message into multiple](https://discuss.elastic.co/t/logstash-split-one-message-into-multiple/321496)
**Author:** [@mail2shanth](https://discuss.elastic.co/u/mail2shanth)\ **Replies:** 10\ **Last updated:** [December 20, 2022, 7:10pm UTC](https://discuss.elastic.co/t/logstash-split-one-message-into-multiple/321496 "2022-12-20T19:10:54Z")
Hi, I've the following message and I managed to create the output from only message, {"@timestamp":"2022-12-01T13:30:00.004Z","message":"\<190\>Dec 1 14:29:59 10.62.161.199 AA-AMG3U: 0950198238 NN \[MDA 8/4\]: LN44 SA 20… --- ## [LogStash Conf | Drop Empty Lines](https://discuss.elastic.co/t/logstash-conf-drop-empty-lines/321674)
**Author:** [@srii](https://discuss.elastic.co/u/srii)\ **Replies:** 5\ **Last updated:** [December 20, 2022, 6:37pm UTC](https://discuss.elastic.co/t/logstash-conf-drop-empty-lines/321674 "2022-12-20T18:37:29Z")
The contents of LogStash's conf file looks like this: input { beats { port =\> 5044 } file { path =\> "/usr/share/logstash/iway\_logs/\*" start\_position =\> "beginning" sincedb\_path =\> "/dev/null" … --- ## [Logstash conf file for Email Alert](https://discuss.elastic.co/t/logstash-conf-file-for-email-alert/319914)
**Author:** [@Ravi\_Vishwakarma](https://discuss.elastic.co/u/Ravi_Vishwakarma)\ **Replies:** 5\ **Last updated:** [December 20, 2022, 7:42am UTC](https://discuss.elastic.co/t/logstash-conf-file-for-email-alert/319914 "2022-12-20T07:42:19Z")
Hi, Can anyone help me with this I have created two conf file under /etc/logstash/conf.d/ syslog.conf === Working fine input { beats { port =\> "5044" } } filter { grok { match =\> { "messag… --- ## [Logstash Mongo input error](https://discuss.elastic.co/t/logstash-mongo-input-error/321573)
**Author:** [@kmz161](https://discuss.elastic.co/u/kmz161)\ **Replies:** 0\ **Last updated:** [December 19, 2022, 2:59pm UTC](https://discuss.elastic.co/t/logstash-mongo-input-error/321573 "2022-12-19T14:59:59Z")
Hello! I need read info from MongoDB I use JDBC in Logstash input jdbc { jdbc\_driver\_library =\> "/usr/share/logstash/logstash-core/lib/jars/mongojdbc4.8.jar" jdbc\_driver\_class =\> "Java::com.dbschem… --- ## [Cant send sysmon logs via logstash output to apache nifi](https://discuss.elastic.co/t/cant-send-sysmon-logs-via-logstash-output-to-apache-nifi/321487)
**Author:** [@roie](https://discuss.elastic.co/u/roie)\ **Replies:** 6\ **Last updated:** [December 19, 2022, 12:00pm UTC](https://discuss.elastic.co/t/cant-send-sysmon-logs-via-logstash-output-to-apache-nifi/321487 "2022-12-19T12:00:42Z")
hi . im trying to send sysmon logs via logstash and there is no data getin , these are my logs error: {"log.level":"error","@timestamp":"2022-12-18T08:15:16.318Z","log.logger":"publisher\_pipeline\_output","log.origin":{… --- ## [Limited output from logstash](https://discuss.elastic.co/t/limited-output-from-logstash/321538)
**Author:** [@INS](https://discuss.elastic.co/u/INS)\ **Replies:** 4\ **Last updated:** [December 19, 2022, 10:35am UTC](https://discuss.elastic.co/t/limited-output-from-logstash/321538 "2022-12-19T10:35:26Z")
Hi I'm wondering if it's possible to limited output event form logstash? I've observed many of circuit\_breaking\_exception :error=\>{"type"=\>"circuit\_breaking\_exception", "reason"=\>"\[parent\] Data too large, data for \[in… --- ## [Dynamic email sending with logstash](https://discuss.elastic.co/t/dynamic-email-sending-with-logstash/321536)
**Author:** [@pramila\_niroshan](https://discuss.elastic.co/u/pramila_niroshan)\ **Replies:** 0\ **Last updated:** [December 19, 2022, 9:31am UTC](https://discuss.elastic.co/t/dynamic-email-sending-with-logstash/321536 "2022-12-19T09:31:45Z")
Is there a way to send a email from logs? That's meant whenever we found a email address from our log then we extract that email address and send to that address to mail. (We use log4j2 and MDC to write the log file) --- ## [Timing Questions with Logstash pipelines v. API Data](https://discuss.elastic.co/t/timing-questions-with-logstash-pipelines-v-api-data/321235)
**Author:** [@Broken08](https://discuss.elastic.co/u/Broken08)\ **Replies:** 0\ **Last updated:** [December 14, 2022, 6:46pm UTC](https://discuss.elastic.co/t/timing-questions-with-logstash-pipelines-v-api-data/321235 "2022-12-14T18:46:27Z")
I have 9 logstash Pipelines that run to ingest data into ES. I was trying to do some math initially by taking the time the pipeline starts ingest (Looking at the \*.pipeline.log file looking at the time the query started… --- ## [Is it possible to create multiple index with multiple folder name](https://discuss.elastic.co/t/is-it-possible-to-create-multiple-index-with-multiple-folder-name/321457)
**Author:** [@guhi\_rockky](https://discuss.elastic.co/u/guhi_rockky)\ **Replies:** 18\ **Last updated:** [December 18, 2022, 2:43pm UTC](https://discuss.elastic.co/t/is-it-possible-to-create-multiple-index-with-multiple-folder-name/321457 "2022-12-18T14:43:05Z")
Am monitoring logs files in different folders. I have 3 folders, folder1,folder2,folder3. Is it possible to Create different index name with name of folders using logstash.? --- ## [Paser logs with grok](https://discuss.elastic.co/t/paser-logs-with-grok/321446)
**Author:** [@CodeRed](https://discuss.elastic.co/u/CodeRed)\ **Replies:** 9\ **Last updated:** [December 18, 2022, 12:45pm UTC](https://discuss.elastic.co/t/paser-logs-with-grok/321446 "2022-12-18T12:45:28Z")
I'm researching about ELK for my company's monitoring work, with the passer log I'm wondering that will only add grok-patterns to the grok-patterns config file but can't be added via kibana, I have find out Qradar has a … --- ## [Passing store procedure parameters using jdbc connector in the logstash conf file](https://discuss.elastic.co/t/passing-store-procedure-parameters-using-jdbc-connector-in-the-logstash-conf-file/321485)
**Author:** [@Kamal\_Hamzat](https://discuss.elastic.co/u/Kamal_Hamzat)\ **Replies:** 0\ **Last updated:** [December 18, 2022, 8:55am UTC](https://discuss.elastic.co/t/passing-store-procedure-parameters-using-jdbc-connector-in-the-logstash-conf-file/321485 "2022-12-18T08:55:06Z")
Hi all, How do I add store procedure parameters when using jdbc connector in the logstash conf file. Regards Kamal --- ## [Logstash Encoding mistach](https://discuss.elastic.co/t/logstash-encoding-mistach/320084)
**Author:** [@SilasMuniz1](https://discuss.elastic.co/u/SilasMuniz1)\ **Replies:** 2\ **Last updated:** [December 17, 2022, 1:43am UTC](https://discuss.elastic.co/t/logstash-encoding-mistach/320084 "2022-12-17T01:43:17Z")
Hello, I am trying resolve an encoding problem using mutate gsub. There are some words that display with error: For instance Confirma\\u00e7\\u00eo, correct is Confirmacao. Then a used the following gsub: mutate { … --- ## [Calculate time difference between 2 events with unique id](https://discuss.elastic.co/t/calculate-time-difference-between-2-events-with-unique-id/321388)
**Author:** [@yago82](https://discuss.elastic.co/u/yago82)\ **Replies:** 1\ **Last updated:** [December 16, 2022, 5:48pm UTC](https://discuss.elastic.co/t/calculate-time-difference-between-2-events-with-unique-id/321388 "2022-12-16T17:48:28Z")
Hi, I have two events with same field idRda like the following, I need to calculate the difference between the timestamp "message": "2022-12-13 14:52:00.399 {\[ACTIVE\] ExecuteThread: 5 for queue: weblogic.kernel.Default… --- ## [I started logstash successfully but I don't see anything in my kibana](https://discuss.elastic.co/t/i-started-logstash-successfully-but-i-dont-see-anything-in-my-kibana/321417)
**Author:** [@Whazaza](https://discuss.elastic.co/u/Whazaza)\ **Replies:** 2\ **Last updated:** [December 16, 2022, 5:02pm UTC](https://discuss.elastic.co/t/i-started-logstash-successfully-but-i-dont-see-anything-in-my-kibana/321417 "2022-12-16T17:02:22Z")
I have a problem with logstash it boots with no errors in the logs but I don't see anything in kibana Kibana and elastic are on another node, both apps are running on docker at version 8.5.3 logstash is on a separate … --- ## [Arbitrary hash to CEF custom fields?](https://discuss.elastic.co/t/arbitrary-hash-to-cef-custom-fields/321422)
**Author:** [@j00bar](https://discuss.elastic.co/u/j00bar)\ **Replies:** 1\ **Last updated:** [December 16, 2022, 4:37pm UTC](https://discuss.elastic.co/t/arbitrary-hash-to-cef-custom-fields/321422 "2022-12-16T16:37:14Z")
I'm using the cef codec for encoding data. CEF lets you add custom attributes with a series of deviceCustom\* fields. If you've got an attribute confidence and its value is high, you can say in CEF: deviceCustomString1La… --- ## [Duplication Due to rollover policy in kibana,data coming from logstash pipeline](https://discuss.elastic.co/t/duplication-due-to-rollover-policy-in-kibana-data-coming-from-logstash-pipeline/319959)
**Author:** [@PRASHANT\_MEHTA](https://discuss.elastic.co/u/PRASHANT_MEHTA)\ **Replies:** 9\ **Last updated:** [December 5, 2022, 5:52pm UTC](https://discuss.elastic.co/t/duplication-due-to-rollover-policy-in-kibana-data-coming-from-logstash-pipeline/319959 "2022-12-05T17:52:25Z")
Hello All, I am ingesting data into a rollover index. The data comes from perl scripts running every 15 min,20 min,4hour,12 hour,16 hour etc and gives updates on previously ingested events through doc\_as\_upsert used in … --- ## [Which codec parameter we can use for logstash http input section](https://discuss.elastic.co/t/which-codec-parameter-we-can-use-for-logstash-http-input-section/321405)
**Author:** [@prashant1](https://discuss.elastic.co/u/prashant1)\ **Replies:** 0\ **Last updated:** [December 16, 2022, 11:30am UTC](https://discuss.elastic.co/t/which-codec-parameter-we-can-use-for-logstash-http-input-section/321405 "2022-12-16T11:30:31Z")
We are sending logs from fluentd with http to logstash having version 7.10.2. But looks like codec =\> fluent does not work for http for logstash as we got some \_fluentparse error. We have tried also using codec =\> json… --- ## [Upsert a document with unique id but same fields and different values](https://discuss.elastic.co/t/upsert-a-document-with-unique-id-but-same-fields-and-different-values/321403)
**Author:** [@yago82](https://discuss.elastic.co/u/yago82)\ **Replies:** 0\ **Last updated:** [December 16, 2022, 11:24am UTC](https://discuss.elastic.co/t/upsert-a-document-with-unique-id-but-same-fields-and-different-values/321403 "2022-12-16T11:24:08Z")
Hi, I have two events with same id idRda like the following, I need to unify the logs in one document, but adding two fields "timestamp1" and "timestamp2" example: idRda:\[4040477\] timestamp1:\[1670939520399\] timestam… --- ## [If statement not working as expected](https://discuss.elastic.co/t/if-statement-not-working-as-expected/321054)
**Author:** [@Mistral](https://discuss.elastic.co/u/Mistral)\ **Replies:** 3\ **Last updated:** [December 16, 2022, 7:49am UTC](https://discuss.elastic.co/t/if-statement-not-working-as-expected/321054 "2022-12-16T07:49:45Z")
Hi guys, I know this topic has been posted a few times, but I'm unable to find any tracks to help me since a few days. I'm trying to adapt the logstash 5.8 configuration provided by stormshield to a higher (latest) vers… --- ## [Can I use update statement in jdbc output plugin logstash](https://discuss.elastic.co/t/can-i-use-update-statement-in-jdbc-output-plugin-logstash/321358)
**Author:** [@Big\_Man](https://discuss.elastic.co/u/Big_Man)\ **Replies:** 1\ **Last updated:** [December 16, 2022, 4:21am UTC](https://discuss.elastic.co/t/can-i-use-update-statement-in-jdbc-output-plugin-logstash/321358 "2022-12-16T04:21:51Z")
I need to update specific row in my database. Can i use update statement? If yes how do that Or any other solution to achieve that. --- ## [TCP-input Receiving an encoding error](https://discuss.elastic.co/t/tcp-input-receiving-an-encoding-error/321344)
**Author:** [@elrozario](https://discuss.elastic.co/u/elrozario)\ **Replies:** 1\ **Last updated:** [December 15, 2022, 8:49pm UTC](https://discuss.elastic.co/t/tcp-input-receiving-an-encoding-error/321344 "2022-12-15T20:49:24Z")
Hello, I have a TCP input and getting data with some special characters. How do I resolve this issue? Here is my config input { tcp { host =\> "0.0.0.0" port =\> "0000" mode =\> "server" … --- ## [Not getting output with CEF codec](https://discuss.elastic.co/t/not-getting-output-with-cef-codec/321341)
**Author:** [@j00bar](https://discuss.elastic.co/u/j00bar)\ **Replies:** 2\ **Last updated:** [December 15, 2022, 8:18pm UTC](https://discuss.elastic.co/t/not-getting-output-with-cef-codec/321341 "2022-12-15T20:18:56Z")
I've got a working pipeline in Logstash where non-ECS JSON (I have ecs\_compatibility disabled in my pipeline) is coming in from SQS, getting transformed using mutate filters, and then output using stdout and the rubydebu… --- ## [Logstash convert existing timestamp format in message](https://discuss.elastic.co/t/logstash-convert-existing-timestamp-format-in-message/321309)
**Author:** [@Groove](https://discuss.elastic.co/u/Groove)\ **Replies:** 0\ **Last updated:** [December 15, 2022, 1:10pm UTC](https://discuss.elastic.co/t/logstash-convert-existing-timestamp-format-in-message/321309 "2022-12-15T13:10:09Z")
Hello, I have 2 different application logs. OUTPUT: \<135\>1 2022-12-12T16:28:02Z HOSTNAME EvntSLog - - - Le service Service de licences de client (ClipSVC) est entré dans l’état : arrêté. \<134\>Dec 12 16:28:02 HOSTNAME… --- ## [Problem configure output Email in logstash.conf?](https://discuss.elastic.co/t/problem-configure-output-email-in-logstash-conf/321269)
**Author:** [@tpot\_IT](https://discuss.elastic.co/u/tpot_IT)\ **Replies:** 0\ **Last updated:** [December 15, 2022, 8:07am UTC](https://discuss.elastic.co/t/problem-configure-output-email-in-logstash-conf/321269 "2022-12-15T08:07:10Z")
Hello, My goal is to get email alerts from the Tpot . Do I need to install alerta before configure the logstash.conf to send emails ? I'm trying to follow those instructions: (gitHub) and insert the Email output to… --- ## [Getting data from lagging database](https://discuss.elastic.co/t/getting-data-from-lagging-database/321244)
**Author:** [@raunakraje](https://discuss.elastic.co/u/raunakraje)\ **Replies:** 0\ **Last updated:** [December 14, 2022, 9:46pm UTC](https://discuss.elastic.co/t/getting-data-from-lagging-database/321244 "2022-12-14T21:46:51Z")
Hi Guys, I have 2 MSSQL databases a Production db(A) and a Backup db (B). Data from Production DB gets updated in Backup DB with a lag of 15 mins. I am trying to get data from Backup DB but due to this lag I am unable t… --- ## [Logstash jdbc input state](https://discuss.elastic.co/t/logstash-jdbc-input-state/321192)
**Author:** [@Mahdi\_Moazami](https://discuss.elastic.co/u/Mahdi_Moazami)\ **Replies:** 1\ **Last updated:** [December 14, 2022, 3:43pm UTC](https://discuss.elastic.co/t/logstash-jdbc-input-state/321192 "2022-12-14T15:43:48Z")
Hi there elastic team, Suppose i have multiple pipelines each with one jdbc input. each of them execute a sql query that uses the value of sql\_last\_value to fetch only updated data and each pipeline file targets a speci… --- ## [Logstash - Reading files from Windows Network Share](https://discuss.elastic.co/t/logstash-reading-files-from-windows-network-share/321105)
**Author:** [@anon99430464](https://discuss.elastic.co/u/anon99430464)\ **Replies:** 4\ **Last updated:** [December 14, 2022, 7:05am UTC](https://discuss.elastic.co/t/logstash-reading-files-from-windows-network-share/321105 "2022-12-14T07:05:50Z")
I'm trying to read log files from a network share on Windows. I know this isn't best practice, but currently there is no way around it, and from the documentation it sounds like it is supported. However I can not get th… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=98) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=100)