# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=60 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 61 --- ## [Remove event.original from logstash as it comes in every document of logstash version 8.8.2(ECS)](https://discuss.elastic.co/t/remove-event-original-from-logstash-as-it-comes-in-every-document-of-logstash-version-8-8-2-ecs/342612)
**Author:** [@PRASHANT\_MEHTA](https://discuss.elastic.co/u/PRASHANT_MEHTA)\ **Replies:** 2\ **Last updated:** [September 8, 2023, 3:12pm UTC](https://discuss.elastic.co/t/remove-event-original-from-logstash-as-it-comes-in-every-document-of-logstash-version-8-8-2-ecs/342612 "2023-09-08T15:12:18Z")
Hello All, I am facing major issue with logstash after migration from 7.9.1 to 8.8.2 version. Elasticsearch/Logstash 8.X version has ECS compatibility enabled by default.This adds new field event.original in every do… --- ## [Removing fields from logstash](https://discuss.elastic.co/t/removing-fields-from-logstash/341782)
**Author:** [@bharti](https://discuss.elastic.co/u/bharti)\ **Replies:** 66\ **Last updated:** [September 8, 2023, 11:06am UTC](https://discuss.elastic.co/t/removing-fields-from-logstash/341782 "2023-09-08T11:06:15Z")
input { file { path =\> "/var/log/abc.log" } beats { port =\> 5044 } } filter { mutate { remove\_field =\> \[ "agent.version.keyword" \] } } --- ## [Logstash output Elastic upsert](https://discuss.elastic.co/t/logstash-output-elastic-upsert/341549)
**Author:** [@mathur7vidit](https://discuss.elastic.co/u/mathur7vidit)\ **Replies:** 6\ **Last updated:** [September 8, 2023, 11:08am UTC](https://discuss.elastic.co/t/logstash-output-elastic-upsert/341549 "2023-09-08T11:08:55Z")
Hi Team, i am looking for clarity on logstash's Elasticsearch output attribute docs\_as\_upsert and action =\> update. Now for this action to work properly and update the existing document, does the document should be on … --- ## [I m running logstash in a container](https://discuss.elastic.co/t/i-m-running-logstash-in-a-container/342564)
**Author:** [@ranjini](https://discuss.elastic.co/u/ranjini)\ **Replies:** 9\ **Last updated:** [September 8, 2023, 9:07am UTC](https://discuss.elastic.co/t/i-m-running-logstash-in-a-container/342564 "2023-09-08T09:07:26Z")
Why does this runs pipelines.yml. while passing the configuration using -f. I use startup.sh which in turn calls the /usr/share/logstash/bin/logstash -f /usr/share/logstash/conf.d/ -w 2 Within the container I see 2 pro… --- ## [Receiving messages from remote syslog using logstash](https://discuss.elastic.co/t/receiving-messages-from-remote-syslog-using-logstash/342559)
**Author:** [@d14](https://discuss.elastic.co/u/d14)\ **Replies:** 0\ **Last updated:** [September 8, 2023, 1:47am UTC](https://discuss.elastic.co/t/receiving-messages-from-remote-syslog-using-logstash/342559 "2023-09-08T01:47:47Z")
I am trying to receive data from a remote syslog server using logstash and the syslog input plugin but unsure how it works. I have a custom domain I want to use for this communication, do I use the custom IP/domain in t… --- ## [Question elk](https://discuss.elastic.co/t/question-elk/342529)
**Author:** [@Farah\_Bannour](https://discuss.elastic.co/u/Farah_Bannour)\ **Replies:** 0\ **Last updated:** [September 7, 2023, 1:49pm UTC](https://discuss.elastic.co/t/question-elk/342529 "2023-09-07T13:49:46Z")
Bonjour ,je voulais dans cette cas supprimer seulement la premier numero comme par exemple ici "create\_uid" : \[ 1, "Support" \], je voudrais supprimer 1 dans le champs create\_uid comment je peux faire ca --- ## [IP match failed](https://discuss.elastic.co/t/ip-match-failed/342475)
**Author:** [@javierelastic](https://discuss.elastic.co/u/javierelastic)\ **Replies:** 4\ **Last updated:** [September 7, 2023, 11:27am UTC](https://discuss.elastic.co/t/ip-match-failed/342475 "2023-09-07T11:27:38Z")
Hi everyone! Something strange happens to me. I'm trying to see if a source ip matches a pattern I indicate. The ip is 100.44.1.128 and it tells me that it matches "^10.\*" How is it possible? if \[IPorigen\] =~ "^10.\*"… --- ## [Alternative grok with API](https://discuss.elastic.co/t/alternative-grok-with-api/342265)
**Author:** [@sam1975](https://discuss.elastic.co/u/sam1975)\ **Replies:** 3\ **Last updated:** [September 7, 2023, 9:11am UTC](https://discuss.elastic.co/t/alternative-grok-with-api/342265 "2023-09-07T09:11:30Z")
Hello, I've some pipeline which use grok to parse logs and apply some modifications. As i collect in input data from Elastic index, make some modifications and send it directly data transformed in an Elastic index, is … --- ## [Logstash 8.9.0](https://discuss.elastic.co/t/logstash-8-9-0/342362)
**Author:** [@ranjini](https://discuss.elastic.co/u/ranjini)\ **Replies:** 2\ **Last updated:** [September 7, 2023, 8:53am UTC](https://discuss.elastic.co/t/logstash-8-9-0/342362 "2023-09-07T08:53:59Z")
Please help me. I m running logstash 8.9.0. I recieve the below error in the pod logs. \[2023-09-05T16:25:32,904\]\[ERROR\]\[logstash.javapipeline \]\[main\]\[d9383b2c5e755b975c5f06446fd24ec66c0265c309ed53bd78ed6e05939579ec\] … --- ## [Creating custom grok pattern](https://discuss.elastic.co/t/creating-custom-grok-pattern/342303)
**Author:** [@Cruz](https://discuss.elastic.co/u/Cruz)\ **Replies:** 4\ **Last updated:** [September 7, 2023, 6:30am UTC](https://discuss.elastic.co/t/creating-custom-grok-pattern/342303 "2023-09-07T06:30:00Z")
I was working with Logstash to structure the following types of logs: 2023-09-05 11:53:25 (152.32.73.6)-Logistics Request Approved: {"id":7355,"lr\_number":"LR-M006108","lr\_type":"2","lr\_type\_list":"1","lr\_type\_others":n… --- ## [Logstash lumberjack output kept on using IP instead of hostname](https://discuss.elastic.co/t/logstash-lumberjack-output-kept-on-using-ip-instead-of-hostname/342493)
**Author:** [@mikhatanu](https://discuss.elastic.co/u/mikhatanu)\ **Replies:** 0\ **Last updated:** [September 7, 2023, 3:20am UTC](https://discuss.elastic.co/t/logstash-lumberjack-output-kept-on-using-ip-instead-of-hostname/342493 "2023-09-07T03:20:14Z")
Hello, i currently have a logstash in openshift exposed through openshift route with beats input. I tried to send some logs using powershell with logstash for windows: bin/logstash -e 'input { generator { count =\> 5 } }… --- ## [Question elk stack](https://discuss.elastic.co/t/question-elk-stack/342411)
**Author:** [@Farah\_Bannour](https://discuss.elastic.co/u/Farah_Bannour)\ **Replies:** 3\ **Last updated:** [September 6, 2023, 12:07pm UTC](https://discuss.elastic.co/t/question-elk-stack/342411 "2023-09-06T12:07:52Z")
Hello , I HAVE A QUESTION IF ANY ONE khnwo please help me how i drop an empty value in the field elk stack (i have a field resultat contain value recu,ajourné,admis ,false) i wont to drop only the value false but with … --- ## [How to get the field value from an object](https://discuss.elastic.co/t/how-to-get-the-field-value-from-an-object/342246)
**Author:** [@uma\_parvathy](https://discuss.elastic.co/u/uma_parvathy)\ **Replies:** 3\ **Last updated:** [September 6, 2023, 9:57am UTC](https://discuss.elastic.co/t/how-to-get-the-field-value-from-an-object/342246 "2023-09-06T09:57:43Z")
i've a case\_number which has a field "task\_id". i need to use that id to get all task details. i tried to get the value , it is empty. please help me out. Here is the Elasticsearch pulled data { "\_index": "logs… --- ## [Logstash SSL/TLS error](https://discuss.elastic.co/t/logstash-ssl-tls-error/342368)
**Author:** [@Kvoyce2023](https://discuss.elastic.co/u/Kvoyce2023)\ **Replies:** 0\ **Last updated:** [September 5, 2023, 5:56pm UTC](https://discuss.elastic.co/t/logstash-ssl-tls-error/342368 "2023-09-05T17:56:55Z")
My ELK stack got 3 ES nodes and 2 logstash nodes. Kibana is installed on one of the Logstash nodes. I was able to generate CA and all certificates. Distributed the certificates to all nodes.Confirmed Elasticsearch nodes … --- ## [Logstash in k8s - parsing nested json from MongoDB and get every nested json as separated field](https://discuss.elastic.co/t/logstash-in-k8s-parsing-nested-json-from-mongodb-and-get-every-nested-json-as-separated-field/341755)
**Author:** [@Denis\_Lezgin](https://discuss.elastic.co/u/Denis_Lezgin)\ **Replies:** 2\ **Last updated:** [September 5, 2023, 7:11am UTC](https://discuss.elastic.co/t/logstash-in-k8s-parsing-nested-json-from-mongodb-and-get-every-nested-json-as-separated-field/341755 "2023-09-05T07:11:15Z")
Hi there, I'm using Logstash to take documents from specific MongoDB collection, and save it to Elasticsearch. Nested fields are being saved to "log\_entry" as one JSON, starting with "BSON" or "ID", depends on manipul… --- ## [Tomcat access log analysis](https://discuss.elastic.co/t/tomcat-access-log-analysis/341688)
**Author:** [@Brian\_Michelsen](https://discuss.elastic.co/u/Brian_Michelsen)\ **Replies:** 1\ **Last updated:** [September 4, 2023, 5:39pm UTC](https://discuss.elastic.co/t/tomcat-access-log-analysis/341688 "2023-09-04T17:39:14Z")
Hi, I have setup Elasticseach, Logstash and Kibana to analyse response times on a application. The pattern of the access log is: %a %{request.id}r %{request.username}r %t "%m %U%{sanitized.query}r %H" %s %b %… --- ## [Logstash dateparse error](https://discuss.elastic.co/t/logstash-dateparse-error/342209)
**Author:** [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Replies:** 4\ **Last updated:** [September 4, 2023, 5:16pm UTC](https://discuss.elastic.co/t/logstash-dateparse-error/342209 "2023-09-04T17:16:42Z")
I have a logstash pipleline working from years. worked since 7.1 till 8.5.3 (no problem on any upgrade) two month ago when I upgraded to 8.5.3 it was still working fine. suddenly it stop working due to date parsing fai… --- ## [Salesforce logstash input sensible to changes](https://discuss.elastic.co/t/salesforce-logstash-input-sensible-to-changes/341784)
**Author:** [@Samuele\_Lolli](https://discuss.elastic.co/u/Samuele_Lolli)\ **Replies:** 4\ **Last updated:** [September 4, 2023, 12:34pm UTC](https://discuss.elastic.co/t/salesforce-logstash-input-sensible-to-changes/341784 "2023-09-04T12:34:29Z")
Hi everyone, i implemented a pipeline using logstash that is currently working fine but i have some question. Im importing the data from salesforce but im not able to understand if the salesforce plugin is sensible to … --- ## [Lag in logs](https://discuss.elastic.co/t/lag-in-logs/342233)
**Author:** [@kriti\_dabas](https://discuss.elastic.co/u/kriti_dabas)\ **Replies:** 2\ **Last updated:** [September 4, 2023, 11:35am UTC](https://discuss.elastic.co/t/lag-in-logs/342233 "2023-09-04T11:35:01Z")
filter { json { source =\> "message" } grok { match =\> { "message" =\> \[ "%{DATA:description} default %{DATA:connection\_details} : SPCBId %{DATA:spcbId} - ClientIP %{DATA:clientIP} - Client… --- ## [Logstash is not getting whole table data](https://discuss.elastic.co/t/logstash-is-not-getting-whole-table-data/341945)
**Author:** [@uma\_parvathy](https://discuss.elastic.co/u/uma_parvathy)\ **Replies:** 2\ **Last updated:** [September 4, 2023, 10:28am UTC](https://discuss.elastic.co/t/logstash-is-not-getting-whole-table-data/341945 "2023-09-04T10:28:42Z")
Hi All, i've been working on Elasticsearch recently. the logstash pipeline pulls the data only 10k records from the db table. How shall i make it to pull whole table data ? i tried both jdbc\_page\_size and jdbc\_fetch\_s… --- ## [Separating pipeline logs issues](https://discuss.elastic.co/t/separating-pipeline-logs-issues/340199)
**Author:** [@Siavash\_Fazli](https://discuss.elastic.co/u/Siavash_Fazli)\ **Replies:** 6\ **Last updated:** [September 4, 2023, 8:46am UTC](https://discuss.elastic.co/t/separating-pipeline-logs-issues/340199 "2023-09-04T08:46:00Z")
Hi guys. I am running Logstash version 8.8.2 on docker with more than 10 pipelines. I want to separate pipeline logs to separate log files, according to logstash document: document said set path.logs and pipeline.sep… --- ## [Extract some words in a keyword field](https://discuss.elastic.co/t/extract-some-words-in-a-keyword-field/342135)
**Author:** [@Claudio\_Ract\_Costa](https://discuss.elastic.co/u/Claudio_Ract_Costa)\ **Replies:** 4\ **Last updated:** [September 2, 2023, 3:36am UTC](https://discuss.elastic.co/t/extract-some-words-in-a-keyword-field/342135 "2023-09-02T03:36:42Z")
Hi, I have a field (keyword) with the following value (this value can be in different order and different values): {"sessRules":{"SetLTEQoS":{"authSessAmbr":{"uplink":"4200 Mbps","downlink":"4200 Mbps"},"authDefQos":{… --- ## [Use operator with gsub](https://discuss.elastic.co/t/use-operator-with-gsub/342111)
**Author:** [@sam1975](https://discuss.elastic.co/u/sam1975)\ **Replies:** 7\ **Last updated:** [September 1, 2023, 3:17pm UTC](https://discuss.elastic.co/t/use-operator-with-gsub/342111 "2023-09-01T15:17:56Z")
Hello, I need to change a logstash pipeline which use gsub but i've doubt of the syntax Instead of this mutate {gsub =\> \[ "status", "(?i)added", "plugged" \]} mutate {gsub =\> \[ "status", "(?i)installed", "plugged" … --- ## [Logstash fails to start listener due to Error: failed to create a child event loop in io.netty.util.concurrent.MultithreadEventExecutorGroup.\(io/netty/util/concurrent/MultithreadEventExecutorGroup.java:88)](https://discuss.elastic.co/t/logstash-fails-to-start-listener-due-to-error-failed-to-create-a-child-event-loop-in-io-netty-util-concurrent-multithreadeventexecutorgroup-init-io-netty-util-concurrent-multithreadeventexecutorgroup-java-88/341295)
**Author:** [@tlukac](https://discuss.elastic.co/u/tlukac)\ **Replies:** 7\ **Last updated:** [September 1, 2023, 9:54am UTC](https://discuss.elastic.co/t/logstash-fails-to-start-listener-due-to-error-failed-to-create-a-child-event-loop-in-io-netty-util-concurrent-multithreadeventexecutorgroup-init-io-netty-util-concurrent-multithreadeventexecutorgroup-java-88/341295 "2023-09-01T09:54:32Z")
Logstash continually raises this error when attempting to start LogStash::Inputs::Beats plugin. There is no "Caused by" info in the stack trace so cannot determine what is causing the issue. Any ideas on how to diagnos… --- ## [Logstash Stdout empty](https://discuss.elastic.co/t/logstash-stdout-empty/342073)
**Author:** [@Bountardos](https://discuss.elastic.co/u/Bountardos)\ **Replies:** 3\ **Last updated:** [September 1, 2023, 9:33am UTC](https://discuss.elastic.co/t/logstash-stdout-empty/342073 "2023-09-01T09:33:35Z")
Hi there, i'm having issues with a recent configuration on my logstash and i can't understand why it's not working. I have multiple configuration files running, and working. This one was working also as of a week ago, b… --- ## [Index is not creating in logstash through mule](https://discuss.elastic.co/t/index-is-not-creating-in-logstash-through-mule/342100)
**Author:** [@vikascateina](https://discuss.elastic.co/u/vikascateina)\ **Replies:** 0\ **Last updated:** [September 1, 2023, 7:53am UTC](https://discuss.elastic.co/t/index-is-not-creating-in-logstash-through-mule/342100 "2023-09-01T07:53:06Z")
Hi I am not able to push the logs from my mule application to logstash which is running in ecs in aws and it is up . input { tcp { port =\> 4560 codec =\> json } } filter { date { match =\> \[ "timeMillis", "UNIX\_MS… --- ## [logstash Handling exception: io.netty.handler.codec.DecoderException:](https://discuss.elastic.co/t/logstash-handling-exception-io-netty-handler-codec-decoderexception/342054)
**Author:** [@zuoseven](https://discuss.elastic.co/u/zuoseven)\ **Replies:** 5\ **Last updated:** [September 1, 2023, 5:54am UTC](https://discuss.elastic.co/t/logstash-handling-exception-io-netty-handler-codec-decoderexception/342054 "2023-09-01T05:54:15Z")
Handling exception: io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: Empty server certificate chain (caused by: javax.net.ssl.SSLHandshakeException: Empty server certificate chain) \[2023-08-3… --- ## [Open SSL vulnerability in logstash directory](https://discuss.elastic.co/t/open-ssl-vulnerability-in-logstash-directory/341982)
**Author:** [@Supriyo](https://discuss.elastic.co/u/Supriyo)\ **Replies:** 4\ **Last updated:** [September 1, 2023, 5:01am UTC](https://discuss.elastic.co/t/open-ssl-vulnerability-in-logstash-directory/341982 "2023-09-01T05:01:56Z")
Security scans have found this open SSL vulnerability in logstash directory. We are trying to upgrade OpenSSL version 3.0.8 or later in the production server. The current version on the server is 3.0.3. Could you plea… --- ## [Json codec vs. json\_lines codec for collecting mongoexport output JSON?](https://discuss.elastic.co/t/json-codec-vs-json-lines-codec-for-collecting-mongoexport-output-json/341616)
**Author:** [@paolovalladolid](https://discuss.elastic.co/u/paolovalladolid)\ **Replies:** 18\ **Last updated:** [August 31, 2023, 2:59pm UTC](https://discuss.elastic.co/t/json-codec-vs-json-lines-codec-for-collecting-mongoexport-output-json/341616 "2023-08-31T14:59:47Z")
I'm trying to get Logstash to ingest a JSON file created by a mongoexport call. The file looks like this: { "\_id": "3c51d008add94422abf107f0", "name": "Pulse Get SVN By ID", "type": "automation", "tasks": { "53… --- ## [Snowflake to Elasticsearch Using Logtsash](https://discuss.elastic.co/t/snowflake-to-elasticsearch-using-logtsash/341785)
**Author:** [@ksaimohan2k](https://discuss.elastic.co/u/ksaimohan2k)\ **Replies:** 3\ **Last updated:** [August 31, 2023, 7:01am UTC](https://discuss.elastic.co/t/snowflake-to-elasticsearch-using-logtsash/341785 "2023-08-31T07:01:17Z")
We are migrating data from Snowflake to Elasticsearch using the Logtsash driver. I took the logstash conf file template from \[Pull data from Snowflake with logstash | by Izek Chen | Medium\]. Below is the Logstash confi… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=59) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=61)