# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=106 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 107 --- ## [How to divide a single field of a log in elasticsearch into multiple fields in a single event only using logstash filter plugin](https://discuss.elastic.co/t/how-to-divide-a-single-field-of-a-log-in-elasticsearch-into-multiple-fields-in-a-single-event-only-using-logstash-filter-plugin/318538)
**Author:** [@khushi](https://discuss.elastic.co/u/khushi)\ **Replies:** 3\ **Last updated:** [November 10, 2022, 4:32am UTC](https://discuss.elastic.co/t/how-to-divide-a-single-field-of-a-log-in-elasticsearch-into-multiple-fields-in-a-single-event-only-using-logstash-filter-plugin/318538 "2022-11-10T04:32:39Z")
I am sending emails from a mail service to elasticsearch. Now I want to have multiple fields of a single field. --- ## [Mapping Conflict with Several Types](https://discuss.elastic.co/t/mapping-conflict-with-several-types/318366)
**Author:** [@JakBains](https://discuss.elastic.co/u/JakBains)\ **Replies:** 2\ **Last updated:** [November 9, 2022, 8:44pm UTC](https://discuss.elastic.co/t/mapping-conflict-with-several-types/318366 "2022-11-09T20:44:37Z")
Hello - my organization recently had an issue where the latest index in our datastream from logstash has the wrong mappings. This has made our IP addresses and geopoints appear as text and keyword types, as a result many… --- ## [Logstash input json splitted by newline "\\n"](https://discuss.elastic.co/t/logstash-input-json-splitted-by-newline-n/318146)
**Author:** [@dmrlixos](https://discuss.elastic.co/u/dmrlixos)\ **Replies:** 7\ **Last updated:** [November 9, 2022, 4:49pm UTC](https://discuss.elastic.co/t/logstash-input-json-splitted-by-newline-n/318146 "2022-11-09T16:49:45Z")
Hi I have a logstash pipeline running on tcp port 8080, that receive a log from another application. These application send logs with header "application/json;charset=UTF-8" and maybe send 1 json line or many json sppl… --- ## [Remove domain from username](https://discuss.elastic.co/t/remove-domain-from-username/318464)
**Author:** [@aivazisd](https://discuss.elastic.co/u/aivazisd)\ **Replies:** 4\ **Last updated:** [November 9, 2022, 3:05pm UTC](https://discuss.elastic.co/t/remove-domain-from-username/318464 "2022-11-09T15:05:29Z")
Working with a string: user-identity: Delete IP-User mapping 555.55.55.555 - LOCAL\\user Succeeded - VPN user logout I'm trying to extract the user field without the 'LOCAL'. I can capture the IP and string 'logout' for… --- ## [How to check old pipeline is terminated or not in logstash during config reload](https://discuss.elastic.co/t/how-to-check-old-pipeline-is-terminated-or-not-in-logstash-during-config-reload/318514)
**Author:** [@teja\_tata](https://discuss.elastic.co/u/teja_tata)\ **Replies:** 0\ **Last updated:** [November 9, 2022, 7:25am UTC](https://discuss.elastic.co/t/how-to-check-old-pipeline-is-terminated-or-not-in-logstash-during-config-reload/318514 "2022-11-09T07:25:14Z")
From the above documentation we could see that Logstash swaps the existing pipeline with the new pipeline. If the checks fail, the old pipeline continues to function, and the errors are propagated to the console. How … --- ## [Send syslogs in batches/an array of events records?](https://discuss.elastic.co/t/send-syslogs-in-batches-an-array-of-events-records/318477)
**Author:** [@ianbv](https://discuss.elastic.co/u/ianbv)\ **Replies:** 0\ **Last updated:** [November 8, 2022, 9:41pm UTC](https://discuss.elastic.co/t/send-syslogs-in-batches-an-array-of-events-records/318477 "2022-11-08T21:41:03Z")
Is there a way to group linux syslogs events together in to an array with a logstash filter or output plugin? Here is the example schema I need to get the logs into for sending to ADX: \[ { "records": \[ { r1 … --- ## [Send Ubuntu Syslogs to Azure Event Hub with Logstash - json issue](https://discuss.elastic.co/t/send-ubuntu-syslogs-to-azure-event-hub-with-logstash-json-issue/317990)
**Author:** [@ianbv](https://discuss.elastic.co/u/ianbv)\ **Replies:** 1\ **Last updated:** [November 8, 2022, 9:11pm UTC](https://discuss.elastic.co/t/send-ubuntu-syslogs-to-azure-event-hub-with-logstash-json-issue/317990 "2022-11-08T21:11:23Z")
Situation: Using Logstash to forward ubuntu (azure vm) syslogs to an azure event hub. Problem: Using "json" or "json\_batch" results in adding the \_jsonparseerror tag to events because the syslog isn't in json format. Is… --- ## [Split HTTP Poller Responses to Multiple Documents using Split Filter](https://discuss.elastic.co/t/split-http-poller-responses-to-multiple-documents-using-split-filter/318418)
**Author:** [@sajid](https://discuss.elastic.co/u/sajid)\ **Replies:** 3\ **Last updated:** [November 8, 2022, 8:33pm UTC](https://discuss.elastic.co/t/split-http-poller-responses-to-multiple-documents-using-split-filter/318418 "2022-11-08T20:33:14Z")
We have multiple healthcheck endpoint of different applications which give response in json about the health status of that application and its dependencies. Below are the response of two of the hc URLs: URL 1 Response… --- ## [Logstash failed to start (org.jruby.RubyKernel.exit(org/jruby/RubyKernel.java:790) ~\[jruby.jar:?\])](https://discuss.elastic.co/t/logstash-failed-to-start-org-jruby-rubykernel-exit-org-jruby-rubykernel-java-790-jruby-jar/318457)
**Author:** [@Rootlente](https://discuss.elastic.co/u/Rootlente)\ **Replies:** 12\ **Last updated:** [November 8, 2022, 8:18pm UTC](https://discuss.elastic.co/t/logstash-failed-to-start-org-jruby-rubykernel-exit-org-jruby-rubykernel-java-790-jruby-jar/318457 "2022-11-08T20:18:04Z")
Hello, I am building home lab environment to collect winevent logs using filebat and send them on my Ubuntu machine where i installed everything and configured them by documentation but every time i get this error i d… --- ## [Logstash 7.17 update OpenJDK to 11.0.17](https://discuss.elastic.co/t/logstash-7-17-update-openjdk-to-11-0-17/317350)
**Author:** [@brilong](https://discuss.elastic.co/u/brilong)\ **Replies:** 2\ **Last updated:** [November 8, 2022, 10:56am UTC](https://discuss.elastic.co/t/logstash-7-17-update-openjdk-to-11-0-17/317350 "2022-11-08T10:56:29Z")
Is Logstash 7.17 still maintained and is there a plan to upgrade the built-in JDK to 11.0.17? This fixes various CVEs for those of us unable to upgrade to logstash 8.4 at this time. https://openjdk.org/groups/vulnerabi… --- ## [8.4.2: "incompatible encodings: CP850 and UTF-8" - Issue sending DB records to Elastic after upgrade to Elastic 8.4.2 from 8.3.2](https://discuss.elastic.co/t/8-4-2-incompatible-encodings-cp850-and-utf-8-issue-sending-db-records-to-elastic-after-upgrade-to-elastic-8-4-2-from-8-3-2/315697)
**Author:** [@stevedearl](https://discuss.elastic.co/u/stevedearl)\ **Replies:** 4\ **Last updated:** [November 8, 2022, 8:59am UTC](https://discuss.elastic.co/t/8-4-2-incompatible-encodings-cp850-and-utf-8-issue-sending-db-records-to-elastic-after-upgrade-to-elastic-8-4-2-from-8-3-2/315697 "2022-11-08T08:59:40Z")
Hi All, I've been running the 8.3.2 Elastic products for some months (ELK), using logstash to poll an MSSQL DB regularly and pull the data into Elasticsearch. This has been working without any significant problems. I … --- ## [Need Help to extract custom log data using gork](https://discuss.elastic.co/t/need-help-to-extract-custom-log-data-using-gork/318350)
**Author:** [@ipasa](https://discuss.elastic.co/u/ipasa)\ **Replies:** 1\ **Last updated:** [November 7, 2022, 4:20pm UTC](https://discuss.elastic.co/t/need-help-to-extract-custom-log-data-using-gork/318350 "2022-11-07T16:20:43Z")
We want to extract some custom logs like this, \[2022-10-19 07:45:21\] test\_site.INFO: REQUEST: \[{"username":"6HANT","password":"u5469!230","grant\_type":"password"},{"Content-Type":"application/json"},"https://abc.SD.com:… --- ## [Add array of object into an array through logstash](https://discuss.elastic.co/t/add-array-of-object-into-an-array-through-logstash/318335)
**Author:** [@yashveer](https://discuss.elastic.co/u/yashveer)\ **Replies:** 0\ **Last updated:** [November 7, 2022, 1:45pm UTC](https://discuss.elastic.co/t/add-array-of-object-into-an-array-through-logstash/318335 "2022-11-07T13:45:57Z")
NEED TO DO THIS THOROUGH LOGSTASH ONLY ''' index =\> "test-voq-local" script\_type =\> "inline" document\_id =\> "%{\[i\_odino\]}" scripted\_upsert =\> true action =\> "update" script =\> "ctx.\_source.record.add(params.data); pa… --- ## [Logstash output plugin data changes when stored to file as compared to standard output stdout{ }](https://discuss.elastic.co/t/logstash-output-plugin-data-changes-when-stored-to-file-as-compared-to-standard-output-stdout/318186)
**Author:** [@brbhanushali](https://discuss.elastic.co/u/brbhanushali)\ **Replies:** 3\ **Last updated:** [November 7, 2022, 1:50pm UTC](https://discuss.elastic.co/t/logstash-output-plugin-data-changes-when-stored-to-file-as-compared-to-standard-output-stdout/318186 "2022-11-07T13:50:59Z")
Hello All, I am new to logstash and working on a task to get syslogs from GitHub to Logstash to Elastic Search. I am facing a strange problem in Logstash. To understand the format that is coming on to input port of th… --- ## [Malformed escape pair at index 54](https://discuss.elastic.co/t/malformed-escape-pair-at-index-54/318312)
**Author:** [@reed](https://discuss.elastic.co/u/reed)\ **Replies:** 0\ **Last updated:** [November 7, 2022, 10:06am UTC](https://discuss.elastic.co/t/malformed-escape-pair-at-index-54/318312 "2022-11-07T10:06:16Z")
Hi all, i have my http output plugin called so: http { url =\> "http://10.98.144.73:9200/loyd-strategyone-processcode-%{process-code}/\_update\_by\_query?conflicts=proceed" headers =\> { "Authorization" =\> "Basic Z… --- ## [Error 8.5.0 running as systemctl service](https://discuss.elastic.co/t/error-8-5-0-running-as-systemctl-service/318278)
**Author:** [@georgios.mpouras](https://discuss.elastic.co/u/georgios.mpouras)\ **Replies:** 2\ **Last updated:** [November 7, 2022, 8:37am UTC](https://discuss.elastic.co/t/error-8-5-0-running-as-systemctl-service/318278 "2022-11-07T08:37:28Z")
I have a new installation of logstash 8.5.0 at Alma 9 from the repo When I run logstash as user logstash from the command line everything works as expocted sudo -u logstash /usr/share/logstash/bin/logstash "--path.sett… --- ## [How to remove transferred logs with Logstash?](https://discuss.elastic.co/t/how-to-remove-transferred-logs-with-logstash/318249)
**Author:** [@Vladimir\_Routine](https://discuss.elastic.co/u/Vladimir_Routine)\ **Replies:** 1\ **Last updated:** [November 7, 2022, 5:58am UTC](https://discuss.elastic.co/t/how-to-remove-transferred-logs-with-logstash/318249 "2022-11-07T05:58:21Z")
I want to use Logstash to move my log files from Mongodb to Elasticsearch. Is there any option to remove transferred logs from mongo? --- ## [Logstash not working fine](https://discuss.elastic.co/t/logstash-not-working-fine/318284)
**Author:** [@greeklegend](https://discuss.elastic.co/u/greeklegend)\ **Replies:** 5\ **Last updated:** [November 7, 2022, 5:27am UTC](https://discuss.elastic.co/t/logstash-not-working-fine/318284 "2022-11-07T05:27:48Z")
When I am trying to run as I am transferring data from csv file to Elasticsearch using logstash. My logstash.conf as follows input { file { path =\> "C:/Users/user/Downloads/abc/abc.csv" start\_position =\> "be… --- ## [Why logstash crash](https://discuss.elastic.co/t/why-logstash-crash/318286)
**Author:** [@jin\_zhao](https://discuss.elastic.co/u/jin_zhao)\ **Replies:** 0\ **Last updated:** [November 7, 2022, 2:32am UTC](https://discuss.elastic.co/t/why-logstash-crash/318286 "2022-11-07T02:32:12Z")
error log \[2022-11-06T10:33:45,283\]\[ERROR\]\[logstash.javapipeline \]\[main\] Pipeline worker error, the pipeline will be stopped {:pipeline\_id=\>"main", :error=\>"(IOError) Resource temporarily unavailable", :exception=\>Ja… --- ## [Logstash issues/ is not working](https://discuss.elastic.co/t/logstash-issues-is-not-working/318265)
**Author:** [@Zay\_Lin\_Htun](https://discuss.elastic.co/u/Zay_Lin_Htun)\ **Replies:** 4\ **Last updated:** [November 7, 2022, 2:19am UTC](https://discuss.elastic.co/t/logstash-issues-is-not-working/318265 "2022-11-07T02:19:21Z")
There are three AWS EC2 instances. First one is Elasticsearch+ Kibana, Second one is logstash and third one UbuntuOS which will send logs to logstash. I deployed Elasticsearch and Kibana together on AWS EC2 instancens a… --- ## [Splitting for Logstash working intermittently](https://discuss.elastic.co/t/splitting-for-logstash-working-intermittently/318280)
**Author:** [@eleong](https://discuss.elastic.co/u/eleong)\ **Replies:** 2\ **Last updated:** [November 7, 2022, 2:08am UTC](https://discuss.elastic.co/t/splitting-for-logstash-working-intermittently/318280 "2022-11-07T02:08:49Z")
Hi, Currently, I have Logstash configured for splitting. The issue is that, sometimes it works, sometimes it goes about 10-15 minutes without any output (it suppose to have an output every 5 minutes). At the backend, I… --- ## [Help extracting logs from message field with /t delimiter](https://discuss.elastic.co/t/help-extracting-logs-from-message-field-with-t-delimiter/318212)
**Author:** [@RaonyO](https://discuss.elastic.co/u/RaonyO)\ **Replies:** 3\ **Last updated:** [November 4, 2022, 7:50pm UTC](https://discuss.elastic.co/t/help-extracting-logs-from-message-field-with-t-delimiter/318212 "2022-11-04T19:50:03Z")
hello, I would like to know how I can extract the fields from this log, using grok or another way, I've tried with grok but I completely messed up, the question is how can i separate the fields with the /t delimiter? I … --- ## [Logstash fails to start with SQS Input](https://discuss.elastic.co/t/logstash-fails-to-start-with-sqs-input/318142)
**Author:** [@garohde](https://discuss.elastic.co/u/garohde)\ **Replies:** 6\ **Last updated:** [November 4, 2022, 6:04pm UTC](https://discuss.elastic.co/t/logstash-fails-to-start-with-sqs-input/318142 "2022-11-04T18:04:02Z")
Hey everyone. New to the forum; hope I'm creating this thread in the right place. Been battling this odd problem for a few days and could really use some help. We have a Logstash server on an AWS EC2 server currently … --- ## [Logstash-Twitter input error "undefined method \`filter' for nil:NilClass"](https://discuss.elastic.co/t/logstash-twitter-input-error-undefined-method-filter-for-nil-nilclass/317874)
**Author:** [@True](https://discuss.elastic.co/u/True)\ **Replies:** 5\ **Last updated:** [November 4, 2022, 9:44am UTC](https://discuss.elastic.co/t/logstash-twitter-input-error-undefined-method-filter-for-nil-nilclass/317874 "2022-11-04T09:44:42Z")
Hello. I have been facing issues using the 'Twitter input' in Logstash. LS version : 8.2.2 ES version : 8.2.2 When I run Logstash, Twit are not indexed and the error message below continues to appear. Did I write th… --- ## [Error Running Logstash multiple config](https://discuss.elastic.co/t/error-running-logstash-multiple-config/318167)
**Author:** [@Rakhshunda\_Noorein\_J](https://discuss.elastic.co/u/Rakhshunda_Noorein_J)\ **Replies:** 2\ **Last updated:** [November 4, 2022, 7:51am UTC](https://discuss.elastic.co/t/error-running-logstash-multiple-config/318167 "2022-11-04T07:51:29Z")
Hello, I want to run multiple configs from logstash. So I wrote two pipelines in my piepeline.yml file. # List of pipelines to be loaded by Logstash # # This document must be a list of dictionaries/hashes, where the ke… --- ## [Logstash stuck when lumberjack server is down](https://discuss.elastic.co/t/logstash-stuck-when-lumberjack-server-is-down/318166)
**Author:** [@ferdose\_shaik](https://discuss.elastic.co/u/ferdose_shaik)\ **Replies:** 0\ **Last updated:** [November 4, 2022, 7:25am UTC](https://discuss.elastic.co/t/logstash-stuck-when-lumberjack-server-is-down/318166 "2022-11-04T07:25:23Z")
We are using Logstash 7.15.2 with following lumberjack output configured. output { lumberjack { id =\> "dcae5gcprod" hosts =\> \["dcae-5gc-prod.ecomp.idns.cci.att.com"\] codec =\> json port =\> 31135 ssl\_cert… --- ## [Logstash SNMP input plugin](https://discuss.elastic.co/t/logstash-snmp-input-plugin/318158)
**Author:** [@Aniket\_Pant](https://discuss.elastic.co/u/Aniket_Pant)\ **Replies:** 0\ **Last updated:** [November 4, 2022, 5:20am UTC](https://discuss.elastic.co/t/logstash-snmp-input-plugin/318158 "2022-11-04T05:20:51Z")
Hi Community, We have multiple IPs for monitoring SNMP OIDs is there any method we can put all ip in a file and use it in a snmp input plugin of logstash. --- ## [Elasticsearch mapping issue for ES version 7.\*](https://discuss.elastic.co/t/elasticsearch-mapping-issue-for-es-version-7/318114)
**Author:** [@sssamant](https://discuss.elastic.co/u/sssamant)\ **Replies:** 1\ **Last updated:** [November 4, 2022, 1:29am UTC](https://discuss.elastic.co/t/elasticsearch-mapping-issue-for-es-version-7/318114 "2022-11-04T01:29:07Z")
Hello, We are facing some issue related to mapping when we ingest data. Here is error we are getting: "error"=\>{"type"=\>"illegal\_argument\_exception", "reason"=\>"Mapper for \[fn.keyword\] conflicts with existing mapper:\\… --- ## [Security vulnerability CVE-2021-46848 in Logstash base OS image](https://discuss.elastic.co/t/security-vulnerability-cve-2021-46848-in-logstash-base-os-image/318128)
**Author:** [@NeoELK](https://discuss.elastic.co/u/NeoELK)\ **Replies:** 0\ **Last updated:** [November 3, 2022, 5:18pm UTC](https://discuss.elastic.co/t/security-vulnerability-cve-2021-46848-in-logstash-base-os-image/318128 "2022-11-03T17:18:39Z")
Hi there! I was trying to deploy Logstash by using this Docker image: docker.elastic.co/logstash/logstash:7.17.7 But my AquaSec scanner is throwing an error related to this vulnerability: CVE-2021-46848 | Ubuntu, So I … --- ## [Logstash ack and persistent queue checkpoint order](https://discuss.elastic.co/t/logstash-ack-and-persistent-queue-checkpoint-order/317699)
**Author:** [@bonyolult](https://discuss.elastic.co/u/bonyolult)\ **Replies:** 2\ **Last updated:** [November 3, 2022, 2:38pm UTC](https://discuss.elastic.co/t/logstash-ack-and-persistent-queue-checkpoint-order/317699 "2022-11-03T14:38:24Z")
Hi, read the documentation forth and back and it's not obvious when a message is acked (e.g. consuming from RabbitMQ) . Is the message acked after written to Logstash PQ or acked after reading the input source but befor… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=105) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=107)