# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=160 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 161 --- ## [Errors with event splitting using the input tcp plugin](https://discuss.elastic.co/t/errors-with-event-splitting-using-the-input-tcp-plugin/296157)
**Author:** [@opajaron](https://discuss.elastic.co/u/opajaron)\ **Replies:** 4\ **Last updated:** [February 18, 2022, 8:30am UTC](https://discuss.elastic.co/t/errors-with-event-splitting-using-the-input-tcp-plugin/296157 "2022-02-18T08:30:11Z")
Hi, we have a problem using the logstash tcp input plugin. What is happening is that it is not splitting correctly with "\\n" each event that arrives. With the tests we are doing, we are sending events with a rate of ~60 … --- ## [How to parse the multiline and nested json file](https://discuss.elastic.co/t/how-to-parse-the-multiline-and-nested-json-file/297542)
**Author:** [@ycui56](https://discuss.elastic.co/u/ycui56)\ **Replies:** 1\ **Last updated:** [February 17, 2022, 8:22pm UTC](https://discuss.elastic.co/t/how-to-parse-the-multiline-and-nested-json-file/297542 "2022-02-17T20:22:31Z")
Hi, I read lots of posts on the similar topics, but I still have problems to figure it out. So I have to post this one and hopefully get help from here. Thanks a lot in advance. Jason file: { "lable1": { d1: 0, d2… --- ## [Unable to Migrate Custom Index to Cloud Through Logstash](https://discuss.elastic.co/t/unable-to-migrate-custom-index-to-cloud-through-logstash/297419)
**Author:** [@daniel.talbot](https://discuss.elastic.co/u/daniel.talbot)\ **Replies:** 4\ **Last updated:** [February 17, 2022, 6:08pm UTC](https://discuss.elastic.co/t/unable-to-migrate-custom-index-to-cloud-through-logstash/297419 "2022-02-17T18:08:24Z")
Trying to use a basic configuration to write a custom index to a cloud instance but upon doing so I keep getting an error addressing Ruby. Posting the config and results of starting with systemctl betlow: Config: inpu… --- ## [Resolve mapping conflicts (kinda) automatically?](https://discuss.elastic.co/t/resolve-mapping-conflicts-kinda-automatically/297412)
**Author:** [@TheEvilDonut](https://discuss.elastic.co/u/TheEvilDonut)\ **Replies:** 2\ **Last updated:** [February 17, 2022, 2:25pm UTC](https://discuss.elastic.co/t/resolve-mapping-conflicts-kinda-automatically/297412 "2022-02-17T14:25:59Z")
So here's our situation. Most of our applications use an object field named (fictional) "foo" so :slight\_smile: foo.field\_a foo.field\_b foo.field\_c etc... But from time to time a "rogue log" will come in and will c… --- ## [Scheduler Java plugin Logstash](https://discuss.elastic.co/t/scheduler-java-plugin-logstash/297482)
**Author:** [@aalagia](https://discuss.elastic.co/u/aalagia)\ **Replies:** 6\ **Last updated:** [February 17, 2022, 2:07pm UTC](https://discuss.elastic.co/t/scheduler-java-plugin-logstash/297482 "2022-02-17T14:07:42Z")
Hi, I need to use scheduler in custom plugin written in Java for Logstash. How is possible to implement this thing? Thank you Antonio --- ## [Migration from old ELK to Elastic Cloud](https://discuss.elastic.co/t/migration-from-old-elk-to-elastic-cloud/297369)
**Author:** [@mgfeal](https://discuss.elastic.co/u/mgfeal)\ **Replies:** 3\ **Last updated:** [February 17, 2022, 1:38pm UTC](https://discuss.elastic.co/t/migration-from-old-elk-to-elastic-cloud/297369 "2022-02-17T13:38:55Z")
Hello, We are in the process of migrating an older version of ELK to Elastic Cloud. We are already evaluating pricing, subscription model, etc. and I am getting ahead of the process with some question I have. Right now… --- ## [Logstash send data to 2 elastic search clusters](https://discuss.elastic.co/t/logstash-send-data-to-2-elastic-search-clusters/297468)
**Author:** [@gangireddy\_l](https://discuss.elastic.co/u/gangireddy_l)\ **Replies:** 0\ **Last updated:** [February 17, 2022, 9:57am UTC](https://discuss.elastic.co/t/logstash-send-data-to-2-elastic-search-clusters/297468 "2022-02-17T09:57:09Z")
HI Team, I have a scenario to send same logs to both production and preproduction elastic cluster, Is there any possibility to do that. Regards, Reddy --- ## [Logstash could not receive log after several minutes](https://discuss.elastic.co/t/logstash-could-not-receive-log-after-several-minutes/297462)
**Author:** [@Stevenssssss](https://discuss.elastic.co/u/Stevenssssss)\ **Replies:** 1\ **Last updated:** [February 17, 2022, 9:37am UTC](https://discuss.elastic.co/t/logstash-could-not-receive-log-after-several-minutes/297462 "2022-02-17T09:37:02Z")
Hi, my logstash is able to receive the log when the service is started. However after several minutes, it could not receive the log. I could not find any error message in syslog or logstash-plain.log. I then run the /us… --- ## [Not loading pipeline - ConfigurationError](https://discuss.elastic.co/t/not-loading-pipeline-configurationerror/297422)
**Author:** [@Sconic](https://discuss.elastic.co/u/Sconic)\ **Replies:** 2\ **Last updated:** [February 16, 2022, 11:56pm UTC](https://discuss.elastic.co/t/not-loading-pipeline-configurationerror/297422 "2022-02-16T23:56:16Z")
I've updated the output of our pipeline to change the index that some documents get added to but I'm getting a configuration error when trying to start Logstash, I can't for the life of me find what I'm missing. I'm usin… --- ## [Logstash monitoring data not showing up in Kibana](https://discuss.elastic.co/t/logstash-monitoring-data-not-showing-up-in-kibana/297408)
**Author:** [@Pedro\_Osorio](https://discuss.elastic.co/u/Pedro_Osorio)\ **Replies:** 0\ **Last updated:** [February 16, 2022, 7:14pm UTC](https://discuss.elastic.co/t/logstash-monitoring-data-not-showing-up-in-kibana/297408 "2022-02-16T19:14:56Z")
Hi all, I've been trying to see Logstash metrics in Kibana, but with no success. My setup is as follows: metricbeat daemonset collecting host/pods metrics shipping to elastic metricbeat deployment collecting cluster/l… --- ## [Logstash Pipeline Filter (\*)](https://discuss.elastic.co/t/logstash-pipeline-filter/297266)
**Author:** [@max\_cyril](https://discuss.elastic.co/u/max_cyril)\ **Replies:** 3\ **Last updated:** [February 16, 2022, 6:40pm UTC](https://discuss.elastic.co/t/logstash-pipeline-filter/297266 "2022-02-16T18:40:46Z")
RE: Hi,I am struggling to write a logstash pipeline filter section i have used "TRANSFORMS" from kibana interface, it did not solve the case. \*\*\* i want to filter and only output event that correspond to the maxim… --- ## [Need to Extract field](https://discuss.elastic.co/t/need-to-extract-field/297154)
**Author:** [@SRIGURU\_VEL](https://discuss.elastic.co/u/SRIGURU_VEL)\ **Replies:** 7\ **Last updated:** [February 16, 2022, 6:36pm UTC](https://discuss.elastic.co/t/need-to-extract-field/297154 "2022-02-16T18:36:23Z")
Hi Experts, I am using logstash 7.9.3 and i have written the MongoDb connection with Elasticsearch pipeline. Data's are perfectly pushed into Elasticsearch. In that index I need to split one field. My logstash configu… --- ## [Get lines in one line logstash filter](https://discuss.elastic.co/t/get-lines-in-one-line-logstash-filter/297263)
**Author:** [@alex\_vermex](https://discuss.elastic.co/u/alex_vermex)\ **Replies:** 3\ **Last updated:** [February 16, 2022, 6:32pm UTC](https://discuss.elastic.co/t/get-lines-in-one-line-logstash-filter/297263 "2022-02-16T18:32:32Z")
Hi, log file example: 0101-15:17:39:012|aze|exp|id|aa|bb|ee|rr|zz|tt|jj 0101-15:17:39:012|aze|exp|id....|ERR exception nom du para: sss à System... à Web.............|Ref|lev|15|Code\_Status = 50 0101-15:17:39:0… --- ## [Logstsh parce grok add teg \_grokparsefailure](https://discuss.elastic.co/t/logstsh-parce-grok-add-teg-grokparsefailure/297251)
**Author:** [@bsilsy](https://discuss.elastic.co/u/bsilsy)\ **Replies:** 3\ **Last updated:** [February 16, 2022, 6:30pm UTC](https://discuss.elastic.co/t/logstsh-parce-grok-add-teg-grokparsefailure/297251 "2022-02-16T18:30:27Z")
good day everyone! I'm trying to parse field "message" with grock but, i get \_grokparsefailure tag on 100% of documents send data using filebeat message log example "message": "10.10.10.10 - text \[15/Feb/2022:15:27:2… --- ## [Logstash runner error](https://discuss.elastic.co/t/logstash-runner-error/297378)
**Author:** [@Bigboy0706](https://discuss.elastic.co/u/Bigboy0706)\ **Replies:** 3\ **Last updated:** [February 16, 2022, 3:41pm UTC](https://discuss.elastic.co/t/logstash-runner-error/297378 "2022-02-16T15:41:43Z")
\[2022-02-16T14:02:47,966\]\[INFO \]\[logstash.runner \] Log4j configuration path used is: /etc/logstash/log4j2.properties \[2022-02-16T14:02:47,980\]\[INFO \]\[logstash.runner \] Starting Logstash {"logstash.versi… --- ## [Logstash ingest and export to elasticsearch files twice](https://discuss.elastic.co/t/logstash-ingest-and-export-to-elasticsearch-files-twice/297247)
**Author:** [@Greninja\_San](https://discuss.elastic.co/u/Greninja_San)\ **Replies:** 15\ **Last updated:** [February 16, 2022, 2:06pm UTC](https://discuss.elastic.co/t/logstash-ingest-and-export-to-elasticsearch-files-twice/297247 "2022-02-16T14:06:26Z")
Hello! I have a logstash config that gather lines from CSVs and then send them to Elasticsearch. However, for some unknown reason, lines are duplicated in Elasticsearch, taking twice the storage space, and making statis… --- ## [My JDBC SQL statement have $ sign how do i supply it in JDBC input plugin?](https://discuss.elastic.co/t/my-jdbc-sql-statement-have-sign-how-do-i-supply-it-in-jdbc-input-plugin/297326)
**Author:** [@cans](https://discuss.elastic.co/u/cans)\ **Replies:** 0\ **Last updated:** [February 16, 2022, 6:12am UTC](https://discuss.elastic.co/t/my-jdbc-sql-statement-have-sign-how-do-i-supply-it-in-jdbc-input-plugin/297326 "2022-02-16T06:12:40Z")
Hi, My JDBC input conf statement involves selecting a view from an oracle database. This view involves a $ sign. therefore I get an SQL command not properly ended error when I run the logstash script. how do I escape th… --- ## [How to compare the log values and drop the deduplicated log in Logstash](https://discuss.elastic.co/t/how-to-compare-the-log-values-and-drop-the-deduplicated-log-in-logstash/297239)
**Author:** [@Bigboy0706](https://discuss.elastic.co/u/Bigboy0706)\ **Replies:** 2\ **Last updated:** [February 16, 2022, 4:16am UTC](https://discuss.elastic.co/t/how-to-compare-the-log-values-and-drop-the-deduplicated-log-in-logstash/297239 "2022-02-16T04:16:37Z")
Feb 15, 2022 @ 16:16:01.743 columns.url:https://mail.google.com/mail/u/0/#inbox name:chrome\_browser\_history @timestamp:Feb 15, 2022 @ 16:16:01.743 @version:1 action:added calendarTime:Tue Feb 15 08:15:58 2022 UTC columns… --- ## [Reindex data is multiplying docs](https://discuss.elastic.co/t/reindex-data-is-multiplying-docs/296920)
**Author:** [@cris](https://discuss.elastic.co/u/cris)\ **Replies:** 13\ **Last updated:** [February 16, 2022, 12:21am UTC](https://discuss.elastic.co/t/reindex-data-is-multiplying-docs/296920 "2022-02-16T00:21:19Z")
Hello community! I am working with logstash to reindex some data, I have a problem when I run logstash.service the documents increase a lot in the new index, in the original index I have 3000 documents and I expect like… --- ## [Logstatsh Time Zone issue](https://discuss.elastic.co/t/logstatsh-time-zone-issue/297280)
**Author:** [@hanumant](https://discuss.elastic.co/u/hanumant)\ **Replies:** 1\ **Last updated:** [February 15, 2022, 5:55pm UTC](https://discuss.elastic.co/t/logstatsh-time-zone-issue/297280 "2022-02-15T17:55:39Z")
Hi Team, environment logstatsh and elastic server are running in IST System DataBase is in UTC time zone I'm using JDBC logstatsh to push the information I'm seeing the date related all columns values are getting alter… --- ## [Self-managed App Search with SSL enabled](https://discuss.elastic.co/t/self-managed-app-search-with-ssl-enabled/297245)
**Author:** [@Amine27](https://discuss.elastic.co/u/Amine27)\ **Replies:** 0\ **Last updated:** [February 15, 2022, 1:06pm UTC](https://discuss.elastic.co/t/self-managed-app-search-with-ssl-enabled/297245 "2022-02-15T13:06:28Z")
Hello, I am trying to connect Logstash to App Search output with SSL enabled. I am getting this error: Pipeline error {:pipeline\_id=\>"main", :exception=\>#\ **Author:** [@Wonder\_Garance](https://discuss.elastic.co/u/Wonder_Garance)\ **Replies:** 4\ **Last updated:** [February 15, 2022, 5:13pm UTC](https://discuss.elastic.co/t/how-to-add-optional-json-field-in-logstash/296650 "2022-02-15T17:13:47Z") Hi, I wrote a Logstash pipeline to modify some fields of json input files. The keys introduction.text, introduction.html and body in json are optional, I don't want to create empty fields in json output files. How to add… --- ## [Logstash remove field](https://discuss.elastic.co/t/logstash-remove-field/297209)
**Author:** [@chandima\_jayamina](https://discuss.elastic.co/u/chandima_jayamina)\ **Replies:** 3\ **Last updated:** [February 15, 2022, 4:54pm UTC](https://discuss.elastic.co/t/logstash-remove-field/297209 "2022-02-15T16:54:10Z")
Hi I am new to ELK. I am thinking what is the best way to remove field. Just use mutate filter and remove field First set field to nil and then use mutate filter and remove What is the best way to remove the field. An… --- ## [Possible bug in logstash 7.16.2 with degug](https://discuss.elastic.co/t/possible-bug-in-logstash-7-16-2-with-degug/297260)
**Author:** [@mtudisco](https://discuss.elastic.co/u/mtudisco)\ **Replies:** 0\ **Last updated:** [February 15, 2022, 2:51pm UTC](https://discuss.elastic.co/t/possible-bug-in-logstash-7-16-2-with-degug/297260 "2022-02-15T14:51:36Z")
Hi, We recently upgraded our stack to elastic 7.16.2, and after that I introduced some changes in my configuracion. That configuration worked in test, but when i put it in production it does not get any error, however i… --- ## [Schedule not working sometimes | gaps in records](https://discuss.elastic.co/t/schedule-not-working-sometimes-gaps-in-records/296442)
**Author:** [@Nasser](https://discuss.elastic.co/u/Nasser)\ **Replies:** 1\ **Last updated:** [February 15, 2022, 1:53pm UTC](https://discuss.elastic.co/t/schedule-not-working-sometimes-gaps-in-records/296442 "2022-02-15T13:53:44Z")
Hi i have 10 jdbc.conf files when i start the logstash all queries executed successfully but after the first shot some of the queries not execute. i tried to find the reason in logstash.log but nothing show PFB screen… --- ## [Not able to write into log file on running logstash](https://discuss.elastic.co/t/not-able-to-write-into-log-file-on-running-logstash/296574)
**Author:** [@RaviK](https://discuss.elastic.co/u/RaviK)\ **Replies:** 2\ **Last updated:** [February 15, 2022, 10:06am UTC](https://discuss.elastic.co/t/not-able-to-write-into-log-file-on-running-logstash/296574 "2022-02-15T10:06:07Z")
Hi Can anyone please help. My logstash service is running fine and able to parse the log lines from input files. But i can see due to running logstash service, my logfile is not getting updated. But when i'm stopping… --- ## [Auto\_flush\_interval in exec module](https://discuss.elastic.co/t/auto-flush-interval-in-exec-module/295863)
**Author:** [@jeleb](https://discuss.elastic.co/u/jeleb)\ **Replies:** 1\ **Last updated:** [February 15, 2022, 9:03am UTC](https://discuss.elastic.co/t/auto-flush-interval-in-exec-module/295863 "2022-02-15T09:03:39Z")
Hi, I have a problem with auto\_flush\_interval I want to check the command result logstash.conf: input { … --- ## [Geo IP Location Mapping issue in Logstash Filter](https://discuss.elastic.co/t/geo-ip-location-mapping-issue-in-logstash-filter/297168)
**Author:** [@jancodenew](https://discuss.elastic.co/u/jancodenew)\ **Replies:** 1\ **Last updated:** [February 15, 2022, 5:22am UTC](https://discuss.elastic.co/t/geo-ip-location-mapping-issue-in-logstash-filter/297168 "2022-02-15T05:22:00Z")
I am using geoIP logstash filter for getting the geo location details using IPs in logstash. For manually updating the maxmind geolite2 databse, we have downloaded the database files(GeoLite2-City.mmdb and GeoLite2-ASN.… --- ## [Delay between Logstash and ES? ELK 7.16.1 \[ECK\]](https://discuss.elastic.co/t/delay-between-logstash-and-es-elk-7-16-1-eck/295882)
**Author:** [@rudyfaile](https://discuss.elastic.co/u/rudyfaile)\ **Replies:** 15\ **Last updated:** [February 14, 2022, 7:59pm UTC](https://discuss.elastic.co/t/delay-between-logstash-and-es-elk-7-16-1-eck/295882 "2022-02-14T19:59:30Z")
Howdy, Similar to and We're experiencing a 1-2 minute delay between logstash and ES and can't figure out why. We're relatively sure the problem is between logstash and ES. If I manually trigger an error it appear… --- ## [Pipeline stopped with "Index 83 out of bounds for length 16"](https://discuss.elastic.co/t/pipeline-stopped-with-index-83-out-of-bounds-for-length-16/296898)
**Author:** [@mtudisco](https://discuss.elastic.co/u/mtudisco)\ **Replies:** 4\ **Last updated:** [February 14, 2022, 6:07pm UTC](https://discuss.elastic.co/t/pipeline-stopped-with-index-83-out-of-bounds-for-length-16/296898 "2022-02-14T18:07:01Z")
Hi, We have a configuration of logstash with several pipelines, having a main pipeline that receives the information, identifies the type of information and sends to different pipelines for procesing before sending the … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=159) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=161)