# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=291 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 292 --- ## [Logstash mapping wrong field types](https://discuss.elastic.co/t/logstash-mapping-wrong-field-types/252460)
**Author:** [@JFC](https://discuss.elastic.co/u/JFC)\ **Replies:** 3\ **Last updated:** [October 19, 2020, 9:22pm UTC](https://discuss.elastic.co/t/logstash-mapping-wrong-field-types/252460 "2020-10-19T21:22:48Z")
Hi , I'm new to logstash and ELK stack. So apologies in advance if this question sounds obvious. Problem: I have fields that are being detected as strings by Kibana even after the template.json file explicitly maps the… --- ## [Mutate rename multiple fields, how to improve performance?](https://discuss.elastic.co/t/mutate-rename-multiple-fields-how-to-improve-performance/252606)
**Author:** [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Replies:** 1\ **Last updated:** [October 19, 2020, 9:05pm UTC](https://discuss.elastic.co/t/mutate-rename-multiple-fields-how-to-improve-performance/252606 "2020-10-19T21:05:02Z")
Hello, I have a pipeline where I need to rename something around 1100 fields, the documents do not have all the fields, but in total I need to rename near 1100 fields. Currently I have one mutate filter with a rename a… --- ## [Logstash multiline codec with xml filter](https://discuss.elastic.co/t/logstash-multiline-codec-with-xml-filter/252593)
**Author:** [@Arvind\_Ks](https://discuss.elastic.co/u/Arvind_Ks)\ **Replies:** 1\ **Last updated:** [October 19, 2020, 7:00pm UTC](https://discuss.elastic.co/t/logstash-multiline-codec-with-xml-filter/252593 "2020-10-19T19:00:08Z")
Hello Everyone i have a input xml file, that i am using to send to elasticsearch. i want all fields within and . However i am getting below error. pls suggest. Input file 2020-09-30 11:59:54,755 (Test worker) JobSch… --- ## [Doubts about extract field](https://discuss.elastic.co/t/doubts-about-extract-field/252558)
**Author:** [@rildo](https://discuss.elastic.co/u/rildo)\ **Replies:** 2\ **Last updated:** [October 19, 2020, 6:31pm UTC](https://discuss.elastic.co/t/doubts-about-extract-field/252558 "2020-10-19T18:31:34Z")
Hello People , I have this code ´ ´ ´ input { stdin { } } output { stdout { codec =\> rubydebug } } filter { json { source =\> "message" } mutate { gsub =\> \[ "message" , "\\n", "," \] } mutate { gsub =\> \[ "messa… --- ## [Where should I drop records? And how?](https://discuss.elastic.co/t/where-should-i-drop-records-and-how/252383)
**Author:** [@mbuster](https://discuss.elastic.co/u/mbuster)\ **Replies:** 2\ **Last updated:** [October 19, 2020, 6:18pm UTC](https://discuss.elastic.co/t/where-should-i-drop-records-and-how/252383 "2020-10-19T18:18:59Z")
Background: Sending Sonicwall syslog to Logstash in order to keep records on SSL VPN logins/logouts Sonicwall indices are accounting for a majority of my storage usage Need: Drop Sonicwall records based on specific … --- ## [Logstash Filebeat input](https://discuss.elastic.co/t/logstash-filebeat-input/251992)
**Author:** [@jafri6](https://discuss.elastic.co/u/jafri6)\ **Replies:** 3\ **Last updated:** [October 19, 2020, 9:36am UTC](https://discuss.elastic.co/t/logstash-filebeat-input/251992 "2020-10-19T09:36:39Z")
My goal is to setup filebeat on at least 5 systems, and get the data to elasticsearch via logstash. I imported data via the file plugin earlier for one of my trials, and the data was imported successfully. I tried the … --- ## [Logstash truncating original message while applying json filter](https://discuss.elastic.co/t/logstash-truncating-original-message-while-applying-json-filter/252500)
**Author:** [@pks01](https://discuss.elastic.co/u/pks01)\ **Replies:** 0\ **Last updated:** [October 19, 2020, 8:14am UTC](https://discuss.elastic.co/t/logstash-truncating-original-message-while-applying-json-filter/252500 "2020-10-19T08:14:27Z")
I have logstash running on kubernetes which gets the logs from azure-eventhub using eventhub plugin. In eventhub, the messages are logged in JSON format. While reading events from the topic in logstash and applying JSON … --- ## [Logstash and Elasticsearch ILM bootstrap index](https://discuss.elastic.co/t/logstash-and-elasticsearch-ilm-bootstrap-index/252310)
**Author:** [@Mikele](https://discuss.elastic.co/u/Mikele)\ **Replies:** 1\ **Last updated:** [October 19, 2020, 7:29am UTC](https://discuss.elastic.co/t/logstash-and-elasticsearch-ilm-bootstrap-index/252310 "2020-10-19T07:29:35Z")
I need to dynamically create several types (with different index names) indexes in Elasticsearch from logstash. I need to use lifecycle (ILM) feature. My questions and either problems are: How can I from logstash creat… --- ## [Is my systemctl restart logstash not working?](https://discuss.elastic.co/t/is-my-systemctl-restart-logstash-not-working/252464)
**Author:** [@yulaika](https://discuss.elastic.co/u/yulaika)\ **Replies:** 2\ **Last updated:** [October 19, 2020, 3:51am UTC](https://discuss.elastic.co/t/is-my-systemctl-restart-logstash-not-working/252464 "2020-10-19T03:51:22Z")
the messages stopped halfway. it’s not even completed. when i systemctl restart logstash, it didn’t even get restarted for some reason. --- ## [Launch logstash conf files automatically without "logstash -f" command](https://discuss.elastic.co/t/launch-logstash-conf-files-automatically-without-logstash-f-command/252462)
**Author:** [@kumar4](https://discuss.elastic.co/u/kumar4)\ **Replies:** 0\ **Last updated:** [October 19, 2020, 12:03am UTC](https://discuss.elastic.co/t/launch-logstash-conf-files-automatically-without-logstash-f-command/252462 "2020-10-19T00:03:47Z")
hello my friends, I have .conf files in my directory, and I have to use the logtsash -f command to start reading, is it possible to start them automatically without using the command and also at OS boot thank you --- ## [Metricbeat mapper error in 7.9.2](https://discuss.elastic.co/t/metricbeat-mapper-error-in-7-9-2/252428)
**Author:** [@frankfoti](https://discuss.elastic.co/u/frankfoti)\ **Replies:** 2\ **Last updated:** [October 18, 2020, 9:58pm UTC](https://discuss.elastic.co/t/metricbeat-mapper-error-in-7-9-2/252428 "2020-10-18T21:58:21Z")
I have a metricbeat pipeline that works fine sending data to my on premis cluster but getting this error sending to ES cloud: "mapper \[index\_recovery.shards.index.size.reused\_in\_bytes\] cannot be changed from type \[long\]… --- ## [Logstash grep and grok](https://discuss.elastic.co/t/logstash-grep-and-grok/252441)
**Author:** [@tarunmendon](https://discuss.elastic.co/u/tarunmendon)\ **Replies:** 3\ **Last updated:** [October 18, 2020, 3:16pm UTC](https://discuss.elastic.co/t/logstash-grep-and-grok/252441 "2020-10-18T15:16:24Z")
I am new to Elasticstack. I am trying to implement a logstash pipeline in which the a file would be processed and it would filter(grep) and output if the line of file contains following keyword - java.lang.Exception - … --- ## [Making Logstash and/or Filebeats process one entry at a time](https://discuss.elastic.co/t/making-logstash-and-or-filebeats-process-one-entry-at-a-time/251822)
**Author:** [@jonVR](https://discuss.elastic.co/u/jonVR)\ **Replies:** 9\ **Last updated:** [October 16, 2020, 10:04pm UTC](https://discuss.elastic.co/t/making-logstash-and-or-filebeats-process-one-entry-at-a-time/251822 "2020-10-16T22:04:18Z")
I currently have a setup where Filebeats is tracking a folder (in which there is only 1 file), and sends entries to Logstash. In my logstash.yml, I have pipeline.workers set to 1. Here is a snippet of my logstash.conf, f… --- ## [Pipeline not starting](https://discuss.elastic.co/t/pipeline-not-starting/252112)
**Author:** [@dorcas](https://discuss.elastic.co/u/dorcas)\ **Replies:** 0\ **Last updated:** [October 14, 2020, 10:41pm UTC](https://discuss.elastic.co/t/pipeline-not-starting/252112 "2020-10-14T22:41:35Z")
kindly help pipeline action failing :exception=\>#\ Could not execute action: PipelineAction::Create after add truststore index =\> "Sample" user =\> "Log… --- ## [Logstash Filter Error](https://discuss.elastic.co/t/logstash-filter-error/252252)
**Author:** [@iccMe](https://discuss.elastic.co/u/iccMe)\ **Replies:** 2\ **Last updated:** [October 16, 2020, 1:59pm UTC](https://discuss.elastic.co/t/logstash-filter-error/252252 "2020-10-16T13:59:50Z")
Hi, I am trying to pull out relevant fields from a log file message and pass it into elasticsearch. I am still in testing and have managed to use the grok debugger to pull out the fields I need. An example of the log me… --- ## [MongoDB elasticsearch using logstash error](https://discuss.elastic.co/t/mongodb-elasticsearch-using-logstash-error/252347)
**Author:** [@Manuang](https://discuss.elastic.co/u/Manuang)\ **Replies:** 0\ **Last updated:** [October 16, 2020, 1:05pm UTC](https://discuss.elastic.co/t/mongodb-elasticsearch-using-logstash-error/252347 "2020-10-16T13:05:28Z")
I am trying to connect mongodb to elastic search. I am using Mongodb compass and in that the database is MongoPOC and collection is Json\_file. My config file is as below: MongoDB 4.2.10 Enterprise Cluster Standalone i… --- ## [Logstas-output-mongodb DB creation](https://discuss.elastic.co/t/logstas-output-mongodb-db-creation/252305)
**Author:** [@tamilarasanbravo](https://discuss.elastic.co/u/tamilarasanbravo)\ **Replies:** 0\ **Last updated:** [October 16, 2020, 6:36am UTC](https://discuss.elastic.co/t/logstas-output-mongodb-db-creation/252305 "2020-10-16T06:36:14Z")
Hi Team, I am trying to have logstash output for MongoDB and I am unsure whether if i can customize the Mongo DB name based upon the input logstash receives. To be be more precise, I am unaware of the logstash capabili… --- ## [Importing json file using logstash stuck with Successfully started Logstash API endpoint {:port=\>9600}](https://discuss.elastic.co/t/importing-json-file-using-logstash-stuck-with-successfully-started-logstash-api-endpoint-port-9600/252342)
**Author:** [@Manuang](https://discuss.elastic.co/u/Manuang)\ **Replies:** 0\ **Last updated:** [October 16, 2020, 11:50am UTC](https://discuss.elastic.co/t/importing-json-file-using-logstash-stuck-with-successfully-started-logstash-api-endpoint-port-9600/252342 "2020-10-16T11:50:21Z")
{"request\_id":1,"requestDetail\_id":0,"enrichedContent":"enrich cont example","statusCode":"P","reasonDesc":"This is a test","createdBy":"Sarah","createdDate":"Sep 24, 2010"} Json data Conf file is input { file{ pa… --- ## [What Grok pattern will be?](https://discuss.elastic.co/t/what-grok-pattern-will-be/252340)
**Author:** [@Bhavin\_Varsur](https://discuss.elastic.co/u/Bhavin_Varsur)\ **Replies:** 0\ **Last updated:** [October 16, 2020, 11:40am UTC](https://discuss.elastic.co/t/what-grok-pattern-will-be/252340 "2020-10-16T11:40:15Z")
what should grok pattern will be of this log? log sample 2020-10-16 16:53:17.4009 \[57244\] ERROR Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware 192.168.43.244 bvarsur@gmail.com http://localhost/Tran… --- ## [Aggregation not working in Logstash input elasticsearch filter](https://discuss.elastic.co/t/aggregation-not-working-in-logstash-input-elasticsearch-filter/252298)
**Author:** [@Vijay\_Barai](https://discuss.elastic.co/u/Vijay_Barai)\ **Replies:** 0\ **Last updated:** [October 16, 2020, 5:07am UTC](https://discuss.elastic.co/t/aggregation-not-working-in-logstash-input-elasticsearch-filter/252298 "2020-10-16T05:07:57Z")
Hi Team, I am trying to do a aggregation using ES query. Query is giving a result in develper tool but when I use it in logstash config file I am not getting a actual result. I am getting only few selected field, at lea… --- ## [Error stacktrace not showing on the linux server](https://discuss.elastic.co/t/error-stacktrace-not-showing-on-the-linux-server/252303)
**Author:** [@Bhavin\_Varsur](https://discuss.elastic.co/u/Bhavin_Varsur)\ **Replies:** 0\ **Last updated:** [October 16, 2020, 5:59am UTC](https://discuss.elastic.co/t/error-stacktrace-not-showing-on-the-linux-server/252303 "2020-10-16T05:59:53Z")
I've installed kibana,elasticsearch and logstash version 7.9.2 on the linux server. In local windows machine it working fine and showing error stracktrace of message. but over the linux it's not showing error stracktra… --- ## [What is the grok pattern for find userid from message?](https://discuss.elastic.co/t/what-is-the-grok-pattern-for-find-userid-from-message/252301)
**Author:** [@Bhavin\_Varsur](https://discuss.elastic.co/u/Bhavin_Varsur)\ **Replies:** 0\ **Last updated:** [October 16, 2020, 5:50am UTC](https://discuss.elastic.co/t/what-is-the-grok-pattern-for-find-userid-from-message/252301 "2020-10-16T05:50:49Z")
I've two layouts one is error layout.in this layout have two additional fields like.requestUrl and requestmethod. the second one is common layout which is for warn,debug,info log levels. i've also added userId which is… --- ## [Custom Logstash pipeline configuration file](https://discuss.elastic.co/t/custom-logstash-pipeline-configuration-file/252277)
**Author:** [@Adriann](https://discuss.elastic.co/u/Adriann)\ **Replies:** 5\ **Last updated:** [October 16, 2020, 12:29am UTC](https://discuss.elastic.co/t/custom-logstash-pipeline-configuration-file/252277 "2020-10-16T00:29:08Z")
Hi, I want to make a custom pipeline using SNMP walk plugin. I need some help in the first steps to properly understand syntax. So far I have something like this filter { mutate { add\_field =\> … --- ## [How do i create field Types such as geo\_point in a logstash conf file?](https://discuss.elastic.co/t/how-do-i-create-field-types-such-as-geo-point-in-a-logstash-conf-file/251922)
**Author:** [@fallenreaper](https://discuss.elastic.co/u/fallenreaper)\ **Replies:** 5\ **Last updated:** [October 15, 2020, 7:17pm UTC](https://discuss.elastic.co/t/how-do-i-create-field-types-such-as-geo-point-in-a-logstash-conf-file/251922 "2020-10-15T19:17:58Z")
I have location data I have been testing with various formats related to geo\_point in order for the machine to say "this is a geo\_point" but it doesnt. I said to myself "logstash handles parsing and should take the burd… --- ## [Increment field value when event occur in logstash output](https://discuss.elastic.co/t/increment-field-value-when-event-occur-in-logstash-output/252259)
**Author:** [@anja1](https://discuss.elastic.co/u/anja1)\ **Replies:** 0\ **Last updated:** [October 15, 2020, 7:16pm UTC](https://discuss.elastic.co/t/increment-field-value-when-event-occur-in-logstash-output/252259 "2020-10-15T19:16:16Z")
Is it possible to increment a field value everytime an event occur? Inputs are received from a filebeat and i want to store the values in elasticsearch I have following in my logstash pipeline: \> filter \> { \> … --- ## [Parse log using logstash and make ecs format](https://discuss.elastic.co/t/parse-log-using-logstash-and-make-ecs-format/251980)
**Author:** [@111387](https://discuss.elastic.co/u/111387)\ **Replies:** 4\ **Last updated:** [October 15, 2020, 6:36pm UTC](https://discuss.elastic.co/t/parse-log-using-logstash-and-make-ecs-format/251980 "2020-10-15T18:36:37Z")
i Receiving WAF(Web Application FireWall) log, Network Scan Result log file(xml) And try to send this log to ElasticSearch using logstash i want to parse this log according to ECS Format. but, "https://github.com/elas… --- ## [How to check Logstash logs appearing in Elasticsearch?](https://discuss.elastic.co/t/how-to-check-logstash-logs-appearing-in-elasticsearch/252174)
**Author:** [@deanwarrenuk](https://discuss.elastic.co/u/deanwarrenuk)\ **Replies:** 2\ **Last updated:** [October 15, 2020, 6:14pm UTC](https://discuss.elastic.co/t/how-to-check-logstash-logs-appearing-in-elasticsearch/252174 "2020-10-15T18:14:54Z")
Okay I am a noob I have ELK installed in Docker containers and am sending logs from a Python program using python-logstash. I can see Logstash receiving my logs when I do docker-compose logs . How do I check these are … --- ## [Logstash Unable to renew Kerberos ticket](https://discuss.elastic.co/t/logstash-unable-to-renew-kerberos-ticket/252241)
**Author:** [@burkeg](https://discuss.elastic.co/u/burkeg)\ **Replies:** 0\ **Last updated:** [October 15, 2020, 4:58pm UTC](https://discuss.elastic.co/t/logstash-unable-to-renew-kerberos-ticket/252241 "2020-10-15T16:58:16Z")
Hi, I'm trying to setup logstash agent with Kafka output that has Kerberos based authentication and uses SASL\_SSL mechanism. Logstash agent works fine when started but fails to renew Kerberos ticket. I've configured th… --- ## [File not readable (please check user and group permissions for the path)](https://discuss.elastic.co/t/file-not-readable-please-check-user-and-group-permissions-for-the-path/252130)
**Author:** [@BH\_BV](https://discuss.elastic.co/u/BH_BV)\ **Replies:** 1\ **Last updated:** [October 15, 2020, 2:43pm UTC](https://discuss.elastic.co/t/file-not-readable-please-check-user-and-group-permissions-for-the-path/252130 "2020-10-15T14:43:49Z")
im using docker-compose and suddenly logstash now unable to read the driver, but driver is in mounted folder inside the container with full read write permissions. whats possible the problem? logstash\_1 | Error: un… --- ## [Logstash elasticsearch input, size and schedule params](https://discuss.elastic.co/t/logstash-elasticsearch-input-size-and-schedule-params/252203)
**Author:** [@feichau](https://discuss.elastic.co/u/feichau)\ **Replies:** 0\ **Last updated:** [October 15, 2020, 1:19pm UTC](https://discuss.elastic.co/t/logstash-elasticsearch-input-size-and-schedule-params/252203 "2020-10-15T13:19:47Z")
I want to use elasticsearch input to query documents that is still missing some data and I didn't find how to limit the amount of documents returned by elasticsearch. The size param defines the amount that is returned ea… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=290) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=292)