# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=214 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 215 --- ## [How to use throttle filter to add tag to only one record](https://discuss.elastic.co/t/how-to-use-throttle-filter-to-add-tag-to-only-one-record/277640)
**Author:** [@venkat\_s](https://discuss.elastic.co/u/venkat_s)\ **Replies:** 0\ **Last updated:** [July 2, 2021, 11:13am UTC](https://discuss.elastic.co/t/how-to-use-throttle-filter-to-add-tag-to-only-one-record/277640 "2021-07-02T11:13:29Z")
Hi, I am using Logstash to process a csv file that contains more that one Lakh rows. I want to add a tag to only one record so that I can send email if any record contains this tag. I am trying to use throttle but its… --- ## [Logstash when started using nohup, it is logging too much resulting in huge size](https://discuss.elastic.co/t/logstash-when-started-using-nohup-it-is-logging-too-much-resulting-in-huge-size/276560)
**Author:** [@mastersmit](https://discuss.elastic.co/u/mastersmit)\ **Replies:** 8\ **Last updated:** [July 2, 2021, 4:50am UTC](https://discuss.elastic.co/t/logstash-when-started-using-nohup-it-is-logging-too-much-resulting-in-huge-size/276560 "2021-07-02T04:50:02Z")
we are running logstash and its output has file and elasticsearch for every opening and closing the file plugin is logging in... resulting the nohup.out file being very huge... How do i avoid so much of logging, or is t… --- ## [LogStash, GeoJSON and Kibana](https://discuss.elastic.co/t/logstash-geojson-and-kibana/277450)
**Author:** [@f4d0](https://discuss.elastic.co/u/f4d0)\ **Replies:** 4\ **Last updated:** [July 2, 2021, 2:24am UTC](https://discuss.elastic.co/t/logstash-geojson-and-kibana/277450 "2021-07-02T02:24:43Z")
I have successfully (I think) ingested logs using Logstash. Used the geoip filter and all the data looks good and well strucutred. When I index it in Kibana, the coordinates are set as float. This is what I can observe i… --- ## [Testing ruby filter for logstash](https://discuss.elastic.co/t/testing-ruby-filter-for-logstash/277599)
**Author:** [@blackberrySherbet](https://discuss.elastic.co/u/blackberrySherbet)\ **Replies:** 0\ **Last updated:** [July 1, 2021, 9:08pm UTC](https://discuss.elastic.co/t/testing-ruby-filter-for-logstash/277599 "2021-07-01T21:08:50Z")
How would you write a test to check if the code described in Indent any json file nested fields and make it flat (@Badger ) is working? I am having a hard time using the inline testing framework described in Ruby filter… --- ## [Using Field Data in Output Hosts?](https://discuss.elastic.co/t/using-field-data-in-output-hosts/277594)
**Author:** [@Dan\_L](https://discuss.elastic.co/u/Dan_L)\ **Replies:** 1\ **Last updated:** [July 1, 2021, 8:09pm UTC](https://discuss.elastic.co/t/using-field-data-in-output-hosts/277594 "2021-07-01T20:09:57Z")
Hello, I am trying to use the value of a field when I am defining hosts in logstash output. I've been able to do this in other logstash outputs like index. Is this possible? For example: output { elasticsearch { hosts … --- ## [Inrepolation logstash 7.11.1](https://discuss.elastic.co/t/inrepolation-logstash-7-11-1/277571)
**Author:** [@zakabluk](https://discuss.elastic.co/u/zakabluk)\ **Replies:** 1\ **Last updated:** [July 1, 2021, 4:28pm UTC](https://discuss.elastic.co/t/inrepolation-logstash-7-11-1/277571 "2021-07-01T16:28:09Z")
I haw been logstash version 6.X and my logstash output S3 worked. I use prefix at output: prefix =\> “dillinger-stat-message/%{+YYYY.MM.dd}/%{host\[name\]}/%{source}” when I migrate to logstash version 7.11.1(docker, file… --- ## [Function from filter block](https://discuss.elastic.co/t/function-from-filter-block/277547)
**Author:** [@Velly](https://discuss.elastic.co/u/Velly)\ **Replies:** 0\ **Last updated:** [July 1, 2021, 11:46am UTC](https://discuss.elastic.co/t/function-from-filter-block/277547 "2021-07-01T11:46:10Z")
Hello! I am a new user of ELK. I am trying to understand how works SIEM parser «1». In filter block I see: filter { load\_source\_mapper\_filters(mcsevt). Mcs is service of agent that picks up Windows logs. I don’t un… --- ## [Logstash problems with fields (Point to Point)](https://discuss.elastic.co/t/logstash-problems-with-fields-point-to-point/277196)
**Author:** [@Michael\_Dylan\_McAloo](https://discuss.elastic.co/u/Michael_Dylan_McAloo)\ **Replies:** 2\ **Last updated:** [July 1, 2021, 11:23am UTC](https://discuss.elastic.co/t/logstash-problems-with-fields-point-to-point/277196 "2021-07-01T11:23:29Z")
I have a problem, I have two IPs one source and one destination, for the source IP Logstash automatically creates a field called "geoip.location" that contains the latitude and longitude of the specific IP, but for the d… --- ## [Logstash not listening to 5044](https://discuss.elastic.co/t/logstash-not-listening-to-5044/277432)
**Author:** [@azid](https://discuss.elastic.co/u/azid)\ **Replies:** 2\ **Last updated:** [July 1, 2021, 4:21am UTC](https://discuss.elastic.co/t/logstash-not-listening-to-5044/277432 "2021-07-01T04:21:54Z")
Hi Im new to ELK, when I try to transfert data filebeat -\>logstash I got my logstash not listening to port 5044 any help would be appreciated Pipeline : input { beats { port =\> 5044 } } filter { gro… --- ## [Jdbc\_static prepared statement error](https://discuss.elastic.co/t/jdbc-static-prepared-statement-error/277507)
**Author:** [@metalshanked](https://discuss.elastic.co/u/metalshanked)\ **Replies:** 0\ **Last updated:** [July 1, 2021, 3:37am UTC](https://discuss.elastic.co/t/jdbc-static-prepared-statement-error/277507 "2021-07-01T03:37:51Z")
Hello, I have the below jdbc\_static lookup statement but it seems to fail with the error:- exception=\>#\,… --- ## [Update few Attributes in Nested Object of index using logstash](https://discuss.elastic.co/t/update-few-attributes-in-nested-object-of-index-using-logstash/277500)
**Author:** [@kamalgunda](https://discuss.elastic.co/u/kamalgunda)\ **Replies:** 0\ **Last updated:** [June 30, 2021, 8:49pm UTC](https://discuss.elastic.co/t/update-few-attributes-in-nested-object-of-index-using-logstash/277500 "2021-06-30T20:49:35Z")
Hi All, Good Day! I have data in elastic search index and want to update the nested object two attributes values out of 20 attributes. Input table has n number of records and whatever matches with document\_id , it sho… --- ## [Indexing Xml subfields as a new field](https://discuss.elastic.co/t/indexing-xml-subfields-as-a-new-field/277215)
**Author:** [@anon33720113](https://discuss.elastic.co/u/anon33720113)\ **Replies:** 11\ **Last updated:** [June 30, 2021, 5:01pm UTC](https://discuss.elastic.co/t/indexing-xml-subfields-as-a-new-field/277215 "2021-06-30T17:01:18Z")
Hi all, I have a xml-File with windows eventlogs that is structured like this: \ \ \...\ ... \ \ **Author:** [@Vijaykumar\_Gundavara](https://discuss.elastic.co/u/Vijaykumar_Gundavara)\ **Replies:** 0\ **Last updated:** [June 30, 2021, 4:53pm UTC](https://discuss.elastic.co/t/http-output-plugin-to-update-datastreams-entire-document/277487 "2021-06-30T16:53:24Z") Hi, MY use case is as follows. I have a document which is getting inserted into elasticsearch datastreams. The same document is getting updated multiple times in the application and I would like to use \_update\_query API… --- ## [Logstash Keystore While Running As A Service](https://discuss.elastic.co/t/logstash-keystore-while-running-as-a-service/277262)
**Author:** [@ankitdevnalkar](https://discuss.elastic.co/u/ankitdevnalkar)\ **Replies:** 5\ **Last updated:** [June 30, 2021, 4:15pm UTC](https://discuss.elastic.co/t/logstash-keystore-while-running-as-a-service/277262 "2021-06-30T16:15:16Z")
Hello Team, I have created Logstash Keystore with command sudo -E /usr/share/logstash/bin/logstash-keystore --path.settings /etc/logstash create but while running /usr/share/logstash/bin/logstash-keystore list it is thro… --- ## [Logstash (Unable to retrieve license information from license server {:message=\>"Unsupported or unrecognized SSL message"})](https://discuss.elastic.co/t/logstash-unable-to-retrieve-license-information-from-license-server-message-unsupported-or-unrecognized-ssl-message/277484)
**Author:** [@Ashwin\_Patil1](https://discuss.elastic.co/u/Ashwin_Patil1)\ **Replies:** 0\ **Last updated:** [June 30, 2021, 4:02pm UTC](https://discuss.elastic.co/t/logstash-unable-to-retrieve-license-information-from-license-server-message-unsupported-or-unrecognized-ssl-message/277484 "2021-06-30T16:02:49Z")
I am getting the following error, does anyone know what is happening? localhost:/etc/logstash # tail -f /var/log/logstash/logstash-plain.log \[2021-06-30T16:44:31,287\]\[ERROR\]\[logstash.licensechecker.licensereader\] Unabl… --- ## [Compare a collection/array of 2 numeric field values within same document](https://discuss.elastic.co/t/compare-a-collection-array-of-2-numeric-field-values-within-same-document/277461)
**Author:** [@Munazza](https://discuss.elastic.co/u/Munazza)\ **Replies:** 0\ **Last updated:** [June 30, 2021, 12:52pm UTC](https://discuss.elastic.co/t/compare-a-collection-array-of-2-numeric-field-values-within-same-document/277461 "2021-06-30T12:52:58Z")
I am trying to load sql server data to elasticsearch using logstash and have the watcher rule to alert if one field value is greater than the other field value within the same document. I am looking for a condition in … --- ## [Logstash compare field with quotedstring fails](https://discuss.elastic.co/t/logstash-compare-field-with-quotedstring-fails/277385)
**Author:** [@mtudisco](https://discuss.elastic.co/u/mtudisco)\ **Replies:** 2\ **Last updated:** [June 30, 2021, 12:32pm UTC](https://discuss.elastic.co/t/logstash-compare-field-with-quotedstring-fails/277385 "2021-06-30T12:32:17Z")
Hi, I'm parsing modsecurity messages from apache error log, one of the fields i get is ruleid that is parsed with a grok filter using the pattern QUOTEDSTRING. So in the field for instance i get a string with quotes: ""… --- ## [Grok Pattern for line - custom date](https://discuss.elastic.co/t/grok-pattern-for-line-custom-date/277412)
**Author:** [@Kamikaze\_K](https://discuss.elastic.co/u/Kamikaze_K)\ **Replies:** 1\ **Last updated:** [June 30, 2021, 12:27pm UTC](https://discuss.elastic.co/t/grok-pattern-for-line-custom-date/277412 "2021-06-30T12:27:12Z")
I am trying to use a grok pattern to get the fields from the following. any help with this especially with the custom date? 0E7 10JUN21 23:37:53.8 Track 1 EZ: 100 EX: 72 Speed: 36 mph I need to get the fields; "date" … --- ## [Parsing data presented in the form of a table](https://discuss.elastic.co/t/parsing-data-presented-in-the-form-of-a-table/273143)
**Author:** [@San9](https://discuss.elastic.co/u/San9)\ **Replies:** 8\ **Last updated:** [June 30, 2021, 9:58am UTC](https://discuss.elastic.co/t/parsing-data-presented-in-the-form-of-a-table/273143 "2021-06-30T09:58:12Z")
Hello team. More and more often you have to work with logos that come to logstash in the form of a table: Network Management Card AOS v3.7.3 Symmetra APP v1.1.2 Date Time Name Contact Location System IP 23.0… --- ## [If Else condition based on input log lines in logstash](https://discuss.elastic.co/t/if-else-condition-based-on-input-log-lines-in-logstash/277336)
**Author:** [@sudo-ranjith](https://discuss.elastic.co/u/sudo-ranjith)\ **Replies:** 2\ **Last updated:** [June 30, 2021, 8:22am UTC](https://discuss.elastic.co/t/if-else-condition-based-on-input-log-lines-in-logstash/277336 "2021-06-30T08:22:52Z")
Hi There, I would like to write a logstash config file with an if else condition. line 1: "Severity","ThreadID","Date","Time","Application","Message" GROK for Line 1: %{DATA:Severity}","%{DATA:ThreadID}","%{DATA:Dat… --- ## [Error: can’t merge a non object mapping with an object mapping](https://discuss.elastic.co/t/error-can-t-merge-a-non-object-mapping-with-an-object-mapping/276773)
**Author:** [@jofr](https://discuss.elastic.co/u/jofr)\ **Replies:** 1\ **Last updated:** [June 30, 2021, 7:45am UTC](https://discuss.elastic.co/t/error-can-t-merge-a-non-object-mapping-with-an-object-mapping/276773 "2021-06-30T07:45:50Z")
Somehow I can not define a geo\_point field type in the Elasticsearch mapping and then import data with Logstash. The details are described here in this Stackoverflow question. If I do not define an Elasticsearch mapping… --- ## [Gsub - Replace windows line terminators (\\r\\n) with unix (\\n)](https://discuss.elastic.co/t/gsub-replace-windows-line-terminators-r-n-with-unix-n/277394)
**Author:** [@metalshanked](https://discuss.elastic.co/u/metalshanked)\ **Replies:** 2\ **Last updated:** [June 30, 2021, 2:50am UTC](https://discuss.elastic.co/t/gsub-replace-windows-line-terminators-r-n-with-unix-n/277394 "2021-06-30T02:50:59Z")
Hi, I am trying to format a csv file so that it can be used in the translate filter. the csv file has only two columns (comma separated) with windows line ending format (CRLF) which somehow causes the translate filter… --- ## [Translate filter multiple values in same label ECS field](https://discuss.elastic.co/t/translate-filter-multiple-values-in-same-label-ecs-field/277391)
**Author:** [@metalshanked](https://discuss.elastic.co/u/metalshanked)\ **Replies:** 3\ **Last updated:** [June 30, 2021, 2:22am UTC](https://discuss.elastic.co/t/translate-filter-multiple-values-in-same-label-ecs-field/277391 "2021-06-30T02:22:31Z")
Hi, I am looking to populate the \[labels\]\[feed\] field (labels is the ECS field) with contents from two separate CSV fields as below. I am hoping to have it as an array. Is this a correct approach? If not, what would be… --- ## [Logstash stopped processing because of an error: (SystemExit) exit](https://discuss.elastic.co/t/logstash-stopped-processing-because-of-an-error-systemexit-exit/275744)
**Author:** [@its-ogawa](https://discuss.elastic.co/u/its-ogawa)\ **Replies:** 27\ **Last updated:** [June 30, 2021, 2:20am UTC](https://discuss.elastic.co/t/logstash-stopped-processing-because-of-an-error-systemexit-exit/275744 "2021-06-30T02:20:16Z")
Logstash has suddenly stopped collecting. If you look at the log, you will see the following message Logstash stopped processing because of an error: (SystemExit) exit What does this mean? I have checked that the con… --- ## [How to configure syslog (input) in logstash conf file](https://discuss.elastic.co/t/how-to-configure-syslog-input-in-logstash-conf-file/277349)
**Author:** [@MdRashid](https://discuss.elastic.co/u/MdRashid)\ **Replies:** 7\ **Last updated:** [June 29, 2021, 10:26pm UTC](https://discuss.elastic.co/t/how-to-configure-syslog-input-in-logstash-conf-file/277349 "2021-06-29T22:26:30Z")
Hi All, Im new to ELK i have config ELK in ubuntu as a docker container when i check kibana URL No Elasticsearch indices match your pattern. Note: - My VM is store in google cloud Here is my logstash.conf file in… --- ## [\[ERROR\] Ruby exception occurred: undefined method \`+' for nil:NilClass](https://discuss.elastic.co/t/error-ruby-exception-occurred-undefined-method-for-nil-nilclass/277377)
**Author:** [@Juan\_David\_Jaramillo](https://discuss.elastic.co/u/Juan_David_Jaramillo)\ **Replies:** 4\ **Last updated:** [June 29, 2021, 6:32pm UTC](https://discuss.elastic.co/t/error-ruby-exception-occurred-undefined-method-for-nil-nilclass/277377 "2021-06-29T18:32:35Z")
Hi everyone, I have a problem with a filter in ruby, I am doing an operation between 2 metrics, but I get the following error: \[ERROR\] 2021-06-29 14:02:24.847 \[\[main\]\>worker0\] ruby - Ruby exception occurred: undefined m… --- ## [Настройки маппинга](https://discuss.elastic.co/t/topic/277223)
**Author:** [@\_LA](https://discuss.elastic.co/u/_LA)\ **Replies:** 2\ **Last updated:** [June 29, 2021, 1:57pm UTC](https://discuss.elastic.co/t/topic/277223 "2021-06-29T13:57:58Z")
Подскажите пожалуйста, где настраивается маппинг для входящих json файлов? --- ## [Logstash Periodically Failing](https://discuss.elastic.co/t/logstash-periodically-failing/277353)
**Author:** [@jtillman2020](https://discuss.elastic.co/u/jtillman2020)\ **Replies:** 2\ **Last updated:** [June 29, 2021, 1:42pm UTC](https://discuss.elastic.co/t/logstash-periodically-failing/277353 "2021-06-29T13:42:33Z")
Every few days I notice my logstash stops ingesting data. When I check the log I find this: \[2021-06-29T03:30:39,236\]\[ERROR\]\[logstash.javapipeline \]\[main\] Pipeline worker error, the pipeline will be stopped {:pipeli… --- ## [How to send logs to multiple UDP destination using logstash from elasticsearch](https://discuss.elastic.co/t/how-to-send-logs-to-multiple-udp-destination-using-logstash-from-elasticsearch/277308)
**Author:** [@Vedagiri\_Balaji](https://discuss.elastic.co/u/Vedagiri_Balaji)\ **Replies:** 3\ **Last updated:** [June 29, 2021, 10:04am UTC](https://discuss.elastic.co/t/how-to-send-logs-to-multiple-udp-destination-using-logstash-from-elasticsearch/277308 "2021-06-29T10:04:20Z")
Hi , Currently we sending logs to a UDP destination, we intend send the same logs to multiple UDP destination ( logs are read from elasticsearch ) udp { id =\> "udp\_exporter" codec =\> plain { format =\> "%{message}" … --- ## [Problem parsing date with date filter](https://discuss.elastic.co/t/problem-parsing-date-with-date-filter/277329)
**Author:** [@Neelu\_Chandrasekhar](https://discuss.elastic.co/u/Neelu_Chandrasekhar)\ **Replies:** 0\ **Last updated:** [June 29, 2021, 9:47am UTC](https://discuss.elastic.co/t/problem-parsing-date-with-date-filter/277329 "2021-06-29T09:47:49Z")
I have a log like this {"log":"{\\"@timestamp\\":\\"2021-06-01T02:08:25.7387857-04:00\\",\\"level\\":\\"Information\\",\\"messageTemplate\\":\\"Got patients request\\",\\"message\\":\\"Got patients request I am trying to parse date w… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=213) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=215)