# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=247 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 248 --- ## [Logstash Json Parse Error](https://discuss.elastic.co/t/logstash-json-parse-error/267058)
**Author:** [@Anu6006](https://discuss.elastic.co/u/Anu6006)\ **Replies:** 2\ **Last updated:** [March 16, 2021, 4:30am UTC](https://discuss.elastic.co/t/logstash-json-parse-error/267058 "2021-03-16T04:30:36Z")
Hi Team, I'm trying to get NSG Flow logs through Azure Blob Storage plugin version: logstash-input-azureblob (0.9.13). logstash-codec-json (3.0.5) I am trying to save my Logstash output in JSON format. But it is sho… --- ## [How can I check event.field value from a file?](https://discuss.elastic.co/t/how-can-i-check-event-field-value-from-a-file/267292)
**Author:** [@Ishaan](https://discuss.elastic.co/u/Ishaan)\ **Replies:** 4\ **Last updated:** [March 16, 2021, 1:27am UTC](https://discuss.elastic.co/t/how-can-i-check-event-field-value-from-a-file/267292 "2021-03-16T01:27:19Z")
How can I check if the event.field\_name value is present in the file? If yes, drop the event otherwise send the event to elasticsearch. Note: The field\_name could have multiple values in that file, and I want to check i… --- ## [Nested fields in syslog output plugin](https://discuss.elastic.co/t/nested-fields-in-syslog-output-plugin/267268)
**Author:** [@dsv](https://discuss.elastic.co/u/dsv)\ **Replies:** 2\ **Last updated:** [March 15, 2021, 4:54pm UTC](https://discuss.elastic.co/t/nested-fields-in-syslog-output-plugin/267268 "2021-03-15T16:54:33Z")
Hey guys, i'm not able to send a nested field in the syslog output plugin. syslog { host =\> "1.7.3.9" rfc =\> "rfc5424" codec =\> cef … --- ## [Unable to ignore @timestamp in logs](https://discuss.elastic.co/t/unable-to-ignore-timestamp-in-logs/267141)
**Author:** [@Rakesh\_B](https://discuss.elastic.co/u/Rakesh_B)\ **Replies:** 6\ **Last updated:** [March 15, 2021, 4:22pm UTC](https://discuss.elastic.co/t/unable-to-ignore-timestamp-in-logs/267141 "2021-03-15T16:22:59Z")
Hi, We send multiple types of logs to Logstash and one of them uploaded JSON logs with an "@timestamp" field. Logstash tries to parse that field and when it fails it copies the value into "\_@timestamp" and throws a warn… --- ## [Error Parsing JSON data in Logstash](https://discuss.elastic.co/t/error-parsing-json-data-in-logstash/267249)
**Author:** [@atharvak](https://discuss.elastic.co/u/atharvak)\ **Replies:** 3\ **Last updated:** [March 15, 2021, 3:16pm UTC](https://discuss.elastic.co/t/error-parsing-json-data-in-logstash/267249 "2021-03-15T15:16:18Z")
Hello guys, I am trying to debug this issue showing up related to Json Parsing. When my filebeat sends data to logstash, following error is showing up. My Logstash Filter filter { if "cost\_management" in \[tags\] { f… --- ## [Logstash avro codec multiple hostnames under schema\_uri](https://discuss.elastic.co/t/logstash-avro-codec-multiple-hostnames-under-schema-uri/267278)
**Author:** [@Sebastian\_Rodak](https://discuss.elastic.co/u/Sebastian_Rodak)\ **Replies:** 0\ **Last updated:** [March 15, 2021, 2:42pm UTC](https://discuss.elastic.co/t/logstash-avro-codec-multiple-hostnames-under-schema-uri/267278 "2021-03-15T14:42:26Z")
Hi, We are checking avro schema for the required information / format of the event. I'd like to know because I couldn't found how to setup multiple schema\_uri for avro schema e.g output{ codec =\> avro { schema\_uri… --- ## [How to format number in logstash?](https://discuss.elastic.co/t/how-to-format-number-in-logstash/267182)
**Author:** [@padamrai](https://discuss.elastic.co/u/padamrai)\ **Replies:** 3\ **Last updated:** [March 15, 2021, 2:21pm UTC](https://discuss.elastic.co/t/how-to-format-number-in-logstash/267182 "2021-03-15T14:21:30Z")
Suppose I want to sum field its sum range 1 to 10000 then shows like 10k, sum range from 10000 to million it shows in million and sum range more than millions then it shows in billions. Is it possible to perform such … --- ## [Add a field starting from a substring of the file name](https://discuss.elastic.co/t/add-a-field-starting-from-a-substring-of-the-file-name/266337)
**Author:** [@JuanLuis\_Santiago\_de](https://discuss.elastic.co/u/JuanLuis_Santiago_de)\ **Replies:** 2\ **Last updated:** [March 15, 2021, 2:10pm UTC](https://discuss.elastic.co/t/add-a-field-starting-from-a-substring-of-the-file-name/266337 "2021-03-15T14:10:50Z")
Hello. I have several logs with the following names, where \[E-1\].\[P-28\], \[E-1\].\[P-45\] and \[E-1\].\[P-51\] are operators that generate these logs (I only can identify them by obtaining from the file name) p2sajava131.srv.g… --- ## [IP reputation ELK integration](https://discuss.elastic.co/t/ip-reputation-elk-integration/265959)
**Author:** [@tarekilani](https://discuss.elastic.co/u/tarekilani)\ **Replies:** 2\ **Last updated:** [March 15, 2021, 11:57am UTC](https://discuss.elastic.co/t/ip-reputation-elk-integration/265959 "2021-03-15T11:57:21Z")
Hello, Please i'm trying to integrate abuseipdb API to elasticsearch so i can have a reputation score of the IP adressess. When i searched on the internet i only found tutorials talking about integrating abuseipdb with … --- ## [Implementing utf-8-sig in Logstash conf file](https://discuss.elastic.co/t/implementing-utf-8-sig-in-logstash-conf-file/267251)
**Author:** [@shafeeka](https://discuss.elastic.co/u/shafeeka)\ **Replies:** 0\ **Last updated:** [March 15, 2021, 11:45am UTC](https://discuss.elastic.co/t/implementing-utf-8-sig-in-logstash-conf-file/267251 "2021-03-15T11:45:50Z")
Hi everyone, I was trying to implement skip\_header within my Csv filter plugin. However, I'm unsuccessful as my first column's header has this invisible \\ufeff character and hence it does not match the name I specified i… --- ## [Logstash S3 input plugin prefix with full path is ignored](https://discuss.elastic.co/t/logstash-s3-input-plugin-prefix-with-full-path-is-ignored/267236)
**Author:** [@noako](https://discuss.elastic.co/u/noako)\ **Replies:** 0\ **Last updated:** [March 15, 2021, 9:17am UTC](https://discuss.elastic.co/t/logstash-s3-input-plugin-prefix-with-full-path-is-ignored/267236 "2021-03-15T09:17:48Z")
I am using logstash docker image with s3 input plugin for input, the issue I am having is: if I set the prefix to the full file path, then the file is ignored, with this log: 2021-03-14T11:43:56,233\]\[DEBUG\]\[logstash.i… --- ## [How can resume logstash pipeline after terminated or crashed?](https://discuss.elastic.co/t/how-can-resume-logstash-pipeline-after-terminated-or-crashed/267029)
**Author:** [@Sage](https://discuss.elastic.co/u/Sage)\ **Replies:** 2\ **Last updated:** [March 15, 2021, 7:22am UTC](https://discuss.elastic.co/t/how-can-resume-logstash-pipeline-after-terminated-or-crashed/267029 "2021-03-15T07:22:35Z")
I designed a pipeline with elasticsearch as a input plugin and s3 as a output plugin. In my case, pipeline started successfully and push the data on s3 bucket but in case of termination or crash it does not start from wh… --- ## [ES 7.7.0 / LS 7.7.0 - Logstash not writing to a green index when some data nodes are down](https://discuss.elastic.co/t/es-7-7-0-ls-7-7-0-logstash-not-writing-to-a-green-index-when-some-data-nodes-are-down/267205)
**Author:** [@ld\_pvl](https://discuss.elastic.co/u/ld_pvl)\ **Replies:** 0\ **Last updated:** [March 14, 2021, 11:49pm UTC](https://discuss.elastic.co/t/es-7-7-0-ls-7-7-0-logstash-not-writing-to-a-green-index-when-some-data-nodes-are-down/267205 "2021-03-14T23:49:55Z")
Hi Team, I had a strange problem the other day and could not figure out what was happening. In my cluster, about 30% of my data nodes were brought down (intentionally). The cluster became red as expected but still had … --- ## [Csv to float from logstash to elasticsearch](https://discuss.elastic.co/t/csv-to-float-from-logstash-to-elasticsearch/267105)
**Author:** [@ofitz](https://discuss.elastic.co/u/ofitz)\ **Replies:** 4\ **Last updated:** [March 14, 2021, 3:48pm UTC](https://discuss.elastic.co/t/csv-to-float-from-logstash-to-elasticsearch/267105 "2021-03-14T15:48:39Z")
I have some problems to read CSV and convert the values as float in Array My Demo Dataset: 215900;216100;216300;216430; 216750;216950;217230;217500; 217780;217930;218100;218250; My logstash Config: input { file { … --- ## [Filebeat error](https://discuss.elastic.co/t/filebeat-error/267193)
**Author:** [@Boris\_Becker](https://discuss.elastic.co/u/Boris_Becker)\ **Replies:** 0\ **Last updated:** [March 14, 2021, 3:21pm UTC](https://discuss.elastic.co/t/filebeat-error/267193 "2021-03-14T15:21:15Z")
Do filebeat connected to logstash if it got the error (how it can be fixed?): 2021-03-14T15:06:08.306Z ERROR \[publisher\_pipeline\_output\] pipeline/output.go:154 Failed to connect to backoff(elasticsearch(https://vpc-au… --- ## [Help with equation in ruby](https://discuss.elastic.co/t/help-with-equation-in-ruby/267172)
**Author:** [@Sketchy](https://discuss.elastic.co/u/Sketchy)\ **Replies:** 1\ **Last updated:** [March 14, 2021, 2:43pm UTC](https://discuss.elastic.co/t/help-with-equation-in-ruby/267172 "2021-03-14T14:43:14Z")
I'm doing some calculations in ruby and works well when doing basic stuff like subtracting a from b but I have something a little more complex that I just cant get the syntax right or work out the most efficient way to d… --- ## [How to split array json into multiple fields](https://discuss.elastic.co/t/how-to-split-array-json-into-multiple-fields/267112)
**Author:** [@prabhakar\_talari](https://discuss.elastic.co/u/prabhakar_talari)\ **Replies:** 3\ **Last updated:** [March 13, 2021, 2:39pm UTC](https://discuss.elastic.co/t/how-to-split-array-json-into-multiple-fields/267112 "2021-03-13T14:39:27Z")
Hi All, I have a array json which am getting from http end point "summary": { "result": \[ { "status": { "offlineCount": 0, "warningCount": 0, "onlineCount": 1… --- ## [Processing old newsgroup messages](https://discuss.elastic.co/t/processing-old-newsgroup-messages/267147)
**Author:** [@Jack\_Judge](https://discuss.elastic.co/u/Jack_Judge)\ **Replies:** 0\ **Last updated:** [March 13, 2021, 11:33am UTC](https://discuss.elastic.co/t/processing-old-newsgroup-messages/267147 "2021-03-13T11:33:15Z")
Hi, I'm working on a project to index millions of newsgroups messages dating from the early 80s to the present day. The messages are supplied as individual files. I'm using logstash in "read" mode with the multiline code… --- ## [Configuring logstash for multiple input (port and log file path)](https://discuss.elastic.co/t/configuring-logstash-for-multiple-input-port-and-log-file-path/267140)
**Author:** [@asan](https://discuss.elastic.co/u/asan)\ **Replies:** 0\ **Last updated:** [March 13, 2021, 6:03am UTC](https://discuss.elastic.co/t/configuring-logstash-for-multiple-input-port-and-log-file-path/267140 "2021-03-13T06:03:13Z")
Hi I've Configured three of my network devices to write their logs in separated file in /var/log and also i configured my Logstash to read these file as input and it's working well and i could create index for them and … --- ## [Connecting to logstash with Basic Authentication](https://discuss.elastic.co/t/connecting-to-logstash-with-basic-authentication/267121)
**Author:** [@alicango](https://discuss.elastic.co/u/alicango)\ **Replies:** 3\ **Last updated:** [March 12, 2021, 8:28pm UTC](https://discuss.elastic.co/t/connecting-to-logstash-with-basic-authentication/267121 "2021-03-12T20:28:01Z")
Hello, I am using HTTP input plugin in logstash config. I have been always sending data to logstash over HTTP POST method by using logstash API like this; curl -XPOST http://\:9605 -d '{"my": "data"}' also … --- ## [Event.get(message).bytesize](https://discuss.elastic.co/t/event-get-message-bytesize/267004)
**Author:** [@jchaves506](https://discuss.elastic.co/u/jchaves506)\ **Replies:** 3\ **Last updated:** [March 12, 2021, 6:07pm UTC](https://discuss.elastic.co/t/event-get-message-bytesize/267004 "2021-03-12T18:07:31Z")
Continuing the discussion from How to get entire enriched "event" size (not message size): I'm using metricbeat, and I'm looking to know what's the size of the message, I tried using that but I'm getting: \[ERROR\]\[logst… --- ## [How can I drop the domain from host.name field in logstash?](https://discuss.elastic.co/t/how-can-i-drop-the-domain-from-host-name-field-in-logstash/267031)
**Author:** [@Ishaan](https://discuss.elastic.co/u/Ishaan)\ **Replies:** 3\ **Last updated:** [March 12, 2021, 4:03pm UTC](https://discuss.elastic.co/t/how-can-i-drop-the-domain-from-host-name-field-in-logstash/267031 "2021-03-12T16:03:48Z")
I want to remove domain from host.name field coming from metricbeat. For example: host.name = abcd.xyz.com host.name = abc@xyz.com I want to remove everything after . or @ or any other delimiter to save the value of … --- ## [Capitalize Every Word In String](https://discuss.elastic.co/t/capitalize-every-word-in-string/266997)
**Author:** [@wwalker](https://discuss.elastic.co/u/wwalker)\ **Replies:** 8\ **Last updated:** [March 12, 2021, 3:56pm UTC](https://discuss.elastic.co/t/capitalize-every-word-in-string/266997 "2021-03-12T15:56:27Z")
Trying to capitalize each word in a fields value. I initially tried the titleize function of Ruby, but that didn't work...I guess because it's not part of Ruby internals or something like that? ruby { code =\> 'ev… --- ## [How do I properly configure input to Logstash for a rewritable file?](https://discuss.elastic.co/t/how-do-i-properly-configure-input-to-logstash-for-a-rewritable-file/267089)
**Author:** [@Andrew\_Foxis](https://discuss.elastic.co/u/Andrew_Foxis)\ **Replies:** 1\ **Last updated:** [March 12, 2021, 3:54pm UTC](https://discuss.elastic.co/t/how-do-i-properly-configure-input-to-logstash-for-a-rewritable-file/267089 "2021-03-12T15:54:32Z")
Hi all! I have csv file, It is overwritten every 5 minutes. I have already configured input, filter,output. But in the current configuration I only get new lines from the file in the index. But I need to have all the… --- ## [Logstash input with filtered output](https://discuss.elastic.co/t/logstash-input-with-filtered-output/266951)
**Author:** [@djehuty](https://discuss.elastic.co/u/djehuty)\ **Replies:** 8\ **Last updated:** [March 12, 2021, 3:48pm UTC](https://discuss.elastic.co/t/logstash-input-with-filtered-output/266951 "2021-03-12T15:48:18Z")
I need to divide Winlogbeat input and push it to two separate indices when a beat has a particular event ID. I used an if statement but the indices seems to have the same data. This is my pipeline configuration: i… --- ## [If(filter,output) Else(filter,output)](https://discuss.elastic.co/t/if-filter-output-else-filter-output/267097)
**Author:** [@aannee](https://discuss.elastic.co/u/aannee)\ **Replies:** 3\ **Last updated:** [March 12, 2021, 3:25pm UTC](https://discuss.elastic.co/t/if-filter-output-else-filter-output/267097 "2021-03-12T15:25:47Z")
Hi Elastic Team, Is this possible in logstash? input { } if "test" in \[mesasge\] { filter {} output {} } else { filter {} output {} } Thank you! --- ## [Logstash-tcp-input having unrecoverable error when logstash is used in systemd](https://discuss.elastic.co/t/logstash-tcp-input-having-unrecoverable-error-when-logstash-is-used-in-systemd/267079)
**Author:** [@Arvind\_Ks](https://discuss.elastic.co/u/Arvind_Ks)\ **Replies:** 0\ **Last updated:** [March 12, 2021, 11:53am UTC](https://discuss.elastic.co/t/logstash-tcp-input-having-unrecoverable-error-when-logstash-is-used-in-systemd/267079 "2021-03-12T11:53:53Z")
Hi Team, I am trying to use tcp plugin in logstash. When i run the config file related to this using logstash cli command , i am able to start it without any issues. However when i try starting it as a systemd service i… --- ## [Docker logstash in ECS receives SIGTERM after ~5 min](https://discuss.elastic.co/t/docker-logstash-in-ecs-receives-sigterm-after-5-min/267012)
**Author:** [@tiberiu89](https://discuss.elastic.co/u/tiberiu89)\ **Replies:** 3\ **Last updated:** [March 12, 2021, 10:06am UTC](https://discuss.elastic.co/t/docker-logstash-in-ecs-receives-sigterm-after-5-min/267012 "2021-03-12T10:06:06Z")
Well, the title says it all, I have logstash container deployed in ECS, with a hard memory limit if 1.5GB, pipeline config is already tested locally so I'm fairly sure that's not the problem. I'm sending container logs … --- ## [Logstash Filter](https://discuss.elastic.co/t/logstash-filter/266182)
**Author:** [@Anu6006](https://discuss.elastic.co/u/Anu6006)\ **Replies:** 2\ **Last updated:** [March 12, 2021, 8:21am UTC](https://discuss.elastic.co/t/logstash-filter/266182 "2021-03-12T08:21:31Z")
Hi Team, I'm trying to get NSG Flow logs through Azure Blob Storage (logstash-input-azureblob (0.9.12)). But the filtering part is not working properly and showing the errors \_jsonparsefailure, \_split\_type\_failure, \_da… --- ## [Unable to index data as casesensitive using jdbc input in logstash](https://discuss.elastic.co/t/unable-to-index-data-as-casesensitive-using-jdbc-input-in-logstash/266820)
**Author:** [@Sammeta\_David\_Raju](https://discuss.elastic.co/u/Sammeta_David_Raju)\ **Replies:** 2\ **Last updated:** [March 12, 2021, 8:10am UTC](https://discuss.elastic.co/t/unable-to-index-data-as-casesensitive-using-jdbc-input-in-logstash/266820 "2021-03-12T08:10:51Z")
\`Unable to index data as case-sensitive using jdbc input in logstash\` even after using lowercase\_column\_names =\> false [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=246) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=248)