# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1053 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1054 --- ## [Logstash - adding new fields](https://discuss.elastic.co/t/logstash-adding-new-fields/71276)
**Author:** [@przecie](https://discuss.elastic.co/u/przecie)\ **Replies:** 2\ **Last updated:** [January 14, 2017, 6:05pm UTC](https://discuss.elastic.co/t/logstash-adding-new-fields/71276 "2017-01-14T18:05:10Z")
Hi, I'm new in Elastic, so my question can be one of the 'stupid' one, but i hope it will help me to understand the connection between Logstash and Elastic search. So... I followed the tutorial on this site: https:… --- ## [Grok extraction of field based on regex match on Index Name of event in question possible?](https://discuss.elastic.co/t/grok-extraction-of-field-based-on-regex-match-on-index-name-of-event-in-question-possible/71578)
**Author:** [@andynz2017](https://discuss.elastic.co/u/andynz2017)\ **Replies:** 2\ **Last updated:** [January 13, 2017, 11:15pm UTC](https://discuss.elastic.co/t/grok-extraction-of-field-based-on-regex-match-on-index-name-of-event-in-question-possible/71578 "2017-01-13T23:15:04Z")
This is a general enquiry. I have read multiple articles around how you can forcefully add a year to an existing date/timestamp that does not contain the year in the input message. What I'm looking to achieve is the e… --- ## [UDP Input getting broken into two entries](https://discuss.elastic.co/t/udp-input-getting-broken-into-two-entries/71572)
**Author:** [@DigiAngel](https://discuss.elastic.co/u/DigiAngel)\ **Replies:** 2\ **Last updated:** [January 13, 2017, 10:53pm UTC](https://discuss.elastic.co/t/udp-input-getting-broken-into-two-entries/71572 "2017-01-13T22:53:21Z")
Pretty much the topic. Here's the input: udp { type =\> "logon10" port =\> 7515 buffer\_size =\> 65536 receive\_buffer\_bytes =\> 65536 } However when … --- ## [Install logstash 5.1.2 from repository fails](https://discuss.elastic.co/t/install-logstash-5-1-2-from-repository-fails/71547)
**Author:** [@asuhan](https://discuss.elastic.co/u/asuhan)\ **Replies:** 3\ **Last updated:** [January 13, 2017, 5:56pm UTC](https://discuss.elastic.co/t/install-logstash-5-1-2-from-repository-fails/71547 "2017-01-13T17:56:00Z")
ubuntu 14.04 configured logstash repo: deb https://artifacts.elastic.co/packages/5.x/apt stable main apt-get install logstash Reading package lists... Done Building dependency tree Reading state information... Done The… --- ## [How do I use environment variables when running as service?](https://discuss.elastic.co/t/how-do-i-use-environment-variables-when-running-as-service/71392)
**Author:** [@astickler](https://discuss.elastic.co/u/astickler)\ **Replies:** 5\ **Last updated:** [January 13, 2017, 5:14pm UTC](https://discuss.elastic.co/t/how-do-i-use-environment-variables-when-running-as-service/71392 "2017-01-13T17:14:57Z")
I am using Logstash v5.1.0 on Ubuntu 16.04 and my logstash conf file includes an environment variable reference to set the output graphite server, but I cannot get it to work when running as a service. My config file … --- ## [Problem with storing coordinates as geo\_point](https://discuss.elastic.co/t/problem-with-storing-coordinates-as-geo-point/71548)
**Author:** [@oschlueter](https://discuss.elastic.co/u/oschlueter)\ **Replies:** 0\ **Last updated:** [January 13, 2017, 5:11pm UTC](https://discuss.elastic.co/t/problem-with-storing-coordinates-as-geo-point/71548 "2017-01-13T17:11:37Z")
Hello everyone, I'm facing an issue regarding storing coordinates as geo\_point. My goal is to add documents to ElasticSearch through a Logstash pipeline which I receive in JSON format. Omitting most of the fields the … --- ## [Is logstash-output-jdbc supported by ELASTIC?](https://discuss.elastic.co/t/is-logstash-output-jdbc-supported-by-elastic/71421)
**Author:** [@iti](https://discuss.elastic.co/u/iti)\ **Replies:** 2\ **Last updated:** [January 13, 2017, 5:10pm UTC](https://discuss.elastic.co/t/is-logstash-output-jdbc-supported-by-elastic/71421 "2017-01-13T17:10:58Z")
It is not under "supported output plugin", while logstash-input-jdbc is. Any reason for this? --- ## [Can logstash-file-plugin listen to "symlink"?](https://discuss.elastic.co/t/can-logstash-file-plugin-listen-to-symlink/2771)
**Author:** [@ohadR](https://discuss.elastic.co/u/ohadR)\ **Replies:** 3\ **Last updated:** [January 13, 2017, 4:34pm UTC](https://discuss.elastic.co/t/can-logstash-file-plugin-listen-to-symlink/2771 "2017-01-13T16:34:02Z")
hi I'm using the "file" as imput plugin, and I direct it to a path in unix machine. This path consists of links to log files that reside in other directories. Can it listen to "symlink"? --- ## [ERROR - Cloudwatch Output plugin - Direct event field references have been disabled](https://discuss.elastic.co/t/error-cloudwatch-output-plugin-direct-event-field-references-have-been-disabled/71231)
**Author:** [@CliveL](https://discuss.elastic.co/u/CliveL)\ **Replies:** 1\ **Last updated:** [January 13, 2017, 4:02pm UTC](https://discuss.elastic.co/t/error-cloudwatch-output-plugin-direct-event-field-references-have-been-disabled/71231 "2017-01-13T16:02:24Z")
Hi, I have been trying to use the Cloudwatch Output plugin in both Logstash 5.0.1 and 5.1.1 and when I set the Cloudwatch Output to DEBUG I see the following lines when it attempts to set a metric: \[2017-01-11T12:41:2… --- ## [Best practices for logstash/beat configs when porting rules to elastic from legacy NMS](https://discuss.elastic.co/t/best-practices-for-logstash-beat-configs-when-porting-rules-to-elastic-from-legacy-nms/71525)
**Author:** [@mattwisner](https://discuss.elastic.co/u/mattwisner)\ **Replies:** 0\ **Last updated:** [January 13, 2017, 1:41pm UTC](https://discuss.elastic.co/t/best-practices-for-logstash-beat-configs-when-porting-rules-to-elastic-from-legacy-nms/71525 "2017-01-13T13:41:07Z")
Hi, I'm brand new to elastic, and contemplating a POC for offloading to and/or full replacement of our legacy NMS systems. This could mean porting 100,000s or even millions of lines of legacy code to logstash filters. … --- ## [Is it possible to substitute the value of a field from env variable in logstash](https://discuss.elastic.co/t/is-it-possible-to-substitute-the-value-of-a-field-from-env-variable-in-logstash/71510)
**Author:** [@G.S](https://discuss.elastic.co/u/G.S)\ **Replies:** 1\ **Last updated:** [January 13, 2017, 12:31pm UTC](https://discuss.elastic.co/t/is-it-possible-to-substitute-the-value-of-a-field-from-env-variable-in-logstash/71510 "2017-01-13T12:31:45Z")
Hi, I am using logstash-2.3.4. In my case, input file contains the ID of the operation, which I need to translate it to its name in the output. For example: Input: 1,SUCCESS 2,SUCCESS 3,ERROR Let's assume the num… --- ## [OpenSSL::SSL::SSLError: Received fatal alert: unknown\_ca](https://discuss.elastic.co/t/openssl-ssl-sslerror-received-fatal-alert-unknown-ca/71518)
**Author:** [@benji](https://discuss.elastic.co/u/benji)\ **Replies:** 0\ **Last updated:** [January 13, 2017, 12:30pm UTC](https://discuss.elastic.co/t/openssl-ssl-sslerror-received-fatal-alert-unknown-ca/71518 "2017-01-13T12:30:46Z")
Hello all, First off - Thanks for LogStash :thumbsup: I am looking for help in solving the error (see title) from my /var/log/logstash/logstash-plain.log log (when debug enabled) I have an SSL cert from LetsEncrypt and h… --- ## [Problem with dynamic log update using http\_poller](https://discuss.elastic.co/t/problem-with-dynamic-log-update-using-http-poller/71482)
**Author:** [@karthikeyan](https://discuss.elastic.co/u/karthikeyan)\ **Replies:** 7\ **Last updated:** [January 13, 2017, 12:27pm UTC](https://discuss.elastic.co/t/problem-with-dynamic-log-update-using-http-poller/71482 "2017-01-13T12:27:07Z")
Hi , This is my configuration file input { http\_poller{ urls =\> { url129server =\> "http://xxx.log" } codec =\> "plain" metadata\_target =\> "http\_poller\_metadata" request\_timeout =\> 60 schedule =\> { "every… --- ## [Logstash keeps doing s3 input task but nerver send ouput events](https://discuss.elastic.co/t/logstash-keeps-doing-s3-input-task-but-nerver-send-ouput-events/71504)
**Author:** [@nixmind](https://discuss.elastic.co/u/nixmind)\ **Replies:** 0\ **Last updated:** [January 13, 2017, 10:13am UTC](https://discuss.elastic.co/t/logstash-keeps-doing-s3-input-task-but-nerver-send-ouput-events/71504 "2017-01-13T10:13:04Z")
Hi, I have issue with my logstash s3 iinput. The last messages I see in my kibana iinterface is from several days earlier In fact I have an AWS elb with logs enable. I've tested from command line and I can see that log… --- ## [Mainframe logs Plugin](https://discuss.elastic.co/t/mainframe-logs-plugin/71450)
**Author:** [@welcome2017](https://discuss.elastic.co/u/welcome2017)\ **Replies:** 1\ **Last updated:** [January 13, 2017, 10:12am UTC](https://discuss.elastic.co/t/mainframe-logs-plugin/71450 "2017-01-13T10:12:31Z")
Hello Guys - I am trying to understand if there are any mainframe plugins available yet for Logstash just like Splunk - Ironstream ? I am trying to evaluate options for logging a distributed eco system. Or is there any… --- ## [Example geoip config that works LS/ES 5.1](https://discuss.elastic.co/t/example-geoip-config-that-works-ls-es-5-1/71394)
**Author:** [@paka](https://discuss.elastic.co/u/paka)\ **Replies:** 3\ **Last updated:** [January 13, 2017, 9:39am UTC](https://discuss.elastic.co/t/example-geoip-config-that-works-ls-es-5-1/71394 "2017-01-13T09:39:04Z")
Hello, please does geoip work for someone in LS and ES 5.1, is there an example of configuration that works ? I'm fighting with it whole day, trying all various things found in many issues also others apparently hav… --- ## [FIlebeat-Redis-Logstash : Filebeat fast and Logstah slow, logstash threading?](https://discuss.elastic.co/t/filebeat-redis-logstash-filebeat-fast-and-logstah-slow-logstash-threading/71052)
**Author:** [@nixmind](https://discuss.elastic.co/u/nixmind)\ **Replies:** 18\ **Last updated:** [January 13, 2017, 9:38am UTC](https://discuss.elastic.co/t/filebeat-redis-logstash-filebeat-fast-and-logstah-slow-logstash-threading/71052 "2017-01-13T09:38:03Z")
Hi all, I'm facing a latency issue with logstash. In fact I have an ELK stack built like this : I have several web front on AWS EC2 in an AWS autoscaling group I have filebeat installed on each front filebeat read… --- ## [How do I use the logstash alter coalesce plugin?](https://discuss.elastic.co/t/how-do-i-use-the-logstash-alter-coalesce-plugin/71494)
**Author:** [@sliddjur](https://discuss.elastic.co/u/sliddjur)\ **Replies:** 0\ **Last updated:** [January 13, 2017, 9:12am UTC](https://discuss.elastic.co/t/how-do-i-use-the-logstash-alter-coalesce-plugin/71494 "2017-01-13T09:12:58Z")
Each of my event contains one type of src addr and one type of dst addr. These variations always produce errors. How do I properly use the coalesce filter in logstash? It isnt clear to me in examples. This is what I've… --- ## [Unable to set heap size via LS\_HEAP\_SIZE on logstash 5.1.1](https://discuss.elastic.co/t/unable-to-set-heap-size-via-ls-heap-size-on-logstash-5-1-1/71085)
**Author:** [@Marco\_Spadoni](https://discuss.elastic.co/u/Marco_Spadoni)\ **Replies:** 3\ **Last updated:** [January 13, 2017, 9:06am UTC](https://discuss.elastic.co/t/unable-to-set-heap-size-via-ls-heap-size-on-logstash-5-1-1/71085 "2017-01-13T09:06:36Z")
I am currently using logstash 5.1.1 on CentOS 7 sending it into execution from a bash shell. When using logstash 2.3.4 I was able to run several instances each one with a different maximum heap size, by making use of the… --- ## [How to get N chars in a string field?](https://discuss.elastic.co/t/how-to-get-n-chars-in-a-string-field/71481)
**Author:** [@Super\_L](https://discuss.elastic.co/u/Super_L)\ **Replies:** 9\ **Last updated:** [January 13, 2017, 7:34am UTC](https://discuss.elastic.co/t/how-to-get-n-chars-in-a-string-field/71481 "2017-01-13T07:34:16Z")
Hi all I need get 4 chars in a field as es type. My conf as follow: filter { if \[driveid\] =~ "^CENNAVIVOMS" { drop { } } else { ruby { code =\> " event\['@met… --- ## [\_grokparsefailure in Logstash?](https://discuss.elastic.co/t/-grokparsefailure-in-logstash/71466)
**Author:** [@Yaswanth](https://discuss.elastic.co/u/Yaswanth)\ **Replies:** 0\ **Last updated:** [January 13, 2017, 4:32am UTC](https://discuss.elastic.co/t/-grokparsefailure-in-logstash/71466 "2017-01-13T04:32:43Z")
Hi, Es 2.4.0 logstash 5.1.1 I have config file like this input { file { path =\> "F:\\logstash-5.1.1\\logstash-5.1.1\\bin\\slowlog.log" start\_position =\> "beginning" } } filter { grok { # parses the commo… --- ## [Not displaying fields specified in the Logstash filter](https://discuss.elastic.co/t/not-displaying-fields-specified-in-the-logstash-filter/42517)
**Author:** [@Sameer\_Dharur](https://discuss.elastic.co/u/Sameer_Dharur)\ **Replies:** 10\ **Last updated:** [January 13, 2017, 6:58am UTC](https://discuss.elastic.co/t/not-displaying-fields-specified-in-the-logstash-filter/42517 "2017-01-13T06:58:00Z")
Hey, I just created an index using Elasticsearch and Logstash but the fields I've specified using the Logstash filter (grok) aren't really reflecting in Kibana. Here's my filter - input { file { path =\> "C:/logstash… --- ## [How to reset / revert Kafka offset of a consumer group used by Logstash?](https://discuss.elastic.co/t/how-to-reset-revert-kafka-offset-of-a-consumer-group-used-by-logstash/71467)
**Author:** [@elasticheart](https://discuss.elastic.co/u/elasticheart)\ **Replies:** 0\ **Last updated:** [January 13, 2017, 5:27am UTC](https://discuss.elastic.co/t/how-to-reset-revert-kafka-offset-of-a-consumer-group-used-by-logstash/71467 "2017-01-13T05:27:23Z")
Hi, I am using Logstash GA 5.0.0 to read data from Kafka. For testing purposes, I need to re-run Logstash several times, and my Kafka topic has limited number of test messages. Everything works fine, but the problem is… --- ## [AWS S3 Logstash configuration with log.gz](https://discuss.elastic.co/t/aws-s3-logstash-configuration-with-log-gz/71465)
**Author:** [@shilong](https://discuss.elastic.co/u/shilong)\ **Replies:** 0\ **Last updated:** [January 13, 2017, 4:06am UTC](https://discuss.elastic.co/t/aws-s3-logstash-configuration-with-log-gz/71465 "2017-01-13T04:06:56Z")
Hi, is there anyone can help with this configuration on logstash? Please keep in mind it is a log.gz files. And it imports the data from the mPulse (A RUM data tool) every hour. The each piece of Json looks like this, … --- ## [I want to know how to avoid parsing error in foreign language](https://discuss.elastic.co/t/i-want-to-know-how-to-avoid-parsing-error-in-foreign-language/71170)
**Author:** [@111146](https://discuss.elastic.co/u/111146)\ **Replies:** 3\ **Last updated:** [January 13, 2017, 12:35am UTC](https://discuss.elastic.co/t/i-want-to-know-how-to-avoid-parsing-error-in-foreign-language/71170 "2017-01-13T00:35:08Z")
I have found that Vietnamese has a parsing error in logStash I want to know how to avoid errors. test.xml Tập 12 - Mười Tội Ác - The Ten Deadly Sins 2016 error message Trouble parsing xml with XmlSimple {:source=\>"m… --- ## [Need to split a field into several fields in a JSON object](https://discuss.elastic.co/t/need-to-split-a-field-into-several-fields-in-a-json-object/71418)
**Author:** [@Lahiru\_Lakshitha\_Sam](https://discuss.elastic.co/u/Lahiru_Lakshitha_Sam)\ **Replies:** 3\ **Last updated:** [January 12, 2017, 10:32pm UTC](https://discuss.elastic.co/t/need-to-split-a-field-into-several-fields-in-a-json-object/71418 "2017-01-12T22:32:19Z")
I need to split the "message": in to several fields in the same event and to remove some stuff in the "message": field Can I do this using split or mutate Need some help? - Version: Logstash 5.1.1 - Operating System: … --- ## [Long field causing pipeline exception](https://discuss.elastic.co/t/long-field-causing-pipeline-exception/71425)
**Author:** [@mhooper](https://discuss.elastic.co/u/mhooper)\ **Replies:** 0\ **Last updated:** [January 12, 2017, 7:19pm UTC](https://discuss.elastic.co/t/long-field-causing-pipeline-exception/71425 "2017-01-12T19:19:32Z")
Hello. When parsing error logs once in a while we get extremely long messages (1M). Obviously this breaks the 32K index limit so I was looking at solutions. I found a rangefilter plugin that would allow me accept a ran… --- ## [How to use CURL command as Logstash input](https://discuss.elastic.co/t/how-to-use-curl-command-as-logstash-input/71414)
**Author:** [@Dan](https://discuss.elastic.co/u/Dan)\ **Replies:** 1\ **Last updated:** [January 12, 2017, 6:27pm UTC](https://discuss.elastic.co/t/how-to-use-curl-command-as-logstash-input/71414 "2017-01-12T18:27:51Z")
CURL command is my input and Elastic search is my output plugin in logstash configuration file. How can give CURL as input plugin. I tried http input plugin, its not worked for me. Please suggest me which plugin i c… --- ## [Collate plugin installation fails](https://discuss.elastic.co/t/collate-plugin-installation-fails/71119)
**Author:** [@solasiva](https://discuss.elastic.co/u/solasiva)\ **Replies:** 1\ **Last updated:** [January 12, 2017, 6:14pm UTC](https://discuss.elastic.co/t/collate-plugin-installation-fails/71119 "2017-01-12T18:14:44Z")
Hi i am trying to install collate plugin with logstash version 5.1.1 on ubuntu 16.04 it gives me the following error can anyone please help me out with this oot@SSP-Desk:/usr/share/logstash/bin# ./logstash-plugi… --- ## [Keep relational data nested in one document](https://discuss.elastic.co/t/keep-relational-data-nested-in-one-document/71220)
**Author:** [@JohnG](https://discuss.elastic.co/u/JohnG)\ **Replies:** 1\ **Last updated:** [January 12, 2017, 6:14pm UTC](https://discuss.elastic.co/t/keep-relational-data-nested-in-one-document/71220 "2017-01-12T18:14:04Z")
Hi, Hope someone can help. I've looked all over Google and this forum without finding the answer. Lets say I have three tables - PRODUCT Columns - productid, productName (productId is unique) COLOUR Columns - c… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1052) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1054)