# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=605 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 606 --- ## [Logstash Reached open files limit](https://discuss.elastic.co/t/logstash-reached-open-files-limit/167331)
**Author:** [@Rocky\_RK](https://discuss.elastic.co/u/Rocky_RK)\ **Replies:** 4\ **Last updated:** [February 6, 2019, 5:53pm UTC](https://discuss.elastic.co/t/logstash-reached-open-files-limit/167331 "2019-02-06T17:53:32Z")
I understand this question with the title has been asked many times before and almost every thread i has opened and learned here, but the doesn't come across any concrete answer so far specially for the Linux with syst… --- ## [Elapsed filter but problem in calculate the real diference](https://discuss.elastic.co/t/elapsed-filter-but-problem-in-calculate-the-real-diference/166760)
**Author:** [@bigster](https://discuss.elastic.co/u/bigster)\ **Replies:** 3\ **Last updated:** [February 6, 2019, 5:50pm UTC](https://discuss.elastic.co/t/elapsed-filter-but-problem-in-calculate-the-real-diference/166760 "2019-02-06T17:50:53Z")
Hi all, I´ve a elapsed filter but the calculation of the elapsed time is based on the timestamp value the occurs when the data enters the Elastic not the value i'm getting from the DB. I've seen a couple of posts sayin… --- ## [Using environment variable on logstash.yml](https://discuss.elastic.co/t/using-environment-variable-on-logstash-yml/167328)
**Author:** [@jvsell](https://discuss.elastic.co/u/jvsell)\ **Replies:** 0\ **Last updated:** [February 6, 2019, 4:29pm UTC](https://discuss.elastic.co/t/using-environment-variable-on-logstash-yml/167328 "2019-02-06T16:29:22Z")
Good afternoon everyone I'm trying to follow the documentation example - https://www.elastic.co/guide/en/logstash/6.5/logstash-settings-file.html - referring to the use of variables in logstash.yml more specifically in … --- ## [Logstash grok For GREEDYDATA Field](https://discuss.elastic.co/t/logstash-grok-for-greedydata-field/167215)
**Author:** [@Rocky\_RK](https://discuss.elastic.co/u/Rocky_RK)\ **Replies:** 2\ **Last updated:** [February 6, 2019, 3:41pm UTC](https://discuss.elastic.co/t/logstash-grok-for-greedydata-field/167215 "2019-02-06T15:41:09Z")
Hi Experts, I have a grok Filter thats working fine but now i have another challenge to split the GREEDYDATA data field by comm(,) that's generally a last field in the GREEDYDATA . Data after comma will be alike /proj… --- ## [Snmp for apc (american power conversion)](https://discuss.elastic.co/t/snmp-for-apc-american-power-conversion/167194)
**Author:** [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Replies:** 5\ **Last updated:** [February 6, 2019, 2:57pm UTC](https://discuss.elastic.co/t/snmp-for-apc-american-power-conversion/167194 "2019-02-06T14:57:05Z")
Hi I have never work with snmp and trying to get power information from apc unit. after a day of digging around I found it's OID. ran snmpwalk command to grab ton of information. how do I interprete this? what is all t… --- ## [Logstash link fileds in aggregate filter](https://discuss.elastic.co/t/logstash-link-fileds-in-aggregate-filter/167272)
**Author:** [@fedwe](https://discuss.elastic.co/u/fedwe)\ **Replies:** 1\ **Last updated:** [February 6, 2019, 2:00pm UTC](https://discuss.elastic.co/t/logstash-link-fileds-in-aggregate-filter/167272 "2019-02-06T14:00:31Z")
I have 2 files one that contains the filed number for example 961123456 and the other file cointains : country\_code 961 country name x can i use the aggregate filter to link the 2 fields to the number filed when indexi… --- ## [Multiple filebeat input in logstash but how to create different set of index for them](https://discuss.elastic.co/t/multiple-filebeat-input-in-logstash-but-how-to-create-different-set-of-index-for-them/166358)
**Author:** [@MiddlewareTeam](https://discuss.elastic.co/u/MiddlewareTeam)\ **Replies:** 5\ **Last updated:** [February 1, 2019, 1:52pm UTC](https://discuss.elastic.co/t/multiple-filebeat-input-in-logstash-but-how-to-create-different-set-of-index-for-them/166358 "2019-02-01T13:52:15Z")
I have two set of environment consider as A and B. I have configured A env in logstash.conf and its working as expected but now I have to add B in the same file but I have to create different index for the A and B. How t… --- ## [Conditional filtering not working in logstash](https://discuss.elastic.co/t/conditional-filtering-not-working-in-logstash/166913)
**Author:** [@MiddlewareTeam](https://discuss.elastic.co/u/MiddlewareTeam)\ **Replies:** 12\ **Last updated:** [February 6, 2019, 12:27pm UTC](https://discuss.elastic.co/t/conditional-filtering-not-working-in-logstash/166913 "2019-02-06T12:27:19Z")
We have below setup in filebeat based on which we are using filtering in logstash but it is not working as expected. Filebeat input: \< -type: log paths: /app/logpath fields: log\_type: apache\_access application: … --- ## [Jmx configuration in logstash docker container](https://discuss.elastic.co/t/jmx-configuration-in-logstash-docker-container/167283)
**Author:** [@azam](https://discuss.elastic.co/u/azam)\ **Replies:** 0\ **Last updated:** [February 6, 2019, 12:24pm UTC](https://discuss.elastic.co/t/jmx-configuration-in-logstash-docker-container/167283 "2019-02-06T12:24:54Z")
Hi I am trying to configure jmx on logstash docker container. My logstash.conf file looks like this: input { jmx { path =\> "/usr/share/logstash/jmx" polling\_frequency =\> 15 nb\_thread =\> 4 type =\> "jmx" } } My j… --- ## [Logstash Kinesis Input Role Arn](https://discuss.elastic.co/t/logstash-kinesis-input-role-arn/167279)
**Author:** [@autarchprinceps](https://discuss.elastic.co/u/autarchprinceps)\ **Replies:** 0\ **Last updated:** [February 6, 2019, 11:46am UTC](https://discuss.elastic.co/t/logstash-kinesis-input-role-arn/167279 "2019-02-06T11:46:17Z")
I added a role\_arn as per the documented configuration option, but it is still trying to use the default credentials. I checked and my version of the plugin 2.0.11 should include that ability. kinesis { id =\> "input.k… --- ## [Logstash ( ver. 6.5.3 ) keep writing logs in /var/log/messages](https://discuss.elastic.co/t/logstash-ver-6-5-3-keep-writing-logs-in-var-log-messages/167271)
**Author:** [@slapper15](https://discuss.elastic.co/u/slapper15)\ **Replies:** 0\ **Last updated:** [February 6, 2019, 11:06am UTC](https://discuss.elastic.co/t/logstash-ver-6-5-3-keep-writing-logs-in-var-log-messages/167271 "2019-02-06T11:06:16Z")
Hello! I find an issue in logstash regarding internal logging mechanism. For some reason logstash keep writing logs in journal , /var/log/messages & /var/log/logstash/logstash-plain.log (i am using centos 7 system ) . … --- ## [Logstash - how to flat json array with ruby filter?](https://discuss.elastic.co/t/logstash-how-to-flat-json-array-with-ruby-filter/165322)
**Author:** [@cheriemilk](https://discuss.elastic.co/u/cheriemilk)\ **Replies:** 10\ **Last updated:** [February 6, 2019, 10:42am UTC](https://discuss.elastic.co/t/logstash-how-to-flat-json-array-with-ruby-filter/165322 "2019-02-06T10:42:50Z")
I have a log file with json format, and there are json arrays in it. for example, a piece of array log is like below. I am using json+ruby fitler to make the array element parsed flatten. but my filter looks like doesn… --- ## [S3 input missing files](https://discuss.elastic.co/t/s3-input-missing-files/166724)
**Author:** [@JonS](https://discuss.elastic.co/u/JonS)\ **Replies:** 3\ **Last updated:** [February 6, 2019, 9:11am UTC](https://discuss.elastic.co/t/s3-input-missing-files/166724 "2019-02-06T09:11:43Z")
Hi, we are trying to set up logstash to read ELB logs stored in an S3 bucket. The logs are split over several folders inside the bucket, and we're having some difficultly configuring the input. When reading from a sing… --- ## [\[ERROR\] 2019-01-08 10:18:42.313 \[main\] Logstash - java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit](https://discuss.elastic.co/t/error-2019-01-08-1042-313-main-logstash-java-lang-illegalstateexception-logstash-stopped-processing-because-of-an-error-systemexit-exit/163302)
**Author:** [@Dv\_Thiyanesh](https://discuss.elastic.co/u/Dv_Thiyanesh)\ **Replies:** 11\ **Last updated:** [February 6, 2019, 6:00am UTC](https://discuss.elastic.co/t/error-2019-01-08-1042-313-main-logstash-java-lang-illegalstateexception-logstash-stopped-processing-because-of-an-error-systemexit-exit/163302 "2019-02-06T06:00:25Z")
I have an error at first i have run the config for beasts but now just for checking it's shows an error like this(in windows) what can i do did i make any mistake correct me C:\\ELK\\logstash-6.5.3\\bin\>logstash -e 'inpu… --- ## [Logs not getting to elasticsearch from logstash](https://discuss.elastic.co/t/logs-not-getting-to-elasticsearch-from-logstash/166050)
**Author:** [@CG3](https://discuss.elastic.co/u/CG3)\ **Replies:** 9\ **Last updated:** [February 5, 2019, 8:35pm UTC](https://discuss.elastic.co/t/logs-not-getting-to-elasticsearch-from-logstash/166050 "2019-02-05T20:35:46Z")
I have an elastic cluster with a logstash server as the first point of entry. In this ecosystem, I have filebeat running on several windows servers. They are sending logs to logstash, which then forwards them to elasti… --- ## [Ingesting data from API](https://discuss.elastic.co/t/ingesting-data-from-api/167192)
**Author:** [@imortalsolitude](https://discuss.elastic.co/u/imortalsolitude)\ **Replies:** 1\ **Last updated:** [February 5, 2019, 8:23pm UTC](https://discuss.elastic.co/t/ingesting-data-from-api/167192 "2019-02-05T20:23:44Z")
I am trying to ingest realtime stock data via an api provided by a third party provider how do i get this data in to ELK stack? --- ## [No results](https://discuss.elastic.co/t/no-results/166038)
**Author:** [@mario\_hvz](https://discuss.elastic.co/u/mario_hvz)\ **Replies:** 0\ **Last updated:** [January 28, 2019, 7:06pm UTC](https://discuss.elastic.co/t/no-results/166038 "2019-01-28T19:06:07Z")
Problem Solved! --- ## [Custom fields visible in stdout, but not propagated to ElasticSearch](https://discuss.elastic.co/t/custom-fields-visible-in-stdout-but-not-propagated-to-elasticsearch/167157)
**Author:** [@jcaballero](https://discuss.elastic.co/u/jcaballero)\ **Replies:** 1\ **Last updated:** [February 5, 2019, 8:14pm UTC](https://discuss.elastic.co/t/custom-fields-visible-in-stdout-but-not-propagated-to-elasticsearch/167157 "2019-02-05T20:14:34Z")
Hi ELK community. I am quite newby, so I apologize in advance if the question is too simple. I have a configuration like this: FileBeat (version 6.6.0) and LogStash (6.6.0) running on host A. For filtering, I use the… --- ## [Logstash translate dictionary\_path using http](https://discuss.elastic.co/t/logstash-translate-dictionary-path-using-http/167159)
**Author:** [@MarcDAFrame](https://discuss.elastic.co/u/MarcDAFrame)\ **Replies:** 4\ **Last updated:** [February 5, 2019, 7:44pm UTC](https://discuss.elastic.co/t/logstash-translate-dictionary-path-using-http/167159 "2019-02-05T19:44:32Z")
is there anyway to provide Logstash with a dictionary path over http? filter { ... translate { dictionary\_path =\> "http://0.0.0.0:8000/dictionary.yml" ... } } or what are the alternatives ? … --- ## [Parse log file with two formats in it](https://discuss.elastic.co/t/parse-log-file-with-two-formats-in-it/166876)
**Author:** [@Abhilash\_B](https://discuss.elastic.co/u/Abhilash_B)\ **Replies:** 5\ **Last updated:** [February 5, 2019, 7:05pm UTC](https://discuss.elastic.co/t/parse-log-file-with-two-formats-in-it/166876 "2019-02-05T19:05:11Z")
I have a log file with two formats in it. The metadata is more like a key value format where the delimiter is '=', the actual data is in a csv format. Both these formats have been bundled into a single file. Can I genera… --- ## [Logstash Syslog @timestamp incorrect](https://discuss.elastic.co/t/logstash-syslog-timestamp-incorrect/166658)
**Author:** [@BenB196](https://discuss.elastic.co/u/BenB196)\ **Replies:** 8\ **Last updated:** [February 5, 2019, 6:59pm UTC](https://discuss.elastic.co/t/logstash-syslog-timestamp-incorrect/166658 "2019-02-05T18:59:10Z")
Hi all, I am running into a bit of an issue with Logstash 6.6.0 and it processing Syslog messages. For some reason it is setting the @timestamp field to America/New\_York time thinking that it is UTC time (Kibana is also… --- ## [Can't import data to elasticsearch from csv, stops at "Successfully started Logstash API endpoint {:port=\>9600}"](https://discuss.elastic.co/t/cant-import-data-to-elasticsearch-from-csv-stops-at-successfully-started-logstash-api-endpoint-port-9600/167173)
**Author:** [@Pablo\_Yanez](https://discuss.elastic.co/u/Pablo_Yanez)\ **Replies:** 4\ **Last updated:** [February 5, 2019, 6:20pm UTC](https://discuss.elastic.co/t/cant-import-data-to-elasticsearch-from-csv-stops-at-successfully-started-logstash-api-endpoint-port-9600/167173 "2019-02-05T18:20:38Z")
I'm working on CentOS 7, this is my configuration file for Logstash: input { file { path =\> "/etc/logstash/files/Chile.csv" start\_position =\> "beginning" } } filter { csv { columns =\> \["ID","region","population… --- ## [Logstash Ports 5044, 9600 and 9700](https://discuss.elastic.co/t/logstash-ports-5044-9600-and-9700/165096)
**Author:** [@kaushalshriyan](https://discuss.elastic.co/u/kaushalshriyan)\ **Replies:** 2\ **Last updated:** [February 5, 2019, 5:27pm UTC](https://discuss.elastic.co/t/logstash-ports-5044-9600-and-9700/165096 "2019-02-05T17:27:49Z")
Hi, I am confused about Logstash Ports 5044, 9600 and 9700. Can somebody please explain to me about these ports what all process are mapped to it. I have two logstash servers which are set up behind AWS Load Balancer. W… --- ## [Address is in use?](https://discuss.elastic.co/t/address-is-in-use/167127)
**Author:** [@fribse](https://discuss.elastic.co/u/fribse)\ **Replies:** 0\ **Last updated:** [February 5, 2019, 1:22pm UTC](https://discuss.elastic.co/t/address-is-in-use/167127 "2019-02-05T13:22:07Z")
I have a setup with parallell logstash. The docker-compose file places the second (test) on port 5001, where the production is on 5000. When I look in logs I see these messages: ESC\[33mlogstashdebug |ESC\[0m \[2019-0… --- ## [Import addresses into Elasticsearch using Logstash](https://discuss.elastic.co/t/import-addresses-into-elasticsearch-using-logstash/167098)
**Author:** [@DaniSusB](https://discuss.elastic.co/u/DaniSusB)\ **Replies:** 4\ **Last updated:** [February 5, 2019, 12:41pm UTC](https://discuss.elastic.co/t/import-addresses-into-elasticsearch-using-logstash/167098 "2019-02-05T12:41:35Z")
Hi, I'd like to import addresses into ES using Logstash. The thing is, I'd like to filter out the addresses that already exist in the ES index but I can't find a ES filter plugin that allows me to first check if the ad… --- ## [New logstash 6.6 fail to start](https://discuss.elastic.co/t/new-logstash-6-6-fail-to-start/166854)
**Author:** [@yossiv](https://discuss.elastic.co/u/yossiv)\ **Replies:** 8\ **Last updated:** [February 5, 2019, 12:39pm UTC](https://discuss.elastic.co/t/new-logstash-6-6-fail-to-start/166854 "2019-02-05T12:39:28Z")
Hi Guys, i have new logstash installed. but for some reason it faile to start. 1 input and output ( the input and output works fine to my on other logstash system i have) plugin were installed already appreciate … --- ## [Ship jvm performance metrics using jmx from AIX to logstash running on Linux](https://discuss.elastic.co/t/ship-jvm-performance-metrics-using-jmx-from-aix-to-logstash-running-on-linux/167106)
**Author:** [@azam](https://discuss.elastic.co/u/azam)\ **Replies:** 0\ **Last updated:** [February 5, 2019, 11:24am UTC](https://discuss.elastic.co/t/ship-jvm-performance-metrics-using-jmx-from-aix-to-logstash-running-on-linux/167106 "2019-02-05T11:24:45Z")
Hey I am trying to ship my application (running on AIX server) performance metrics using JMX to logstash (running on Linux) instance. I followed a guide as exactly: It seems like there is no index pattern being crea… --- ## [Logstash grok for change field value](https://discuss.elastic.co/t/logstash-grok-for-change-field-value/167069)
**Author:** [@lukatoni](https://discuss.elastic.co/u/lukatoni)\ **Replies:** 6\ **Last updated:** [February 5, 2019, 12:21pm UTC](https://discuss.elastic.co/t/logstash-grok-for-change-field-value/167069 "2019-02-05T12:21:45Z")
Hi, I just wanna change my field value like this: I have a log json with field: "source\_host": "dev-host-test.com.pl", and i need to remove only .com.pl leave the old field and create new field with value: "new\_fiel… --- ## [Nxlog to Logstash problem](https://discuss.elastic.co/t/nxlog-to-logstash-problem/165896)
**Author:** [@Mayaya](https://discuss.elastic.co/u/Mayaya)\ **Replies:** 3\ **Last updated:** [February 5, 2019, 11:03am UTC](https://discuss.elastic.co/t/nxlog-to-logstash-problem/165896 "2019-02-05T11:03:42Z")
Hi, Please I am seeing logs of "No connection could be made because the target machine actively refused it.". The logstash pipeline is running. Any guidance is much appreciated. Thanks. My Logstash pipeline config is… --- ## [Logstash-output-s3 line termination](https://discuss.elastic.co/t/logstash-output-s3-line-termination/166946)
**Author:** [@raaone7](https://discuss.elastic.co/u/raaone7)\ **Replies:** 2\ **Last updated:** [February 5, 2019, 4:24am UTC](https://discuss.elastic.co/t/logstash-output-s3-line-termination/166946 "2019-02-05T04:24:56Z")
I am using the below config, problem with the config is , the events are written in the s3 file, without any line separator. making it hard to read and replay later if required. s3 { access\_key\_id =\> "xxx" secre… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=604) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=606)