# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1051 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1052 --- ## [Metrics to graphite output not flushing at proper intervals when also using ES output](https://discuss.elastic.co/t/metrics-to-graphite-output-not-flushing-at-proper-intervals-when-also-using-es-output/72104)
**Author:** [@dai](https://discuss.elastic.co/u/dai)\ **Replies:** 0\ **Last updated:** [January 19, 2017, 1:10am UTC](https://discuss.elastic.co/t/metrics-to-graphite-output-not-flushing-at-proper-intervals-when-also-using-es-output/72104 "2017-01-19T01:10:32Z")
I'm trying to use logstash to tail a log file, parse it, and both measure the stats on how many times a particular type of line shows up per minute and send that stat to graphite, and also send all parsed lines to elasti… --- ## [New pattern won't work cisco vpn](https://discuss.elastic.co/t/new-pattern-wont-work-cisco-vpn/72054)
**Author:** [@tomi](https://discuss.elastic.co/u/tomi)\ **Replies:** 2\ **Last updated:** [January 18, 2017, 8:35pm UTC](https://discuss.elastic.co/t/new-pattern-wont-work-cisco-vpn/72054 "2017-01-18T20:35:24Z")
I have an ASA that record some vpn logs. those aren't included in the core-patterns. so I defined a simple patterns its seem to be working fine in testing envi which is running 5.1.1 logstash. but when moving the same c… --- ## [No 'Access-Control-Allow-Origin- http plugin logstash](https://discuss.elastic.co/t/no-access-control-allow-origin-http-plugin-logstash/69453)
**Author:** [@adhikz](https://discuss.elastic.co/u/adhikz)\ **Replies:** 2\ **Last updated:** [January 18, 2017, 3:04pm UTC](https://discuss.elastic.co/t/no-access-control-allow-origin-http-plugin-logstash/69453 "2017-01-18T15:04:54Z")
I am current trying to do a JavaScript post to Logstash by using a tcp input. JavaScript Post xhr = new XMLHttpRequest(); var url = "http://localhost:5043"; xhr.open("POST", url, true); xhr.setRequestHeader("Cont… --- ## [5.1.2 : logstash-input-zeromq](https://discuss.elastic.co/t/5-1-2-logstash-input-zeromq/72022)
**Author:** [@dawiro](https://discuss.elastic.co/u/dawiro)\ **Replies:** 0\ **Last updated:** [January 18, 2017, 2:19pm UTC](https://discuss.elastic.co/t/5-1-2-logstash-input-zeromq/72022 "2017-01-18T14:19:16Z")
Hi, I'm having issues with installing this plugin. Here's the output from the install attempt. Many of the conflicts were duplicated and so have been removed for brevity: Validating logstash-input-zeromq Installing log… --- ## [Logstash output: dynamically multiple indexes](https://discuss.elastic.co/t/logstash-output-dynamically-multiple-indexes/72008)
**Author:** [@asp](https://discuss.elastic.co/u/asp)\ **Replies:** 1\ **Last updated:** [January 18, 2017, 1:52pm UTC](https://discuss.elastic.co/t/logstash-output-dynamically-multiple-indexes/72008 "2017-01-18T13:52:55Z")
Hi, we have logs of our production, qa and dev systems in the same elasticsearch database. Currently we have one index for all, and I want to separate it, with the target: - higher performance - lower storage usage - … --- ## [Java heap space error when running Logstash](https://discuss.elastic.co/t/java-heap-space-error-when-running-logstash/71978)
**Author:** [@refai2017](https://discuss.elastic.co/u/refai2017)\ **Replies:** 2\ **Last updated:** [January 18, 2017, 12:10pm UTC](https://discuss.elastic.co/t/java-heap-space-error-when-running-logstash/71978 "2017-01-18T12:10:46Z")
Hi Everyone ,, I am very new to the Elastic Stack and I hope to get your help in the following problem: I am facing the error "Java heap space" when running the Logstash, the config file contains 2 indices (one selec… --- ## [Please delete](https://discuss.elastic.co/t/please-delete/71825)
**Author:** [@Abdullah\_Alaiwat](https://discuss.elastic.co/u/Abdullah_Alaiwat)\ **Replies:** 1\ **Last updated:** [January 18, 2017, 12:10pm UTC](https://discuss.elastic.co/t/please-delete/71825 "2017-01-18T12:10:56Z")
PLEASE DELETE the question --- ## [How to set Logstash to parse logfile as soon as logfile has new data](https://discuss.elastic.co/t/how-to-set-logstash-to-parse-logfile-as-soon-as-logfile-has-new-data/71845)
**Author:** [@sezanawa](https://discuss.elastic.co/u/sezanawa)\ **Replies:** 1\ **Last updated:** [January 18, 2017, 10:39am UTC](https://discuss.elastic.co/t/how-to-set-logstash-to-parse-logfile-as-soon-as-logfile-has-new-data/71845 "2017-01-18T10:39:46Z")
Hallo Everybody I have successfully configured ELK on my local machine. when i run following command, it start sending data to ES for indexing. logstash -f.. \\conf\\Onsurance.conf --path.settings=file://C:/ELK/logsta… --- ## [Indexing Tweets with logstash elasticsearch and kibana](https://discuss.elastic.co/t/indexing-tweets-with-logstash-elasticsearch-and-kibana/71916)
**Author:** [@amilus](https://discuss.elastic.co/u/amilus)\ **Replies:** 2\ **Last updated:** [January 18, 2017, 10:04am UTC](https://discuss.elastic.co/t/indexing-tweets-with-logstash-elasticsearch-and-kibana/71916 "2017-01-18T10:04:01Z")
Hello, i'm trying to index tweets, importing them with logstash into elasticsearch but i'm getting an error that i couldn't resolve this is my twitter.conf file input { twitter { # add your data consumer\_key =\> "xx… --- ## [Sending files to logstash with Ruby](https://discuss.elastic.co/t/sending-files-to-logstash-with-ruby/71882)
**Author:** [@sjc07](https://discuss.elastic.co/u/sjc07)\ **Replies:** 11\ **Last updated:** [January 18, 2017, 9:50am UTC](https://discuss.elastic.co/t/sending-files-to-logstash-with-ruby/71882 "2017-01-18T09:50:46Z")
I am working on some code to send a number of files from a series of folders to logstash to be filtered and then added to elasticsearch. I have some Ruby code that identifies the log file type and I then want to send it … --- ## [I would like to access nested data from an array](https://discuss.elastic.co/t/i-would-like-to-access-nested-data-from-an-array/71432)
**Author:** [@mathias](https://discuss.elastic.co/u/mathias)\ **Replies:** 6\ **Last updated:** [January 18, 2017, 7:21am UTC](https://discuss.elastic.co/t/i-would-like-to-access-nested-data-from-an-array/71432 "2017-01-18T07:21:57Z")
I have a data structure looking like this: printed as codec =\> rubydebug { "sequence\_number" =\> 557, "event\_tags" =\> \[ \[0\] { "value" =\> "nacl-st-qkrrfq-ppu-server-6y35vxntjont", … --- ## [Remove Time Zone from Logstash output](https://discuss.elastic.co/t/remove-time-zone-from-logstash-output/71892)
**Author:** [@rkhapre](https://discuss.elastic.co/u/rkhapre)\ **Replies:** 4\ **Last updated:** [January 18, 2017, 7:15am UTC](https://discuss.elastic.co/t/remove-time-zone-from-logstash-output/71892 "2017-01-18T07:15:43Z")
I am trying following to remove the 'Z' from logstash output but its not working Do we have any working solution here In my source the date format is is yyyy-MM-dd HH:mm:ss.SSS the Actual output i am getting from … --- ## [Failed to connect to elasticsearch from logstash](https://discuss.elastic.co/t/failed-to-connect-to-elasticsearch-from-logstash/71909)
**Author:** [@nanshan](https://discuss.elastic.co/u/nanshan)\ **Replies:** 1\ **Last updated:** [January 18, 2017, 6:45am UTC](https://discuss.elastic.co/t/failed-to-connect-to-elasticsearch-from-logstash/71909 "2017-01-18T06:45:39Z")
I have ELB: elasticsearch-data.dev-ore, and there are 3 easlticsearch-nodes behind it. 20:58:33 dev-ore-logstash-i-bef580a6 root ~ \> telnet elasticsearch-data.dev-ore 9200 Trying 10.0.19.78... Connected to elasticsea… --- ## [Not able to parse Custom Date](https://discuss.elastic.co/t/not-able-to-parse-custom-date/70892)
**Author:** [@Ajit\_Singh](https://discuss.elastic.co/u/Ajit_Singh)\ **Replies:** 2\ **Last updated:** [January 18, 2017, 6:32am UTC](https://discuss.elastic.co/t/not-able-to-parse-custom-date/70892 "2017-01-18T06:32:25Z")
I have a date in the format 06/30/2016 00:21:49, I am trying to parse this date using date { match =\> \[ "time", "MM/dd/YYYY HH:mm:ss" \] } But the date is still a string value. So how can I parse a date if it is not… --- ## [How to check and fix Logstash performance](https://discuss.elastic.co/t/how-to-check-and-fix-logstash-performance/53495)
**Author:** [@ori.rubinfeld](https://discuss.elastic.co/u/ori.rubinfeld)\ **Replies:** 4\ **Last updated:** [January 18, 2017, 6:31am UTC](https://discuss.elastic.co/t/how-to-check-and-fix-logstash-performance/53495 "2017-01-18T06:31:34Z")
Hi All, I am having several servers (windows) running filebeat to get log files and send events to the Logstash. Logstash and Elasticsearch runs on the same Linux machine. I have developed a monitor scheduled script … --- ## [Converting milliseconds into seconds](https://discuss.elastic.co/t/converting-milliseconds-into-seconds/71837)
**Author:** [@bini](https://discuss.elastic.co/u/bini)\ **Replies:** 2\ **Last updated:** [January 18, 2017, 4:02am UTC](https://discuss.elastic.co/t/converting-milliseconds-into-seconds/71837 "2017-01-18T04:02:23Z")
Hi, I have captured a field Response time from the log file, but it is in milliseconds, is there a way to convert it in seconds in logstash using any plugin. I tried using mutate but it won't work. Kindly help me out. --- ## [Need to use a if condition](https://discuss.elastic.co/t/need-to-use-a-if-condition/71445)
**Author:** [@Lahiru\_Lakshitha\_Sam](https://discuss.elastic.co/u/Lahiru_Lakshitha_Sam)\ **Replies:** 1\ **Last updated:** [January 17, 2017, 8:49pm UTC](https://discuss.elastic.co/t/need-to-use-a-if-condition/71445 "2017-01-17T20:49:19Z")
Hi I need to use a if condition in my application for example "monitoredCIName" =\> "icdudautmutl23" if monitoredCINme is start with "icd" I need to add new field how do I do that??? OS: CentOs logstash Version 5.1.1 E… --- ## [Clustering logstash 5](https://discuss.elastic.co/t/clustering-logstash-5/70813)
**Author:** [@jstar](https://discuss.elastic.co/u/jstar)\ **Replies:** 6\ **Last updated:** [January 17, 2017, 8:41pm UTC](https://discuss.elastic.co/t/clustering-logstash-5/70813 "2017-01-17T20:41:04Z")
Hi Everybody, I wanted to know if it is possible to install logstash 5 as a cluster? Thanks. --- ## [Elasticsearch output manage\_template permissions](https://discuss.elastic.co/t/elasticsearch-output-manage-template-permissions/71879)
**Author:** [@Brian\_G](https://discuss.elastic.co/u/Brian_G)\ **Replies:** 0\ **Last updated:** [January 17, 2017, 4:19pm UTC](https://discuss.elastic.co/t/elasticsearch-output-manage-template-permissions/71879 "2017-01-17T16:19:39Z")
I am using logstash to manage my templates using the elasticsearch output and manage\_template. I have given the following role to the logstash user (see screenshot) This is elasticsearch 5.1.1 hosted on elastic.co Yo… --- ## [Newbie mutate filter failure](https://discuss.elastic.co/t/newbie-mutate-filter-failure/71550)
**Author:** [@mhooper](https://discuss.elastic.co/u/mhooper)\ **Replies:** 2\ **Last updated:** [January 17, 2017, 4:01pm UTC](https://discuss.elastic.co/t/newbie-mutate-filter-failure/71550 "2017-01-17T16:01:40Z")
t source /mnt/logs/core1ui1/Log\_2017-01-13-12.log My message gets inserted as above. I would like to add a tag (or a field) that tells me that source contained core1ui1. My current filter is as below but I have … --- ## [Logstash error while importing datas from kafka](https://discuss.elastic.co/t/logstash-error-while-importing-datas-from-kafka/71855)
**Author:** [@Grenouille06](https://discuss.elastic.co/u/Grenouille06)\ **Replies:** 7\ **Last updated:** [January 17, 2017, 2:50pm UTC](https://discuss.elastic.co/t/logstash-error-while-importing-datas-from-kafka/71855 "2017-01-17T14:50:24Z")
Hi friends, I use logstash 2.4, kafka 0.10.1 and a cluster Elasticsearch 5.1.1 I import from Active Directory three logs: Application logs, System logs and Security logs. I create three topics in zookeeper from that lo… --- ## [Failed (Connection refused - Connection refused). Sleeping for 2.0](https://discuss.elastic.co/t/failed-connection-refused-connection-refused-sleeping-for-2-0/71848)
**Author:** [@spyros9](https://discuss.elastic.co/u/spyros9)\ **Replies:** 0\ **Last updated:** [January 17, 2017, 10:59am UTC](https://discuss.elastic.co/t/failed-connection-refused-connection-refused-sleeping-for-2-0/71848 "2017-01-17T10:59:48Z")
Hey gang, I'm running filebeat 1.3.1 and logstash 2.4.1 (haven't upgraded to newest because we are still using java7...) on an ubuntu instance on AWS. Once I deploy (we do a green-blue deployment scheme) with the AMI th… --- ## [Using logstash how can i get the values that are in \[ \] to .csv file?](https://discuss.elastic.co/t/using-logstash-how-can-i-get-the-values-that-are-in-to-csv-file/71846)
**Author:** [@Yaswanth](https://discuss.elastic.co/u/Yaswanth)\ **Replies:** 0\ **Last updated:** [January 17, 2017, 10:41am UTC](https://discuss.elastic.co/t/using-logstash-how-can-i-get-the-values-that-are-in-to-csv-file/71846 "2017-01-17T10:41:53Z")
Hi, i have logs in the format \[2017-01-13 21:21:03,956\]\[WARN \]\[index.search.slowlog.query\] \[Yaswanth\] \[bank\]\[0\] took\[28.9ms\], took\_millis\[28\], types\[\], stats\[\], search\_type\[QUERY\_THEN\_FETCH\], total\_shards\[5\], source\[\],… --- ## [Sending logs to .csv file using logstash?](https://discuss.elastic.co/t/sending-logs-to-csv-file-using-logstash/71767)
**Author:** [@Yaswanth](https://discuss.elastic.co/u/Yaswanth)\ **Replies:** 8\ **Last updated:** [January 17, 2017, 10:09am UTC](https://discuss.elastic.co/t/sending-logs-to-csv-file-using-logstash/71767 "2017-01-17T10:09:34Z")
Hi ES 2.4.0 Logstash 5.1.1 I want to send the slowlogs to .csv file using logstash . my config file is like this input { file { path =\> "D:\\logstash-5.1.1\\logstash-5.1.1\\bin\\slowlog.log" start\_position =\> "beginni… --- ## [Logstash Oracle JDBC issue](https://discuss.elastic.co/t/logstash-oracle-jdbc-issue/71814)
**Author:** [@anusha\_samarakkodi](https://discuss.elastic.co/u/anusha_samarakkodi)\ **Replies:** 6\ **Last updated:** [January 17, 2017, 9:19am UTC](https://discuss.elastic.co/t/logstash-oracle-jdbc-issue/71814 "2017-01-17T09:19:33Z")
could you please help to resolve this issue???? Here is my logstash configuration file input { jdbc { jdbc\_validate\_connection =\> true jdbc\_connection\_string =\> "jdbc:oracle:thin:@172.22.28.52:1521/prod" … --- ## [How to avoid using BigDecimal scientific data format](https://discuss.elastic.co/t/how-to-avoid-using-bigdecimal-scientific-data-format/71712)
**Author:** [@georgiana.ciuchi](https://discuss.elastic.co/u/georgiana.ciuchi)\ **Replies:** 0\ **Last updated:** [January 16, 2017, 10:17am UTC](https://discuss.elastic.co/t/how-to-avoid-using-bigdecimal-scientific-data-format/71712 "2017-01-16T10:17:21Z")
I am using Logstash 1.5.5, Kafka as input using JSON codec, a structured message is received by Logstash: { "app\_id" ": "myapp" ...... "log\_message": { "id": "1234", "code": "whatever", "amount":… --- ## [Logstash snmptrap dynamic fields with ruby](https://discuss.elastic.co/t/logstash-snmptrap-dynamic-fields-with-ruby/71336)
**Author:** [@aydinnmu](https://discuss.elastic.co/u/aydinnmu)\ **Replies:** 3\ **Last updated:** [January 17, 2017, 7:45am UTC](https://discuss.elastic.co/t/logstash-snmptrap-dynamic-fields-with-ruby/71336 "2017-01-17T07:45:11Z")
Hi everyone, i have a problem with dynamic fields. i saw a few topics but they are not useful for me. when you use logstash as snmptrap receiver, you will see that datas are not a static values. They will change wh… --- ## [How to load data from AWS Dynamo DB to ElasticSearch](https://discuss.elastic.co/t/how-to-load-data-from-aws-dynamo-db-to-elasticsearch/71498)
**Author:** [@prasad\_kp](https://discuss.elastic.co/u/prasad_kp)\ **Replies:** 1\ **Last updated:** [January 17, 2017, 7:11am UTC](https://discuss.elastic.co/t/how-to-load-data-from-aws-dynamo-db-to-elasticsearch/71498 "2017-01-17T07:11:49Z")
Hi, I want to load data from AWS dynamo Db to elastic search using log stash, any working sample .conf file is there it will very help ful for me.Can we use log stash? I searched and found https://github.com/awslabs… --- ## [How to name elasticsearch index based on time in result set from JDBC input?](https://discuss.elastic.co/t/how-to-name-elasticsearch-index-based-on-time-in-result-set-from-jdbc-input/71810)
**Author:** [@elasticheart](https://discuss.elastic.co/u/elasticheart)\ **Replies:** 1\ **Last updated:** [January 17, 2017, 6:29am UTC](https://discuss.elastic.co/t/how-to-name-elasticsearch-index-based-on-time-in-result-set-from-jdbc-input/71810 "2017-01-17T06:29:39Z")
Hi, I am using jdbc input having a query like; SELECT id,name,address,timea,timeb FROM mytable where TO\_CHAR(timea,'yyyy-mm-dd') \> '2017-01-01' and TO\_CHAR(timeb,'yyyy-mm-dd') \< '2017-01-16' I have configured my ela… --- ## [Faild to install the plugin “logstash-output-influxdb” with logstash 5.1.1](https://discuss.elastic.co/t/faild-to-install-the-plugin-logstash-output-influxdb-with-logstash-5-1-1/71807)
**Author:** [@aybaylyn](https://discuss.elastic.co/u/aybaylyn)\ **Replies:** 0\ **Last updated:** [January 17, 2017, 4:24am UTC](https://discuss.elastic.co/t/faild-to-install-the-plugin-logstash-output-influxdb-with-logstash-5-1-1/71807 "2017-01-17T04:24:44Z")
Hello , I am writing this message to ask for help. The problem is that when I install logstash-ouput-influxdb with the command “bin/logstash-plugin insall logstash-output-influxdb” , It just has no result , the iTerm on … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1050) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1052)