# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=935 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 936 --- ## [Take input from more than 5 servers](https://discuss.elastic.co/t/take-input-from-more-than-5-servers/95330)
**Author:** [@shrikantgulia](https://discuss.elastic.co/u/shrikantgulia)\ **Replies:** 4\ **Last updated:** [August 2, 2017, 1:07pm UTC](https://discuss.elastic.co/t/take-input-from-more-than-5-servers/95330 "2017-08-02T13:07:22Z")
I am new to ELK Pls Help As i have to observe 5 different Servers(5 metricbeat & 5 Filebeat) on ec2 instance . Than do i need to run different logstash config. file or i can take input in a single logstash file and run… --- ## [Setting the value of variable in ruby logstash](https://discuss.elastic.co/t/setting-the-value-of-variable-in-ruby-logstash/95130)
**Author:** [@deepanshu\_mittal](https://discuss.elastic.co/u/deepanshu_mittal)\ **Replies:** 3\ **Last updated:** [August 2, 2017, 12:54pm UTC](https://discuss.elastic.co/t/setting-the-value-of-variable-in-ruby-logstash/95130 "2017-08-02T12:54:47Z")
I am trying to set the value in ruby logstash variable but it is saying me undefined variable. if “el5” in \[message\] { ruby { init =\> "@@linuxversion=‘5’" code =\> “event.set(‘linux’,’(@@linuxversion)’) puts linux” … --- ## [Increase LS\_OPEN\_FILES beyond 16384](https://discuss.elastic.co/t/increase-ls-open-files-beyond-16384/95478)
**Author:** [@mitabrev](https://discuss.elastic.co/u/mitabrev)\ **Replies:** 1\ **Last updated:** [August 2, 2017, 12:11pm UTC](https://discuss.elastic.co/t/increase-ls-open-files-beyond-16384/95478 "2017-08-02T12:11:17Z")
Hi guys, Hope my question isn’t too trivial but I can’t seem to find any other setting to increase this value. PID for logstash service is XXX and proc limits: cat /proc/XXX/limits Max open files 163… --- ## [Throttle filter and multiline filter](https://discuss.elastic.co/t/throttle-filter-and-multiline-filter/95514)
**Author:** [@abc1](https://discuss.elastic.co/u/abc1)\ **Replies:** 0\ **Last updated:** [August 2, 2017, 11:53am UTC](https://discuss.elastic.co/t/throttle-filter-and-multiline-filter/95514 "2017-08-02T11:53:41Z")
when i am using both throttle filter and multiline filter together then kibana is not showing multiline filter result. kindly help me with this issue. --- ## [Multiple pipelines - logstash-6.0.0-alpha2](https://discuss.elastic.co/t/multiple-pipelines-logstash-6-0-0-alpha2/95503)
**Author:** [@directorkuro](https://discuss.elastic.co/u/directorkuro)\ **Replies:** 0\ **Last updated:** [August 2, 2017, 10:18am UTC](https://discuss.elastic.co/t/multiple-pipelines-logstash-6-0-0-alpha2/95503 "2017-08-02T10:18:14Z")
Hi all, Hope you can help. I’m keen to try the multiple pipelines functionality in logstash-6.0.0-alpha2. So far I’ve been unable to get it to work. I’ve edited the “pipelines.yml” file in the config folder. I’ve added … --- ## [Ruby filter event API](https://discuss.elastic.co/t/ruby-filter-event-api/95081)
**Author:** [@dtko](https://discuss.elastic.co/u/dtko)\ **Replies:** 1\ **Last updated:** [August 2, 2017, 10:23am UTC](https://discuss.elastic.co/t/ruby-filter-event-api/95081 "2017-08-02T10:23:35Z")
Hi, I’m recently having some issues with adjusting logstash ruby filter to logstash 5.0’s event API. here’s my old conf file (with suspected lines): filter { ruby { code =\> " require 'json' require 'diges… --- ## [Netflow High CPU utilization and poor throughput](https://discuss.elastic.co/t/netflow-high-cpu-utilization-and-poor-throughput/94854)
**Author:** [@jasonkeller](https://discuss.elastic.co/u/jasonkeller)\ **Replies:** 3\ **Last updated:** [August 2, 2017, 10:23am UTC](https://discuss.elastic.co/t/netflow-high-cpu-utilization-and-poor-throughput/94854 "2017-08-02T10:23:01Z")
I’ve got an issue with Logstash 5.x on an RHEL7 VM (8 vCPU, 2GB memory) on vSphere 6 hosts with E5-2698 v4 Xeon CPUs. I’ve installed the filter plugin and the Netflow plugin - basic gist is that I’m ingesting Netflow an… --- ## [Logstash Lumberjack SSL Error](https://discuss.elastic.co/t/logstash-lumberjack-ssl-error/95041)
**Author:** [@DanMa](https://discuss.elastic.co/u/DanMa)\ **Replies:** 1\ **Last updated:** [August 2, 2017, 10:21am UTC](https://discuss.elastic.co/t/logstash-lumberjack-ssl-error/95041 "2017-08-02T10:21:22Z")
Hey, since a upgrade from Logstash (i didn't know the version which i had before) i get this ssl failure in the logstash log. \[2017-07-29T18:30:26,562\]\[ERROR\]\[logstash.outputs.lumberjack\] All hosts unavailable, sleepi… --- ## [Error while running wmi input plugin](https://discuss.elastic.co/t/error-while-running-wmi-input-plugin/95494)
**Author:** [@maxf](https://discuss.elastic.co/u/maxf)\ **Replies:** 0\ **Last updated:** [August 2, 2017, 9:31am UTC](https://discuss.elastic.co/t/error-while-running-wmi-input-plugin/95494 "2017-08-02T09:31:49Z")
Hello. Im trying to run wmi input plugin, but i got only errors "Bad file descriptor" My config: input { wmi { query =\> "SELECT \* FROM Win32\_Process" host =\> "hostname" user =\> "DOMAIN\\USER" password =\> "mypassw… --- ## [Cloud trail integration in Logstash 5.5.1](https://discuss.elastic.co/t/cloud-trail-integration-in-logstash-5-5-1/95485)
**Author:** [@vijjun123](https://discuss.elastic.co/u/vijjun123)\ **Replies:** 0\ **Last updated:** [August 2, 2017, 8:56am UTC](https://discuss.elastic.co/t/cloud-trail-integration-in-logstash-5-5-1/95485 "2017-08-02T08:56:12Z")
Hi All, facing issues during cloud trail integration in logstash 5.5.1 could you please suggest me further. Please find the error details below. \[2017-08-02T08:45:37,521\]\[FATAL\]\[logstash.runner \] The given c… --- ## [Logstash data rotation](https://discuss.elastic.co/t/logstash-data-rotation/94433)
**Author:** [@mkain](https://discuss.elastic.co/u/mkain)\ **Replies:** 2\ **Last updated:** [August 2, 2017, 7:49am UTC](https://discuss.elastic.co/t/logstash-data-rotation/94433 "2017-08-02T07:49:59Z")
Hi, I am running ELK 5.4.3 to capture syslog and SNMP traps. To manage space on disk I want to delete older data. Can someone pls give me hints for: 1) deleting data older than 90 days 2) moving data older than 90 d… --- ## [Questions about the Logstash Configuration](https://discuss.elastic.co/t/questions-about-the-logstash-configuration/95437)
**Author:** [@LeonSun](https://discuss.elastic.co/u/LeonSun)\ **Replies:** 2\ **Last updated:** [August 2, 2017, 4:03am UTC](https://discuss.elastic.co/t/questions-about-the-logstash-configuration/95437 "2017-08-02T04:03:14Z")
Hi, I am new to Logstash, I built up 2 configuration file on my server for 2 types of log; I put the 2 types of logs into different folder:/FW\_Log and /VAL-LOG; , and put the 2 config files under same folder: /logstash/c… --- ## [\_grokparsefailure in the file](https://discuss.elastic.co/t/-grokparsefailure-in-the-file/95123)
**Author:** [@varunverma](https://discuss.elastic.co/u/varunverma)\ **Replies:** 5\ **Last updated:** [August 2, 2017, 12:37am UTC](https://discuss.elastic.co/t/-grokparsefailure-in-the-file/95123 "2017-08-02T00:37:55Z")
I have the below grok filter in my logstash conf file filter { grok { match =\> { ‘message’ =\> ’ %{IPORHOST:clientip} - %{USERNAME:user} %{HTTPDATE:timestamp} \\"%{WORD:method} %{NOTSPACE:request} %{NOTSPACE:httpversion… --- ## [Execute logstash script as batch](https://discuss.elastic.co/t/execute-logstash-script-as-batch/95236)
**Author:** [@amjad.hussain](https://discuss.elastic.co/u/amjad.hussain)\ **Replies:** 5\ **Last updated:** [August 1, 2017, 7:18pm UTC](https://discuss.elastic.co/t/execute-logstash-script-as-batch/95236 "2017-08-01T19:18:35Z")
I am using Logstash 5.4.1. I need to execute one of my logstash script only once. I don’t want logstash thread to run in background. I want to use logstash script like a batch. Once thread reach end of the file in in… --- ## [Converting a number for use in geo-point](https://discuss.elastic.co/t/converting-a-number-for-use-in-geo-point/95252)
**Author:** [@elaair](https://discuss.elastic.co/u/elaair)\ **Replies:** 9\ **Last updated:** [August 1, 2017, 4:08pm UTC](https://discuss.elastic.co/t/converting-a-number-for-use-in-geo-point/95252 "2017-08-01T16:08:37Z")
I have data that contains src\_latitude, src\_longitude, dst\_latitiude, and dst\_longitude. Some of the data is zero which seems to throw off elastic. \[2017-07-31T15:17:54,787\]\[DEBUG\]\[o.e.a.b.TransportShardBulkAction\] \[UzW… --- ## [Using "key/value with nested fields" with logstash](https://discuss.elastic.co/t/using-key-value-with-nested-fields-with-logstash/95340)
**Author:** [@widhalmt](https://discuss.elastic.co/u/widhalmt)\ **Replies:** 1\ **Last updated:** [August 1, 2017, 3:25pm UTC](https://discuss.elastic.co/t/using-key-value-with-nested-fields-with-logstash/95340 "2017-08-01T15:25:20Z")
Hi, I have a problem where someone logs lots of different JSON logs into Logstash which forwards them to Elasticsearch. There are so many fields that we hit the 1000 threshold of fields per index. To prevent mapping exp… --- ## [Uploading Json file to ES using LogStash](https://discuss.elastic.co/t/uploading-json-file-to-es-using-logstash/95370)
**Author:** [@SKim](https://discuss.elastic.co/u/SKim)\ **Replies:** 0\ **Last updated:** [August 1, 2017, 3:11pm UTC](https://discuss.elastic.co/t/uploading-json-file-to-es-using-logstash/95370 "2017-08-01T15:11:40Z")
I have a very simple Json file: {“ABC”:“123”,“DEF”:“456”} I’m trying to upload that to Elastic Search using LogStash. My LogStash config file is: input { file { start\_position =\> "beginning" path =\> "C:\\Docs\\Elast… --- ## [Gc info (LastGcInfo)](https://discuss.elastic.co/t/gc-info-lastgcinfo/95347)
**Author:** [@abhishekattlalance](https://discuss.elastic.co/u/abhishekattlalance)\ **Replies:** 0\ **Last updated:** [August 1, 2017, 1:26pm UTC](https://discuss.elastic.co/t/gc-info-lastgcinfo/95347 "2017-08-01T13:26:26Z")
Hi , i am using logstash for collecting JMX values. i want to use GC info in kibana dashboard. but its coming like below: {\=\>#J… --- ## [Will elasticsearch convert files of different types](https://discuss.elastic.co/t/will-elasticsearch-convert-files-of-different-types/94342)
**Author:** [@sudi\_2611](https://discuss.elastic.co/u/sudi_2611)\ **Replies:** 6\ **Last updated:** [August 1, 2017, 12:50pm UTC](https://discuss.elastic.co/t/will-elasticsearch-convert-files-of-different-types/94342 "2017-08-01T12:50:47Z")
Hi All, I am new to elasticserach....the logstash is pulling the data from cluster and in kibana when viewing the data total filesystem size is different from the actual box...any suggestions as how to have to standar… --- ## [Log stash 5.5.0 and 5.5.1 Unable to start in windows](https://discuss.elastic.co/t/log-stash-5-5-0-and-5-5-1-unable-to-start-in-windows/94754)
**Author:** [@prabushankar79](https://discuss.elastic.co/u/prabushankar79)\ **Replies:** 2\ **Last updated:** [August 1, 2017, 12:52pm UTC](https://discuss.elastic.co/t/log-stash-5-5-0-and-5-5-1-unable-to-start-in-windows/94754 "2017-08-01T12:52:49Z")
Now again Published Logstash core is not working . I am a new bee to Elastic, when i am trying to start Logstash in my windows machine, I am getting the following error . I have referred few older post and referred the b… --- ## [PKI between Logstash and Winlogbeat](https://discuss.elastic.co/t/pki-between-logstash-and-winlogbeat/94851)
**Author:** [@socdev](https://discuss.elastic.co/u/socdev)\ **Replies:** 3\ **Last updated:** [August 1, 2017, 11:59am UTC](https://discuss.elastic.co/t/pki-between-logstash-and-winlogbeat/94851 "2017-08-01T11:59:06Z")
I am having a frustrating problem with setting up PKI between Logstash and Winlogbeat. I have tried two (X-Pack, EASY-RSA) methods both successfully creating a CA, Private and Public Key, however both result in the same … --- ## [Filtering Metricbeat data in LogStash](https://discuss.elastic.co/t/filtering-metricbeat-data-in-logstash/95332)
**Author:** [@vinay.t](https://discuss.elastic.co/u/vinay.t)\ **Replies:** 0\ **Last updated:** [August 1, 2017, 11:47am UTC](https://discuss.elastic.co/t/filtering-metricbeat-data-in-logstash/95332 "2017-08-01T11:47:43Z")
Hello, I have installed metricbeat on my windows servers and when I noticed it was shipping all resource utilisation but I do not want everything. Based on my requirement I modified metricbeat config file so that I sho… --- ## [Logstash crash issue](https://discuss.elastic.co/t/logstash-crash-issue/95313)
**Author:** [@mingfeng.zhang](https://discuss.elastic.co/u/mingfeng.zhang)\ **Replies:** 0\ **Last updated:** [August 1, 2017, 10:26am UTC](https://discuss.elastic.co/t/logstash-crash-issue/95313 "2017-08-01T10:26:35Z")
Exception in thread “\[main\]\>worker1” java.lang.ArrayIndexOutOfBoundsException: -1 at java.util.ArrayList.elementData(ArrayList.java:418) at java.util.ArrayList.remove(ArrayList.java:495) at org.logstash.FieldReference… --- ## [Insert CSV to ES](https://discuss.elastic.co/t/insert-csv-to-es/95304)
**Author:** [@Tals](https://discuss.elastic.co/u/Tals)\ **Replies:** 0\ **Last updated:** [August 1, 2017, 9:21am UTC](https://discuss.elastic.co/t/insert-csv-to-es/95304 "2017-08-01T09:21:11Z")
Hi I’m trying to insert a new CSV file to ES. Got Successfully started Logstash API endpoint {:port=\>9600} But Nothing Happen…It seems to me that Logstash uses a previous mapping I made for other file… . Is it ? T… --- ## [Convert a date format in logstash pipeline](https://discuss.elastic.co/t/convert-a-date-format-in-logstash-pipeline/95000)
**Author:** [@marwa](https://discuss.elastic.co/u/marwa)\ **Replies:** 16\ **Last updated:** [August 1, 2017, 8:22am UTC](https://discuss.elastic.co/t/convert-a-date-format-in-logstash-pipeline/95000 "2017-08-01T08:22:31Z")
hello , i am trying to load a csv file into elasticsearch via logstash .this is a snippet of my data sets transaction\_date,customer\_name,product\_name,product\_id,price\_unit,quantity,fidelity\_card\_id,discount,category 2-… --- ## [Last line of log not picked up by logstash](https://discuss.elastic.co/t/last-line-of-log-not-picked-up-by-logstash/95286)
**Author:** [@Vivek\_Sharma](https://discuss.elastic.co/u/Vivek_Sharma)\ **Replies:** 2\ **Last updated:** [August 1, 2017, 7:24am UTC](https://discuss.elastic.co/t/last-line-of-log-not-picked-up-by-logstash/95286 "2017-08-01T07:24:59Z")
Hi I am using logstash version 5 and trying to parse a log file. When there are 10 records in the log file, logstash processes only 9. Not picking up the last line of the log. I picked the last record and tried to par… --- ## [Can Not Reserve Enough Space For Object Heap](https://discuss.elastic.co/t/can-not-reserve-enough-space-for-object-heap/94498)
**Author:** [@infamous](https://discuss.elastic.co/u/infamous)\ **Replies:** 3\ **Last updated:** [August 1, 2017, 7:10am UTC](https://discuss.elastic.co/t/can-not-reserve-enough-space-for-object-heap/94498 "2017-08-01T07:10:49Z")
Hi all, I've been using Logstash and Elasticsearch for a while now and have survived using a 1g object heap for development. However, I can not get the heap to go over 1150Mb before it says I can not reserve enough spa… --- ## [Logstash not reading more than 2 input file](https://discuss.elastic.co/t/logstash-not-reading-more-than-2-input-file/94874)
**Author:** [@P\_Kumar](https://discuss.elastic.co/u/P_Kumar)\ **Replies:** 1\ **Last updated:** [August 1, 2017, 3:04am UTC](https://discuss.elastic.co/t/logstash-not-reading-more-than-2-input-file/94874 "2017-08-01T03:04:04Z")
Hi , I am new to ELK , i am trying to read the logs for IBM dmgr ,node and Application logs . but it is reading top 2 file input and creating index but 3rd file is not creating index and not reading could you please he… --- ## [Unable to get log files from directory in logstash](https://discuss.elastic.co/t/unable-to-get-log-files-from-directory-in-logstash/95226)
**Author:** [@bjsprat](https://discuss.elastic.co/u/bjsprat)\ **Replies:** 1\ **Last updated:** [July 31, 2017, 11:43pm UTC](https://discuss.elastic.co/t/unable-to-get-log-files-from-directory-in-logstash/95226 "2017-07-31T23:43:26Z")
I am trying to get logstash to read json logs from a directory. Here is my config file: input { file { path =\> "/home/ast/nifi-files-cp/\*.json" } tcp { port =\> 10000 } } filter{ json{ source =\> "m… --- ## [Add extra fields for doc\_as\_upsert](https://discuss.elastic.co/t/add-extra-fields-for-doc-as-upsert/95239)
**Author:** [@amjad.hussain](https://discuss.elastic.co/u/amjad.hussain)\ **Replies:** 0\ **Last updated:** [July 31, 2017, 7:51pm UTC](https://discuss.elastic.co/t/add-extra-fields-for-doc-as-upsert/95239 "2017-07-31T19:51:29Z")
I have written one Logstash script for update. This script will update or insert if the record is not present in elasticsearch. I am using doc\_as\_upsert as true for that. input { file { path =\> "C:/PADS\_ELK/Logstas… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=934) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=936)