# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=901 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 902 --- ## [Converting bytes to gb in logstash configuration file](https://discuss.elastic.co/t/converting-bytes-to-gb-in-logstash-configuration-file/102026)
**Author:** [@somu411](https://discuss.elastic.co/u/somu411)\ **Replies:** 8\ **Last updated:** [September 28, 2017, 2:51pm UTC](https://discuss.elastic.co/t/converting-bytes-to-gb-in-logstash-configuration-file/102026 "2017-09-28T14:51:09Z")
Hi All, I have logstash conf file, where I have field with bytes but when I Load to Kibana I should see new field with bytes coverted to GB. I tried using ruby filter but it is failing with error "\[2017-09-27T15:16:43,… --- ## [Trouble using elasticsearch filter plugin](https://discuss.elastic.co/t/trouble-using-elasticsearch-filter-plugin/99981)
**Author:** [@cito.ets](https://discuss.elastic.co/u/cito.ets)\ **Replies:** 7\ **Last updated:** [September 28, 2017, 2:47pm UTC](https://discuss.elastic.co/t/trouble-using-elasticsearch-filter-plugin/99981 "2017-09-28T14:47:24Z")
Hi, A bit new into SSL/TLS. I do have managed to get Logstash to work with Searchguard/Elasticsearch (elasticsearch output) on a test server but when trying to use the elasticsearch filter plugin I get a ConnectionFaile… --- ## [Creating arrays in ruby](https://discuss.elastic.co/t/creating-arrays-in-ruby/102109)
**Author:** [@fluffierbump](https://discuss.elastic.co/u/fluffierbump)\ **Replies:** 4\ **Last updated:** [September 28, 2017, 1:52pm UTC](https://discuss.elastic.co/t/creating-arrays-in-ruby/102109 "2017-09-28T13:52:28Z")
I am trying to set an event with multiple records in an array as follows ruby { code =\> " begin event.set('\[fieldname\]\[0\]', 'This is record 0') event.set('\[fieldna… --- ## [Logstash duration quastion](https://discuss.elastic.co/t/logstash-duration-quastion/102049)
**Author:** [@kiwi0217](https://discuss.elastic.co/u/kiwi0217)\ **Replies:** 3\ **Last updated:** [September 28, 2017, 1:07pm UTC](https://discuss.elastic.co/t/logstash-duration-quastion/102049 "2017-09-28T13:07:05Z")
I write some logstash filter the count the user online period. but i got something wrong in my filter. And i don't know how to filter this by different users. log format is "|2017-09-28-09:49:18|INFO|JupyterHub|User l… --- ## [PKI between Logstash and Winlogbeat v2](https://discuss.elastic.co/t/pki-between-logstash-and-winlogbeat-v2/102107)
**Author:** [@logallthethings](https://discuss.elastic.co/u/logallthethings)\ **Replies:** 0\ **Last updated:** [September 28, 2017, 11:46am UTC](https://discuss.elastic.co/t/pki-between-logstash-and-winlogbeat-v2/102107 "2017-09-28T11:46:07Z")
Continuing the discussion from PKI between Logstash and Winlogbeat: I'm also facing the same issue as described in the link, this question was never answered. I've working on this for the past 1.5 days and I always get… --- ## [Logstash log says "Pushing flushed events" but I can't see any output in ES](https://discuss.elastic.co/t/logstash-log-says-pushing-flushed-events-but-i-cant-see-any-output-in-es/101988)
**Author:** [@tomer](https://discuss.elastic.co/u/tomer)\ **Replies:** 5\ **Last updated:** [September 28, 2017, 11:35am UTC](https://discuss.elastic.co/t/logstash-log-says-pushing-flushed-events-but-i-cant-see-any-output-in-es/101988 "2017-09-28T11:35:48Z")
Hi, I am getting the following log in Logstash, but from some reason in Elasticsearch I can't see any output. How can this be? \[2017-09-27T16:38:40,961\]\[DEBUG\]\[logstash.pipeline \] Pushing flushed events {:event… --- ## [Can statement in jdbc\_streaming filter using a field from input?](https://discuss.elastic.co/t/can-statement-in-jdbc-streaming-filter-using-a-field-from-input/102099)
**Author:** [@Jin\_Guo](https://discuss.elastic.co/u/Jin_Guo)\ **Replies:** 0\ **Last updated:** [September 28, 2017, 10:42am UTC](https://discuss.elastic.co/t/can-statement-in-jdbc-streaming-filter-using-a-field-from-input/102099 "2017-09-28T10:42:18Z")
Hi, I have a pipeline like below: input { stdin {} } filter { kv {} jdbc\_streaming { jdbc\_driver\_library =\> "D:\\Work\\mysql-connector-java-5.1.43-bin.jar" jdbc\_driver\_class =\> "com.mysql.jdbc.Driver" … --- ## [Date aaray in logstash csv file](https://discuss.elastic.co/t/date-aaray-in-logstash-csv-file/102098)
**Author:** [@DILIP\_SHARMA](https://discuss.elastic.co/u/DILIP_SHARMA)\ **Replies:** 0\ **Last updated:** [September 28, 2017, 10:37am UTC](https://discuss.elastic.co/t/date-aaray-in-logstash-csv-file/102098 "2017-09-28T10:37:10Z")
i trying to upload csv file in logstash where i have dateaaray field . my sample csv file - d1,1,2017/09/27 10:00:00,2017/09/27 10:06:00,6,"2017/09/27 10:00:00,2017/09/27 10:01:00,2017/09/27 10:02:00,2017/09/27 10:03:… --- ## [Cant start logstash](https://discuss.elastic.co/t/cant-start-logstash/101999)
**Author:** [@111126](https://discuss.elastic.co/u/111126)\ **Replies:** 5\ **Last updated:** [September 28, 2017, 10:17am UTC](https://discuss.elastic.co/t/cant-start-logstash/101999 "2017-09-28T10:17:02Z")
\[ERROR\]\[logstash.agent \] Cannot create pipeline {:reason=\>"Cannot evaluate${DATA}. Environment variableDATAis not set and there is no default value given."} I dont understand this error and I dont know what is it ${DATA… --- ## [One logstash node can't stop](https://discuss.elastic.co/t/one-logstash-node-cant-stop/102073)
**Author:** [@KeithTt](https://discuss.elastic.co/u/KeithTt)\ **Replies:** 4\ **Last updated:** [September 28, 2017, 9:19am UTC](https://discuss.elastic.co/t/one-logstash-node-cant-stop/102073 "2017-09-28T09:19:18Z")
ELK stack version：5.5.1 There is one logstash can not be restarted, I have used kill -9 PID to kill the process, but it respawn a new one automatically. There are three logstash nodes in my ELK stack, the other two are… --- ## [Logstash-output-mongodb](https://discuss.elastic.co/t/logstash-output-mongodb/102089)
**Author:** [@Nikparab](https://discuss.elastic.co/u/Nikparab)\ **Replies:** 0\ **Last updated:** [September 28, 2017, 9:49am UTC](https://discuss.elastic.co/t/logstash-output-mongodb/102089 "2017-09-28T09:49:58Z")
Not able to install output plugin for logstash 5.6.2 version. giving following error Plugin version conflict, aborting ERROR: Installation Aborted, message: Bundler could not find compatible versions for gem "mongo": … --- ## [How to parse logs on startup and merge files per folder?](https://discuss.elastic.co/t/how-to-parse-logs-on-startup-and-merge-files-per-folder/101291)
**Author:** [@vrider](https://discuss.elastic.co/u/vrider)\ **Replies:** 5\ **Last updated:** [September 28, 2017, 4:25am UTC](https://discuss.elastic.co/t/how-to-parse-logs-on-startup-and-merge-files-per-folder/101291 "2017-09-28T04:25:42Z")
Dear Community, I had troubles with parsing of logs with logstash during startup Once I add an exta line to log file -\> parsed; It seems that parsing only is triggerred once change is observer Is there a setting fo… --- ## [Logstash can not tracking oracle timestamp correctly](https://discuss.elastic.co/t/logstash-can-not-tracking-oracle-timestamp-correctly/99776)
**Author:** [@Trident](https://discuss.elastic.co/u/Trident)\ **Replies:** 1\ **Last updated:** [September 28, 2017, 12:35am UTC](https://discuss.elastic.co/t/logstash-can-not-tracking-oracle-timestamp-correctly/99776 "2017-09-28T00:35:03Z")
I use logstash-jdbc-input import data to es from oracle, and want use a timestamp column as tracking column. below is my conf \[41 PM\] when first run, the :sql\_last\_value was “N” and logstash failed can anyone help me… --- ## [Logstash: grok pars failure with multiple custom patterns](https://discuss.elastic.co/t/logstash-grok-pars-failure-with-multiple-custom-patterns/101992)
**Author:** [@erikkn](https://discuss.elastic.co/u/erikkn)\ **Replies:** 1\ **Last updated:** [September 27, 2017, 8:07pm UTC](https://discuss.elastic.co/t/logstash-grok-pars-failure-with-multiple-custom-patterns/101992 "2017-09-27T20:07:47Z")
Hi guys, I am trying to pars a log file of ~500 lines to find the following two lines: flaked=yes && platform=aws so that I can add these lines as attributes for the given event/document. The configuration (logstash.co… --- ## [Elasticsearch output plugin does not write to elasticsearch](https://discuss.elastic.co/t/elasticsearch-output-plugin-does-not-write-to-elasticsearch/101401)
**Author:** [@Jeffrey\_Blayney](https://discuss.elastic.co/u/Jeffrey_Blayney)\ **Replies:** 10\ **Last updated:** [September 27, 2017, 7:58pm UTC](https://discuss.elastic.co/t/elasticsearch-output-plugin-does-not-write-to-elasticsearch/101401 "2017-09-27T19:58:17Z")
Hello! I've just upgraded logstash from 2.2 to 5.5. I have the current version of the elasticsearch output plugin installed (8.8.2) Logstash no longer outputs anything to elasticsearch with the same config file. The on… --- ## [Logstash Persistent Queues throws exception when trying to read BigInteger values from the queue](https://discuss.elastic.co/t/logstash-persistent-queues-throws-exception-when-trying-to-read-biginteger-values-from-the-queue/101566)
**Author:** [@subhasdan](https://discuss.elastic.co/u/subhasdan)\ **Replies:** 6\ **Last updated:** [September 27, 2017, 7:32pm UTC](https://discuss.elastic.co/t/logstash-persistent-queues-throws-exception-when-trying-to-read-biginteger-values-from-the-queue/101566 "2017-09-27T19:32:36Z")
Logstash version: any version with PQs enabled - currently tested on 5.5 upto 5.6.1 Platform: any Simple test case: https://github.com/rdsubhas/logstash-queue-corruption logstash.yml: http.host: "0.0.0.0" path.config… --- ## [Logstash Grok Filter breaks Single log line as 2 lines](https://discuss.elastic.co/t/logstash-grok-filter-breaks-single-log-line-as-2-lines/102020)
**Author:** [@vijay.sangha](https://discuss.elastic.co/u/vijay.sangha)\ **Replies:** 1\ **Last updated:** [September 27, 2017, 7:30pm UTC](https://discuss.elastic.co/t/logstash-grok-filter-breaks-single-log-line-as-2-lines/102020 "2017-09-27T19:30:29Z")
I am working to write a simple grok pattern and I have generated a grok filter which works awsome in grok debugger but when I use it in logstash, it generates 2 lines out of single line of log. these message also not pro… --- ## [Rubydebug is not displayed](https://discuss.elastic.co/t/rubydebug-is-not-displayed/101743)
**Author:** [@sam281](https://discuss.elastic.co/u/sam281)\ **Replies:** 12\ **Last updated:** [September 27, 2017, 7:23pm UTC](https://discuss.elastic.co/t/rubydebug-is-not-displayed/101743 "2017-09-27T19:23:30Z")
Hello, I am running logstash on a windows server and have enabled Rubydebug to see how the data is being processed, but I don't see anything being logged on to console or in the rubydebug file. I have tried both the op… --- ## [Logstash ruby convert seconds to milliseconds](https://discuss.elastic.co/t/logstash-ruby-convert-seconds-to-milliseconds/101508)
**Author:** [@ktella](https://discuss.elastic.co/u/ktella)\ **Replies:** 2\ **Last updated:** [September 27, 2017, 7:05pm UTC](https://discuss.elastic.co/t/logstash-ruby-convert-seconds-to-milliseconds/101508 "2017-09-27T19:05:24Z")
I'm trying to convert responsetime from nginx access logs which is in seconds to milliseconds using ruby. I was able to successfully GROK the access log. grok { match =\> { "message" =\> "%{IPV4:clientip} - - \[%{HTTPDAT… --- ## [Logstach SSL issue](https://discuss.elastic.co/t/logstach-ssl-issue/102019)
**Author:** [@fouzmtl](https://discuss.elastic.co/u/fouzmtl)\ **Replies:** 0\ **Last updated:** [September 27, 2017, 6:59pm UTC](https://discuss.elastic.co/t/logstach-ssl-issue/102019 "2017-09-27T18:59:28Z")
Hello, I m new to Logstach/ElasticSearch. I m trying to configure the system to use SSL: I have downloaded xpack to create the SSL documentation . I m having hard time to configure it. Logstach is logstach-5.6.1. I h… --- ## [Parent Child Relationships within a single index with same document types](https://discuss.elastic.co/t/parent-child-relationships-within-a-single-index-with-same-document-types/102014)
**Author:** [@Stephen\_Le\_Bel](https://discuss.elastic.co/u/Stephen_Le_Bel)\ **Replies:** 0\ **Last updated:** [September 27, 2017, 5:40pm UTC](https://discuss.elastic.co/t/parent-child-relationships-within-a-single-index-with-same-document-types/102014 "2017-09-27T17:40:40Z")
Is there a way to have a parent child relationship built between 2 documents of the same type in the same index? I used this example, but type is the same as my\_parent PUT my\_index { "mappings": { "my\_parent": {}, … --- ## [Best practices for indexing log data](https://discuss.elastic.co/t/best-practices-for-indexing-log-data/101891)
**Author:** [@noky](https://discuss.elastic.co/u/noky)\ **Replies:** 5\ **Last updated:** [September 27, 2017, 3:56pm UTC](https://discuss.elastic.co/t/best-practices-for-indexing-log-data/101891 "2017-09-27T15:56:17Z")
Hi, I'm new to the Elastic/ELK stack and am using it for log aggregation/analysis. Basically, the classic use case of using Filebeat to ship log files to Logstash and output the data to Elasticsearch. My main problem is… --- ## [Logstash Filter meter is writing output to tag\_field variable instead of its value](https://discuss.elastic.co/t/logstash-filter-meter-is-writing-output-to-tag-field-variable-instead-of-its-value/101980)
**Author:** [@kapil514](https://discuss.elastic.co/u/kapil514)\ **Replies:** 1\ **Last updated:** [September 27, 2017, 2:46pm UTC](https://discuss.elastic.co/t/logstash-filter-meter-is-writing-output-to-tag-field-variable-instead-of-its-value/101980 "2017-09-27T14:46:43Z")
Below is my logstash configuration. cat /etc/logstash/conf.d/logstash.conf input { beats { port =\> "5044" } } filter { mutate { gsub =\> \["host", '.', '\_'\] } if \[type\] == "log" { metrics { meter =\> \["logstas… --- ## [Problem with output elasticsearch data duplicate on index](https://discuss.elastic.co/t/problem-with-output-elasticsearch-data-duplicate-on-index/101981)
**Author:** [@eldiablo62](https://discuss.elastic.co/u/eldiablo62)\ **Replies:** 4\ **Last updated:** [September 27, 2017, 2:16pm UTC](https://discuss.elastic.co/t/problem-with-output-elasticsearch-data-duplicate-on-index/101981 "2017-09-27T14:16:04Z")
Hi, I have a problem, i have configured 2 datasources and i want to index the data different index but my data are not on the good index. my /etc/logstash/conf.d/config\_mysql\_glpi\_tickets.conf input { jdbc { jdbc… --- ## [JDBC input multiple tables with table name as a parameter](https://discuss.elastic.co/t/jdbc-input-multiple-tables-with-table-name-as-a-parameter/101970)
**Author:** [@Maxclac](https://discuss.elastic.co/u/Maxclac)\ **Replies:** 4\ **Last updated:** [September 27, 2017, 2:03pm UTC](https://discuss.elastic.co/t/jdbc-input-multiple-tables-with-table-name-as-a-parameter/101970 "2017-09-27T14:03:27Z")
Hello everyone! I am working currently with a Postgres database and trying to convert it to a json file for Elasticsearch. I found out that Logstash is perfect for doing this with the JDBC input plugin. I have several… --- ## [Logstash input elasticsearch plugin throws error](https://discuss.elastic.co/t/logstash-input-elasticsearch-plugin-throws-error/101956)
**Author:** [@kabali12345](https://discuss.elastic.co/u/kabali12345)\ **Replies:** 1\ **Last updated:** [September 27, 2017, 1:46pm UTC](https://discuss.elastic.co/t/logstash-input-elasticsearch-plugin-throws-error/101956 "2017-09-27T13:46:00Z")
Hi all here i am trying to get data from elasticsearch and send it stdout but the problem is logstash does not stopped after fetching all records in index its repeatedly fetching all records . my logstash.conf i am… --- ## [Netflow - Forwarding Netflow Data to offsite provider](https://discuss.elastic.co/t/netflow-forwarding-netflow-data-to-offsite-provider/101987)
**Author:** [@sarlacpit](https://discuss.elastic.co/u/sarlacpit)\ **Replies:** 0\ **Last updated:** [September 27, 2017, 1:31pm UTC](https://discuss.elastic.co/t/netflow-forwarding-netflow-data-to-offsite-provider/101987 "2017-09-27T13:31:49Z")
Hi, Version info: ELK version 5.6.1 on Ubuntu 16.01 LTS. I am trying to come up with a Netflow soultion where as the ELK stack receives Netflow data then forwards on the raw unformatted Netflow data to an offsite provi… --- ## [GIT Pipelines Integration](https://discuss.elastic.co/t/git-pipelines-integration/101121)
**Author:** [@paul1243](https://discuss.elastic.co/u/paul1243)\ **Replies:** 11\ **Last updated:** [September 27, 2017, 12:04pm UTC](https://discuss.elastic.co/t/git-pipelines-integration/101121 "2017-09-27T12:04:04Z")
Hi - We are trying to integrate with GIT CI/CD with ELK , I'd like to pull the Pipelines job status and send it to Elastic Search and then display it on the dashboard. Would somone please provide your comments on this. … --- ## [Transfer Unix messages to ELS](https://discuss.elastic.co/t/transfer-unix-messages-to-els/101927)
**Author:** [@bigsol](https://discuss.elastic.co/u/bigsol)\ **Replies:** 6\ **Last updated:** [September 27, 2017, 11:25am UTC](https://discuss.elastic.co/t/transfer-unix-messages-to-els/101927 "2017-09-27T11:25:28Z")
Hello all, I need your help to understand how I forward UNIX SYSTEM (SOLARIS) messages to ELS? Thanks --- ## [Logstash expecting \\n for it to process an XML](https://discuss.elastic.co/t/logstash-expecting-n-for-it-to-process-an-xml/101941)
**Author:** [@aggarwalJitesh94](https://discuss.elastic.co/u/aggarwalJitesh94)\ **Replies:** 7\ **Last updated:** [September 27, 2017, 11:11am UTC](https://discuss.elastic.co/t/logstash-expecting-n-for-it-to-process-an-xml/101941 "2017-09-27T11:11:43Z")
Hello Team, I am trying to parse through an XML file and read only a single tag out of the entire XML using the Grok Patterns. My grok pattern looks like this. Its able to parse through an XML when its properly indente… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=900) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=902)