# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=974 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 975 --- ## [How to access beats data in logstash](https://discuss.elastic.co/t/how-to-access-beats-data-in-logstash/87340)
**Author:** [@ludo](https://discuss.elastic.co/u/ludo)\ **Replies:** 2\ **Last updated:** [May 30, 2017, 4:34pm UTC](https://discuss.elastic.co/t/how-to-access-beats-data-in-logstash/87340 "2017-05-30T16:34:57Z")
Hi there, I started learning Elastic stack a few weeks ago by using beats + logstash + statsd (because I already have an existing graphite/Grafana stack). I'm stuck with metricbeats and heartbeats because I'm not abl… --- ## [How can I make the Logstash TCP input to separate messages that it listens on a TCP port?](https://discuss.elastic.co/t/how-can-i-make-the-logstash-tcp-input-to-separate-messages-that-it-listens-on-a-tcp-port/87500)
**Author:** [@Ramuyko](https://discuss.elastic.co/u/Ramuyko)\ **Replies:** 4\ **Last updated:** [May 30, 2017, 4:47pm UTC](https://discuss.elastic.co/t/how-can-i-make-the-logstash-tcp-input-to-separate-messages-that-it-listens-on-a-tcp-port/87500 "2017-05-30T16:47:27Z")
I want to send data to Logstash using the TCP protocol. And to send the data I'm using the Node-RED. A simple configuration for doing that looks like this: In the Logstash folder, I created a file called nodered.conf … --- ## [Read REST API logs and push them to elasticsearch](https://discuss.elastic.co/t/read-rest-api-logs-and-push-them-to-elasticsearch/85638)
**Author:** [@ayush\_rastogi1](https://discuss.elastic.co/u/ayush_rastogi1)\ **Replies:** 2\ **Last updated:** [May 30, 2017, 4:27pm UTC](https://discuss.elastic.co/t/read-rest-api-logs-and-push-them-to-elasticsearch/85638 "2017-05-30T16:27:07Z")
Hi Guys, I am trying to process the Service API request logs and trying to data to elasticsearch A sample line from my logs look like "111.125.230.58 - - \[13/May/2017:08:50:35 +0000\] \\"GET /ContentStorageService/… --- ## [Logstash Filter Verifier - Multiline Messages](https://discuss.elastic.co/t/logstash-filter-verifier-multiline-messages/84463)
**Author:** [@dawi](https://discuss.elastic.co/u/dawi)\ **Replies:** 4\ **Last updated:** [May 30, 2017, 3:28pm UTC](https://discuss.elastic.co/t/logstash-filter-verifier-multiline-messages/84463 "2017-05-30T15:28:18Z")
Logstash Filter Verifier works great for testing even complex Logstash Pipelines. However it seems that it currently does not support testing of multiline messages properly. Whenever I add a "\\n" into the message strin… --- ## [Logstash not able to access the custom field from Filebeat](https://discuss.elastic.co/t/logstash-not-able-to-access-the-custom-field-from-filebeat/87553)
**Author:** [@Dattatray\_Kulkarni](https://discuss.elastic.co/u/Dattatray_Kulkarni)\ **Replies:** 1\ **Last updated:** [May 30, 2017, 3:25pm UTC](https://discuss.elastic.co/t/logstash-not-able-to-access-the-custom-field-from-filebeat/87553 "2017-05-30T15:25:50Z")
Hi, I have multiple types of log files from different servers. So I am using multiple filebeat agents. In the Filebeat configuration, I am adding a custom field - log\_type Here is the snippet of the filebeat configur… --- ## [Parse string data with ; separator in logstash](https://discuss.elastic.co/t/parse-string-data-with-separator-in-logstash/87524)
**Author:** [@Charlotte\_EGM](https://discuss.elastic.co/u/Charlotte_EGM)\ **Replies:** 5\ **Last updated:** [May 30, 2017, 3:02pm UTC](https://discuss.elastic.co/t/parse-string-data-with-separator-in-logstash/87524 "2017-05-30T15:02:39Z")
Hi, I'm trying to parse data that looks like this : “timestamp1;value1; timestamp2;value2;… timestampN;valueN;” I'm trying csv filter (as they originally comes from csv), but it is not working well,as it doesn't u… --- ## [\_dateparsefailure again](https://discuss.elastic.co/t/-dateparsefailure-again/87475)
**Author:** [@santihago](https://discuss.elastic.co/u/santihago)\ **Replies:** 2\ **Last updated:** [May 30, 2017, 3:00pm UTC](https://discuss.elastic.co/t/-dateparsefailure-again/87475 "2017-05-30T15:00:31Z")
Hi everyone! I have this configuration: filter{ grok{ match =\> { "message" =\> "%{TIMESTAMP\_ISO8601:timestamp} %{LOGLEVEL:severity} \[%{DATA:class}\] %{GREEDYDATA:message}" } } date{ match =\>… --- ## [Create a TimeStamp without having a real Date from a csv with a time-index](https://discuss.elastic.co/t/create-a-timestamp-without-having-a-real-date-from-a-csv-with-a-time-index/87584)
**Author:** [@S3nda8a](https://discuss.elastic.co/u/S3nda8a)\ **Replies:** 3\ **Last updated:** [May 30, 2017, 2:54pm UTC](https://discuss.elastic.co/t/create-a-timestamp-without-having-a-real-date-from-a-csv-with-a-time-index/87584 "2017-05-30T14:54:43Z")
Hello there, i want to visualize a lot of old csv files. The files contains a lot of data each. Structure: Time;DataSet1;\[...\];DataSet48 0.009618018;\[...\];0 0.019466060;\[...\];1 \[...\] 720.946525393;\[...\];0.012807 The … --- ## [Jdbc-streaming and jdbc-input gets different numeric results](https://discuss.elastic.co/t/jdbc-streaming-and-jdbc-input-gets-different-numeric-results/85778)
**Author:** [@endersonmaia](https://discuss.elastic.co/u/endersonmaia)\ **Replies:** 8\ **Last updated:** [May 30, 2017, 2:44pm UTC](https://discuss.elastic.co/t/jdbc-streaming-and-jdbc-input-gets-different-numeric-results/85778 "2017-05-30T14:44:04Z")
I'm using jdbc-input to get some sales data at a SQL Server satabase, and then I run a jdbc-streaming to complement the data. One thing I noticed is that for NUMERIC(18,4) data type, at the jdbc-input I get comma for d… --- ## [Mutate beats field isnt working in logstash5.0.2](https://discuss.elastic.co/t/mutate-beats-field-isnt-working-in-logstash5-0-2/87069)
**Author:** [@sgunnala](https://discuss.elastic.co/u/sgunnala)\ **Replies:** 2\ **Last updated:** [May 30, 2017, 1:39pm UTC](https://discuss.elastic.co/t/mutate-beats-field-isnt-working-in-logstash5-0-2/87069 "2017-05-30T13:39:41Z")
This pattern in the mutate in logstash.conf isnt working as expected to add hostname as a field. referring to this discussion https://discuss.elastic.co/t/mutate-beats-field/74940 it shows working as expected. Any help … --- ## [Requesting for installation guide for ELK on production server](https://discuss.elastic.co/t/requesting-for-installation-guide-for-elk-on-production-server/87435)
**Author:** [@sravankumarch](https://discuss.elastic.co/u/sravankumarch)\ **Replies:** 4\ **Last updated:** [May 30, 2017, 1:01pm UTC](https://discuss.elastic.co/t/requesting-for-installation-guide-for-elk-on-production-server/87435 "2017-05-30T13:01:45Z")
Hi friends can any one help me out for providing installation document ELK on ubuntu 16.0 production server, i am following digitalocean documents but them were not write procedure. --- ## [Compute duration between two ISO8601 fields in the same event](https://discuss.elastic.co/t/compute-duration-between-two-iso8601-fields-in-the-same-event/87497)
**Author:** [@Myles](https://discuss.elastic.co/u/Myles)\ **Replies:** 2\ **Last updated:** [May 30, 2017, 12:46pm UTC](https://discuss.elastic.co/t/compute-duration-between-two-iso8601-fields-in-the-same-event/87497 "2017-05-30T12:46:35Z")
I have an oracle ETL table that I am polling with the JDBC input and wanted to add a field as a calculation between the start and end time of the event which is logged in the same event. Here is what I've tried: fi… --- ## [Logstash returns 0 (success) on JDBC error](https://discuss.elastic.co/t/logstash-returns-0-success-on-jdbc-error/86658)
**Author:** [@dwjvaughan](https://discuss.elastic.co/u/dwjvaughan)\ **Replies:** 4\ **Last updated:** [May 30, 2017, 12:30pm UTC](https://discuss.elastic.co/t/logstash-returns-0-success-on-jdbc-error/86658 "2017-05-30T12:30:56Z")
Hi, We use logstash 5.2.2 on CentOS 7 to extract data from an Oracle database to put into Elasticsearch (5.2.2). I've noticed that if there is a failure (username/password etc.) on the Oracle end logstash still returns… --- ## [No logs sent from Logstash to Elasticsearch. Failed to create monitoring even](https://discuss.elastic.co/t/no-logs-sent-from-logstash-to-elasticsearch-failed-to-create-monitoring-even/87278)
**Author:** [@Vinuja1](https://discuss.elastic.co/u/Vinuja1)\ **Replies:** 0\ **Last updated:** [May 26, 2017, 3:43pm UTC](https://discuss.elastic.co/t/no-logs-sent-from-logstash-to-elasticsearch-failed-to-create-monitoring-even/87278 "2017-05-26T15:43:49Z")
Hi I am having the error : 017-02-14T12:51:07,865\]\[ERROR\]\[logstash.inputs.metrics \] Failed to create monitoring event {:message=\>"For path: events", :error=\>"LogStash::Instrument::MetricStore::MetricNotFound"}" I ve ad… --- ## ["Overwritten" files & logstash](https://discuss.elastic.co/t/overwritten-files-logstash/87405)
**Author:** [@mitabrev](https://discuss.elastic.co/u/mitabrev)\ **Replies:** 2\ **Last updated:** [May 30, 2017, 11:29am UTC](https://discuss.elastic.co/t/overwritten-files-logstash/87405 "2017-05-30T11:29:43Z")
Hi guys, Got one question. I would like to create bash scripts from my elastic server to harvest log files from remote directories. It would be probably very often, that one log file would be overwritten with the same… --- ## [Reading logs with text format and showing the count of success and failure](https://discuss.elastic.co/t/reading-logs-with-text-format-and-showing-the-count-of-success-and-failure/86663)
**Author:** [@BIRENDRA](https://discuss.elastic.co/u/BIRENDRA)\ **Replies:** 1\ **Last updated:** [May 30, 2017, 11:25am UTC](https://discuss.elastic.co/t/reading-logs-with-text-format-and-showing-the-count-of-success-and-failure/86663 "2017-05-30T11:25:43Z")
I am trying to read the log file given as text file.I need to take the count of success and fail lines and capture the reason for that, which is in the same line where we have succes and failure message. eg. 20170506-0… --- ## [Issue 411 Alert with Logstash](https://discuss.elastic.co/t/issue-411-alert-with-logstash/86657)
**Author:** [@SeifAllah\_Karaa](https://discuss.elastic.co/u/SeifAllah_Karaa)\ **Replies:** 1\ **Last updated:** [May 30, 2017, 11:10am UTC](https://discuss.elastic.co/t/issue-411-alert-with-logstash/86657 "2017-05-30T11:10:35Z")
I have problem when i would like to connect Logstash with 411 . I have any problem in log files of apache server. $config\['elasticsearch'\] = \[ # Configuration for the 411 Alerts index. 'alerts' =\> \[ 'hosts' =\> \['h… --- ## [Parse csv file with file path](https://discuss.elastic.co/t/parse-csv-file-with-file-path/86649)
**Author:** [@Mouad\_Mimouni](https://discuss.elastic.co/u/Mouad_Mimouni)\ **Replies:** 1\ **Last updated:** [May 30, 2017, 11:00am UTC](https://discuss.elastic.co/t/parse-csv-file-with-file-path/86649 "2017-05-30T11:00:40Z")
I use logstash on docker, i want to parse a csv file with file plugin : input { file { path =\> "$file/path" start\_position =\> "beginning" } but i have nothing when i run logstash : The stdin plugin is now waiting f… --- ## [How can I make collectd report which device/mount point it is looking at?](https://discuss.elastic.co/t/how-can-i-make-collectd-report-which-device-mount-point-it-is-looking-at/32728)
**Author:** [@jerrac](https://discuss.elastic.co/u/jerrac)\ **Replies:** 9\ **Last updated:** [May 30, 2017, 10:04am UTC](https://discuss.elastic.co/t/how-can-i-make-collectd-report-which-device-mount-point-it-is-looking-at/32728 "2017-05-30T10:04:20Z")
Using collectd i want to report disk space usage to ELK. I successfully enabled CollectD's df plugin, and it is sending data to ELK. But I don't see any field that says "this is the usage for /boot", or "this is the usag… --- ## [My timestamp can't get converted to a date type](https://discuss.elastic.co/t/my-timestamp-cant-get-converted-to-a-date-type/87476)
**Author:** [@gregbk](https://discuss.elastic.co/u/gregbk)\ **Replies:** 7\ **Last updated:** [May 30, 2017, 10:01am UTC](https://discuss.elastic.co/t/my-timestamp-cant-get-converted-to-a-date-type/87476 "2017-05-30T10:01:09Z")
Hello, I have some working timestamp detected as date in elasticsearch. But can't make this one works. Grok is matching well, and logstash is parsing well the line: 2017-05-29 10:27:41,756 | soft-1 | user1 | | 10.x.x.… --- ## [It is possible to create multiple groks for multiple paths?](https://discuss.elastic.co/t/it-is-possible-to-create-multiple-groks-for-multiple-paths/87530)
**Author:** [@tabs11](https://discuss.elastic.co/u/tabs11)\ **Replies:** 2\ **Last updated:** [May 30, 2017, 9:48am UTC](https://discuss.elastic.co/t/it-is-possible-to-create-multiple-groks-for-multiple-paths/87530 "2017-05-30T09:48:17Z")
I mean, define two paths, which the files (csv or log) in one path have 9 columns and in other path the files have 11 columns, then apply two different groks, one for the file with 9 columns and other for the file with 1… --- ## [Salesforce EventLogFIle Object impossible to retreive](https://discuss.elastic.co/t/salesforce-eventlogfile-object-impossible-to-retreive/85716)
**Author:** [@saversano](https://discuss.elastic.co/u/saversano)\ **Replies:** 13\ **Last updated:** [May 30, 2017, 8:35am UTC](https://discuss.elastic.co/t/salesforce-eventlogfile-object-impossible-to-retreive/85716 "2017-05-30T08:35:30Z")
Hello Team,, i'm trying to retreive Logfiles from EventLogFile using Logstash and Salesforce plugin for logstash on my sandbox. I am able to retreive some objects (as per Opportunity, Account, Contact, etc.) but not Eve… --- ## [Significant difference in size output because of an if statment (that shouldn't happend)](https://discuss.elastic.co/t/significant-difference-in-size-output-because-of-an-if-statment-that-shouldnt-happend/87461)
**Author:** [@sahar\_q](https://discuss.elastic.co/u/sahar_q)\ **Replies:** 3\ **Last updated:** [May 30, 2017, 8:09am UTC](https://discuss.elastic.co/t/significant-difference-in-size-output-because-of-an-if-statment-that-shouldnt-happend/87461 "2017-05-30T08:09:47Z")
Hi, For some reason that i dont get the follow input creates twice less the size(27KB) on the created index in elastic: output { if "ArpTable" in \[tags\] { elasticsearch { hosts =\> \["x"\] action =\> "index" index =\> … --- ## [Problem to convert lat and long for Kibana](https://discuss.elastic.co/t/problem-to-convert-lat-and-long-for-kibana/87449)
**Author:** [@FlorianMtin](https://discuss.elastic.co/u/FlorianMtin)\ **Replies:** 4\ **Last updated:** [May 30, 2017, 7:51am UTC](https://discuss.elastic.co/t/problem-to-convert-lat-and-long-for-kibana/87449 "2017-05-30T07:51:26Z")
Hello, I'm using Elastic Stack since few days and I'm having some difficulties to convert latitude and longitude into geo\_point type, I also try many solution on this forum but I can't resolve my problem :confused: . I… --- ## [Help with correct big config file characterization and output to multiple index's](https://discuss.elastic.co/t/help-with-correct-big-config-file-characterization-and-output-to-multiple-indexs/87366)
**Author:** [@sahar\_q](https://discuss.elastic.co/u/sahar_q)\ **Replies:** 13\ **Last updated:** [May 30, 2017, 7:00am UTC](https://discuss.elastic.co/t/help-with-correct-big-config-file-characterization-and-output-to-multiple-indexs/87366 "2017-05-30T07:00:39Z")
Hi guys, I want to know what is the best way to maintain a big configuration file. My plan is to get Different artifacts data through one port, apply filters to every artifact depending on what artifact it is and out… --- ## [Translate filter not working](https://discuss.elastic.co/t/translate-filter-not-working/87505)
**Author:** [@Sjaak01](https://discuss.elastic.co/u/Sjaak01)\ **Replies:** 1\ **Last updated:** [May 30, 2017, 6:35am UTC](https://discuss.elastic.co/t/translate-filter-not-working/87505 "2017-05-30T06:35:41Z")
Hi, I'm using the translation filter to look up some values. This works but the issue I'm running into is that I want to keep the original field untouched so I need to copy the field and replace the value in the new fi… --- ## [Since\_db](https://discuss.elastic.co/t/since-db/87459)
**Author:** [@Renu\_Sharma](https://discuss.elastic.co/u/Renu_Sharma)\ **Replies:** 3\ **Last updated:** [May 30, 2017, 5:45am UTC](https://discuss.elastic.co/t/since-db/87459 "2017-05-30T05:45:29Z")
Hi, I have doubt in since\_db which log-stash generate. is it necessary to delete since\_db file each time or we have any other way? --- ## [Logstash not updating documents JDBC Postgres](https://discuss.elastic.co/t/logstash-not-updating-documents-jdbc-postgres/87322)
**Author:** [@ImTheDeveloper](https://discuss.elastic.co/u/ImTheDeveloper)\ **Replies:** 4\ **Last updated:** [May 30, 2017, 5:33am UTC](https://discuss.elastic.co/t/logstash-not-updating-documents-jdbc-postgres/87322 "2017-05-30T05:33:05Z")
Hi All, Every hour my data is updated in Postgres. The data in my table is updated each hour with a full drop of the table and a new bulk insert of the data. Right now, I am running logstash JDBC on a schedule so that 5… --- ## [How to run logstash conf continuously](https://discuss.elastic.co/t/how-to-run-logstash-conf-continuously/87443)
**Author:** [@kabali12345](https://discuss.elastic.co/u/kabali12345)\ **Replies:** 2\ **Last updated:** [May 30, 2017, 1:01am UTC](https://discuss.elastic.co/t/how-to-run-logstash-conf-continuously/87443 "2017-05-30T01:01:48Z")
Hi....all Here i am trying to push elasticsearch data into RabbitMq here is my conf input { elasticsearch { hosts =\> "192.168.1.23:9200" index =\> "task-\*" query =\> '{ "query":{ "match\_all": {} }, "sort… --- ## [How the logstash s3 input plugin handle load balance?](https://discuss.elastic.co/t/how-the-logstash-s3-input-plugin-handle-load-balance/87064)
**Author:** [@hanks](https://discuss.elastic.co/u/hanks)\ **Replies:** 2\ **Last updated:** [May 30, 2017, 12:29am UTC](https://discuss.elastic.co/t/how-the-logstash-s3-input-plugin-handle-load-balance/87064 "2017-05-30T00:29:32Z")
Hi, the question is if I have multi logstash servers and set the same conf to collect the s3 bucket, is the result become duplicate? And how to setup load balance for s3 input plugin? Thank you for your time. [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=973) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=975)