# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1207 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1208 --- ## [Lumberjack input + grok and multiline](https://discuss.elastic.co/t/lumberjack-input-grok-and-multiline/37739)
**Author:** [@nkoleff](https://discuss.elastic.co/u/nkoleff)\ **Replies:** 0\ **Last updated:** [December 22, 2015, 9:09am UTC](https://discuss.elastic.co/t/lumberjack-input-grok-and-multiline/37739 "2015-12-22T09:09:29Z")
Hello, I have to combine logstash's lumberjack input, which in my case is the main shipper for all types of logs from centralized location to logstash and then I need to filter it with grok. So far so good - but the is… --- ## [Mongo::Auth::Unauthorized: User rhone is not authorized to access rhone\_log](https://discuss.elastic.co/t/mongo-auth-unauthorized-user-rhone-is-not-authorized-to-access-rhone-log/37733)
**Author:** [@dabing](https://discuss.elastic.co/u/dabing)\ **Replies:** 0\ **Last updated:** [December 22, 2015, 5:30am UTC](https://discuss.elastic.co/t/mongo-auth-unauthorized-user-rhone-is-not-authorized-to-access-rhone-log/37733 "2015-12-22T05:30:20Z")
who can help me? I config the mongo output like this: mongodb { collection =\> "%{indexCollection}" database =\> "rhone\_log" uri =\> "mongodb://rhone:h… --- ## [Logstash and logstash-output-mongodb](https://discuss.elastic.co/t/logstash-and-logstash-output-mongodb/36777)
**Author:** [@sampasei](https://discuss.elastic.co/u/sampasei)\ **Replies:** 3\ **Last updated:** [December 22, 2015, 5:27am UTC](https://discuss.elastic.co/t/logstash-and-logstash-output-mongodb/36777 "2015-12-22T05:27:11Z")
Hi, i'm trying to use logstash with mongodb output plugin but i have some problems with floating point numbers. Below an example Failed to send event to MongoDB {:event=\>#\ **Author:** [@spuder](https://discuss.elastic.co/u/spuder)\ **Replies:** 2\ **Last updated:** [December 22, 2015, 1:31am UTC](https://discuss.elastic.co/t/logstash-parse-date-with-custom-timezone/37721 "2015-12-22T01:31:56Z") I have logs that have a field "timestamp" with the following format { "foo":"bar", "timeStamp":"2015/12/12 04:18:15.839Z" } Logstash doesn't recognize this as a time, so it imports it as a string and give… --- ## [Issues installing custom plugin](https://discuss.elastic.co/t/issues-installing-custom-plugin/36908)
**Author:** [@Russ\_Lavoie](https://discuss.elastic.co/u/Russ_Lavoie)\ **Replies:** 1\ **Last updated:** [December 21, 2015, 9:54pm UTC](https://discuss.elastic.co/t/issues-installing-custom-plugin/36908 "2015-12-21T21:54:42Z")
I am having issues installing a custom plugin and it is driving me nuts. Environment Ubuntu 12.0.4 64 Bit Installed Logstash via provided deb package (1.5.2) Followed the link https://www.elastic.co/guide/en/logstash… --- ## [Log4j SocketAppender. Logstash : tcp input --\> OK, log4j input --\> KO - SOLVED](https://discuss.elastic.co/t/log4j-socketappender-logstash-tcp-input-ok-log4j-input-ko-solved/36184)
**Author:** [@mturonne](https://discuss.elastic.co/u/mturonne)\ **Replies:** 20\ **Last updated:** [December 21, 2015, 11:14am UTC](https://discuss.elastic.co/t/log4j-socketappender-logstash-tcp-input-ok-log4j-input-ko-solved/36184 "2015-12-21T11:14:44Z")
Hello there, I am currently facing an issue when using SocketAppender in Log4j (v1.2) with log4j input in Logstash (v2.1). Issue is simple, I configured Log4j like this : Log4jConfiguration.xml : \< log4j:configurat… --- ## [Ingesting Elasticsearch Logs](https://discuss.elastic.co/t/ingesting-elasticsearch-logs/37671)
**Author:** [@dawiro](https://discuss.elastic.co/u/dawiro)\ **Replies:** 0\ **Last updated:** [December 21, 2015, 9:11am UTC](https://discuss.elastic.co/t/ingesting-elasticsearch-logs/37671 "2015-12-21T09:11:04Z")
Hi, If I send elasticsearch logs to logstash with filebeat will I need to do any grokking on those logs or does logstash recognise and parse the elasticsearch log and slow query log format by default? If there are filter… --- ## [Can we use logStach in Windows?](https://discuss.elastic.co/t/can-we-use-logstach-in-windows/37659)
**Author:** [@Amit2015](https://discuss.elastic.co/u/Amit2015)\ **Replies:** 2\ **Last updated:** [December 21, 2015, 5:42am UTC](https://discuss.elastic.co/t/can-we-use-logstach-in-windows/37659 "2015-12-21T05:42:37Z")
Hi, I have Windows 7 operating system. Can I configure Logstach on Windows??? If any tutorial please share --- ## [Logstash to read log files from remote Windows network drives](https://discuss.elastic.co/t/logstash-to-read-log-files-from-remote-windows-network-drives/37652)
**Author:** [@mguiwin](https://discuss.elastic.co/u/mguiwin)\ **Replies:** 1\ **Last updated:** [December 21, 2015, 3:31am UTC](https://discuss.elastic.co/t/logstash-to-read-log-files-from-remote-windows-network-drives/37652 "2015-12-21T03:31:31Z")
Hello, I'm new to ELK, and I would like to set up a solution to index Microsoft IIS and applicative .NET logs with ES. I'm aware about different approaches: 1) \[app servers: log files ➔ Logstash\] ➔ \[collecting ser… --- ## [Completely lost](https://discuss.elastic.co/t/completely-lost/37632)
**Author:** [@new2logstash](https://discuss.elastic.co/u/new2logstash)\ **Replies:** 5\ **Last updated:** [December 21, 2015, 2:38am UTC](https://discuss.elastic.co/t/completely-lost/37632 "2015-12-21T02:38:02Z")
Hello all. I have followed the excellent Digital Ocean filebeat/logstash/ES/Kibana (How To Install Elasticsearch, Logstash, and Kibana (ELK Stack) on CentOS 7 | DigitalOcean). I am getting messages from one of my files i… --- ## [Problem to starting logstash](https://discuss.elastic.co/t/problem-to-starting-logstash/37656)
**Author:** [@Sangheon](https://discuss.elastic.co/u/Sangheon)\ **Replies:** 0\ **Last updated:** [December 21, 2015, 12:16am UTC](https://discuss.elastic.co/t/problem-to-starting-logstash/37656 "2015-12-21T00:16:35Z")
Hi. I set up ELK for logging apache log. I installed a filebeat at apache server and a logstash 2.1 at log server when I started a logstash, i got an error message Logstash startup completed Beats input: unhandled… --- ## [Logstash 1.5.3 with Kafka 0.9's quota](https://discuss.elastic.co/t/logstash-1-5-3-with-kafka-0-9s-quota/37614)
**Author:** [@Deb](https://discuss.elastic.co/u/Deb)\ **Replies:** 1\ **Last updated:** [December 20, 2015, 4:44pm UTC](https://discuss.elastic.co/t/logstash-1-5-3-with-kafka-0-9s-quota/37614 "2015-12-20T16:44:03Z")
In Kafka 0.9 support for quota is added as mentioned here. I am having some doubts about the behavior of quota in accordance with logstash. So in Kafka 0.9 quota can be configured for each client-id. If one of the cli… --- ## [JDBC does not create .logstash\_jdbc\_last\_run](https://discuss.elastic.co/t/jdbc-does-not-create-logstash-jdbc-last-run/37334)
**Author:** [@aviv.ratzon](https://discuss.elastic.co/u/aviv.ratzon)\ **Replies:** 0\ **Last updated:** [December 16, 2015, 9:56am UTC](https://discuss.elastic.co/t/jdbc-does-not-create-logstash-jdbc-last-run/37334 "2015-12-16T09:56:00Z")
Everything worked OK, until I made some change I guess. I tried everything - different computer, different conf, redownloading the logstash and the plugin. It just won't create the file. I tried specifying another path … --- ## [Graylog2 didn't recognize gelf output from logstash，but try in nc test is ok， Thanks for any suggestion](https://discuss.elastic.co/t/graylog2-didnt-recognize-gelf-output-from-logstash-but-try-in-nc-test-is-ok-thanks-for-any-suggestion/37628)
**Author:** [@hong\_zhou](https://discuss.elastic.co/u/hong_zhou)\ **Replies:** 0\ **Last updated:** [December 19, 2015, 6:36pm UTC](https://discuss.elastic.co/t/graylog2-didnt-recognize-gelf-output-from-logstash-but-try-in-nc-test-is-ok-thanks-for-any-suggestion/37628 "2015-12-19T18:36:14Z")
graylog2 didn't recognize gelf output from logstash， and still cann't find the way to fix it 。 but when i try nc test direct to graylog2 ，everything is ok。 just not work when get from logstash gelf output when i try… --- ## [Can't filter Input on Filter Logstash](https://discuss.elastic.co/t/cant-filter-input-on-filter-logstash/37618)
**Author:** [@tatdat](https://discuss.elastic.co/u/tatdat)\ **Replies:** 0\ **Last updated:** [December 19, 2015, 11:09am UTC](https://discuss.elastic.co/t/cant-filter-input-on-filter-logstash/37618 "2015-12-19T11:09:48Z")
Hi, i try to install filebeat on web server with config filebeat: prospectors: - paths: /var/log/nginx/access.log input\_type: nginx-access-2 logstash: hosts: \["10.1.11.171:5044"\] index: filebeat-kibana-srv A… --- ## [Logstash-jdbc last\_run\_metadata\_path](https://discuss.elastic.co/t/logstash-jdbc-last-run-metadata-path/37617)
**Author:** [@liweigao](https://discuss.elastic.co/u/liweigao)\ **Replies:** 0\ **Last updated:** [December 19, 2015, 10:01am UTC](https://discuss.elastic.co/t/logstash-jdbc-last-run-metadata-path/37617 "2015-12-19T10:01:01Z")
Hello everyone ,Now I think of the data in my home and abroad real-time update, I tried many times last\_run\_metadata\_path will not change, I look at the official website found only to restart the service when he will upd… --- ## [Logstash jdbc for mysql](https://discuss.elastic.co/t/logstash-jdbc-for-mysql/36738)
**Author:** [@liweigao](https://discuss.elastic.co/u/liweigao)\ **Replies:** 6\ **Last updated:** [December 19, 2015, 9:53am UTC](https://discuss.elastic.co/t/logstash-jdbc-for-mysql/36738 "2015-12-19T09:53:07Z")
I have 500W data in the database, I use logstash input myspl to es, the service will die, what do I need? Exception when executing JDBC query {:exception=\>#, :level=\>:warn} --- ## [Best way to truncate a date field?](https://discuss.elastic.co/t/best-way-to-truncate-a-date-field/37507)
**Author:** [@CraigFoote](https://discuss.elastic.co/u/CraigFoote)\ **Replies:** 4\ **Last updated:** [December 18, 2015, 3:47pm UTC](https://discuss.elastic.co/t/best-way-to-truncate-a-date-field/37507 "2015-12-18T15:47:40Z")
I'm ingesting old logs and want to use a data field value in the index name: elasticsearch{ index =\> "logstash-systemname-%{day}" ... } It's a LogStash::TimeStamp field but I want to copy it and truncate the c… --- ## [Logstash input - access all files from a folder via HTTP](https://discuss.elastic.co/t/logstash-input-access-all-files-from-a-folder-via-http/37579)
**Author:** [@luis.carvalho](https://discuss.elastic.co/u/luis.carvalho)\ **Replies:** 0\ **Last updated:** [December 18, 2015, 1:46pm UTC](https://discuss.elastic.co/t/logstash-input-access-all-files-from-a-folder-via-http/37579 "2015-12-18T13:46:02Z")
Hi! I would like to know if it's possible to get all files from a folder (stored in a HTTP server) through the logstash input (e.g. using http\_poller). And if it's possible, what would be a good solution. Thanks, Luís … --- ## [JDBC Importer for Logstash is adding timezone to date time field](https://discuss.elastic.co/t/jdbc-importer-for-logstash-is-adding-timezone-to-date-time-field/37537)
**Author:** [@Senz79](https://discuss.elastic.co/u/Senz79)\ **Replies:** 3\ **Last updated:** [December 18, 2015, 12:13pm UTC](https://discuss.elastic.co/t/jdbc-importer-for-logstash-is-adding-timezone-to-date-time-field/37537 "2015-12-18T12:13:38Z")
Hello I am using jdbc importer to extract data from SQL Server to Elasticsearch. I am sharing the structure and i got configuration OK with logstash configtest. Now i am getting an weird issue there is an UsageDate fie… --- ## [How to filter out an xml response with a specific root element](https://discuss.elastic.co/t/how-to-filter-out-an-xml-response-with-a-specific-root-element/37557)
**Author:** [@davinders](https://discuss.elastic.co/u/davinders)\ **Replies:** 0\ **Last updated:** [December 18, 2015, 9:46am UTC](https://discuss.elastic.co/t/how-to-filter-out-an-xml-response-with-a-specific-root-element/37557 "2015-12-18T09:46:49Z")
In my logs many xml response are printing, I want to filter-out/extract specific xml responses in logstash. In below example there are two xml response, HtlResponse and SOAResult, I want to extract out say HtlResponse, … --- ## [Failed parsing date from field](https://discuss.elastic.co/t/failed-parsing-date-from-field/37542)
**Author:** [@nivetha](https://discuss.elastic.co/u/nivetha)\ **Replies:** 1\ **Last updated:** [December 18, 2015, 8:07am UTC](https://discuss.elastic.co/t/failed-parsing-date-from-field/37542 "2015-12-18T08:07:26Z")
I am trying to parse the date with different timezone, but I get the "Failed parsing date from field". I dont know where I go wrong. Here are the conf I use: filter{ date{ match =\> \[ "eventTime", "YYYY/MM/dd HH:mm:… --- ## [Predefined elasticsearch schema, not mapped with logstash data upload from mysql table column](https://discuss.elastic.co/t/predefined-elasticsearch-schema-not-mapped-with-logstash-data-upload-from-mysql-table-column/37541)
**Author:** [@krushnat\_khawale](https://discuss.elastic.co/u/krushnat_khawale)\ **Replies:** 0\ **Last updated:** [December 18, 2015, 7:18am UTC](https://discuss.elastic.co/t/predefined-elasticsearch-schema-not-mapped-with-logstash-data-upload-from-mysql-table-column/37541 "2015-12-18T07:18:21Z")
I'm using logstash to upload data from mysql to elasticsearch. I've created a 'index' named testindex as follows POST /insights{ "settings" : { "number\_of\_shards" : 1 }, "mappings" : { "type1" : { "\_source" :… --- ## [Logstash undefined method \`bson\_type' for #\ **Author:** [@dabing](https://discuss.elastic.co/u/dabing)\ **Replies:** 2\ **Last updated:** [December 18, 2015, 6:56am UTC](https://discuss.elastic.co/t/logstash-undefined-method-bson-type-for-bigdecimal/37531 "2015-12-18T06:56:46Z") when i use logstash-mongodb-output plugin, my log data include floating number,then have this problem?who can help me? --- ## [How to receive the file over a socket which is a input to logstash](https://discuss.elastic.co/t/how-to-receive-the-file-over-a-socket-which-is-a-input-to-logstash/37529)
**Author:** [@sharath3185](https://discuss.elastic.co/u/sharath3185)\ **Replies:** 1\ **Last updated:** [December 18, 2015, 4:50am UTC](https://discuss.elastic.co/t/how-to-receive-the-file-over-a-socket-which-is-a-input-to-logstash/37529 "2015-12-18T04:50:01Z")
Hi, I need to send a file over a socket connection (in 1024 bytes blocks) to a logstash service. I am trying to achieve this through python. The logstash service should use this as an input and parse the same. Is the… --- ## [Delete files and permanently deleting them](https://discuss.elastic.co/t/delete-files-and-permanently-deleting-them/37511)
**Author:** [@Don\_Pich](https://discuss.elastic.co/u/Don_Pich)\ **Replies:** 0\ **Last updated:** [December 17, 2015, 8:29pm UTC](https://discuss.elastic.co/t/delete-files-and-permanently-deleting-them/37511 "2015-12-17T20:29:45Z")
I have a Debian Wheezy server that has logstash-forwarder running on it. Periodically, my PRTG reports that the /var file system is full. When I look at it and do a lsof, it reveals that the process is holding the fil… --- ## [Parsing syslog data: Need help understanding documentation example](https://discuss.elastic.co/t/parsing-syslog-data-need-help-understanding-documentation-example/37416)
**Author:** [@gaurav1424](https://discuss.elastic.co/u/gaurav1424)\ **Replies:** 4\ **Last updated:** [December 17, 2015, 8:01pm UTC](https://discuss.elastic.co/t/parsing-syslog-data-need-help-understanding-documentation-example/37416 "2015-12-17T20:01:58Z")
Hello All, I am referring logstash example of parsing syslog data : https://www.elastic.co/guide/en/logstash/current/config-examples.html#\_processing\_syslog\_messages My input line is : Dec 3 01:22:48 arista1.lab Pr… --- ## [Does Logstash's RabbitMQ output plugin produce messages in bulk?](https://discuss.elastic.co/t/does-logstashs-rabbitmq-output-plugin-produce-messages-in-bulk/37368)
**Author:** [@sharon.c](https://discuss.elastic.co/u/sharon.c)\ **Replies:** 5\ **Last updated:** [December 17, 2015, 6:35pm UTC](https://discuss.elastic.co/t/does-logstashs-rabbitmq-output-plugin-produce-messages-in-bulk/37368 "2015-12-17T18:35:48Z")
I am a new user for Logstash, and I want to use Logstash's RabbitMQ output plugin to feed data to RabbitMQ, the event speed I am processing is 3000-4000 events per second, I want to know if RabbitMQ output plugin is doin… --- ## [How to handle multiple app logs using single log-stash instance](https://discuss.elastic.co/t/how-to-handle-multiple-app-logs-using-single-log-stash-instance/37472)
**Author:** [@sampath\_methuku](https://discuss.elastic.co/u/sampath_methuku)\ **Replies:** 1\ **Last updated:** [December 17, 2015, 3:37pm UTC](https://discuss.elastic.co/t/how-to-handle-multiple-app-logs-using-single-log-stash-instance/37472 "2015-12-17T15:37:03Z")
we have 5 types apps which generates log in different format , and I want to store these 5 apps logs in 5 different indexes , but I want to use only single log-stash instance . to differentiate the input at logstash l… --- ## [GUI for reading logs](https://discuss.elastic.co/t/gui-for-reading-logs/26157)
**Author:** [@Mark\_van\_Rossum](https://discuss.elastic.co/u/Mark_van_Rossum)\ **Replies:** 5\ **Last updated:** [December 17, 2015, 3:27pm UTC](https://discuss.elastic.co/t/gui-for-reading-logs/26157 "2015-12-17T15:27:08Z")
I have a typical ELK stack, and am a huge fan of Kibana for analysing structured logs like e.g. Apache access logs. I can draw graphs to my hearts content showing real time performace of our webservers which is awesome. … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1206) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1208)