# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1050 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1051 --- ## [Failed to Import JSON data to elastic via logstash](https://discuss.elastic.co/t/failed-to-import-json-data-to-elastic-via-logstash/72269)
**Author:** [@gutasaputra](https://discuss.elastic.co/u/gutasaputra)\ **Replies:** 1\ **Last updated:** [January 20, 2017, 8:10am UTC](https://discuss.elastic.co/t/failed-to-import-json-data-to-elastic-via-logstash/72269 "2017-01-20T08:10:26Z")
hi, i tried to import json data to my elastic via logstash. this is my config : config\_online.conf input { file { path =\> \["/usr/local/Cellar/logstash/5.1.1/config\_oneline.json"\] start\_position =\> "beginning" } … --- ## [OUI issues](https://discuss.elastic.co/t/oui-issues/72182)
**Author:** [@zen\_spartan](https://discuss.elastic.co/u/zen_spartan)\ **Replies:** 1\ **Last updated:** [January 20, 2017, 8:08am UTC](https://discuss.elastic.co/t/oui-issues/72182 "2017-01-20T08:08:43Z")
Successfully installed the OUI plugin but the following error keeps occuring within the logstash log: \[2017-01-19T14:48:07,125\]\[ERROR\]\[logstash.pipeline \] Exception in pipelineworker, the pipeline stopped proces… --- ## [Filebeat: Wrong index structure](https://discuss.elastic.co/t/filebeat-wrong-index-structure/38700)
**Author:** [@pbmsys](https://discuss.elastic.co/u/pbmsys)\ **Replies:** 13\ **Last updated:** [January 20, 2017, 7:24am UTC](https://discuss.elastic.co/t/filebeat-wrong-index-structure/38700 "2017-01-20T07:24:30Z")
Hi. I am having some difficulties with the indexing of filebeat instead of using the "filebeat-YYYY.MM.dd" it sporadically indexes as %{\[@metadata\]\[beat\]}-%{+YYYY.MM.dd}.. like it does not handle the metadata: \*\* curl… --- ## [Adding specific Log Format to logstash](https://discuss.elastic.co/t/adding-specific-log-format-to-logstash/72187)
**Author:** [@Maher.Glenza](https://discuss.elastic.co/u/Maher.Glenza)\ **Replies:** 3\ **Last updated:** [January 20, 2017, 7:16am UTC](https://discuss.elastic.co/t/adding-specific-log-format-to-logstash/72187 "2017-01-20T07:16:17Z")
hi i m working with logstash 5.1.2 and i m having specific logs from my application i want to know if i can create a log format for my logs so that i can subdivise them then make a customized one for elasticsearch outp… --- ## [Spreadsheet\_safe setting in csv output plugin?](https://discuss.elastic.co/t/spreadsheet-safe-setting-in-csv-output-plugin/72251)
**Author:** [@Yaswanth](https://discuss.elastic.co/u/Yaswanth)\ **Replies:** 1\ **Last updated:** [January 20, 2017, 7:14am UTC](https://discuss.elastic.co/t/spreadsheet-safe-setting-in-csv-output-plugin/72251 "2017-01-20T07:14:08Z")
HI, Why this spreadsheet\_safe plugin is used in logstash and how does it effect my output? I went through this site but didn't get a clear picture of why it is used? https://www.elastic.co/guide/en/logstash/current/p… --- ## [Logstash Config Variables](https://discuss.elastic.co/t/logstash-config-variables/72203)
**Author:** [@kopacko](https://discuss.elastic.co/u/kopacko)\ **Replies:** 1\ **Last updated:** [January 20, 2017, 6:48am UTC](https://discuss.elastic.co/t/logstash-config-variables/72203 "2017-01-20T06:48:36Z")
In the logstash.yml file, there is a line: node.name: Is this field, node.name a variable that can be used within Logstash config files? --- ## [Logstash service issues (LOGSTASH\_HOME / path.settings)](https://discuss.elastic.co/t/logstash-service-issues-logstash-home-path-settings/72259)
**Author:** [@mrbanzai](https://discuss.elastic.co/u/mrbanzai)\ **Replies:** 1\ **Last updated:** [January 20, 2017, 5:26am UTC](https://discuss.elastic.co/t/logstash-service-issues-logstash-home-path-settings/72259 "2017-01-20T05:26:19Z")
In attempting to debug why the Logstash 5.1.2 wasn't starting after an installation / upgrade using the Puppet module (https://github.com/elastic/puppet-logstash), I discovered some inconsistencies between the error mess… --- ## [Logstash/Elasticsearch 5.x Errors](https://discuss.elastic.co/t/logstash-elasticsearch-5-x-errors/71781)
**Author:** [@kopacko](https://discuss.elastic.co/u/kopacko)\ **Replies:** 20\ **Last updated:** [January 20, 2017, 1:22am UTC](https://discuss.elastic.co/t/logstash-elasticsearch-5-x-errors/71781 "2017-01-20T01:22:10Z")
Forgive me, for I am a n00b when I comes to Unix or even Logstash/Elasticsearch. I had several localized clusters online for almost a year when I began upgrading them to 5.x. My environment is such that, each location o… --- ## [Pipe output plugin not passing event](https://discuss.elastic.co/t/pipe-output-plugin-not-passing-event/28130)
**Author:** [@jimmydigital](https://discuss.elastic.co/u/jimmydigital)\ **Replies:** 6\ **Last updated:** [January 19, 2017, 11:13pm UTC](https://discuss.elastic.co/t/pipe-output-plugin-not-passing-event/28130 "2017-01-19T23:13:12Z")
Hello.. I've written a python script to create tickets when certain SNMP traps are received. It works great at the command line when I cat a sample event into it.. but when run from logstash with the pipe plugin it's … --- ## [Logstash influxdb](https://discuss.elastic.co/t/logstash-influxdb/72175)
**Author:** [@irom77](https://discuss.elastic.co/u/irom77)\ **Replies:** 4\ **Last updated:** [January 19, 2017, 8:30pm UTC](https://discuss.elastic.co/t/logstash-influxdb/72175 "2017-01-19T20:30:03Z")
Having problems with setup of logstash-output-influxdb. My config is below: input { udp { port =\> 11514 type =\> "threat" } } filter { if \[type\] =="threat" { … --- ## [Could not start Logstash 5.0 as a service on CentOS 7](https://discuss.elastic.co/t/could-not-start-logstash-5-0-as-a-service-on-centos-7/64791)
**Author:** [@chuenl](https://discuss.elastic.co/u/chuenl)\ **Replies:** 4\ **Last updated:** [January 19, 2017, 8:25pm UTC](https://discuss.elastic.co/t/could-not-start-logstash-5-0-as-a-service-on-centos-7/64791 "2017-01-19T20:25:33Z")
I am running CentOS 7. I had followed the instruction and used yum to install: https://www.elastic.co/guide/en/logstash/5.0/installing-logstash.html I put my conf file at /etc/logstash/conf.d. Launching logstash man… --- ## [Logstash 5.1.2 not working with Java 8](https://discuss.elastic.co/t/logstash-5-1-2-not-working-with-java-8/72201)
**Author:** [@mohsin106](https://discuss.elastic.co/u/mohsin106)\ **Replies:** 0\ **Last updated:** [January 19, 2017, 5:49pm UTC](https://discuss.elastic.co/t/logstash-5-1-2-not-working-with-java-8/72201 "2017-01-19T17:49:27Z")
Hi, I'm new to elastic and was trying to get Logstash working on my RHEL 6.8 server. The Java I have installed is (openjdk version "1.8.0\_111") however when I try running the "/usr/share/logstash/bin/logstash -e 'input {… --- ## [Replace @timestamp with actual timestamp from log file](https://discuss.elastic.co/t/replace-timestamp-with-actual-timestamp-from-log-file/72017)
**Author:** [@ivten](https://discuss.elastic.co/u/ivten)\ **Replies:** 15\ **Last updated:** [January 19, 2017, 4:59pm UTC](https://discuss.elastic.co/t/replace-timestamp-with-actual-timestamp-from-log-file/72017 "2017-01-19T16:59:07Z")
Hi, I have json log in the following format: {"@timestamp":"2017-01-18T11:41:28.753Z","source":"host1","level":"INFO","message":"Some log event"} the log is read by filebeat and sent to redis/logstash/elasticsearch/… --- ## [Basic nested JSON parse failure](https://discuss.elastic.co/t/basic-nested-json-parse-failure/41013)
**Author:** [@Patrick\_Barker](https://discuss.elastic.co/u/Patrick_Barker)\ **Replies:** 10\ **Last updated:** [January 19, 2017, 4:24pm UTC](https://discuss.elastic.co/t/basic-nested-json-parse-failure/41013 "2017-01-19T16:24:14Z")
Hey, I am new to log stash and surely I must be doing someone wrong. When I sending basic nested json from python 3 it fails: Message=json.dumps({'default': 'default', 'time': 'again', 'this': {'go': 'by'}}) My log s… --- ## [Type of logs and errors](https://discuss.elastic.co/t/type-of-logs-and-errors/72174)
**Author:** [@Maher.Glenza](https://discuss.elastic.co/u/Maher.Glenza)\ **Replies:** 0\ **Last updated:** [January 19, 2017, 1:57pm UTC](https://discuss.elastic.co/t/type-of-logs-and-errors/72174 "2017-01-19T13:57:02Z")
hi , plz i m a new user of ELK Stack and i m working with logstash 5.1.2 to receive my specefic logs from a java ee application this is one line example of the log : 1e045e2f-a06b-40c9-954e-cc26b0ead93a|20170118160224… --- ## [Data loading in real time using logstash csv input](https://discuss.elastic.co/t/data-loading-in-real-time-using-logstash-csv-input/69552)
**Author:** [@sabyasachi\_mallick](https://discuss.elastic.co/u/sabyasachi_mallick)\ **Replies:** 9\ **Last updated:** [January 19, 2017, 1:13pm UTC](https://discuss.elastic.co/t/data-loading-in-real-time-using-logstash-csv-input/69552 "2017-01-19T13:13:15Z")
i am fetching data using a python script and dumping into a csv file. And using that csv file as input in my logstash config file.Each time my python code runs , it opens and overwrites all that csv fil.But sometimes i m… --- ## [Logstah throttle filter dropping messages indefinitely](https://discuss.elastic.co/t/logstah-throttle-filter-dropping-messages-indefinitely/64585)
**Author:** [@rcanzanese](https://discuss.elastic.co/u/rcanzanese)\ **Replies:** 3\ **Last updated:** [January 19, 2017, 12:12pm UTC](https://discuss.elastic.co/t/logstah-throttle-filter-dropping-messages-indefinitely/64585 "2017-01-19T12:12:05Z")
Trying to use the throttle plugin in logstash to drop messages if I am receiving them at too high a rate. My goal is to: drop excess messages if more than 100 received in a 10 second window To accomplish this, I use… --- ## [Logstash grokparsefailure - message pattern problem](https://discuss.elastic.co/t/logstash-grokparsefailure-message-pattern-problem/72157)
**Author:** [@elasticheart](https://discuss.elastic.co/u/elasticheart)\ **Replies:** 2\ **Last updated:** [January 19, 2017, 11:56am UTC](https://discuss.elastic.co/t/logstash-grokparsefailure-message-pattern-problem/72157 "2017-01-19T11:56:21Z")
Hi, I am trying to parse the following message; ##SEVERE 05-Jan-2017 06:02:25.562 Fatal error## I am using grok in Logstash like below; filter { grok { match =\> { "message" =\> "^##(?\(SEVERE|INFO|WARN)… --- ## [Performance Tips For Logstash And Monitoring Its Queue](https://discuss.elastic.co/t/performance-tips-for-logstash-and-monitoring-its-queue/72141)
**Author:** [@Nitz](https://discuss.elastic.co/u/Nitz)\ **Replies:** 2\ **Last updated:** [January 19, 2017, 9:55am UTC](https://discuss.elastic.co/t/performance-tips-for-logstash-and-monitoring-its-queue/72141 "2017-01-19T09:55:45Z")
Hi, We have logs infrastructure, built in this form: 1. Filebeat agents consume logs and output them to one topic in Kafka. We are talking about ~50k messages per sec. This part of the architecture works great. 2. We h… --- ## [Not able to convert the Grokked value to timestamp](https://discuss.elastic.co/t/not-able-to-convert-the-grokked-value-to-timestamp/72115)
**Author:** [@Prateek\_Kshtriya](https://discuss.elastic.co/u/Prateek_Kshtriya)\ **Replies:** 2\ **Last updated:** [January 19, 2017, 9:36am UTC](https://discuss.elastic.co/t/not-able-to-convert-the-grokked-value-to-timestamp/72115 "2017-01-19T09:36:34Z")
Hi All, I am trying to capture the time in the below message- 02 Sep 2016 06:31:03:Master: WARNING - \>: OL:0-1232 GHOW: error found in waiting table: Problem 1- The time stamp is not getting picked by "DATESTAMP" g… --- ## [Logstash with web interface](https://discuss.elastic.co/t/logstash-with-web-interface/69803)
**Author:** [@udaykirankona](https://discuss.elastic.co/u/udaykirankona)\ **Replies:** 10\ **Last updated:** [January 19, 2017, 9:28am UTC](https://discuss.elastic.co/t/logstash-with-web-interface/69803 "2017-01-19T09:28:45Z")
Hi, Is there any possibility of using logstash from web interface. Like, can we create/modify logstash configuration from front end application. My requirement is to create application to process "any logs/ any log… --- ## [Data loading into ES index is very slow](https://discuss.elastic.co/t/data-loading-into-es-index-is-very-slow/69164)
**Author:** [@udaykirankona](https://discuss.elastic.co/u/udaykirankona)\ **Replies:** 12\ **Last updated:** [January 19, 2017, 9:17am UTC](https://discuss.elastic.co/t/data-loading-into-es-index-is-very-slow/69164 "2017-01-19T09:17:47Z")
HI, I am trying to load data from Mysql to elasticsearch using logstash, but its loading very slow. 6500/minute into elasticsearch. The following is my Logstash config input { jdbc { type =\> "type" jdbc\_dri… --- ## [Jdbc - logstash service changes timezone affecting data in kibana](https://discuss.elastic.co/t/jdbc-logstash-service-changes-timezone-affecting-data-in-kibana/72137)
**Author:** [@gilisade](https://discuss.elastic.co/u/gilisade)\ **Replies:** 0\ **Last updated:** [January 19, 2017, 9:05am UTC](https://discuss.elastic.co/t/jdbc-logstash-service-changes-timezone-affecting-data-in-kibana/72137 "2017-01-19T09:05:31Z")
hi there i have a jdbc configuration to get data from oracle tables. when running the configuration with the logstash service (meaning putting it in /etc/logstash/conf.d ) and restarting the service i get data with ti… --- ## [LogstashJ : Java bindings for Logstash](https://discuss.elastic.co/t/logstashj-java-bindings-for-logstash/41892)
**Author:** [@Mogztter](https://discuss.elastic.co/u/Mogztter)\ **Replies:** 5\ **Last updated:** [January 19, 2017, 7:47am UTC](https://discuss.elastic.co/t/logstashj-java-bindings-for-logstash/41892 "2017-01-19T07:47:22Z")
Hello, I've just started working on a Java binding for Logstash. My initial goal was to be able to write unit tests in Java but once we have a binding in place we could do so much more. To give you some context, I'm … --- ## [Logstash command not found](https://discuss.elastic.co/t/logstash-command-not-found/72112)
**Author:** [@gutasaputra](https://discuss.elastic.co/u/gutasaputra)\ **Replies:** 5\ **Last updated:** [January 19, 2017, 7:28am UTC](https://discuss.elastic.co/t/logstash-command-not-found/72112 "2017-01-19T07:28:51Z")
hi there, i want to deploy logstash in ubuntu server. i follow this tutorial : Installing Logstash | Logstash Reference \[8.11\] | Elastic i start running logstash by using this comand : sudo systemctl start logstas… --- ## [MSSQL datetime column parsing good format mapping and good date filter?](https://discuss.elastic.co/t/mssql-datetime-column-parsing-good-format-mapping-and-good-date-filter/70946)
**Author:** [@Lemec\_Cinq](https://discuss.elastic.co/u/Lemec_Cinq)\ **Replies:** 13\ **Last updated:** [January 19, 2017, 6:38am UTC](https://discuss.elastic.co/t/mssql-datetime-column-parsing-good-format-mapping-and-good-date-filter/70946 "2017-01-19T06:38:24Z")
I am importing a MS SQL table with a datetime column. It makes stored values of this type: 2008-02-18 00:00:00.000 2009-04-28 00:00:00.000 2000-05-03 00:00:00.000 ... by default jdbc input plugin does not detect thi… --- ## [Logstash JDBC Output to postgreSQL](https://discuss.elastic.co/t/logstash-jdbc-output-to-postgresql/72119)
**Author:** [@Phi1001](https://discuss.elastic.co/u/Phi1001)\ **Replies:** 0\ **Last updated:** [January 19, 2017, 6:36am UTC](https://discuss.elastic.co/t/logstash-jdbc-output-to-postgresql/72119 "2017-01-19T06:36:32Z")
i would like to put a json in my database It works fine but i would like to convert the json "properties" in string to store it in 1 column "name":"Product", "properties": { … --- ## [What's better, multi config or one config(multi input)](https://discuss.elastic.co/t/whats-better-multi-config-or-one-config-multi-input/71925)
**Author:** [@leewig](https://discuss.elastic.co/u/leewig)\ **Replies:** 3\ **Last updated:** [January 19, 2017, 6:26am UTC](https://discuss.elastic.co/t/whats-better-multi-config-or-one-config-multi-input/71925 "2017-01-19T06:26:25Z")
hi, everybody. i have used logstash for collecting the data of mysql. the data of mysql have several tables, and i created config files like below. tb\_a.config input { jdbc { jdbc\_driver\_l… --- ## [SSL/Authentication: logstash -\> logstash](https://discuss.elastic.co/t/ssl-authentication-logstash-logstash/71541)
**Author:** [@Maria\_Delarosa](https://discuss.elastic.co/u/Maria_Delarosa)\ **Replies:** 1\ **Last updated:** [January 19, 2017, 2:39am UTC](https://discuss.elastic.co/t/ssl-authentication-logstash-logstash/71541 "2017-01-19T02:39:07Z")
I have two servers and each has a logstash instance running. On Server A, the apache logs are captured by logstash and sent to Server B. I am using the TCP pluging and it is listening/recieving data on port 5000 on Serve… --- ## [Delete elasticsearch document with logstash-jdbc-input](https://discuss.elastic.co/t/delete-elasticsearch-document-with-logstash-jdbc-input/47490)
**Author:** [@suntuo](https://discuss.elastic.co/u/suntuo)\ **Replies:** 12\ **Last updated:** [January 19, 2017, 1:43am UTC](https://discuss.elastic.co/t/delete-elasticsearch-document-with-logstash-jdbc-input/47490 "2017-01-19T01:43:48Z")
Insert and update work with below config file but elasticseach has no response as i excute delete-sql on mysql input { jdbc { jdbc\_driver\_library =\> "/opt/logstash-2.2.2/mysql-connector-java-5.1.38/mysql-connecto… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1049) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1051)