# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=855 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 856 --- ## [Why i have this error :: \_jsonparsefailure](https://discuss.elastic.co/t/why-i-have-this-error-jsonparsefailure/111190)
**Author:** [@yaser](https://discuss.elastic.co/u/yaser)\ **Replies:** 2\ **Last updated:** [December 15, 2017, 4:34pm UTC](https://discuss.elastic.co/t/why-i-have-this-error-jsonparsefailure/111190 "2017-12-15T16:34:07Z")
i send string like this message filed from rabbitmq to logstash but i cant parse my json to my mapping! "message": "{DeviceID :2155CA", "tags": \[ "\_jsonparsefailure" \] }, --- ## [Logstash Tunings for Large Config (1000 Conditional Statements)](https://discuss.elastic.co/t/logstash-tunings-for-large-config-1000-conditional-statements/112012)
**Author:** [@bgeels](https://discuss.elastic.co/u/bgeels)\ **Replies:** 3\ **Last updated:** [December 15, 2017, 4:31pm UTC](https://discuss.elastic.co/t/logstash-tunings-for-large-config-1000-conditional-statements/112012 "2017-12-15T16:31:15Z")
Hi All, Are there any config tunings other than heap size that can be modified to allow logstash to load a large config file ( on the order of 1000 conditional statements )? Having anything more than about 200 conditio… --- ## [Repeating field name seems to silently fail](https://discuss.elastic.co/t/repeating-field-name-seems-to-silently-fail/111832)
**Author:** [@ccullen](https://discuss.elastic.co/u/ccullen)\ **Replies:** 2\ **Last updated:** [December 15, 2017, 4:18pm UTC](https://discuss.elastic.co/t/repeating-field-name-seems-to-silently-fail/111832 "2017-12-15T16:18:03Z")
For the last 6 weeks or so I have been developing Logstash filters for the various logs our network produces, and I have been on a steep learning curve. It's possible that I am simply ignorant of some basic fact, but I'… --- ## [Can you create a standard log entry that logstash will automatically parse](https://discuss.elastic.co/t/can-you-create-a-standard-log-entry-that-logstash-will-automatically-parse/111952)
**Author:** [@Sniipe](https://discuss.elastic.co/u/Sniipe)\ **Replies:** 8\ **Last updated:** [December 15, 2017, 4:13pm UTC](https://discuss.elastic.co/t/can-you-create-a-standard-log-entry-that-logstash-will-automatically-parse/111952 "2017-12-15T16:13:04Z")
I'm creating new logs (using log4net) and I was wondering if there is a standard where you can create an entry that Kibana will be able to come up with the available fields automatically? I just spotted this in another … --- ## [Find Elasticsearch Documents by JDBC input and update a field](https://discuss.elastic.co/t/find-elasticsearch-documents-by-jdbc-input-and-update-a-field/112006)
**Author:** [@wanna](https://discuss.elastic.co/u/wanna)\ **Replies:** 0\ **Last updated:** [December 15, 2017, 3:26pm UTC](https://discuss.elastic.co/t/find-elasticsearch-documents-by-jdbc-input-and-update-a-field/112006 "2017-12-15T15:26:37Z")
Hi, i have a jdbc input with user data. So i would like find all documents in Elasticsearch index with the user\_id of current input. Then i would like update field values of all the documents wich find with the user\_i… --- ## [JDBC issue when using paging](https://discuss.elastic.co/t/jdbc-issue-when-using-paging/111818)
**Author:** [@arisbanach](https://discuss.elastic.co/u/arisbanach)\ **Replies:** 0\ **Last updated:** [December 14, 2017, 4:57pm UTC](https://discuss.elastic.co/t/jdbc-issue-when-using-paging/111818 "2017-12-14T16:57:46Z")
I am not sure why this is happening, but I have a large Postgres database I'm importing from Amazon RDS. I have set the statement =\> "SELECT \* from mytable" and jdbc\_paging\_enabled =\> true and jdbc\_page\_size =\> 10000. T… --- ## [Problem with logstash input kinesis plugin while reading data from the stream](https://discuss.elastic.co/t/problem-with-logstash-input-kinesis-plugin-while-reading-data-from-the-stream/111995)
**Author:** [@kabali12345](https://discuss.elastic.co/u/kabali12345)\ **Replies:** 1\ **Last updated:** [December 15, 2017, 2:11pm UTC](https://discuss.elastic.co/t/problem-with-logstash-input-kinesis-plugin-while-reading-data-from-the-stream/111995 "2017-12-15T14:11:29Z")
Hi all.. I am trying to access kinesis stream i mean to read data from the stream here is my conf input { kinesis { application\_name =\> "logstash" kinesis\_stream\_name =\> "apple" region =\> "xxxxxx" code… --- ## [Use multiline and grok for splitting complex json input](https://discuss.elastic.co/t/use-multiline-and-grok-for-splitting-complex-json-input/111987)
**Author:** [@gassogba](https://discuss.elastic.co/u/gassogba)\ **Replies:** 1\ **Last updated:** [December 15, 2017, 1:59pm UTC](https://discuss.elastic.co/t/use-multiline-and-grok-for-splitting-complex-json-input/111987 "2017-12-15T13:59:35Z")
Hi, Here is the structure of my json input: { "searchMeta": { "interval": "S10", "sourceIds": \[ "test1" \], "startTimestamp": 1505558633, "endTimestamp": 1513334633, "startIndex": 0, "maxResult": 500, "prevRes… --- ## [Problems securing communication between filebeat and logstash](https://discuss.elastic.co/t/problems-securing-communication-between-filebeat-and-logstash/111993)
**Author:** [@strt](https://discuss.elastic.co/u/strt)\ **Replies:** 0\ **Last updated:** [December 15, 2017, 1:50pm UTC](https://discuss.elastic.co/t/problems-securing-communication-between-filebeat-and-logstash/111993 "2017-12-15T13:50:17Z")
I have the ELK stack installed with filebeat configured on the client machine. How do I go about securing the connection. I've tried here:- This comes with a message advising that the key should be certificate type p… --- ## [Stop logstash when finish import](https://discuss.elastic.co/t/stop-logstash-when-finish-import/111969)
**Author:** [@afonso.ribeiro](https://discuss.elastic.co/u/afonso.ribeiro)\ **Replies:** 4\ **Last updated:** [December 15, 2017, 1:11pm UTC](https://discuss.elastic.co/t/stop-logstash-when-finish-import/111969 "2017-12-15T13:11:16Z")
Hi guys, I know this topic has already been answered but I wanted to know if with these new release is there any news about this? If not, can anyone give an example of how to stop logstash with stdin or similar, automa… --- ## [Netflow missing field](https://discuss.elastic.co/t/netflow-missing-field/110520)
**Author:** [@tim-timmer](https://discuss.elastic.co/u/tim-timmer)\ **Replies:** 3\ **Last updated:** [December 15, 2017, 1:07pm UTC](https://discuss.elastic.co/t/netflow-missing-field/110520 "2017-12-15T13:07:22Z")
Hi everyone! I'm very new to ELK... So, I've got the following: Elasticsearch v6.0.0 + X-Pack Logstash v6.0.0 + X-Pack Kibana v6.0.0 + X-Pack I want to collect netflow data. The issue is: I've executed /usr/share/… --- ## [Questions about dead letter queue](https://discuss.elastic.co/t/questions-about-dead-letter-queue/110798)
**Author:** [@system](https://discuss.elastic.co/u/system)\ **Replies:** 1\ **Last updated:** [December 15, 2017, 12:38pm UTC](https://discuss.elastic.co/t/questions-about-dead-letter-queue/110798 "2017-12-15T12:38:14Z")
Hi there, we've recently started looking into setting up a dead\_letter\_queue pipeline at Discourse but I had some questions which wasn't immediately clear from the documentation. Does the DLQ log files get deleted at … --- ## [Problem with logstash input kinesis plugin](https://discuss.elastic.co/t/problem-with-logstash-input-kinesis-plugin/111981)
**Author:** [@kabali12345](https://discuss.elastic.co/u/kabali12345)\ **Replies:** 0\ **Last updated:** [December 15, 2017, 12:22pm UTC](https://discuss.elastic.co/t/problem-with-logstash-input-kinesis-plugin/111981 "2017-12-15T12:22:57Z")
Hi all.. I am trying to access kinesis stream i mean to read data from the stream here is my conf input { kinesis { application\_name =\> "logstash" kinesis\_stream\_name =\> "apple" region =\> "xxxxxx" code… --- ## [How to filter kafka topics based on their names in logstash conf in the output section using if-else condition for elastic search output](https://discuss.elastic.co/t/how-to-filter-kafka-topics-based-on-their-names-in-logstash-conf-in-the-output-section-using-if-else-condition-for-elastic-search-output/111863)
**Author:** [@KUMAR\_SATYAM](https://discuss.elastic.co/u/KUMAR_SATYAM)\ **Replies:** 1\ **Last updated:** [December 15, 2017, 10:43am UTC](https://discuss.elastic.co/t/how-to-filter-kafka-topics-based-on-their-names-in-logstash-conf-in-the-output-section-using-if-else-condition-for-elastic-search-output/111863 "2017-12-15T10:43:59Z")
I am using the new version of Kafka plugin for elkstack input { kafka { bootstrap\_servers =\> "1XX.X.X.X:9092" topics =\> \["test\_all\_logs","test\_apiserver\_logs","test\_orchestrator\_logs","test\_credentialstore\_log… --- ## [Ruby script files question](https://discuss.elastic.co/t/ruby-script-files-question/111947)
**Author:** [@Dan\_Markhasin](https://discuss.elastic.co/u/Dan_Markhasin)\ **Replies:** 1\ **Last updated:** [December 15, 2017, 10:34am UTC](https://discuss.elastic.co/t/ruby-script-files-question/111947 "2017-12-15T10:34:20Z")
Hi, Following the releast of Logstash 6.1 with support for loading ruby code from external files, I'd like to know what is the process for making changes to these external files? If we change the code in one of the fil… --- ## [Get back original message after grokking](https://discuss.elastic.co/t/get-back-original-message-after-grokking/111513)
**Author:** [@shwesinhan](https://discuss.elastic.co/u/shwesinhan)\ **Replies:** 7\ **Last updated:** [December 15, 2017, 10:30am UTC](https://discuss.elastic.co/t/get-back-original-message-after-grokking/111513 "2017-12-15T10:30:55Z")
hi all, I'm a beginner of elk. i'm trying to extract fields from log message event in logstash. after logstash extract fields from message, can i get back original message? and is there anyway to get back original mes… --- ## [Trigger Logstash JDBC Plugin](https://discuss.elastic.co/t/trigger-logstash-jdbc-plugin/111913)
**Author:** [@Sahul\_Hameed\_B](https://discuss.elastic.co/u/Sahul_Hameed_B)\ **Replies:** 5\ **Last updated:** [December 15, 2017, 10:28am UTC](https://discuss.elastic.co/t/trigger-logstash-jdbc-plugin/111913 "2017-12-15T10:28:27Z")
Once the new record is updated or insterted, i need to call this command logstash -f JDBCConnector.conf --debug again. Other than schedule command. is it possible to do that?? --- ## [\_source field gets: "http\_request\_failure"](https://discuss.elastic.co/t/source-field-gets-http-request-failure/111281)
**Author:** [@Rym\_Guerbi\_Michaut](https://discuss.elastic.co/u/Rym_Guerbi_Michaut)\ **Replies:** 5\ **Last updated:** [December 15, 2017, 10:22am UTC](https://discuss.elastic.co/t/source-field-gets-http-request-failure/111281 "2017-12-15T10:22:28Z")
Hi there, I want to fetch JSON formatted logs from a URL. I'm using HTTP\_POLLER. Sending it to Elasticsearch output under the name: "json\_10meilleurs\_logs" Here's my config file: input { http\_poller { urls =\> { … --- ## [Logstash one single point failure](https://discuss.elastic.co/t/logstash-one-single-point-failure/111220)
**Author:** [@f26227279](https://discuss.elastic.co/u/f26227279)\ **Replies:** 5\ **Last updated:** [December 15, 2017, 10:16am UTC](https://discuss.elastic.co/t/logstash-one-single-point-failure/111220 "2017-12-15T10:16:54Z")
Hi,everyone: this is my ELK structure: \[image\] I am afraid that if the logstash host crash, and all the system can't run. so if there's a way to prevent the situation occur, such as joining a logstash to stand by. t… --- ## [Logstash cant connect to Elasticsearch on localhost](https://discuss.elastic.co/t/logstash-cant-connect-to-elasticsearch-on-localhost/111699)
**Author:** [@bittujindani](https://discuss.elastic.co/u/bittujindani)\ **Replies:** 6\ **Last updated:** [December 15, 2017, 10:15am UTC](https://discuss.elastic.co/t/logstash-cant-connect-to-elasticsearch-on-localhost/111699 "2017-12-15T10:15:03Z")
Hi, I have setup Logstash, ES and Kibana on EC2 server and used localhost:9200 in all three setup yml file. curl command is also working fine. but once I am pushing msg using jdbc I am getting below error. JDBC working… --- ## [Grok Failure with Logstash 6.0](https://discuss.elastic.co/t/grok-failure-with-logstash-6-0/108381)
**Author:** [@holiday-sunrise](https://discuss.elastic.co/u/holiday-sunrise)\ **Replies:** 8\ **Last updated:** [December 15, 2017, 9:00am UTC](https://discuss.elastic.co/t/grok-failure-with-logstash-6-0/108381 "2017-12-15T09:00:41Z")
hi i´m trying to filter MSReportserver logs. But i always get an "Grokfalure" library!WindowsService\_20!37d8!11/06/2017-11:00:13:: i INFO: Cleaned 0 batch records, 0 policies, 0 sessions, 0 cache entries, 0 snapshots, … --- ## [Unable to start logstash on windows](https://discuss.elastic.co/t/unable-to-start-logstash-on-windows/111910)
**Author:** [@edna](https://discuss.elastic.co/u/edna)\ **Replies:** 2\ **Last updated:** [December 15, 2017, 7:50am UTC](https://discuss.elastic.co/t/unable-to-start-logstash-on-windows/111910 "2017-12-15T07:50:29Z")
I am unable to start Logstash. Each time I run a Logstash command, I keep getting the below error. Can someone please help me? \[image\] --- ## [How to include the host name in the graphite metric name](https://discuss.elastic.co/t/how-to-include-the-host-name-in-the-graphite-metric-name/111881)
**Author:** [@vaichidrewar](https://discuss.elastic.co/u/vaichidrewar)\ **Replies:** 0\ **Last updated:** [December 15, 2017, 12:53am UTC](https://discuss.elastic.co/t/how-to-include-the-host-name-in-the-graphite-metric-name/111881 "2017-12-15T00:53:20Z")
I want to deploy logstash config on multiple machines and push the metrics to graphite. So hostname needs to be part of the graphite key Here is the config I tried. %{host} is not getting replaced by the ip address or… --- ## [Logstash cloudtrail plugin](https://discuss.elastic.co/t/logstash-cloudtrail-plugin/111871)
**Author:** [@piyush](https://discuss.elastic.co/u/piyush)\ **Replies:** 2\ **Last updated:** [December 14, 2017, 10:47pm UTC](https://discuss.elastic.co/t/logstash-cloudtrail-plugin/111871 "2017-12-14T22:47:00Z")
Hi Team, I am forwarding aws cloudtrail logs to ES-5.5 and it is forwarded by logstash-5 (using cloudtrail plugin). While preparing visualizations, i realized that for some events "responseElements" field is not organiz… --- ## [Metrics for multiple log messages](https://discuss.elastic.co/t/metrics-for-multiple-log-messages/111860)
**Author:** [@vaichidrewar](https://discuss.elastic.co/u/vaichidrewar)\ **Replies:** 0\ **Last updated:** [December 14, 2017, 9:06pm UTC](https://discuss.elastic.co/t/metrics-for-multiple-log-messages/111860 "2017-12-14T21:06:57Z")
Hi, I want to parse log file and keep the count of various messages per minute I got the following config working for counting one type of message -. So below config is counting log lines containing "PixelLogParseAndEn… --- ## [Logstash 6.0.0 multiple pipelines not working](https://discuss.elastic.co/t/logstash-6-0-0-multiple-pipelines-not-working/110894)
**Author:** [@kyle\_che](https://discuss.elastic.co/u/kyle_che)\ **Replies:** 3\ **Last updated:** [December 14, 2017, 8:48pm UTC](https://discuss.elastic.co/t/logstash-6-0-0-multiple-pipelines-not-working/110894 "2017-12-14T20:48:12Z")
I am trying to configure multiple pipelines on logstash 6.0.0 and can not get it to work. here are my configurations. What am I missing? On top of that, all of the beats info being sent is being logged into the syslog… --- ## [JSON file processing - looking for direction](https://discuss.elastic.co/t/json-file-processing-looking-for-direction/111811)
**Author:** [@Blake\_2112](https://discuss.elastic.co/u/Blake_2112)\ **Replies:** 2\ **Last updated:** [December 14, 2017, 7:45pm UTC](https://discuss.elastic.co/t/json-file-processing-looking-for-direction/111811 "2017-12-14T19:45:04Z")
To start, I have been successful procssing single JSON files with the following LS config file. I use the 'exec' command to cat the JSON file. Works fine. The issue I have is that I want to be able to process 100 differ… --- ## [Logstash filter spec passing, but same message fails in real environment](https://discuss.elastic.co/t/logstash-filter-spec-passing-but-same-message-fails-in-real-environment/111810)
**Author:** [@anthony-o](https://discuss.elastic.co/u/anthony-o)\ **Replies:** 0\ **Last updated:** [December 14, 2017, 4:19pm UTC](https://discuss.elastic.co/t/logstash-filter-spec-passing-but-same-message-fails-in-real-environment/111810 "2017-12-14T16:19:30Z")
I'm trying to create a logstash filter to parse SPARQL queries and store them in ES. I've got very strange results: some messages tested in rspec tests and correctly passing them are failing in Logstash real runtime env… --- ## ["The source does not contain any versions of 'logstash'" when trying to create a logstash filter plugin](https://discuss.elastic.co/t/the-source-does-not-contain-any-versions-of-logstash-when-trying-to-create-a-logstash-filter-plugin/110399)
**Author:** [@anthony-o](https://discuss.elastic.co/u/anthony-o)\ **Replies:** 3\ **Last updated:** [December 14, 2017, 4:31pm UTC](https://discuss.elastic.co/t/the-source-does-not-contain-any-versions-of-logstash-when-trying-to-create-a-logstash-filter-plugin/110399 "2017-12-14T16:31:48Z")
If I follow the current official doc to create a logstash filter plugin, it ends up with the following error: Fetching https://github.com/elastic/logstash Fetching gem metadata from https://rubygems.org/......... Could … --- ## [Reading logs from remote OS](https://discuss.elastic.co/t/reading-logs-from-remote-os/111803)
**Author:** [@LogMe](https://discuss.elastic.co/u/LogMe)\ **Replies:** 1\ **Last updated:** [December 14, 2017, 4:06pm UTC](https://discuss.elastic.co/t/reading-logs-from-remote-os/111803 "2017-12-14T16:06:19Z")
Hi, I have logstash configured in windows, but all my servers are in Linux. So in the conf file, I like to directly access the logs from remote location. can any one advice on this? [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=854) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=856)