# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=572 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 573 --- ## [Logstash: Input from SQS does not get inserted to InfluxDB](https://discuss.elastic.co/t/logstash-input-from-sqs-does-not-get-inserted-to-influxdb/174996)
**Author:** [@smchae](https://discuss.elastic.co/u/smchae)\ **Replies:** 20\ **Last updated:** [April 4, 2019, 8:14am UTC](https://discuss.elastic.co/t/logstash-input-from-sqs-does-not-get-inserted-to-influxdb/174996 "2019-04-04T08:14:31Z")
I am trying to insert data to InfluxDB using Logstash with input method SQS: I send message to Sqs and Logstash reads the messages from Sqs and tries to put data to InfluxDB. I have set logstash like below, however, the… --- ## [JDBC postgres multiple statement syntax](https://discuss.elastic.co/t/jdbc-postgres-multiple-statement-syntax/175193)
**Author:** [@henriot-gricad](https://discuss.elastic.co/u/henriot-gricad)\ **Replies:** 2\ **Last updated:** [April 4, 2019, 7:55am UTC](https://discuss.elastic.co/t/jdbc-postgres-multiple-statement-syntax/175193 "2019-04-04T07:55:16Z")
Hi all, How do you specify multiple statements for JDBC postgres queries in logstash? As per the doc it's possible : \_configuring\_multiple\_sql\_statements but there is no detail as to how to achieve this. I have one DB… --- ## [Old vs new configuration settings](https://discuss.elastic.co/t/old-vs-new-configuration-settings/175330)
**Author:** [@rog\_NB](https://discuss.elastic.co/u/rog_NB)\ **Replies:** 1\ **Last updated:** [April 4, 2019, 7:25am UTC](https://discuss.elastic.co/t/old-vs-new-configuration-settings/175330 "2019-04-04T07:25:10Z")
Hello I'm upgrade my Elasticsearch/Logstash Cluster form 6.6.x to 6.7.0 I saw in the logs some notice, that I'm using old config settings, which will not work after 7.0.0. Like old: xpack.monitoring.elasticsearch.url … --- ## [Logstash performance with large numbers of pipelines](https://discuss.elastic.co/t/logstash-performance-with-large-numbers-of-pipelines/175319)
**Author:** [@Shishire](https://discuss.elastic.co/u/Shishire)\ **Replies:** 0\ **Last updated:** [April 4, 2019, 5:10am UTC](https://discuss.elastic.co/t/logstash-performance-with-large-numbers-of-pipelines/175319 "2019-04-04T05:10:18Z")
I need to collect data from several hundred different types of logs, each with a slightly different format from each other (organic growth of logging formats). I would prefer to institute a separate pipeline for each ty… --- ## [Write output csv file with header](https://discuss.elastic.co/t/write-output-csv-file-with-header/175311)
**Author:** [@Sanriya\_Krimzn](https://discuss.elastic.co/u/Sanriya_Krimzn)\ **Replies:** 0\ **Last updated:** [April 4, 2019, 4:24am UTC](https://discuss.elastic.co/t/write-output-csv-file-with-header/175311 "2019-04-04T04:24:40Z")
Hi there,,, I am Formatting my input file and writing the formatted input to a .csv file. Using Below code snippet. output { elasticsearch { hosts =\> \["localhost:9200"\] index =\> "Sample\_index" document\_type =\> "sam… --- ## [Nested json to filter](https://discuss.elastic.co/t/nested-json-to-filter/175300)
**Author:** [@koti](https://discuss.elastic.co/u/koti)\ **Replies:** 0\ **Last updated:** [April 4, 2019, 1:43am UTC](https://discuss.elastic.co/t/nested-json-to-filter/175300 "2019-04-04T01:43:26Z")
I have the scenario like if the input log having the nested json i need to skip the log . no need to send the output log . Please help me. Thanks in Advance --- ## [DeadLetterQueue Plugin Error](https://discuss.elastic.co/t/deadletterqueue-plugin-error/175289)
**Author:** [@Karthik\_Ramachandran](https://discuss.elastic.co/u/Karthik_Ramachandran)\ **Replies:** 0\ **Last updated:** [April 4, 2019, 12:00am UTC](https://discuss.elastic.co/t/deadletterqueue-plugin-error/175289 "2019-04-04T00:00:01Z")
Whenever the logstash is restarted, getting plugin error as below 23:46:27.262 \[\[main\]\ **Author:** [@Ashok\_Arbhavi](https://discuss.elastic.co/u/Ashok_Arbhavi)\ **Replies:** 0\ **Last updated:** [April 3, 2019, 10:19pm UTC](https://discuss.elastic.co/t/help-with-the-grok-pattern/175283 "2019-04-03T22:19:45Z") Hi, I am trying to parse the following Log Events in Logstash Using Grok Pattern: \[3/19/19 9:11:44:401 UTC\] 0000017f ServletWrappe I com.ibm.ws.webcontainer.servlet.ServletWrapper init SRVE0242I: \[FileNetEngine\] \[/acce… --- ## [Json Parser failure when the message contains emoji unicodes](https://discuss.elastic.co/t/json-parser-failure-when-the-message-contains-emoji-unicodes/175224)
**Author:** [@Denis\_Galvao](https://discuss.elastic.co/u/Denis_Galvao)\ **Replies:** 5\ **Last updated:** [April 3, 2019, 8:42pm UTC](https://discuss.elastic.co/t/json-parser-failure-when-the-message-contains-emoji-unicodes/175224 "2019-04-03T20:42:56Z")
Hi there. Logstash log a \_jasonparsefailure when the message contains some emojis.. { "@timestamp" =\> 2019-04-03T14:22:08.147Z, "@version" =\> "1", "message" =\> "{\\\\\\"event\\\\\\":\\\\\\"WhatsAppMessagesLog\\\\… --- ## [How to use token based authentication in logstash output and input](https://discuss.elastic.co/t/how-to-use-token-based-authentication-in-logstash-output-and-input/175197)
**Author:** [@r.ganeshbabu](https://discuss.elastic.co/u/r.ganeshbabu)\ **Replies:** 0\ **Last updated:** [April 3, 2019, 12:55pm UTC](https://discuss.elastic.co/t/how-to-use-token-based-authentication-in-logstash-output-and-input/175197 "2019-04-03T12:55:02Z")
Hi All, We are building customized beats application for our environment and in which we are sending data to beats logstash output and we will be receiving the data in logstash input. I read the documentation where we … --- ## [Set Logdate as @timestamp using logstash](https://discuss.elastic.co/t/set-logdate-as-timestamp-using-logstash/175204)
**Author:** [@Vaibhav\_Thapliyal](https://discuss.elastic.co/u/Vaibhav_Thapliyal)\ **Replies:** 6\ **Last updated:** [April 3, 2019, 7:32pm UTC](https://discuss.elastic.co/t/set-logdate-as-timestamp-using-logstash/175204 "2019-04-03T19:32:12Z")
Hi Everyone, I am trying to parse a nginx log file and want to set logdate as @timestamp. I am using the date filter plugin for this. Here's a sample date from my log file: 03/Apr/2019:16:33:03 +0530 The date plugin … --- ## [Analyser les logs en temps réel avec ELK](https://discuss.elastic.co/t/analyser-les-logs-en-temps-reel-avec-elk/175218)
**Author:** [@nejmeddine\_ammar](https://discuss.elastic.co/u/nejmeddine_ammar)\ **Replies:** 2\ **Last updated:** [April 3, 2019, 7:12pm UTC](https://discuss.elastic.co/t/analyser-les-logs-en-temps-reel-avec-elk/175218 "2019-04-03T19:12:48Z")
Hi, I am setting up ELK to analyze logs, but I can Analyze the logs in real time with ELK in windows, without having to re-run the pipeline how can I do that and what do I need? --- ## [How To use date Field in RAW data to used as @timestamp in Grok pattern](https://discuss.elastic.co/t/how-to-use-date-field-in-raw-data-to-used-as-timestamp-in-grok-pattern/174950)
**Author:** [@Nabil\_Mohamed](https://discuss.elastic.co/u/Nabil_Mohamed)\ **Replies:** 6\ **Last updated:** [April 3, 2019, 7:11pm UTC](https://discuss.elastic.co/t/how-to-use-date-field-in-raw-data-to-used-as-timestamp-in-grok-pattern/174950 "2019-04-03T19:11:59Z")
Hi all , i need to use the date field in my Raw Data to used as the @timestamp in my index: RAW record data in the Filebeat --\> 20190402-000037,8120791274.50,7174800715.40,4.18,/run 11.00% / 77.00% /data 55.00% /boot … --- ## [Unable to index nested xml rest api data](https://discuss.elastic.co/t/unable-to-index-nested-xml-rest-api-data/175173)
**Author:** [@Gauti](https://discuss.elastic.co/u/Gauti)\ **Replies:** 10\ **Last updated:** [April 3, 2019, 6:18pm UTC](https://discuss.elastic.co/t/unable-to-index-nested-xml-rest-api-data/175173 "2019-04-03T18:18:35Z")
Hi All, WE are trying to integrate sciencelogic performance data into elasticsearch, for which we are trying through logstash below is my config input { http\_poller { urls =\> { url =\> "https://poc.sciencelo… --- ## [File input doesn t read incoming Json file from web application](https://discuss.elastic.co/t/file-input-doesn-t-read-incoming-json-file-from-web-application/172499)
**Author:** [@hilne](https://discuss.elastic.co/u/hilne)\ **Replies:** 4\ **Last updated:** [April 3, 2019, 6:18pm UTC](https://discuss.elastic.co/t/file-input-doesn-t-read-incoming-json-file-from-web-application/172499 "2019-04-03T18:18:33Z")
Hi everyone, I'm trying to parse incoming json files from my web app stored in folder /tmp/test/\*.json. The problem is that logstash doesnt read my files when there are created. I have to modify them so that logstash p… --- ## [Logstash: Exclude separator lines in logs](https://discuss.elastic.co/t/logstash-exclude-separator-lines-in-logs/175071)
**Author:** [@DougR](https://discuss.elastic.co/u/DougR)\ **Replies:** 1\ **Last updated:** [April 2, 2019, 9:23pm UTC](https://discuss.elastic.co/t/logstash-exclude-separator-lines-in-logs/175071 "2019-04-02T21:23:55Z")
We have several applications, which will frequently insert separator lines into logs. I.e., the log entry looks like: 2019-04-02 06:20:17,453 method\_name INFO ========================================================== I… --- ## [\[Logstash\] \[JDBC\] \[mongoDB\] MissingConverterException for date fields](https://discuss.elastic.co/t/logstash-jdbc-mongodb-missingconverterexception-for-date-fields/175242)
**Author:** [@zaratustra689](https://discuss.elastic.co/u/zaratustra689)\ **Replies:** 0\ **Last updated:** [April 3, 2019, 4:04pm UTC](https://discuss.elastic.co/t/logstash-jdbc-mongodb-missingconverterexception-for-date-fields/175242 "2019-04-03T16:04:45Z")
Hello! I was trying to connect mongoDB with Logstash, but when a collection has fields with dates, Logstash logs show me this error and does not create the index or input data. I'm using Logstash version 6.7.0 \[WARN \]… --- ## [Json parse return filled with NULL](https://discuss.elastic.co/t/json-parse-return-filled-with-null/175175)
**Author:** [@ep4sh](https://discuss.elastic.co/u/ep4sh)\ **Replies:** 9\ **Last updated:** [April 3, 2019, 3:35pm UTC](https://discuss.elastic.co/t/json-parse-return-filled-with-null/175175 "2019-04-03T15:35:24Z")
I parse JSON, and even If JSON is invalid I want to save it. So I created an additional variable "invalid". If there is an exception, the method should return "saved" invalid var, but it doesn't - def jsonParse(str, ev… --- ## [Logstash throwing 406 when running as Service on Linux](https://discuss.elastic.co/t/logstash-throwing-406-when-running-as-service-on-linux/174976)
**Author:** [@kumarvivek633](https://discuss.elastic.co/u/kumarvivek633)\ **Replies:** 8\ **Last updated:** [April 3, 2019, 3:24pm UTC](https://discuss.elastic.co/t/logstash-throwing-406-when-running-as-service-on-linux/174976 "2019-04-03T15:24:31Z")
I had setup Elk on Linux box with Filebeat as data shipper. I am getting very weird issue there. When i am running logstash as service with service logstash start in that case it is getting the data from file beat but th… --- ## [Logstash XML file parsing - awkward Splitting event Problem](https://discuss.elastic.co/t/logstash-xml-file-parsing-awkward-splitting-event-problem/174906)
**Author:** [@NativAt](https://discuss.elastic.co/u/NativAt)\ **Replies:** 10\ **Last updated:** [April 3, 2019, 3:14pm UTC](https://discuss.elastic.co/t/logstash-xml-file-parsing-awkward-splitting-event-problem/174906 "2019-04-03T15:14:10Z")
I'm working with ELK 6.7.0 on docker with official images. This is my conf file: input { file { path =\> "/usr/share/logstash/logs/\*.xml" type =\> "xml" sincedb\_path =\> "/dev/null" codec =\> multiline { … --- ## [Http\_poller input plugin error](https://discuss.elastic.co/t/http-poller-input-plugin-error/174979)
**Author:** [@HaranKumar](https://discuss.elastic.co/u/HaranKumar)\ **Replies:** 18\ **Last updated:** [April 3, 2019, 2:57pm UTC](https://discuss.elastic.co/t/http-poller-input-plugin-error/174979 "2019-04-03T14:57:56Z")
Hello Team, Kindly help me with below error when i try to pull data from a URL in csv format and store it in a csv file output. I have been trying googling around but could not get any answer. last hope is elastic com… --- ## [Logstash: How to stop logstash once the scheduled job is complete](https://discuss.elastic.co/t/logstash-how-to-stop-logstash-once-the-scheduled-job-is-complete/174917)
**Author:** [@Ram18](https://discuss.elastic.co/u/Ram18)\ **Replies:** 2\ **Last updated:** [April 3, 2019, 2:56pm UTC](https://discuss.elastic.co/t/logstash-how-to-stop-logstash-once-the-scheduled-job-is-complete/174917 "2019-04-03T14:56:12Z")
Hi, I have cron scheduled jobs, which would append the data into Elasticsearch indexes. The scheduled job is supposed to run on daily basis. I have different set of files to load. So I need to know, how we can stop the … --- ## [Lumberjack output "Broken Pipe"](https://discuss.elastic.co/t/lumberjack-output-broken-pipe/174936)
**Author:** [@gruselglatz](https://discuss.elastic.co/u/gruselglatz)\ **Replies:** 8\ **Last updated:** [April 3, 2019, 2:12pm UTC](https://discuss.elastic.co/t/lumberjack-output-broken-pipe/174936 "2019-04-03T14:12:52Z")
Hi, i've tried the lumberjack output to utilize the graylog beats input, but it doesn't get a connection. IDK if this is a problem on the logstash or graylog side. Does someone have tried to connect lumberjack to the b… --- ## [Parse XML](https://discuss.elastic.co/t/parse-xml/175150)
**Author:** [@Vivek\_Sharma](https://discuss.elastic.co/u/Vivek_Sharma)\ **Replies:** 1\ **Last updated:** [April 3, 2019, 2:06pm UTC](https://discuss.elastic.co/t/parse-xml/175150 "2019-04-03T14:06:33Z")
Hi For data Order Event, below are the values that are coming in XML: OrderType : Direct OrderDate : 20190401 But the XML contains the field name and field values in below format: \ \OrderType\<… --- ## [Issue parsing xml](https://discuss.elastic.co/t/issue-parsing-xml/175148)
**Author:** [@Vivek\_Sharma](https://discuss.elastic.co/u/Vivek_Sharma)\ **Replies:** 1\ **Last updated:** [April 3, 2019, 1:57pm UTC](https://discuss.elastic.co/t/issue-parsing-xml/175148 "2019-04-03T13:57:00Z")
Hi I have an xml with format as below: \ \ \ \ **Author:** [@tony21177](https://discuss.elastic.co/u/tony21177)\ **Replies:** 1\ **Last updated:** [April 3, 2019, 1:49pm UTC](https://discuss.elastic.co/t/frequently-modifing-the-configuration-of-syslog-output-plugin-randomly-cause-stalling-threads-info/175082 "2019-04-03T13:49:21Z") Hello, My program will frequently modify the configure of syslog output plugin, the host may be reachable or unreachable. I try many times,and it works well,but randomly the log threw out stalling\_threads\_info. At th… --- ## [How to access nested message in logstash KV filter](https://discuss.elastic.co/t/how-to-access-nested-message-in-logstash-kv-filter/175087)
**Author:** [@Vincent\_Chen](https://discuss.elastic.co/u/Vincent_Chen)\ **Replies:** 1\ **Last updated:** [April 3, 2019, 1:44pm UTC](https://discuss.elastic.co/t/how-to-access-nested-message-in-logstash-kv-filter/175087 "2019-04-03T13:44:06Z")
Hi everyone, I want to use KV filter to parse a nested message field, see below for my sample json: { "\_index": "myindex", "\_type": "doc", "\_id": "myid", "\_score": 5.339951, "\_source": { "ms… --- ## [Logstash to es，Deprecated field \[\_routing\] used, expected \[routing\] instead](https://discuss.elastic.co/t/logstash-to-es-deprecated-field-routing-used-expected-routing-instead/175084)
**Author:** [@ZQbd](https://discuss.elastic.co/u/ZQbd)\ **Replies:** 1\ **Last updated:** [April 3, 2019, 1:31pm UTC](https://discuss.elastic.co/t/logstash-to-es-deprecated-field-routing-used-expected-routing-instead/175084 "2019-04-03T13:31:40Z")
I use logstash to es ，logstash output configuration is as follows but has been an error， Deprecated field \[\_routing\] used, expected \[routing\] instead，why and how to solve ？ output { elasticsearch { hosts =\> \["elastics… --- ## [How can ı split url using grok pattern?](https://discuss.elastic.co/t/how-can-i-split-url-using-grok-pattern/174987)
**Author:** [@khergner](https://discuss.elastic.co/u/khergner)\ **Replies:** 5\ **Last updated:** [April 3, 2019, 1:11pm UTC](https://discuss.elastic.co/t/how-can-i-split-url-using-grok-pattern/174987 "2019-04-03T13:11:15Z")
I want to use grok pattern but ı cant't bellow mode . /xx.test.com/hls/media\_600699.ts I'd like result { url:"xx.test.com", media:"hls",....... } How can ı do? --- ## [I can run multiple config file](https://discuss.elastic.co/t/i-can-run-multiple-config-file/175158)
**Author:** [@keep](https://discuss.elastic.co/u/keep)\ **Replies:** 1\ **Last updated:** [April 3, 2019, 12:58pm UTC](https://discuss.elastic.co/t/i-can-run-multiple-config-file/175158 "2019-04-03T12:58:51Z")
logstash -f .\\config1\\sample\_1.conf and logstash -f .\\config1\\sample\_2.conf without a problem : \[2019-04-03T16:51:38,043\]\[WARN \]\[logstash.config.source.multilocal\] Ignoring the 'pipelines.yml' file because modules or… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=571) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=573)