# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=408 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 409 --- ## [Logstash Error to read SQL query for Firebird](https://discuss.elastic.co/t/logstash-error-to-read-sql-query-for-firebird/218388)
**Author:** [@eng.lucianamendes](https://discuss.elastic.co/u/eng.lucianamendes)\ **Replies:** 1\ **Last updated:** [February 7, 2020, 6:04pm UTC](https://discuss.elastic.co/t/logstash-error-to-read-sql-query-for-firebird/218388 "2020-02-07T18:04:56Z")
I´m using Logstash in Raspiberry Pi with OS Raspbian Jesse. It´s works with API, but it don't works with a Firebird. The error is : \[2020-02-07T14:43:11,666\]\[ERROR\]\[logstash.inputs.jdbc \] Java::JavaSql::SQLSyntaxE… --- ## [Reason for creating a custom document\_id?](https://discuss.elastic.co/t/reason-for-creating-a-custom-document-id/218368)
**Author:** [@\_finack](https://discuss.elastic.co/u/_finack)\ **Replies:** 1\ **Last updated:** [February 7, 2020, 3:46pm UTC](https://discuss.elastic.co/t/reason-for-creating-a-custom-document-id/218368 "2020-02-07T15:46:41Z")
I've seen various Logstash filter plugins customizing the document\_id field in pipeline filters (e.g. the HELK project). I'm trying to make sense of the reason for doing this. Doesn't a unique document\_id get applied aut… --- ## [Rename field winlog.event\_data.TargetUserName](https://discuss.elastic.co/t/rename-field-winlog-event-data-targetusername/218345)
**Author:** [@shortcommand](https://discuss.elastic.co/u/shortcommand)\ **Replies:** 1\ **Last updated:** [February 7, 2020, 2:16pm UTC](https://discuss.elastic.co/t/rename-field-winlog-event-data-targetusername/218345 "2020-02-07T14:16:42Z")
Hi, I'm running ELK stack with winlogbeat and I would like to change the name of the field winlog.event\_data.TargetUserName. Currently I've put this in my logstash config: filter{ if \[winlog\]\[event\_data\]\[TargetUse… --- ## [Parsing nested JSON in logstash](https://discuss.elastic.co/t/parsing-nested-json-in-logstash/218306)
**Author:** [@Radha\_Krishna](https://discuss.elastic.co/u/Radha_Krishna)\ **Replies:** 1\ **Last updated:** [February 7, 2020, 2:10pm UTC](https://discuss.elastic.co/t/parsing-nested-json-in-logstash/218306 "2020-02-07T14:10:12Z")
I'm facing an issue with indexing the following event in logstash. { "event\_id" : 132, "thread\_id" : 5664, "record\_number" : "1270538", "beat" : { "hostname" : "my\_host", "version" : "5.4.2", "name" : "events" } … --- ## [Drop filter in logstash with filebeat pipelines](https://discuss.elastic.co/t/drop-filter-in-logstash-with-filebeat-pipelines/218309)
**Author:** [@old\_chocobo](https://discuss.elastic.co/u/old_chocobo)\ **Replies:** 2\ **Last updated:** [February 7, 2020, 12:24pm UTC](https://discuss.elastic.co/t/drop-filter-in-logstash-with-filebeat-pipelines/218309 "2020-02-07T12:24:00Z")
Hello, I have a Filebeat+Logstash+Elasticsearch stack, it is working nice. I am using Filebeat module pipelines in Elasticsearch because I want to use default Filebeat dashboards in Kibana. Now I need to drop some log … --- ## [How to connect mongodb query to logstash plugins](https://discuss.elastic.co/t/how-to-connect-mongodb-query-to-logstash-plugins/218329)
**Author:** [@saravana\_hariharan](https://discuss.elastic.co/u/saravana_hariharan)\ **Replies:** 0\ **Last updated:** [February 7, 2020, 11:10am UTC](https://discuss.elastic.co/t/how-to-connect-mongodb-query-to-logstash-plugins/218329 "2020-02-07T11:10:28Z")
Hi Friends, i am tried to connect mongodb plugins to logstash it worked fine but filters plugins cant be worked there i need some process in inside the filter plugins, can you share logstash code for mongodb statement… --- ## [Help with logstash filter](https://discuss.elastic.co/t/help-with-logstash-filter/218154)
**Author:** [@ademir.andrade](https://discuss.elastic.co/u/ademir.andrade)\ **Replies:** 4\ **Last updated:** [February 7, 2020, 10:59am UTC](https://discuss.elastic.co/t/help-with-logstash-filter/218154 "2020-02-07T10:59:16Z")
Good morning, I would like a help to make a filter in the logstash, I do not want the logstash to process the one of a certain IP range, also including multicast and broadcast. I thank you for your help. --- ## [Filter with spaces and extra pattern](https://discuss.elastic.co/t/filter-with-spaces-and-extra-pattern/218162)
**Author:** [@Ben240489](https://discuss.elastic.co/u/Ben240489)\ **Replies:** 2\ **Last updated:** [February 7, 2020, 10:36am UTC](https://discuss.elastic.co/t/filter-with-spaces-and-extra-pattern/218162 "2020-02-07T10:36:48Z")
Hi guys, I have another parsing question :slight\_smile: My filebeat and logstash are working fine except for a problem in parsing a time in the format HH.MM.SS instead of the double point. My message currently looks l… --- ## [Remote filebeat ingest pipeline working with logstash](https://discuss.elastic.co/t/remote-filebeat-ingest-pipeline-working-with-logstash/218289)
**Author:** [@mahboubeh](https://discuss.elastic.co/u/mahboubeh)\ **Replies:** 1\ **Last updated:** [February 7, 2020, 10:01am UTC](https://discuss.elastic.co/t/remote-filebeat-ingest-pipeline-working-with-logstash/218289 "2020-02-07T10:01:59Z")
Hi i have installed logstash 7.4.1 in my vmware and elasticsearch 7.4.1 . besides that i have installed filebeat 7.4.1 in a remote machine and want to send logs with apache module pipeline into my logstash.then logstash… --- ## [Grock filter](https://discuss.elastic.co/t/grock-filter/218308)
**Author:** [@kau0826](https://discuss.elastic.co/u/kau0826)\ **Replies:** 0\ **Last updated:** [February 7, 2020, 9:50am UTC](https://discuss.elastic.co/t/grock-filter/218308 "2020-02-07T09:50:57Z")
Hi Team, I have a logs as below and its currently view in kinaba, but i need to get logs from start and end point. as a example in below logs i need to get logs between "API CALL" to "API CALL END" in kibana. could y… --- ## [Logstash default document type from logs to doc](https://discuss.elastic.co/t/logstash-default-document-type-from-logs-to-doc/217998)
**Author:** [@alessandro](https://discuss.elastic.co/u/alessandro)\ **Replies:** 2\ **Last updated:** [February 6, 2020, 3:26pm UTC](https://discuss.elastic.co/t/logstash-default-document-type-from-logs-to-doc/217998 "2020-02-06T15:26:08Z")
Hi forum, I would like to ask an advice for a smooth transition from Logstash version 5.6.4 to 6.8.6, given that I have recently updated succesfully both Elasticsearch and Kibana. My main doubts are about the breaking … --- ## [My Logstash throws error stating JavaLang::illegal state exception. I have attached my config file below. can someone help me out?](https://discuss.elastic.co/t/my-logstash-throws-error-stating-javalang-illegal-state-exception-i-have-attached-my-config-file-below-can-someone-help-me-out/218143)
**Author:** [@Varun\_Saraf](https://discuss.elastic.co/u/Varun_Saraf)\ **Replies:** 3\ **Last updated:** [February 7, 2020, 8:01am UTC](https://discuss.elastic.co/t/my-logstash-throws-error-stating-javalang-illegal-state-exception-i-have-attached-my-config-file-below-can-someone-help-me-out/218143 "2020-02-07T08:01:37Z")
input { file { path =\> "/Applications/data/cars.csv" start\_position =\> "beginning" sincedb\_path =\> "/dev/null" } } filter { csv { seperator =\> "," columns =\> \[ "maker", "mode… --- ## [Can logstash use syslog time as @timestamp](https://discuss.elastic.co/t/can-logstash-use-syslog-time-as-timestamp/218270)
**Author:** [@amhulli](https://discuss.elastic.co/u/amhulli)\ **Replies:** 1\ **Last updated:** [February 7, 2020, 6:41am UTC](https://discuss.elastic.co/t/can-logstash-use-syslog-time-as-timestamp/218270 "2020-02-07T06:41:39Z")
We have a server that is currently not forwarding logs for ingesting into elasticsearch. We are about to change that but want to ingest an existing year's worth of logs first. So that we can run time based searches in … --- ## [Alerting thru Logstash](https://discuss.elastic.co/t/alerting-thru-logstash/218266)
**Author:** [@paul1243](https://discuss.elastic.co/u/paul1243)\ **Replies:** 6\ **Last updated:** [February 7, 2020, 6:19am UTC](https://discuss.elastic.co/t/alerting-thru-logstash/218266 "2020-02-07T06:19:21Z")
Hi- I did installed metricbeats on a server, but for some reasons we stop the metricbeats service and start it back after some time. I would need an email alerting thru logstash only when its stopped and if the metricbe… --- ## [How to Ingest data from MongoDB database into ElasticSearch using Logstash](https://discuss.elastic.co/t/how-to-ingest-data-from-mongodb-database-into-elasticsearch-using-logstash/216245)
**Author:** [@jskishor](https://discuss.elastic.co/u/jskishor)\ **Replies:** 7\ **Last updated:** [February 7, 2020, 4:49am UTC](https://discuss.elastic.co/t/how-to-ingest-data-from-mongodb-database-into-elasticsearch-using-logstash/216245 "2020-02-07T04:49:14Z")
Hello Logstash/ES Gurus, I am new to ES and logstash, I need help on ingesting a user collection data from a mongo database into elastic search. I have used logstash-input-mongodb plugin and started the logstash and i… --- ## [Ruby filter "LogStash::ConfigurationError"](https://discuss.elastic.co/t/ruby-filter-logstash-configurationerror/218180)
**Author:** [@dishma237](https://discuss.elastic.co/u/dishma237)\ **Replies:** 6\ **Last updated:** [February 7, 2020, 4:19am UTC](https://discuss.elastic.co/t/ruby-filter-logstash-configurationerror/218180 "2020-02-07T04:19:33Z")
Hi Guys, got a config setup as below to receive snmp walks result and I a script to clean the data. However it fails to run because of an error on the ruby filter. I have tried my script on the output on irb and the scr… --- ## [Please, I need help with grok patterns](https://discuss.elastic.co/t/please-i-need-help-with-grok-patterns/218246)
**Author:** [@Fern148](https://discuss.elastic.co/u/Fern148)\ **Replies:** 1\ **Last updated:** [February 6, 2020, 8:24pm UTC](https://discuss.elastic.co/t/please-i-need-help-with-grok-patterns/218246 "2020-02-06T20:24:08Z")
Hello! All right? Please, I need help with grok patterns. When I test with Dev Tools -\> Grok Debugger it works perfectly, but when I include it in the logstash filter it returns \_grokparsefailure. Is there a way to ca… --- ## [Host data wont populate in dashboards when upgrading beats from 7.0 to 7.3 and using Logstash](https://discuss.elastic.co/t/host-data-wont-populate-in-dashboards-when-upgrading-beats-from-7-0-to-7-3-and-using-logstash/215344)
**Author:** [@Ryan\_Downey](https://discuss.elastic.co/u/Ryan_Downey)\ **Replies:** 7\ **Last updated:** [February 4, 2020, 7:06pm UTC](https://discuss.elastic.co/t/host-data-wont-populate-in-dashboards-when-upgrading-beats-from-7-0-to-7-3-and-using-logstash/215344 "2020-02-04T19:06:34Z")
ECE - Elasticsearch 7.3 Kibana -7 .3 Our metricbeat system overview dashboard will not display a Beat thats running 7.3 after upgrading from 7.0 if we run the beat through Logstash. The beat will populate in the dashb… --- ## [Logstash xml plugin not working](https://discuss.elastic.co/t/logstash-xml-plugin-not-working/217772)
**Author:** [@muireann](https://discuss.elastic.co/u/muireann)\ **Replies:** 3\ **Last updated:** [February 6, 2020, 6:12pm UTC](https://discuss.elastic.co/t/logstash-xml-plugin-not-working/217772 "2020-02-06T18:12:08Z")
hello, I'm trying to use the below logstash config to read from an xml file. The xml is being read into logstash as I can see the different lines from the xml printed in the "message" in the console when I run logstash. … --- ## [What is the equivalent of grock parse failure for a ruby script](https://discuss.elastic.co/t/what-is-the-equivalent-of-grock-parse-failure-for-a-ruby-script/218178)
**Author:** [@Incauto](https://discuss.elastic.co/u/Incauto)\ **Replies:** 1\ **Last updated:** [February 6, 2020, 5:10pm UTC](https://discuss.elastic.co/t/what-is-the-equivalent-of-grock-parse-failure-for-a-ruby-script/218178 "2020-02-06T17:10:47Z")
Hi I want to make shure that what I send to logstash doesnt contain errors so I wan to add something similar to grokparse failure , but for ruby. output { if "\_grokparsefailure" not in \[tags\] { … --- ## [Unable to load large set of records from a table in Servicenow into Elasticsearch using Logstash](https://discuss.elastic.co/t/unable-to-load-large-set-of-records-from-a-table-in-servicenow-into-elasticsearch-using-logstash/218085)
**Author:** [@ramypala](https://discuss.elastic.co/u/ramypala)\ **Replies:** 1\ **Last updated:** [February 6, 2020, 5:00pm UTC](https://discuss.elastic.co/t/unable-to-load-large-set-of-records-from-a-table-in-servicenow-into-elasticsearch-using-logstash/218085 "2020-02-06T17:00:31Z")
Hi Team, I am trying to load Servicenow tables in Elasticsearch using Logstash HTTP Poller Plugin.I am able to load a table with 42 records into Elasticsearch index without any issue.But when I tried to load a table wit… --- ## [Logstash processing order of S3 Buckets](https://discuss.elastic.co/t/logstash-processing-order-of-s3-buckets/218199)
**Author:** [@Reid\_Loper](https://discuss.elastic.co/u/Reid_Loper)\ **Replies:** 0\ **Last updated:** [February 6, 2020, 2:40pm UTC](https://discuss.elastic.co/t/logstash-processing-order-of-s3-buckets/218199 "2020-02-06T14:40:25Z")
I'm architecting a solution where Logstash will process files in an Amazon S3 bucket from multiple sources. Common sense would tell me that Logstash will process these files in a first in, first out order based on the t… --- ## [Pipeline throttle not working properly](https://discuss.elastic.co/t/pipeline-throttle-not-working-properly/218193)
**Author:** [@Henrique\_Gossi](https://discuss.elastic.co/u/Henrique_Gossi)\ **Replies:** 0\ **Last updated:** [February 6, 2020, 2:22pm UTC](https://discuss.elastic.co/t/pipeline-throttle-not-working-properly/218193 "2020-02-06T14:22:49Z")
Using throttle filter, same events get the tag "throttled" and some others don't. follow the content from throttle conf file: filter { throttle { before\_count =\> -1 after\_count =\> 1 period =\> "86400" # 1 … --- ## [Filter Plugins Are not working for Mongodb input](https://discuss.elastic.co/t/filter-plugins-are-not-working-for-mongodb-input/218185)
**Author:** [@saravana\_hariharan](https://discuss.elastic.co/u/saravana_hariharan)\ **Replies:** 0\ **Last updated:** [February 6, 2020, 1:36pm UTC](https://discuss.elastic.co/t/filter-plugins-are-not-working-for-mongodb-input/218185 "2020-02-06T13:36:13Z")
Hi i have used mutate filter ,it does work for oracle sql input , but its not working for mongodb input.please help. --- ## [Translate filter trouble](https://discuss.elastic.co/t/translate-filter-trouble/218007)
**Author:** [@raged](https://discuss.elastic.co/u/raged)\ **Replies:** 2\ **Last updated:** [February 6, 2020, 1:05pm UTC](https://discuss.elastic.co/t/translate-filter-trouble/218007 "2020-02-06T13:05:10Z")
Hello, I am attempting to utilize a pipeline that is for populating a json yml that looks like this: {"name":"PCNAME","collectionname":"Early Adopters - Ring 2 - Office","resourceid":16778707,"@version":"1","@timestamp… --- ## [Missing tags and extra fields](https://discuss.elastic.co/t/missing-tags-and-extra-fields/217992)
**Author:** [@Ben240489](https://discuss.elastic.co/u/Ben240489)\ **Replies:** 3\ **Last updated:** [February 6, 2020, 12:11pm UTC](https://discuss.elastic.co/t/missing-tags-and-extra-fields/217992 "2020-02-06T12:11:14Z")
Hi guys, Once again I have an issue trying out the ELK. In general, in Kibana I cannot see any tags I set and also my extra fields (e.g. number/date) displayed which I parsed. The message itself is seen in Kibana, line … --- ## [\_xpack appending to elasticsearch URI](https://discuss.elastic.co/t/xpack-appending-to-elasticsearch-uri/218167)
**Author:** [@gyana\_nayak](https://discuss.elastic.co/u/gyana_nayak)\ **Replies:** 0\ **Last updated:** [February 6, 2020, 12:06pm UTC](https://discuss.elastic.co/t/xpack-appending-to-elasticsearch-uri/218167 "2020-02-06T12:06:23Z")
Hi All , Keep on getting below error while connecting to elasticsearch 2020-02-06T17:12:19,846\]\[INFO \]\[logstash.outputs.elasticsearch\]\[main\] Using default mapping template \[2020-02-06T17:12:20,150\]\[ERROR\]\[logstash.out… --- ## [MSSQL jdbc jdbc\_user with DomainName](https://discuss.elastic.co/t/mssql-jdbc-jdbc-user-with-domainname/217842)
**Author:** [@ginu](https://discuss.elastic.co/u/ginu)\ **Replies:** 2\ **Last updated:** [February 6, 2020, 10:44am UTC](https://discuss.elastic.co/t/mssql-jdbc-jdbc-user-with-domainname/217842 "2020-02-06T10:44:38Z")
Hi everyone in Logstash pipeline I have the following connection string: jdbc\_user =\> "DomainName\\UserName" But looks like using "" is not working input { jdbc { jdbc\_validate\_connection =\> true jdbc\_drive… --- ## [Logstash 7.5.0 as Windows service using SC commands on command prompt](https://discuss.elastic.co/t/logstash-7-5-0-as-windows-service-using-sc-commands-on-command-prompt/218135)
**Author:** [@sagardhakate](https://discuss.elastic.co/u/sagardhakate)\ **Replies:** 0\ **Last updated:** [February 6, 2020, 10:07am UTC](https://discuss.elastic.co/t/logstash-7-5-0-as-windows-service-using-sc-commands-on-command-prompt/218135 "2020-02-06T10:07:34Z")
I am not able to run logstash 7.5.0 as windows service using SC commands on command prompt. C:\>sc \\VIMPHYT01-22 create "Elasticsearch Logstash-7.5.0" binpath=C:\\Users\\sagar.dhakate\\Software\\ElasticStack\\logstash-7.5.0\\l… --- ## [Elastic search jdbc output plugin](https://discuss.elastic.co/t/elastic-search-jdbc-output-plugin/216166)
**Author:** [@Mohanapriya\_Gopal](https://discuss.elastic.co/u/Mohanapriya_Gopal)\ **Replies:** 1\ **Last updated:** [February 6, 2020, 8:58am UTC](https://discuss.elastic.co/t/elastic-search-jdbc-output-plugin/216166 "2020-02-06T08:58:46Z")
We load data to elastic through logstash by using jdbc driver input plugin. Similarly when we delta load the index after any update or new insert i want to update my jdbc drive check point table after logstash event succ… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=407) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=409)