# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=722 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 723 --- ## [Weird Log Output](https://discuss.elastic.co/t/weird-log-output/139312)
**Author:** [@Marcos\_Felix](https://discuss.elastic.co/u/Marcos_Felix)\ **Replies:** 14\ **Last updated:** [July 13, 2018, 8:40pm UTC](https://discuss.elastic.co/t/weird-log-output/139312 "2018-07-13T20:40:52Z")
I am getting a weird log output and it seems to be an issue with grok Here is a sample of the logs: July 10th 2018, 09:31:00.427 type:random\_logs @timestamp:July 10th 2018, 09:31:00.427 tags:grokparsefailure host:10.1… --- ## [Or check in filter plugin](https://discuss.elastic.co/t/or-check-in-filter-plugin/139979)
**Author:** [@Honda\_fred\_elk](https://discuss.elastic.co/u/Honda_fred_elk)\ **Replies:** 2\ **Last updated:** [July 13, 2018, 4:39pm UTC](https://discuss.elastic.co/t/or-check-in-filter-plugin/139979 "2018-07-13T16:39:57Z")
Hello, I have logstash configuration to get two types of inputs, one from File another from Beats. So I'm setting type variable as "file\_input" and "beats\_input" respectively. Next In the filter plugin, I have to have… --- ## [Logstash error: parsing xml file](https://discuss.elastic.co/t/logstash-error-parsing-xml-file/139720)
**Author:** [@saisimo02](https://discuss.elastic.co/u/saisimo02)\ **Replies:** 3\ **Last updated:** [July 13, 2018, 3:19pm UTC](https://discuss.elastic.co/t/logstash-error-parsing-xml-file/139720 "2018-07-13T15:19:39Z")
Hello, I am new on ELK and I need your help. I would like to get some information about the cpu, memory. Those informative are generated every 30 minutes. My xml file \\\ **Author:** [@n1ur0](https://discuss.elastic.co/u/n1ur0)\ **Replies:** 0\ **Last updated:** [July 13, 2018, 3:17pm UTC](https://discuss.elastic.co/t/send-one-single-datapoint-for-each-timestamp-milisecond/139970 "2018-07-13T15:17:15Z") Hello, Reading through logstash plugins documentation, I couldn't find a way to send single data point for each generated @timestamp (1 datapoint/event per milisecond). Is that even a possibility? Appreciate the help! … --- ## [How to overwrite message (set to minus) if it is not overwrited in grok?](https://discuss.elastic.co/t/how-to-overwrite-message-set-to-minus-if-it-is-not-overwrited-in-grok/138518)
**Author:** [@lek](https://discuss.elastic.co/u/lek)\ **Replies:** 12\ **Last updated:** [July 13, 2018, 3:12pm UTC](https://discuss.elastic.co/t/how-to-overwrite-message-set-to-minus-if-it-is-not-overwrited-in-grok/138518 "2018-07-13T15:12:38Z")
Hello. In message there is json, but in some messages there is no json. I want set message to dash in first case (where there is no json). How to do it? Please, help me. cat /etc/logstash/mytests/test1 input { genera… --- ## [Transfer Docker log with gelf：Could not index event to Elasticsearch](https://discuss.elastic.co/t/transfer-docker-log-with-gelf-could-not-index-event-to-elasticsearch/139852)
**Author:** [@CCH0124](https://discuss.elastic.co/u/CCH0124)\ **Replies:** 1\ **Last updated:** [July 13, 2018, 2:59pm UTC](https://discuss.elastic.co/t/transfer-docker-log-with-gelf-could-not-index-event-to-elasticsearch/139852 "2018-07-13T14:59:12Z")
Logstash has the following error \[2018-07-13T00:28:09,022\]\[WARN \]\[logstash.outputs.elasticsearch\] Could not index event to Elasticsearch. {:status=\>400, :action=\>\["index", {:\_id=\>nil, :\_index=\>"filebeat-gelf-docker-2018… --- ## [Parsing nested json object and make as a single filed](https://discuss.elastic.co/t/parsing-nested-json-object-and-make-as-a-single-filed/139732)
**Author:** [@Charan\_Adabala](https://discuss.elastic.co/u/Charan_Adabala)\ **Replies:** 3\ **Last updated:** [July 13, 2018, 2:55pm UTC](https://discuss.elastic.co/t/parsing-nested-json-object-and-make-as-a-single-filed/139732 "2018-07-13T14:55:21Z")
I have a log file that's an array of objects that looks something like this: \[ { "cate1": "data1a", "cate2": "data2a" }, { "cate1": "data1b", "cate2": "data2b" } \] but I need output like this can you help me p… --- ## [Setting null value without removing the field](https://discuss.elastic.co/t/setting-null-value-without-removing-the-field/139963)
**Author:** [@imaad](https://discuss.elastic.co/u/imaad)\ **Replies:** 0\ **Last updated:** [July 13, 2018, 2:35pm UTC](https://discuss.elastic.co/t/setting-null-value-without-removing-the-field/139963 "2018-07-13T14:35:23Z")
Hello, I'm using a jms input to index data from queues, some fields contains two values ("2018-07-12T23:30:00.000+0000" and "2018-07-12T00:00:00.000+0000") that I have to replace with null value. The input message lo… --- ## [\_grokparsefailure in Kibana although Grok tester returns "matched"](https://discuss.elastic.co/t/grokparsefailure-in-kibana-although-grok-tester-returns-matched/139368)
**Author:** [@ggutsche](https://discuss.elastic.co/u/ggutsche)\ **Replies:** 5\ **Last updated:** [July 13, 2018, 2:37pm UTC](https://discuss.elastic.co/t/grokparsefailure-in-kibana-although-grok-tester-returns-matched/139368 "2018-07-13T14:37:02Z")
Hello I have the following log file entry 2018-07-10 15:24:00.000 DEBUG 5445 --- \[pool-2-thread-1\] d.d.e.g.s.LoginRateLimitingService : Exit: removeOutdatedLoginAttempts() I have written or generated the pattern… --- ## [Error running logsatsh](https://discuss.elastic.co/t/error-running-logsatsh/139952)
**Author:** [@Chiranjeevi\_Medichar](https://discuss.elastic.co/u/Chiranjeevi_Medichar)\ **Replies:** 0\ **Last updated:** [July 13, 2018, 1:26pm UTC](https://discuss.elastic.co/t/error-running-logsatsh/139952 "2018-07-13T13:26:46Z")
Sending Logstash's logs to /root/logstash-6.3.0/logs which is now configured via log4j2.properties \[2018-07-13T14:17:46,014\]\[ERROR\]\[org.logstash.Logstash \] java.lang.IllegalStateException: Logstash stopped processing… --- ## [My grok fikter dont work](https://discuss.elastic.co/t/my-grok-fikter-dont-work/137795)
**Author:** [@DmitryLysenko](https://discuss.elastic.co/u/DmitryLysenko)\ **Replies:** 6\ **Last updated:** [July 13, 2018, 11:35am UTC](https://discuss.elastic.co/t/my-grok-fikter-dont-work/137795 "2018-07-13T11:35:07Z")
Hi, I want the logs of my service to be processed using the filter grok. But instead I get \_grokparsefailure. Could you tell me what the problem is? My logstash configuration: input { file { pa… --- ## [Creat multiple index pattern](https://discuss.elastic.co/t/creat-multiple-index-pattern/137279)
**Author:** [@kimfut](https://discuss.elastic.co/u/kimfut)\ **Replies:** 14\ **Last updated:** [July 13, 2018, 11:32am UTC](https://discuss.elastic.co/t/creat-multiple-index-pattern/137279 "2018-07-13T11:32:19Z")
hello every one , i am trying to make an config to creat a multiple index pattern with logstash , i am using filebeat also , principally i have two types of file , each type locate in separate folder so there is my fi… --- ## [Filter "logstash-filter-dns" doesn't work](https://discuss.elastic.co/t/filter-logstash-filter-dns-doesnt-work/139929)
**Author:** [@da.frolov](https://discuss.elastic.co/u/da.frolov)\ **Replies:** 0\ **Last updated:** [July 13, 2018, 11:20am UTC](https://discuss.elastic.co/t/filter-logstash-filter-dns-doesnt-work/139929 "2018-07-13T11:20:03Z")
Hi, Amazing ElasticSearch Community! English is my second language and I apologize in advance))) I have a problem with filter in logstash "netflow.conf" file. I collect the netflow data from my network devices, and I … --- ## [Pipeline ordering when using the new pipeline output](https://discuss.elastic.co/t/pipeline-ordering-when-using-the-new-pipeline-output/138116)
**Author:** [@Dheeraj\_Gupta](https://discuss.elastic.co/u/Dheeraj_Gupta)\ **Replies:** 2\ **Last updated:** [July 13, 2018, 8:57am UTC](https://discuss.elastic.co/t/pipeline-ordering-when-using-the-new-pipeline-output/138116 "2018-07-13T08:57:52Z")
Hi, I have already posted about the startup/shutdown order when using multiple pipelines. Till v6.2.4, it seems that Logstash arranged the startup/shutdown of pipelines in alphabetic order. I recently upgraded to 6.3.0… --- ## [Input kafka plugin for kafka offset](https://discuss.elastic.co/t/input-kafka-plugin-for-kafka-offset/139914)
**Author:** [@tyb1222](https://discuss.elastic.co/u/tyb1222)\ **Replies:** 0\ **Last updated:** [July 13, 2018, 9:33am UTC](https://discuss.elastic.co/t/input-kafka-plugin-for-kafka-offset/139914 "2018-07-13T09:33:19Z")
I wonder that how should I do if I want to commit kafka offset by setting enable\_auto\_commit = false --- ## [Exclude or replace a specific date value with null value](https://discuss.elastic.co/t/exclude-or-replace-a-specific-date-value-with-null-value/139803)
**Author:** [@imaad](https://discuss.elastic.co/u/imaad)\ **Replies:** 3\ **Last updated:** [July 13, 2018, 9:12am UTC](https://discuss.elastic.co/t/exclude-or-replace-a-specific-date-value-with-null-value/139803 "2018-07-13T09:12:07Z")
I'm using a jms input to index data from queues, some fields contains two values ("2018-07-12T23:30:00.000+0000" and "2018-07-12T00:00:00.000+0000") that I have to replace with null value. How can I resolve this? Thank … --- ## [JDBC input plugin Logstash](https://discuss.elastic.co/t/jdbc-input-plugin-logstash/139718)
**Author:** [@ysumit](https://discuss.elastic.co/u/ysumit)\ **Replies:** 1\ **Last updated:** [July 13, 2018, 7:36am UTC](https://discuss.elastic.co/t/jdbc-input-plugin-logstash/139718 "2018-07-13T07:36:08Z")
Hi Team I have multiple sql queries. For e.g query A for total/B for Average/C for max. a) I need to use the same conf file. b) In output I want to access the data of individual query i.e. A/B/C. How to identify them … --- ## [Parse LDIF format, aggregate several lines into one ES index entry](https://discuss.elastic.co/t/parse-ldif-format-aggregate-several-lines-into-one-es-index-entry/139760)
**Author:** [@izinovik](https://discuss.elastic.co/u/izinovik)\ **Replies:** 2\ **Last updated:** [July 13, 2018, 7:05am UTC](https://discuss.elastic.co/t/parse-ldif-format-aggregate-several-lines-into-one-es-index-entry/139760 "2018-07-13T07:05:04Z")
Hello. I'm trying to solve following task: parse LDIF (LDAP data interchange) with Logstash using 'aggregate' plugin, but currently with no success. Here is an example of LDIF: time: 20180708032751 dn: fqdn=host.acme… --- ## [Change existing timezone](https://discuss.elastic.co/t/change-existing-timezone/139717)
**Author:** [@inhinyera16](https://discuss.elastic.co/u/inhinyera16)\ **Replies:** 2\ **Last updated:** [July 13, 2018, 6:50am UTC](https://discuss.elastic.co/t/change-existing-timezone/139717 "2018-07-13T06:50:49Z")
Hi all, We do have 2 datetime field named @Timestamp (which is UCT by default) and logdate (no timezone set, means default to UTC). Now we would like to map the logdate field to a specific timezone say canadian timezone… --- ## [Issue in Type\_Casting:(long to float and float to long)](https://discuss.elastic.co/t/issue-in-type-casting-long-to-float-and-float-to-long/139530)
**Author:** [@Akhilesh-Tiwari](https://discuss.elastic.co/u/Akhilesh-Tiwari)\ **Replies:** 2\ **Last updated:** [July 13, 2018, 6:03am UTC](https://discuss.elastic.co/t/issue-in-type-casting-long-to-float-and-float-to-long/139530 "2018-07-13T06:03:29Z")
Hi @logstash.chirag @Logstash-lover @logstash\_user @logstash\_oz @logstash @logstashisawesome we have to create index runtime using logstash. we have a table in database,In that we have columns that contains some decim… --- ## [Could not index event to Elasticsearch:cannot be changed from type \[float\] to \[long\]:](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-cannot-be-changed-from-type-float-to-long/138838)
**Author:** [@Akhilesh-Tiwari](https://discuss.elastic.co/u/Akhilesh-Tiwari)\ **Replies:** 9\ **Last updated:** [July 13, 2018, 6:01am UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-cannot-be-changed-from-type-float-to-long/138838 "2018-07-13T06:01:35Z")
Hi Team, I have an issue in creating index using my database table data through logstash JDBC. The issue is. \[2018-07-06T12:37:24,587\]\[WARN \]\[logstash.outputs.elasticsearch\] Could not index event to Elasticsearch. {:s… --- ## [Mongodb update](https://discuss.elastic.co/t/mongodb-update/139869)
**Author:** [@Chiranjeevi\_Medichar](https://discuss.elastic.co/u/Chiranjeevi_Medichar)\ **Replies:** 0\ **Last updated:** [July 13, 2018, 4:50am UTC](https://discuss.elastic.co/t/mongodb-update/139869 "2018-07-13T04:50:02Z")
is there anty option for mongo to update action for set:? --- ## [How to perform checksum](https://discuss.elastic.co/t/how-to-perform-checksum/139634)
**Author:** [@James4](https://discuss.elastic.co/u/James4)\ **Replies:** 2\ **Last updated:** [July 12, 2018, 8:07pm UTC](https://discuss.elastic.co/t/how-to-perform-checksum/139634 "2018-07-12T20:07:13Z")
I have a field that contains a list of "tags/keywords" (around 60) but they can come in any order. I want to be able to search the index for all documents with same combination of "tags/keywords" . So I thought I could… --- ## [Issues connecting to S3 bucket from Logstash](https://discuss.elastic.co/t/issues-connecting-to-s3-bucket-from-logstash/139812)
**Author:** [@jobse.batres](https://discuss.elastic.co/u/jobse.batres)\ **Replies:** 0\ **Last updated:** [July 12, 2018, 6:26pm UTC](https://discuss.elastic.co/t/issues-connecting-to-s3-bucket-from-logstash/139812 "2018-07-12T18:26:16Z")
Hello all, I'm working on setting up a Logstash server in AWS and have that server pull logs from an S3 bucket. When I start up my Logstash instance, I see the below error in the logs. \[2018-07-12T14:25:24,653\]\[INFO \]… --- ## [Split range of dates](https://discuss.elastic.co/t/split-range-of-dates/139725)
**Author:** [@someOne](https://discuss.elastic.co/u/someOne)\ **Replies:** 1\ **Last updated:** [July 12, 2018, 6:11pm UTC](https://discuss.elastic.co/t/split-range-of-dates/139725 "2018-07-12T18:11:32Z")
Can I split from No. date start end detail 1 7/12/2018 7/12/2018 09:00 9/12/2018 14:00 Same detail to No. date start end detail 1 … --- ## [Date format - fused string](https://discuss.elastic.co/t/date-format-fused-string/139808)
**Author:** [@djfab](https://discuss.elastic.co/u/djfab)\ **Replies:** 4\ **Last updated:** [July 12, 2018, 6:03pm UTC](https://discuss.elastic.co/t/date-format-fused-string/139808 "2018-07-12T18:03:12Z")
I'm trying to parse 180711T221704.507399z into @timestamp with date { match =\> \["log\_time","yyMMddHHmmss.SSSSSS"\] } but keep getting "dateparsefailure". I've tried "ISO8601", adding the T, z, and a everything short… --- ## [Csv filter convert](https://discuss.elastic.co/t/csv-filter-convert/139748)
**Author:** [@annk](https://discuss.elastic.co/u/annk)\ **Replies:** 1\ **Last updated:** [July 12, 2018, 5:45pm UTC](https://discuss.elastic.co/t/csv-filter-convert/139748 "2018-07-12T17:45:22Z")
Hi, When I use logstash csv filter convert option it fails to convert integer, I am getting 'long ' instead of integer . Example: csv { separator =\> ";" columns =\> \["Queue","closed\_duplicate","closed\_successful","cl… --- ## [Logstash restarting every 20-30 seconds](https://discuss.elastic.co/t/logstash-restarting-every-20-30-seconds/138940)
**Author:** [@pavan2](https://discuss.elastic.co/u/pavan2)\ **Replies:** 11\ **Last updated:** [July 12, 2018, 5:41pm UTC](https://discuss.elastic.co/t/logstash-restarting-every-20-30-seconds/138940 "2018-07-12T17:41:03Z")
Log file has the following error \[2018-07-06T07:50:00,389\]\[ERROR\]\[org.logstash.Logstash \] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit logstash-simple.conf has … --- ## [\[ERROR\]\[logstash.agent \] Failed to execute action {:action=\>LogStash::PipelineAction::Create/pipeline\_id:main, :exception=\>"LogStash::ConfigurationError", :message=\>"Expected one of #,](https://discuss.elastic.co/t/error-logstash-agent-failed-to-execute-action-action-logstash-create-pipeline-id-main-exception-logstash-configurationerror-message-expected-one-of/139651)
**Author:** [@Shraddha\_Srivastav](https://discuss.elastic.co/u/Shraddha_Srivastav)\ **Replies:** 4\ **Last updated:** [July 12, 2018, 5:36pm UTC](https://discuss.elastic.co/t/error-logstash-agent-failed-to-execute-action-action-logstash-create-pipeline-id-main-exception-logstash-configurationerror-message-expected-one-of/139651 "2018-07-12T17:36:58Z")
Following is my config file: input { file{ path =\> "C:\\Users\\shraddhasrivastav\\Downloads\\cars.csv" start\_position =\> "beginning" sincedb\_path =\> "dev/null" } stdin { } } filter{ csv{ columns =\> \["TimeOfDay… --- ## [Elasticsearch doc type deprecation: "doc" vs. "\_doc"](https://discuss.elastic.co/t/elasticsearch-doc-type-deprecation-doc-vs-doc/139799)
**Author:** [@jmb](https://discuss.elastic.co/u/jmb)\ **Replies:** 0\ **Last updated:** [July 12, 2018, 5:01pm UTC](https://discuss.elastic.co/t/elasticsearch-doc-type-deprecation-doc-vs-doc/139799 "2018-07-12T17:01:16Z")
I'm trying to prepare for the eventual deprecation of document types in Elasticsearch, and it's not easy to nail down all conflicts. One that I'm blinking at at the moment is \[ 'doc', '\_doc' \] ... The Elasticsearch docu… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=721) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=723)