# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=927 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 928 --- ## [Date filter - timestamp & @timestamp are diffrent](https://discuss.elastic.co/t/date-filter-timestamp-timestamp-are-diffrent/95028)
**Author:** [@ssasporta](https://discuss.elastic.co/u/ssasporta)\ **Replies:** 1\ **Last updated:** [August 11, 2017, 6:17pm UTC](https://discuss.elastic.co/t/date-filter-timestamp-timestamp-are-diffrent/95028 "2017-08-11T18:17:15Z")
Hi, The timestamp in the log is : 26/Jul/2017:00:00:00 I defined a pattern: HTTPDATE %{MONTHDAY}/%{MONTH}/%{YEAR}:%{TIME} In the grok I have it : %{HTTPDATE:timestamp} And this is the filter I do: date … --- ## [Logstash failed to start up because of memory issues](https://discuss.elastic.co/t/logstash-failed-to-start-up-because-of-memory-issues/95007)
**Author:** [@shroh](https://discuss.elastic.co/u/shroh)\ **Replies:** 1\ **Last updated:** [August 11, 2017, 6:15pm UTC](https://discuss.elastic.co/t/logstash-failed-to-start-up-because-of-memory-issues/95007 "2017-08-11T18:15:13Z")
My Log stash failed to start up because of memory issues: This is what i see \[root@ip-10-206-118-171 logstash\]# /usr/share/logstash/bin/logstash --path.settings /etc/logstash OpenJDK 64-Bit Server VM warning: INFO: os… --- ## [Run multiple instances of logstash](https://discuss.elastic.co/t/run-multiple-instances-of-logstash/96803)
**Author:** [@ashvita](https://discuss.elastic.co/u/ashvita)\ **Replies:** 1\ **Last updated:** [August 11, 2017, 6:00pm UTC](https://discuss.elastic.co/t/run-multiple-instances-of-logstash/96803 "2017-08-11T18:00:41Z")
Hi, I am new to ELK stack. I am not sure how to run multiple instances of logstash. I have 5 config files with kafka streaming various topics as input and output as elastic search. How do I stream multiple config files … --- ## [Logstash 5.x Mongodb output plugin not workinng](https://discuss.elastic.co/t/logstash-5-x-mongodb-output-plugin-not-workinng/96795)
**Author:** [@arunvijayam](https://discuss.elastic.co/u/arunvijayam)\ **Replies:** 0\ **Last updated:** [August 11, 2017, 3:32pm UTC](https://discuss.elastic.co/t/logstash-5-x-mongodb-output-plugin-not-workinng/96795 "2017-08-11T15:32:49Z")
Hi Team, Is there any work around with mongodb output plugin from logstash 5.x. Do I need to download any other older or new version to support this plugin. Thanks and Regards, Arun. --- ## [How to force/change the order of the fields in a event?](https://discuss.elastic.co/t/how-to-force-change-the-order-of-the-fields-in-a-event/96786)
**Author:** [@Joaquim\_Freitas](https://discuss.elastic.co/u/Joaquim_Freitas)\ **Replies:** 1\ **Last updated:** [August 11, 2017, 3:28pm UTC](https://discuss.elastic.co/t/how-to-force-change-the-order-of-the-fields-in-a-event/96786 "2017-08-11T15:28:39Z")
Hi, It is possible (or how can it be done) to force/change the order of the fields in a event? The reason for my question is simple: I have a field “\_time” in my events that I need to be the very first field in my JSO… --- ## [How to handle multiple match in logstash filter](https://discuss.elastic.co/t/how-to-handle-multiple-match-in-logstash-filter/96647)
**Author:** [@ankeet.jain](https://discuss.elastic.co/u/ankeet.jain)\ **Replies:** 5\ **Last updated:** [August 11, 2017, 1:36pm UTC](https://discuss.elastic.co/t/how-to-handle-multiple-match-in-logstash-filter/96647 "2017-08-11T13:36:41Z")
I want to know what is the best way to handle multiple logs pattern single file. I have created below filter for the my logs. filter { if\[type\] =='AppLog' { grok { break\_on\_match =\> false match =\> { "message" =\> … --- ## [Calculate number of days between two dates logstash \[solved\]](https://discuss.elastic.co/t/calculate-number-of-days-between-two-dates-logstash-solved/96772)
**Author:** [@Chemse](https://discuss.elastic.co/u/Chemse)\ **Replies:** 1\ **Last updated:** [August 11, 2017, 1:07pm UTC](https://discuss.elastic.co/t/calculate-number-of-days-between-two-dates-logstash-solved/96772 "2017-08-11T13:07:22Z")
Hello, I want to calculate number of days between two dates in logstash I have : date { match =\> \["Deb","dd/MM/YYYY"\] target=\> "Deb" } date { match =\> \["toDay","dd/MM/YYYY"\] target=\> "toDay" } and I… --- ## [Status 404 on update](https://discuss.elastic.co/t/status-404-on-update/96432)
**Author:** [@bterhorst](https://discuss.elastic.co/u/bterhorst)\ **Replies:** 6\ **Last updated:** [August 11, 2017, 11:31am UTC](https://discuss.elastic.co/t/status-404-on-update/96432 "2017-08-11T11:31:31Z")
Hi, I have a document that is inserted into elasticsearch under the index: index-%{+YYYY.MM.dd} where %{+YYYY.MM.dd}=2017-08-09 A day later another event comes in and I have to update that document I inserted yesterda… --- ## [Hook one elasticsearch with another ES using logstash](https://discuss.elastic.co/t/hook-one-elasticsearch-with-another-es-using-logstash/95308)
**Author:** [@sujeetKumar](https://discuss.elastic.co/u/sujeetKumar)\ **Replies:** 4\ **Last updated:** [August 11, 2017, 11:12am UTC](https://discuss.elastic.co/t/hook-one-elasticsearch-with-another-es-using-logstash/95308 "2017-08-11T11:12:59Z")
Is there any way we can hook an index from one Elasticsearch to another. input { elasticsearch { index =\> "student\_details" hosts =\> \[“http://localhost:9200”\] } } filter { date { match =\> \[ “joinDate”, “UNIX\_MS”… --- ## [Ruby filter condition](https://discuss.elastic.co/t/ruby-filter-condition/96722)
**Author:** [@sureshboiina](https://discuss.elastic.co/u/sureshboiina)\ **Replies:** 2\ **Last updated:** [August 11, 2017, 9:57am UTC](https://discuss.elastic.co/t/ruby-filter-condition/96722 "2017-08-11T09:57:39Z")
Hi I having a data like status,id 0,A 2,B 3,C A,D B,E 6,N Based on status value need to add one more field and insert good,bad in new field if status is even number then new field value is good else status is a… --- ## [Unexpected mapping?](https://discuss.elastic.co/t/unexpected-mapping/96739)
**Author:** [@TimWard](https://discuss.elastic.co/u/TimWard)\ **Replies:** 0\ **Last updated:** [August 11, 2017, 9:25am UTC](https://discuss.elastic.co/t/unexpected-mapping/96739 "2017-08-11T09:25:39Z")
I discovered that there’s a mapping in Elasticsearch for a field that I wasn’t expecting to be there, as that field is not present in any documents that are indexed (and was not present in any template). The field is (s… --- ## [Logstash Output Avro](https://discuss.elastic.co/t/logstash-output-avro/96245)
**Author:** [@JoseJR](https://discuss.elastic.co/u/JoseJR)\ **Replies:** 2\ **Last updated:** [August 11, 2017, 9:08am UTC](https://discuss.elastic.co/t/logstash-output-avro/96245 "2017-08-11T09:08:05Z")
Is the pipeline config below possible? input { beats { port =\> “5601” } } filter { mutate { remove\_field =\> \["@version","@timestamp",“path”,“host”, “beat”, “message”\] } } output { stdout { codec =\> avro { sc… --- ## [\[ERROR\]\[logstash.instrument.periodicpoller.jvm\] Periodi cPoller: exception](https://discuss.elastic.co/t/error-logstash-instrument-periodicpoller-jvm-periodi-cpoller-exception/92281)
**Author:** [@swatititame](https://discuss.elastic.co/u/swatititame)\ **Replies:** 33\ **Last updated:** [August 11, 2017, 8:59am UTC](https://discuss.elastic.co/t/error-logstash-instrument-periodicpoller-jvm-periodi-cpoller-exception/92281 "2017-08-11T08:59:42Z")
We are running filebeat,logstash, kibana and elasticsearach in Master machine(1 machine) and from slave machines(6 machines). We are just running Filebeat and all the data from those machines is transferred to master mac… --- ## [DNS Sinkhole Grok filter,need your help](https://discuss.elastic.co/t/dns-sinkhole-grok-filter-need-your-help/96730)
**Author:** [@Blason](https://discuss.elastic.co/u/Blason)\ **Replies:** 0\ **Last updated:** [August 11, 2017, 8:30am UTC](https://discuss.elastic.co/t/dns-sinkhole-grok-filter-need-your-help/96730 "2017-08-11T08:30:45Z")
Hi Guys, I have a DNS Sinkhole built on pfsense and its logging DNS resolver logs. Can someone please help me with grok filters for below logs so that I can start building a good dashboards from those? Since this is sin… --- ## [Parse application logs - Help needed](https://discuss.elastic.co/t/parse-application-logs-help-needed/96100)
**Author:** [@ashish235](https://discuss.elastic.co/u/ashish235)\ **Replies:** 1\ **Last updated:** [August 11, 2017, 6:36am UTC](https://discuss.elastic.co/t/parse-application-logs-help-needed/96100 "2017-08-11T06:36:18Z")
Hi team, I 'm new to the ELK stack, I 'm very fond of it and want to implement it for my application logs. The issue is I 'm not able to really make my application logs structured. I 'm using beats to ship the logs to… --- ## [How to write regex pattern on logstash input kakfa topics\_pattern](https://discuss.elastic.co/t/how-to-write-regex-pattern-on-logstash-input-kakfa-topics-pattern/96705)
**Author:** [@sulthonzh](https://discuss.elastic.co/u/sulthonzh)\ **Replies:** 0\ **Last updated:** [August 11, 2017, 6:15am UTC](https://discuss.elastic.co/t/how-to-write-regex-pattern-on-logstash-input-kakfa-topics-pattern/96705 "2017-08-11T06:15:38Z")
kafka { bootstrap\_servers =\> "localhost:9092" # topics =\> \["techsupportforum\_\_listings","black\_\_listings"\] topics\_pattern =\> "\\\\\*\_\_listings" consumer\_threads =\> 1 decorate\_events =\> true } hi, i ha… --- ## [Help on case sensitive kv plugin](https://discuss.elastic.co/t/help-on-case-sensitive-kv-plugin/96508)
**Author:** [@bgardeamesta](https://discuss.elastic.co/u/bgardeamesta)\ **Replies:** 3\ **Last updated:** [August 11, 2017, 6:08am UTC](https://discuss.elastic.co/t/help-on-case-sensitive-kv-plugin/96508 "2017-08-11T06:08:54Z")
Im using the kv plugin to extract some values from syslog messages such as From, to, call-ID, etc. Problem is that some logs come in capitalized and others don’t and from what I can tell kv are case sensitive, any sugges… --- ## [Read logs from Remove Linux sever using Logstash or File beat](https://discuss.elastic.co/t/read-logs-from-remove-linux-sever-using-logstash-or-file-beat/96501)
**Author:** [@Leela\_Kodali](https://discuss.elastic.co/u/Leela_Kodali)\ **Replies:** 5\ **Last updated:** [August 11, 2017, 5:59am UTC](https://discuss.elastic.co/t/read-logs-from-remove-linux-sever-using-logstash-or-file-beat/96501 "2017-08-11T05:59:41Z")
HI, can you please help me on how to read log file on different Linux box, i have ELK and Filebeat installed on one Linux box i am trying to read log file from different Linux box. Here is my Filebeat config file: (He… --- ## [JSON file parsing](https://discuss.elastic.co/t/json-file-parsing/94861)
**Author:** [@elaair](https://discuss.elastic.co/u/elaair)\ **Replies:** 6\ **Last updated:** [August 11, 2017, 5:57am UTC](https://discuss.elastic.co/t/json-file-parsing/94861 "2017-08-11T05:57:31Z")
I have a JSON file from Netscope REST API that I would like to bring into Elastic. I see outout when I use the stdout plugin with rubydebug but when I use elastic output I am getting one entry that really isn't parsed. … --- ## [Datetime Parse \_dateparsefailure YYYY-MM-dd'T'HH:mm:ssZZ](https://discuss.elastic.co/t/datetime-parse--dateparsefailure-yyyy-mm-ddthh-mm-sszz/96691)
**Author:** [@sulthonzh](https://discuss.elastic.co/u/sulthonzh)\ **Replies:** 0\ **Last updated:** [August 11, 2017, 3:27am UTC](https://discuss.elastic.co/t/datetime-parse--dateparsefailure-yyyy-mm-ddthh-mm-sszz/96691 "2017-08-11T03:27:04Z")
I have date string : “2017-08-06T17:01:00+07:00” my filter: date { locale =\> “id” #2017-08-02T12:22:00+07:00 match =\> \[“updatedAt”, “YYYY-MM-dd’T’HH:mm:ssZZ”\] timezone =\> "Asia/Jakarta" target =\> “updatedAt” } r… --- ## [DMZ Best Practice](https://discuss.elastic.co/t/dmz-best-practice/96686)
**Author:** [@Puk](https://discuss.elastic.co/u/Puk)\ **Replies:** 1\ **Last updated:** [August 11, 2017, 1:11am UTC](https://discuss.elastic.co/t/dmz-best-practice/96686 "2017-08-11T01:11:35Z")
Newbie to ELK here and wanting to monitor 3x DMZ hosts I have which run Apache. My single ELK stack server is internal, so I know I “could” open up the port to Logstash and have the DMZ servers send logs via filebeat st… --- ## [Drop message,but still write into elasticsearch](https://discuss.elastic.co/t/drop-message-but-still-write-into-elasticsearch/96614)
**Author:** [@zhangheng](https://discuss.elastic.co/u/zhangheng)\ **Replies:** 3\ **Last updated:** [August 11, 2017, 1:09am UTC](https://discuss.elastic.co/t/drop-message-but-still-write-into-elasticsearch/96614 "2017-08-11T01:09:06Z")
I was confused,because i drop some message in the filter,it really not out put in the stdout ,but it still write into the elasticsearch . here is my config, help me ,please filter { if “GET” in \[message\] { drop { } … --- ## [How to process large data with logstash](https://discuss.elastic.co/t/how-to-process-large-data-with-logstash/96663)
**Author:** [@niraj\_kumar](https://discuss.elastic.co/u/niraj_kumar)\ **Replies:** 0\ **Last updated:** [August 10, 2017, 7:31pm UTC](https://discuss.elastic.co/t/how-to-process-large-data-with-logstash/96663 "2017-08-10T19:31:38Z")
I am having a trouble making my logstash work with around million files in it. I am seeing a max\_open\_file limit warning when running it. \[2017-08-10T12:26:24,844\]\[INFO \]\[logstash.outputs.elasticsearch\] New Elasticsear… --- ## [Logstash Metrics Filter Error "can't convert nil into String"](https://discuss.elastic.co/t/logstash-metrics-filter-error-cant-convert-nil-into-string/96662)
**Author:** [@amarc](https://discuss.elastic.co/u/amarc)\ **Replies:** 0\ **Last updated:** [August 10, 2017, 7:28pm UTC](https://discuss.elastic.co/t/logstash-metrics-filter-error-cant-convert-nil-into-string/96662 "2017-08-10T19:28:52Z")
I am trying to measures events per minute consumed by logstash. In Filebeats, I have a customer field called fwder which is value of filebeat. So to count the messages per minute, I am counting the occurrence of filebe… --- ## [:error =\> can't convert nil into String](https://discuss.elastic.co/t/error-cant-convert-nil-into-string/96500)
**Author:** [@ashvita](https://discuss.elastic.co/u/ashvita)\ **Replies:** 3\ **Last updated:** [August 10, 2017, 6:34pm UTC](https://discuss.elastic.co/t/error-cant-convert-nil-into-string/96500 "2017-08-10T18:34:28Z")
Hi, I am getting this error when I am running 2nd config file in same folder of logstash An unexpected error occurred! :error =\> can’t convert nil into String, :backtrace =\> \[“org/jruby/RubyFileTest.java:95:in director… --- ## [Split xml in multiple events](https://discuss.elastic.co/t/split-xml-in-multiple-events/94923)
**Author:** [@Benjamin\_Carriou](https://discuss.elastic.co/u/Benjamin_Carriou)\ **Replies:** 2\ **Last updated:** [August 10, 2017, 6:14pm UTC](https://discuss.elastic.co/t/split-xml-in-multiple-events/94923 "2017-08-10T18:14:50Z")
Hello ! I know that this subject has been treated several times but I can not solve my problem. Logstash version: 2.4 Split plugin version: 3.1.2 Here is an example of xml that I want split: \ **Author:** [@shroh](https://discuss.elastic.co/u/shroh)\ **Replies:** 0\ **Last updated:** [August 10, 2017, 6:12pm UTC](https://discuss.elastic.co/t/fields-created-using-kv-filter-have-special-characters-and-are-not-available-for-visualisation/96655 "2017-08-10T18:12:52Z") Hi , I have created a field called Client-Correlation-Id using KV filters , the field name is Client-Correlation-Id, the field is properly parsed by logstash and i can see that field being available in the left pane. Bu… --- ## [Logstash/Kibana breaks my log line into multiple events/documets](https://discuss.elastic.co/t/logstash-kibana-breaks-my-log-line-into-multiple-events-documets/96517)
**Author:** [@shroh](https://discuss.elastic.co/u/shroh)\ **Replies:** 6\ **Last updated:** [August 10, 2017, 5:35pm UTC](https://discuss.elastic.co/t/logstash-kibana-breaks-my-log-line-into-multiple-events-documets/96517 "2017-08-10T17:35:10Z")
Its strange though, Logstash/kibana is breaking my single log line into multiple events/documents. Whereas it should be a single event , otherwise it doesnt solve my purpose to even put these logs into ELK. Any idea? ho… --- ## [Help changing Bro logs Epoch time to readable](https://discuss.elastic.co/t/help-changing-bro-logs-epoch-time-to-readable/96338)
**Author:** [@Blaxican707](https://discuss.elastic.co/u/Blaxican707)\ **Replies:** 2\ **Last updated:** [August 10, 2017, 4:53pm UTC](https://discuss.elastic.co/t/help-changing-bro-logs-epoch-time-to-readable/96338 "2017-08-10T16:53:42Z")
Hey all, thanks for the add. I’ve developed my config file that uses filebeat to push bro logs to logstash. Logstash parses pretty well, but my only problem that im running into is that I cannot change the epoch time a … --- ## [Which is the recommended sysbse jdbc driver for logstash sybase connectivity?](https://discuss.elastic.co/t/which-is-the-recommended-sysbse-jdbc-driver-for-logstash-sybase-connectivity/94223)
**Author:** [@Axe](https://discuss.elastic.co/u/Axe)\ **Replies:** 2\ **Last updated:** [August 10, 2017, 4:49pm UTC](https://discuss.elastic.co/t/which-is-the-recommended-sysbse-jdbc-driver-for-logstash-sybase-connectivity/94223 "2017-08-10T16:49:45Z")
I am using JConn3.1(sybase jdbc driver) which is giving error in registring plugin message.Please let me know if any one have tried using sysbase database. [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=926) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=928)