# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1141 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1142 --- ## [Failed to parse @timestamp from UNIX\_MS](https://discuss.elastic.co/t/failed-to-parse-timestamp-from-unix-ms/52732)
**Author:** [@Kevin\_Csuka](https://discuss.elastic.co/u/Kevin_Csuka)\ **Replies:** 3\ **Last updated:** [June 15, 2016, 7:19am UTC](https://discuss.elastic.co/t/failed-to-parse-timestamp-from-unix-ms/52732 "2016-06-15T07:19:24Z")
Going crazy. Been busy / searching all day, but no success. I've got this data which I want to import in Elasticsearch with Logstash: 1465309033156,84,http://test.com/purchase.php,200,OK,ThreadGroup 1-3,true,6,6,84,t… --- ## [Logstash 2.3.1 kafka plugins ssl support](https://discuss.elastic.co/t/logstash-2-3-1-kafka-plugins-ssl-support/50758)
**Author:** [@sundar](https://discuss.elastic.co/u/sundar)\ **Replies:** 7\ **Last updated:** [June 15, 2016, 6:34am UTC](https://discuss.elastic.co/t/logstash-2-3-1-kafka-plugins-ssl-support/50758 "2016-06-15T06:34:30Z")
Hello, We use Kafka input and output plugins with Logstash in our log pipeline. We are using 2.3.1 version of Logstash and Elasticsearch. I see that there is now Kafka 0.9 producer/consumer support in LS 5.0. Any chanc… --- ## [How can we set hosts dynamicaly in elasticsearch plugin in logstash conf file ? I am setting document\_type =\> "%{type}" and type is substitued here. so cant we substitute %{host} ? for hosts parameter?](https://discuss.elastic.co/t/how-can-we-set-hosts-dynamicaly-in-elasticsearch-plugin-in-logstash-conf-file-i-am-setting-document-type-type-and-type-is-substitued-here-so-cant-we-substitute-host-for-hosts-parameter/52731)
**Author:** [@rupal](https://discuss.elastic.co/u/rupal)\ **Replies:** 4\ **Last updated:** [June 15, 2016, 6:18am UTC](https://discuss.elastic.co/t/how-can-we-set-hosts-dynamicaly-in-elasticsearch-plugin-in-logstash-conf-file-i-am-setting-document-type-type-and-type-is-substitued-here-so-cant-we-substitute-host-for-hosts-parameter/52731 "2016-06-15T06:18:06Z")
How can we set hosts dynamicaly in elasticsearch plugin in logstash conf file ? I am setting document\_type =\> "%{type}" and type is substitued here. so cant we substitute %{host} ? for hosts parameter?? --- ## [CSV values matched to same field as array](https://discuss.elastic.co/t/csv-values-matched-to-same-field-as-array/52721)
**Author:** [@Mark\_Rodman](https://discuss.elastic.co/u/Mark_Rodman)\ **Replies:** 1\ **Last updated:** [June 15, 2016, 5:47am UTC](https://discuss.elastic.co/t/csv-values-matched-to-same-field-as-array/52721 "2016-06-15T05:47:35Z")
Hi community. I have an event stream which contains comma separated values which I wish to match into the the same field as an array without the comma. An example might be. \[ value1, value2, val\_ue3, val ue4, value5, v… --- ## [Advice needed multiple logstash instance Vs single](https://discuss.elastic.co/t/advice-needed-multiple-logstash-instance-vs-single/28598)
**Author:** [@achakrab22](https://discuss.elastic.co/u/achakrab22)\ **Replies:** 15\ **Last updated:** [June 15, 2016, 5:46am UTC](https://discuss.elastic.co/t/advice-needed-multiple-logstash-instance-vs-single/28598 "2016-06-15T05:46:26Z")
Hi, I am trying to scan multiple folder paths like dev, qa, prod (maybe 45 paths something like /var/tmp/xyz-dev-abc/\*.logs , or /var/tmp/xyz-qa-abc/\*.logs) I have 15 such different envs(dev/qa/qa1 etc) and 3 such path p… --- ## [Logstash time parsing error](https://discuss.elastic.co/t/logstash-time-parsing-error/52765)
**Author:** [@Kennedy\_Kan1](https://discuss.elastic.co/u/Kennedy_Kan1)\ **Replies:** 3\ **Last updated:** [June 15, 2016, 5:43am UTC](https://discuss.elastic.co/t/logstash-time-parsing-error/52765 "2016-06-15T05:43:03Z")
I have come across with logstash date parsing error like the follow. ←\[33mFailed parsing date from field {:field=\>"Date", :value=\>"Sep 19, 2015 10:16:36 PM HKT", :exception=\>"Invalid format : "Sep 19, 2015 10:16:36 PM… --- ## [\[solved\] Multiple logstash config file](https://discuss.elastic.co/t/solved-multiple-logstash-config-file/51692)
**Author:** [@Prateek\_Kshtriya](https://discuss.elastic.co/u/Prateek_Kshtriya)\ **Replies:** 9\ **Last updated:** [June 15, 2016, 5:39am UTC](https://discuss.elastic.co/t/solved-multiple-logstash-config-file/51692 "2016-06-15T05:39:10Z")
Hi All , Please suggest how to use multiple logstash config file at a time on single command line or as a service. Problem Statement- I have multiple logstash config file(As there is differet data configured in each … --- ## [How to crawler static file by logstash 'file' input plugin without modifying at beginning](https://discuss.elastic.co/t/how-to-crawler-static-file-by-logstash-file-input-plugin-without-modifying-at-beginning/52713)
**Author:** [@Jason\_Zheng](https://discuss.elastic.co/u/Jason_Zheng)\ **Replies:** 1\ **Last updated:** [June 15, 2016, 4:54am UTC](https://discuss.elastic.co/t/how-to-crawler-static-file-by-logstash-file-input-plugin-without-modifying-at-beginning/52713 "2016-06-15T04:54:32Z")
Hi all, there are 3 log files has been archived, and named 'archived.2016.06.12.log', 'archived.2016.06.13.log' and 'archived.2016.06.14.log'. my logstash conf content input { file { path =\> "/home/ubuntu/test\_log/… --- ## [Get JSON Array out of a property value](https://discuss.elastic.co/t/get-json-array-out-of-a-property-value/52667)
**Author:** [@mmarkzon](https://discuss.elastic.co/u/mmarkzon)\ **Replies:** 1\ **Last updated:** [June 14, 2016, 9:09pm UTC](https://discuss.elastic.co/t/get-json-array-out-of-a-property-value/52667 "2016-06-14T21:09:10Z")
I am importing into Elastic. My log file looks like this: {"Transmission":{"Value":"{ \\"Users\\": \[{\\"ArtifactID\\": 123123}\] }"}} I am having difficulty getting the json value out of Users as an array so that it can be … --- ## [Change field to timestamp](https://discuss.elastic.co/t/change-field-to-timestamp/52708)
**Author:** [@Kennedy\_Kan1](https://discuss.elastic.co/u/Kennedy_Kan1)\ **Replies:** 1\ **Last updated:** [June 14, 2016, 8:08pm UTC](https://discuss.elastic.co/t/change-field-to-timestamp/52708 "2016-06-14T20:08:22Z")
I have a csv file which store up cpu usage. There is a field with date format like this "20150101-00:15:00". How can I change it to @timestamp in logstash as shown in kibana? --- ## [Grok filter succeeds in unit test, in grok debugger, but fails in production](https://discuss.elastic.co/t/grok-filter-succeeds-in-unit-test-in-grok-debugger-but-fails-in-production/51685)
**Author:** [@aadamovich](https://discuss.elastic.co/u/aadamovich)\ **Replies:** 2\ **Last updated:** [June 14, 2016, 6:45pm UTC](https://discuss.elastic.co/t/grok-filter-succeeds-in-unit-test-in-grok-debugger-but-fails-in-production/51685 "2016-06-14T18:45:46Z")
Hi! I have the following filter in Logstash: filter { if \[type\] == "weblogic" { multiline { pattern =\> "^####" negate =\> true what =\> previous } grok { match … --- ## [\\r\\n as a row separator](https://discuss.elastic.co/t/r-n-as-a-row-separator/52560)
**Author:** [@baneshbanna](https://discuss.elastic.co/u/baneshbanna)\ **Replies:** 1\ **Last updated:** [June 14, 2016, 6:38pm UTC](https://discuss.elastic.co/t/r-n-as-a-row-separator/52560 "2016-06-14T18:38:25Z")
How to configure \\r\\n as a row separator in logstash file input plugin. path =\> \["/logs/\*"\] start\_position =\> "beginning" type=\>"API" delimiter=\>"\\r\\n" close\_older=\>"300" max\_ope… --- ## [Logstash ouput plugin - Graphite and cluster](https://discuss.elastic.co/t/logstash-ouput-plugin-graphite-and-cluster/52431)
**Author:** [@Rohlik](https://discuss.elastic.co/u/Rohlik)\ **Replies:** 1\ **Last updated:** [June 14, 2016, 6:34pm UTC](https://discuss.elastic.co/t/logstash-ouput-plugin-graphite-and-cluster/52431 "2016-06-14T18:34:26Z")
Hello everyone, We have Graphite in cluster (2 node) and using Logstash for collecting metrics from app. Can we setup multiple nodes in Graphite output plugin? --- ## [Logstash not reading my file, no sincedb file being created either](https://discuss.elastic.co/t/logstash-not-reading-my-file-no-sincedb-file-being-created-either/52363)
**Author:** [@Rohan\_Tirumale](https://discuss.elastic.co/u/Rohan_Tirumale)\ **Replies:** 1\ **Last updated:** [June 14, 2016, 6:26pm UTC](https://discuss.elastic.co/t/logstash-not-reading-my-file-no-sincedb-file-being-created-either/52363 "2016-06-14T18:26:32Z")
I am using logstash 2.0 and this is my config file. For some reason the file isnt being read, but I am able to feed it an input using stdin, for which it shows me a result. input { file{ … --- ## [Modifying Tags in nested JSON array](https://discuss.elastic.co/t/modifying-tags-in-nested-json-array/51716)
**Author:** [@wmikew5672](https://discuss.elastic.co/u/wmikew5672)\ **Replies:** 1\ **Last updated:** [June 14, 2016, 6:10pm UTC](https://discuss.elastic.co/t/modifying-tags-in-nested-json-array/51716 "2016-06-14T18:10:24Z")
I'm looking to modify tags within a nested JSON array via a ruby filter. Basically, the log looks similar to the following: { “message” =\> “”, “@version =\> “1”, “@timestamp” =\> “2016-06-02T13:02:33.966Z”, “host” =\> “… --- ## [Parsing different log format from same application (input)](https://discuss.elastic.co/t/parsing-different-log-format-from-same-application-input/51714)
**Author:** [@T3rm1](https://discuss.elastic.co/u/T3rm1)\ **Replies:** 1\ **Last updated:** [June 14, 2016, 6:08pm UTC](https://discuss.elastic.co/t/parsing-different-log-format-from-same-application-input/51714 "2016-06-14T18:08:13Z")
Hi, I started reading about Logstash some days ago to find out wether it is suitable for my usecase. There is one critical requirement for which I haven't found a solution yet. My application already writes logs to fil… --- ## [Filter embedded JSON](https://discuss.elastic.co/t/filter-embedded-json/51682)
**Author:** [@Andrej\_van\_der\_Zee](https://discuss.elastic.co/u/Andrej_van_der_Zee)\ **Replies:** 1\ **Last updated:** [June 14, 2016, 6:06pm UTC](https://discuss.elastic.co/t/filter-embedded-json/51682 "2016-06-14T18:06:41Z")
Hi, My logs have embedded escaped JSON in one of its input JSON fields: { "log": "{\\"timestamp\\":\\"2016-06-02T07:49:05.850+00:00\\",\\"message\\":\\"Stopping beans in phase 0\\",\\"level\\":\\"INFO\\"}", "stream": "st… --- ## [Logstash rpm install failed with useradd and groupadd error](https://discuss.elastic.co/t/logstash-rpm-install-failed-with-useradd-and-groupadd-error/52760)
**Author:** [@dal](https://discuss.elastic.co/u/dal)\ **Replies:** 0\ **Last updated:** [June 14, 2016, 4:26pm UTC](https://discuss.elastic.co/t/logstash-rpm-install-failed-with-useradd-and-groupadd-error/52760 "2016-06-14T16:26:55Z")
I am trying to install logstash with yum on a red hat vm, I already have the logstash.repo file setup according to the guide and i ran yum install logstash but I get the following error after it downloads everything … --- ## [Logstash: query parameter lower than a value through elasticsearch plugin](https://discuss.elastic.co/t/logstash-query-parameter-lower-than-a-value-through-elasticsearch-plugin/52755)
**Author:** [@sen](https://discuss.elastic.co/u/sen)\ **Replies:** 1\ **Last updated:** [June 14, 2016, 4:23pm UTC](https://discuss.elastic.co/t/logstash-query-parameter-lower-than-a-value-through-elasticsearch-plugin/52755 "2016-06-14T16:23:29Z")
Hello everyone, With executing searches, I know that if I want to query a pamareter lower than a certain value i have to execute the following script: {"query": { "bool": { "must": \[ { "rang… --- ## [Logstash-plugin elasticsearch: apply excuting searches in logstash](https://discuss.elastic.co/t/logstash-plugin-elasticsearch-apply-excuting-searches-in-logstash/52719)
**Author:** [@sen](https://discuss.elastic.co/u/sen)\ **Replies:** 1\ **Last updated:** [June 14, 2016, 3:19pm UTC](https://discuss.elastic.co/t/logstash-plugin-elasticsearch-apply-excuting-searches-in-logstash/52719 "2016-06-14T15:19:36Z")
Hello everyon, Here is my excuting search to query my elasticsearch database (it works fine): curl -XPOST 'localhost:9200/test/\_search?pretty' -d ' { "size":1, "query": { "match": { "log… --- ## [Tags index with logstash and filebeat](https://discuss.elastic.co/t/tags-index-with-logstash-and-filebeat/44682)
**Author:** [@stefanocog](https://discuss.elastic.co/u/stefanocog)\ **Replies:** 3\ **Last updated:** [June 14, 2016, 3:08pm UTC](https://discuss.elastic.co/t/tags-index-with-logstash-and-filebeat/44682 "2016-06-14T15:08:34Z")
I actually use logstash-forwarder and logstash and create a dinamic index with tag with thus configuration: /etc/logstash/conf.d/10-output.conf output { elasticsearch { hosts =\> "localhost:9200" manage\_template =\> … --- ## [As is the data entry logstash?](https://discuss.elastic.co/t/as-is-the-data-entry-logstash/52357)
**Author:** [@Diego\_Gomez](https://discuss.elastic.co/u/Diego_Gomez)\ **Replies:** 8\ **Last updated:** [June 14, 2016, 1:14pm UTC](https://discuss.elastic.co/t/as-is-the-data-entry-logstash/52357 "2016-06-14T13:14:52Z")
As is the data entry by logstash with the host and port? --- ## [Does Logstash is able to support wildcards when specifying JMX object names?](https://discuss.elastic.co/t/does-logstash-is-able-to-support-wildcards-when-specifying-jmx-object-names/38716)
**Author:** [@Sai\_Birada](https://discuss.elastic.co/u/Sai_Birada)\ **Replies:** 5\ **Last updated:** [June 14, 2016, 10:05am UTC](https://discuss.elastic.co/t/does-logstash-is-able-to-support-wildcards-when-specifying-jmx-object-names/38716 "2016-06-14T10:05:22Z")
I am using logstash jmx plugin to log kafka cluster metrics into ELK. But Kafka exposes metrics at granular levels. So i want to know whether i can use wildcards in object names to log metrics at per topic or per clienti… --- ## [Can't input in Elasticsearch](https://discuss.elastic.co/t/cant-input-in-elasticsearch/51972)
**Author:** [@Kevin\_Csuka](https://discuss.elastic.co/u/Kevin_Csuka)\ **Replies:** 8\ **Last updated:** [June 14, 2016, 7:41am UTC](https://discuss.elastic.co/t/cant-input-in-elasticsearch/51972 "2016-06-14T07:41:58Z")
So I'm an all goner here. Got no clue where my mistake is. Here's the deal: I'm trying to input data from a csv file in Elasticsearch 2.3.2. The input, filter and output is done by Logstash 2.3.2. The .csv file: … --- ## [Logstash: NoMethodError: method \`\>=' for nil:NilClass](https://discuss.elastic.co/t/logstash-nomethoderror-method-for-nil-nilclass/52622)
**Author:** [@sen](https://discuss.elastic.co/u/sen)\ **Replies:** 3\ **Last updated:** [June 14, 2016, 8:45am UTC](https://discuss.elastic.co/t/logstash-nomethoderror-method-for-nil-nilclass/52622 "2016-06-14T08:45:59Z")
Hello everyone, Here is my configuration file: input { stdin { } file { path =\> \["myfile.json"\] type =\> "bytel" codec =\> "json" } } filter { if \[log\]\[Frequency\] != \[\] { if … --- ## [Logstash installation on windows](https://discuss.elastic.co/t/logstash-installation-on-windows/52609)
**Author:** [@Surya](https://discuss.elastic.co/u/Surya)\ **Replies:** 10\ **Last updated:** [June 14, 2016, 7:40am UTC](https://discuss.elastic.co/t/logstash-installation-on-windows/52609 "2016-06-14T07:40:08Z")
steps to install logstash in windows. --- ## [Timezone](https://discuss.elastic.co/t/timezone/52690)
**Author:** [@jagan](https://discuss.elastic.co/u/jagan)\ **Replies:** 1\ **Last updated:** [June 14, 2016, 7:02am UTC](https://discuss.elastic.co/t/timezone/52690 "2016-06-14T07:02:18Z")
Hi Gurus, Could you please suggest me how to get the timezone on to the '@timestamp' where i have my configuration which work fine but the time zone is not displayed when i run the config file in logstash(actually i … --- ## [Is there a way to use logstash template without specifying index name as "logstash-\*" when using Elasticsearch output plugin?](https://discuss.elastic.co/t/is-there-a-way-to-use-logstash-template-without-specifying-index-name-as-logstash-when-using-elasticsearch-output-plugin/51906)
**Author:** [@timogoosen](https://discuss.elastic.co/u/timogoosen)\ **Replies:** 3\ **Last updated:** [June 14, 2016, 7:02am UTC](https://discuss.elastic.co/t/is-there-a-way-to-use-logstash-template-without-specifying-index-name-as-logstash-when-using-elasticsearch-output-plugin/51906 "2016-06-14T07:02:12Z")
I know when you output tot Elasticsearch and you specify your index name to output to as "logstash-\*" then your output also contains raw fields. This is because when you use index name "logstash-" then it uses the logsta… --- ## [Logstash in reading files/ documents](https://discuss.elastic.co/t/logstash-in-reading-files-documents/52418)
**Author:** [@Kennedy\_Kan1](https://discuss.elastic.co/u/Kennedy_Kan1)\ **Replies:** 4\ **Last updated:** [June 14, 2016, 6:47am UTC](https://discuss.elastic.co/t/logstash-in-reading-files-documents/52418 "2016-06-14T06:47:42Z")
I would like to know if there are any ways that a logstash configuration file can read through different documents, i.e. docx, pdf, excels, and store them into elasticsearch. Great thanks in advance. --- ## [DATESTAMP\_OTHER seems not parsing properly](https://discuss.elastic.co/t/datestamp-other-seems-not-parsing-properly/52657)
**Author:** [@Adam\_M](https://discuss.elastic.co/u/Adam_M)\ **Replies:** 1\ **Last updated:** [June 14, 2016, 6:47am UTC](https://discuss.elastic.co/t/datestamp-other-seems-not-parsing-properly/52657 "2016-06-14T06:47:36Z")
Hello! I started playing with Logstash and Kibana and I got stuck with my logfile. I tried different approaches but I just can't parse timestamp properly, elastic keeps mapping that field as string... Anybody help, ple… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1140) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1142)