# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=619 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 620 --- ## [Fetch data from MongoDB and feed to Elasticsearch](https://discuss.elastic.co/t/fetch-data-from-mongodb-and-feed-to-elasticsearch/164113)
**Author:** [@Vivek\_Sharma](https://discuss.elastic.co/u/Vivek_Sharma)\ **Replies:** 0\ **Last updated:** [January 14, 2019, 9:10am UTC](https://discuss.elastic.co/t/fetch-data-from-mongodb-and-feed-to-elasticsearch/164113 "2019-01-14T09:10:47Z")
Hi Is there any way that we can fetch data from MongoDB (on some remote server) and feed it into Elasticsearch server. I have a scenario where my data flows as given: SAP -\> Integration Layer -\> MongoDB (Remote server) … --- ## [Include One Identifier for each log entry](https://discuss.elastic.co/t/include-one-identifier-for-each-log-entry/163919)
**Author:** [@mrashid](https://discuss.elastic.co/u/mrashid)\ **Replies:** 6\ **Last updated:** [January 14, 2019, 8:00am UTC](https://discuss.elastic.co/t/include-one-identifier-for-each-log-entry/163919 "2019-01-14T08:00:12Z")
My aim is to include an identifier for logs of a few Jenkins Jobs that run in sequence. Kindly give your opinion on the following -- Is there any way to append an ID such as '1234' to each document entry i.e each lin… --- ## [Input cloudwatch plugin installation](https://discuss.elastic.co/t/input-cloudwatch-plugin-installation/164097)
**Author:** [@sunilmchaudhari](https://discuss.elastic.co/u/sunilmchaudhari)\ **Replies:** 0\ **Last updated:** [January 14, 2019, 7:25am UTC](https://discuss.elastic.co/t/input-cloudwatch-plugin-installation/164097 "2019-01-14T07:25:58Z")
Hi, where Can I get the logstash input cloudwatch plugin to install? I downloaded the zip file from git but its not working, Getting error while installing.. ERROR: Invalid pack for: file:///var/tmp/logstash-input-c… --- ## [Parse Error while parsing JUnit XML](https://discuss.elastic.co/t/parse-error-while-parsing-junit-xml/164037)
**Author:** [@Rohin\_Gopalakrishnan](https://discuss.elastic.co/u/Rohin_Gopalakrishnan)\ **Replies:** 0\ **Last updated:** [January 13, 2019, 12:35pm UTC](https://discuss.elastic.co/t/parse-error-while-parsing-junit-xml/164037 "2019-01-13T12:35:47Z")
I am trying to parse a JUnit XML and I always get the \_xmlparsefailure stating the reason as Preformatted text:exception=\>#\ **Author:** [@Robin\_Guo](https://discuss.elastic.co/u/Robin_Guo)\ **Replies:** 3\ **Last updated:** [January 14, 2019, 4:19am UTC](https://discuss.elastic.co/t/spring-boot-cant-ship-metrics-to-logstash-with-http-post/163320 "2019-01-14T04:19:06Z") Hi Elastic Folks, Recently we have a case to send spring boot metrics to ES, our java developers have deployed the spring boot app with micrometer library(https://micrometer.io/). basically, with micrometer library, spr… --- ## [Logstash can't convert boolean into boolean, it crash](https://discuss.elastic.co/t/logstash-cant-convert-boolean-into-boolean-it-crash/164015)
**Author:** [@Sebastian\_Treu](https://discuss.elastic.co/u/Sebastian_Treu)\ **Replies:** 4\ **Last updated:** [January 13, 2019, 10:54pm UTC](https://discuss.elastic.co/t/logstash-cant-convert-boolean-into-boolean-it-crash/164015 "2019-01-13T22:54:22Z")
Hi, I'm moving data from 1 elasticsearch to another and I'm using logstash for that. The thing is that logstash is randomly crashing in this procedure. I've monitored the heap, the queues and cpu, and so far the were OK… --- ## [Logstash - rename every occurrence of a multi-level nested JSON array element](https://discuss.elastic.co/t/logstash-rename-every-occurrence-of-a-multi-level-nested-json-array-element/163989)
**Author:** [@amitnegi6190](https://discuss.elastic.co/u/amitnegi6190)\ **Replies:** 4\ **Last updated:** [January 13, 2019, 3:35pm UTC](https://discuss.elastic.co/t/logstash-rename-every-occurrence-of-a-multi-level-nested-json-array-element/163989 "2019-01-13T15:35:02Z")
Hi, I am trying to send some JSON data into elasticsearch using logstash: My logstash configuration looks like this: test.config input{ file{ path =\> "/Users/bob/Study/data/test.json" codec =\> json sincedb\_p… --- ## [How to create multiple indexes based on conduction in logstash](https://discuss.elastic.co/t/how-to-create-multiple-indexes-based-on-conduction-in-logstash/164020)
**Author:** [@Vinit\_Kumar](https://discuss.elastic.co/u/Vinit_Kumar)\ **Replies:** 4\ **Last updated:** [January 13, 2019, 3:20pm UTC](https://discuss.elastic.co/t/how-to-create-multiple-indexes-based-on-conduction-in-logstash/164020 "2019-01-13T15:20:11Z")
Trying to create multiple indexes for elasticsearch in logstash. But my "if conduction" is not creating any single index, without if conduction it is working fine. But if I'm using input as file and in logstash without … --- ## [Logstash plugin fails to run from source](https://discuss.elastic.co/t/logstash-plugin-fails-to-run-from-source/164041)
**Author:** [@Sprrw](https://discuss.elastic.co/u/Sprrw)\ **Replies:** 0\ **Last updated:** [January 13, 2019, 2:35pm UTC](https://discuss.elastic.co/t/logstash-plugin-fails-to-run-from-source/164041 "2019-01-13T14:35:16Z")
Hi, First, please note that I'm pretty new to Ruby. I would like to play around with the logstash-output-kafka plugin for educational purposes, but I can't get it to run even before I have made any modifications to it. … --- ## [Expected one of #, input Error](https://discuss.elastic.co/t/expected-one-of-input-error/163968)
**Author:** [@psncius](https://discuss.elastic.co/u/psncius)\ **Replies:** 1\ **Last updated:** [January 13, 2019, 1:28pm UTC](https://discuss.elastic.co/t/expected-one-of-input-error/163968 "2019-01-13T13:28:22Z")
i try monitoring iis use elasticsearch and logstash run in docker and want to use template for ouput elasticsearch but i got this error \[2019-01-12T02:38:28,874\]\[ERROR\]\[logstash.agent \] Failed to execute acti… --- ## [How to set default analyzer for new Logstash indices?](https://discuss.elastic.co/t/how-to-set-default-analyzer-for-new-logstash-indices/163976)
**Author:** [@aqiank](https://discuss.elastic.co/u/aqiank)\ **Replies:** 2\ **Last updated:** [January 13, 2019, 4:34am UTC](https://discuss.elastic.co/t/how-to-set-default-analyzer-for-new-logstash-indices/163976 "2019-01-13T04:34:38Z")
Hi, I'm new to the Elastic stack in general. I would like to know how to set or modify the default analyzer for new indices. I'm currently modifying the analyzer to stop queries from lower-casing the tokens. I have succe… --- ## [JSON Result with 'columns' and 'rows' values - help parsing](https://discuss.elastic.co/t/json-result-with-columns-and-rows-values-help-parsing/163815)
**Author:** [@mseto](https://discuss.elastic.co/u/mseto)\ **Replies:** 12\ **Last updated:** [January 13, 2019, 1:30am UTC](https://discuss.elastic.co/t/json-result-with-columns-and-rows-values-help-parsing/163815 "2019-01-13T01:30:41Z")
I've got a JSON document like so :: { "tables": \[{ "name": "PrimaryResult", "columns": \[{ "name": "test", "type": "string" }, { "name": "test2, "type": "string" }\], "rows": \[ \["testdatarow1", "test2datarow1"\]… --- ## [Load balancing a 3 x Logstash behind PowerDNS setup](https://discuss.elastic.co/t/load-balancing-a-3-x-logstash-behind-powerdns-setup/163991)
**Author:** [@Zerobot](https://discuss.elastic.co/u/Zerobot)\ **Replies:** 1\ **Last updated:** [January 12, 2019, 4:43pm UTC](https://discuss.elastic.co/t/load-balancing-a-3-x-logstash-behind-powerdns-setup/163991 "2019-01-12T16:43:13Z")
HI! So in my setup I'm pushing logs with NXLog (aix) and Filebeat (linux) into 3 logstashes (3 nodes). We have PowerDNS setup to forward everyting from dns "example.logstash.dns" into 3 IP adressess of those logstashes.… --- ## [Logstash ruby filter doesn't parse json and return whole message field in Kibana](https://discuss.elastic.co/t/logstash-ruby-filter-doesnt-parse-json-and-return-whole-message-field-in-kibana/163646)
**Author:** [@cheriemilk](https://discuss.elastic.co/u/cheriemilk)\ **Replies:** 3\ **Last updated:** [January 12, 2019, 2:23am UTC](https://discuss.elastic.co/t/logstash-ruby-filter-doesnt-parse-json-and-return-whole-message-field-in-kibana/163646 "2019-01-12T02:23:10Z")
Hi , I have json log file need to parse like below. {"DP":"dbpool3","CMID":"TALSCMCherie","CMN":"Siemens","UID":"cgrant1","UN":"cgrant","PUID":"lokamoto1","UL":"en\_GB","CIP":"192.168.56.1","SN":"qacandrot\_TALSCMCherie"… --- ## [Can't get text on START\_OBJECT](https://discuss.elastic.co/t/cant-get-text-on-start-object/162881)
**Author:** [@Mark\_Bassett](https://discuss.elastic.co/u/Mark_Bassett)\ **Replies:** 7\ **Last updated:** [January 12, 2019, 12:16am UTC](https://discuss.elastic.co/t/cant-get-text-on-start-object/162881 "2019-01-12T00:16:29Z")
Hi guys, I have some really bad source data that is giving me a mapping conflict when I try to ingest because sometimes it is a string value, and other times it is an array. I've been fighting around with this for a whi… --- ## [JSON parsing question - Elasticsearch+Kibana+Logstash 6.5](https://discuss.elastic.co/t/json-parsing-question-elasticsearch-kibana-logstash-6-5/163255)
**Author:** [@thiagotankian](https://discuss.elastic.co/u/thiagotankian)\ **Replies:** 11\ **Last updated:** [January 11, 2019, 7:13pm UTC](https://discuss.elastic.co/t/json-parsing-question-elasticsearch-kibana-logstash-6-5/163255 "2019-01-11T19:13:00Z")
Hi, I'm a new user from Elastic and I'm trying to parse and index a json file using logstash, but so far I was able to do it but the fields were not indexed as expected - for example - all my fields were not indexed - I… --- ## [JMS listener](https://discuss.elastic.co/t/jms-listener/163930)
**Author:** [@jspeer](https://discuss.elastic.co/u/jspeer)\ **Replies:** 0\ **Last updated:** [January 11, 2019, 4:31pm UTC](https://discuss.elastic.co/t/jms-listener/163930 "2019-01-11T16:31:44Z")
Has anyone found a way of listening for JMS messages in logstash? I know that I can use the input to pull from a JMS queue, but I'm looking for a way to have a JMS message forwarded directly to logstash. --- ## [Unable to parse this timestamp-very strange](https://discuss.elastic.co/t/unable-to-parse-this-timestamp-very-strange/163903)
**Author:** [@Nithani25](https://discuss.elastic.co/u/Nithani25)\ **Replies:** 1\ **Last updated:** [January 11, 2019, 3:03pm UTC](https://discuss.elastic.co/t/unable-to-parse-this-timestamp-very-strange/163903 "2019-01-11T15:03:57Z")
Hi Team, I am working two sets of logs, with different timestamp files and i am not able to parse them Format 1: 2018-Dec-18 08:18:21,531 Logstash conf match =\> { "message" =\> "%{YEAR:year}-%{MONTH:month}-%{MONTHDAY:… --- ## [XML Filter Share value between events](https://discuss.elastic.co/t/xml-filter-share-value-between-events/163360)
**Author:** [@Anabella\_Cristaldi](https://discuss.elastic.co/u/Anabella_Cristaldi)\ **Replies:** 2\ **Last updated:** [January 11, 2019, 2:35pm UTC](https://discuss.elastic.co/t/xml-filter-share-value-between-events/163360 "2019-01-11T14:35:30Z")
Hi all, I need to ingest XML files which are the output of a Nessus scanner. The XML has this structure ?xml version="1.0" ?\> \ \ \ **Author:** [@Harsh\_Sharma](https://discuss.elastic.co/u/Harsh_Sharma)\ **Replies:** 2\ **Last updated:** [January 11, 2019, 1:38pm UTC](https://discuss.elastic.co/t/filebeat-logstash-configuration-6-5-4/163872 "2019-01-11T13:38:13Z") Hi There, I have installed ELK & filebeat version 6.5.4. Kibana & ES configuration is good. From a server I want to send logs (.csv format) from 3 different location but facing problem in configuration somewhere i thin… --- ## [\[logstash.codecs.netflow \] Template length exceeds flowset length, skipping {:template\_id=\>256, :template\_length=\>53, :record\_length=\>48}](https://discuss.elastic.co/t/logstash-codecs-netflow-template-length-exceeds-flowset-length-skipping-template-id-256-template-length-53-record-length-48/163895)
**Author:** [@geriko2019](https://discuss.elastic.co/u/geriko2019)\ **Replies:** 0\ **Last updated:** [January 11, 2019, 11:49am UTC](https://discuss.elastic.co/t/logstash-codecs-netflow-template-length-exceeds-flowset-length-skipping-template-id-256-template-length-53-record-length-48/163895 "2019-01-11T11:49:15Z")
Hi, I have a running implementation of the ELK stack (6.X), with elastiflow module configured (with Elastiflow Dashboard configured on Kibana). I however get the following error on running logstash. The netflow logs ar… --- ## [Upload log file in elasticsearch using logstash](https://discuss.elastic.co/t/upload-log-file-in-elasticsearch-using-logstash/163736)
**Author:** [@Bhavesh\_Padharia](https://discuss.elastic.co/u/Bhavesh_Padharia)\ **Replies:** 10\ **Last updated:** [January 11, 2019, 10:54am UTC](https://discuss.elastic.co/t/upload-log-file-in-elasticsearch-using-logstash/163736 "2019-01-11T10:54:07Z")
Hello Everyone , i am new in elasticsearch I try to upload log file in elasticsearch using logstash i use this curl command system@system-VirtualBox:~/Downloads/logstash-6.5.4$ curl -s -XPOST localhost:9200/\_bulk --… --- ## [Unable to read filebeat output in Logstash](https://discuss.elastic.co/t/unable-to-read-filebeat-output-in-logstash/163852)
**Author:** [@Sripal](https://discuss.elastic.co/u/Sripal)\ **Replies:** 2\ **Last updated:** [January 11, 2019, 10:15am UTC](https://discuss.elastic.co/t/unable-to-read-filebeat-output-in-logstash/163852 "2019-01-11T10:15:16Z")
Dear Team, I tried to read the filebeat output in logstash but am getting error. my Logstash conf file input { beats { port =\> 5044 } } filter{ if\[message\] =~ /^\\\[.\*/ { grok { match =\> \["message" ,'%{SYSLOG54… --- ## [Missing Converter handling for full class name=org.postgresql.util.PGobject, simple name=PGobject](https://discuss.elastic.co/t/missing-converter-handling-for-full-class-name-org-postgresql-util-pgobject-simple-name-pgobject/163338)
**Author:** [@Kit\_Tsang](https://discuss.elastic.co/u/Kit_Tsang)\ **Replies:** 2\ **Last updated:** [January 11, 2019, 9:24am UTC](https://discuss.elastic.co/t/missing-converter-handling-for-full-class-name-org-postgresql-util-pgobject-simple-name-pgobject/163338 "2019-01-11T09:24:01Z")
{ statement=\> "select json\_agg(json\_build\_object('a', value1::int)) from my\_table" // postgres } I got the following error: Exception when executing JDBC query {:exception=\>#\ **Author:** [@mrashid](https://discuss.elastic.co/u/mrashid)\ **Replies:** 4\ **Last updated:** [January 11, 2019, 9:20am UTC](https://discuss.elastic.co/t/copy-data-from-a-field-and-make-it-available-for-all-the-documents/163735 "2019-01-11T09:20:37Z") I am using grok filter to pass the number from a document of field message. For instance, The build ID of the run is : 128 From the above document entry I have used grok to extract "128" to a new field. However , t… --- ## [Output file adding static opening/closing line](https://discuss.elastic.co/t/output-file-adding-static-opening-closing-line/163695)
**Author:** [@thdesy](https://discuss.elastic.co/u/thdesy)\ **Replies:** 2\ **Last updated:** [January 11, 2019, 9:18am UTC](https://discuss.elastic.co/t/output-file-adding-static-opening-closing-line/163695 "2019-01-11T09:18:31Z")
Hi all, probably a pretty newbie question, but I would like to add to output files automatically a static opening and closing line. I.e., I would like to split on a daily basis an output file \[1\] where I have individu… --- ## [Elasticsearch mapping - cannot index a field having name starting with a dot](https://discuss.elastic.co/t/elasticsearch-mapping-cannot-index-a-field-having-name-starting-with-a-dot/163804)
**Author:** [@amitnegi6190](https://discuss.elastic.co/u/amitnegi6190)\ **Replies:** 7\ **Last updated:** [January 11, 2019, 8:47am UTC](https://discuss.elastic.co/t/elasticsearch-mapping-cannot-index-a-field-having-name-starting-with-a-dot/163804 "2019-01-11T08:47:57Z")
Hi, I am trying to send some JSON data into elasticsearch using logstash: My logstash configuration looks like this: test.config input{ file{ path =\> "/Users/bob/data/test.json" codec =\> json sincedb\_path =\> … --- ## [Logstash updating the data not working](https://discuss.elastic.co/t/logstash-updating-the-data-not-working/163862)
**Author:** [@inandi](https://discuss.elastic.co/u/inandi)\ **Replies:** 2\ **Last updated:** [January 11, 2019, 8:42am UTC](https://discuss.elastic.co/t/logstash-updating-the-data-not-working/163862 "2019-01-11T08:42:42Z")
logstash.conf file input { file { path =\> "/var/www/html/review/EsData/review\_data.json" start\_position =\> "beginning" sincedb\_path =\> "/dev/null" codec =\> "json" } } filter { } output { … --- ## [TCP output plugin blocking port across configuration reload](https://discuss.elastic.co/t/tcp-output-plugin-blocking-port-across-configuration-reload/163732)
**Author:** [@vinayakshukre](https://discuss.elastic.co/u/vinayakshukre)\ **Replies:** 2\ **Last updated:** [January 11, 2019, 8:36am UTC](https://discuss.elastic.co/t/tcp-output-plugin-blocking-port-across-configuration-reload/163732 "2019-01-11T08:36:44Z")
Hello Logstash community members, I am relatively new to logstash and want to read data sent by TCP output plugin in my java code. My conf file is shown below. input { tcp { port =\> 5518… --- ## [Logstash updating not working](https://discuss.elastic.co/t/logstash-updating-not-working/163859)
**Author:** [@inandi](https://discuss.elastic.co/u/inandi)\ **Replies:** 2\ **Last updated:** [January 11, 2019, 8:26am UTC](https://discuss.elastic.co/t/logstash-updating-not-working/163859 "2019-01-11T08:26:57Z")
this is logstash.conf file input { file { path =\> "/var/www/html/review/EsData/review\_data.json" start\_position =\> "beginning" sincedb\_path =\> "/dev/null" codec =\> "json" } } filter { } ou… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=618) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=620)