# Pipeline starts but data does not ingest - json or xml files **URL:** \ **Category:** Logstash\ **Created:** [September 22, 2017, 8:44pm UTC](https://discuss.elastic.co/t/pipeline-starts-but-data-does-not-ingest-json-or-xml-files/101538 "2017-09-22T20:44:29Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![cxbarbosa](https://avatars.discourse-cdn.com/v4/letter/c/df705f/32.png) [@cxbarbosa](https://discuss.elastic.co/u/cxbarbosa)\ **Post date:** [September 22, 2017, 8:44pm UTC](https://discuss.elastic.co/t/pipeline-starts-but-data-does-not-ingest-json-or-xml-files/101538/1 "2017-09-22T20:44:29Z")
I'm using API call to pull data into json docs and ingesting through logstash, but when I go to run my config file, I get this in log stash [2017-09-15T16:08:02,579][INFO][logstash.pipeline] Pipeline main started [2017-09-15T16:08:02,658][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=\>9600} No actual data is ingested however and an index is not created. Same thing happens with JSON or XML. I have no clue what I'm doing wrong because the logs do not show an error. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [September 24, 2017, 8:48pm UTC](https://discuss.elastic.co/t/pipeline-starts-but-data-does-not-ingest-json-or-xml-files/101538/2 "2017-09-24T20:48:04Z")
What does your configuration look like? ---
**Author:** ![cxbarbosa](https://avatars.discourse-cdn.com/v4/letter/c/df705f/32.png) [@cxbarbosa](https://discuss.elastic.co/u/cxbarbosa)\ **Post date:** [September 28, 2017, 9:41pm UTC](https://discuss.elastic.co/t/pipeline-starts-but-data-does-not-ingest-json-or-xml-files/101538/3 "2017-09-28T21:41:09Z")
input { file { path =\> "/Users/S.AWDRSRV/Downloads/MDMCSV/json/\*.json" start\_position =\> "beginning" sincedb\_path =\> "/dev/null" } } filter { json { source =\> "message" #filters not added yet } } output { stdout {codec =\> rubydebug } elasticsearch { hosts =\> ["[http://localhost:9200](http://localhost:9200)"] index =\> "test" } } ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [September 29, 2017, 6:10am UTC](https://discuss.elastic.co/t/pipeline-starts-but-data-does-not-ingest-json-or-xml-files/101538/4 "2017-09-29T06:10:10Z")
What if you comment out the elasticsearch output and focus on the stdout output? Are you getting anything? Are you sure Logstash has read access to the JSON files? ---
**Author:** ![Samireddy\_Keerthi](https://avatars.discourse-cdn.com/v4/letter/s/4bbf92/32.png) [@Samireddy\_Keerthi](https://discuss.elastic.co/u/Samireddy_Keerthi)\ **Post date:** [September 29, 2017, 9:48am UTC](https://discuss.elastic.co/t/pipeline-starts-but-data-does-not-ingest-json-or-xml-files/101538/5 "2017-09-29T09:48:45Z")
Even I am facing the same issue. My logstash.conf looks like this input { file { path =\> "C:/logs/msglogin/\*.log" start\_position =\> "beginning" sincedb\_path =\> "null" codec=\>"json" } } filter { json { source =\> "message" } } output { stdout { codec =\> rubydebug } } and json file data is {"\_index": "customer", "\_type": "product", "\_id": 1} ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [September 29, 2017, 10:03am UTC](https://discuss.elastic.co/t/pipeline-starts-but-data-does-not-ingest-json-or-xml-files/101538/6 "2017-09-29T10:03:59Z")
@Samireddy_Keerthi, please start a separate thread for your problem. ---
**Author:** ![cxbarbosa](https://avatars.discourse-cdn.com/v4/letter/c/df705f/32.png) [@cxbarbosa](https://discuss.elastic.co/u/cxbarbosa)\ **Post date:** [September 29, 2017, 11:34pm UTC](https://discuss.elastic.co/t/pipeline-starts-but-data-does-not-ingest-json-or-xml-files/101538/7 "2017-09-29T23:34:24Z")
If I comment it out, will it still pipe to the index? How can I check its read permissions? I run it as admin. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [October 2, 2017, 5:21am UTC](https://discuss.elastic.co/t/pipeline-starts-but-data-does-not-ingest-json-or-xml-files/101538/8 "2017-10-02T05:21:03Z")
> If I comment it out, will it still pipe to the index? No, and that's the whole point. We want to simplify the configuration to make it easier to debug it. > How can I check its read permissions? I run it as admin. Make sure the file itself is readable to the user that runs Logstash and that all directories leading up to the directory are at least executable to that user. The last directory, /Users/S.AWDRSRV/Downloads/MDMCSV/json, also needs to be readable. You can increase Logstash's logging verbosity (there's a command line option for it) and it'll log additional details about what it's doing, including what filename patterns are expanded to. If your filename pattern expands to nothing that usually means that it doesn't have sufficient permissions. ---
**Author:** ![cxbarbosa](https://avatars.discourse-cdn.com/v4/letter/c/df705f/32.png) [@cxbarbosa](https://discuss.elastic.co/u/cxbarbosa)\ **Post date:** [October 6, 2017, 12:07am UTC](https://discuss.elastic.co/t/pipeline-starts-but-data-does-not-ingest-json-or-xml-files/101538/9 "2017-10-06T00:07:36Z")
this is the main output that I get: Successfully started Logstash API endpoint {:port=\>9600} [2017-10-05T17:06:11,384][DEBUG][logstash.inputs.file] each: file grew: /Users/S.AWDRSRV/Downloads/MDMCSV/json/AWoutput.json: old size 0, new size 24162 [2017-10-05T17:06:12,388][DEBUG][logstash.inputs.file] each: file grew: /Users/S.AWDRSRV/Downloads/MDMCSV/json/AWoutput.json: old size 0, new size 24162 [2017-10-05T17:06:13,391][DEBUG][logstash.inputs.file] each: file grew: /Users/S.AWDRSRV/Downloads/MDMCSV/json/AWoutput.json: old size 0, new size 24162 [2017-10-05T17:06:14,396][DEBUG][logstash.inputs.file] each: file grew: /Users/S.AWDRSRV/Downloads/MDMCSV/json/AWoutput.json: old size 0, new size 24162 [2017-10-05T17:06:15,370][DEBUG][logstash.pipeline] Pushing flush onto pipeline [2017-10-05T17:06:15,400][DEBUG][logstash.inputs.file] each: file grew: /Users/S.AWDRSRV/Downloads/MDMCSV/json/AWoutput.json: old size 0, new size 24162 [2017-10-05T17:06:16,402][DEBUG][logstash.inputs.file] each: file grew: /Users/S.AWDRSRV/Downloads/MDMCSV/json/AWoutput.json: old size 0, new size 24162 It just loops this repeatedly and never creates the index. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 3, 2017, 12:08am UTC](https://discuss.elastic.co/t/pipeline-starts-but-data-does-not-ingest-json-or-xml-files/101538/10 "2017-11-03T00:08:13Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.