# Ingesting JSON Data Samples w/ Logstash

**URL:** <https://discuss.elastic.co/t/ingesting-json-data-samples-w-logstash/275185>\
**Category:** Logstash\
**Created:** [June 7, 2021, 6:18pm UTC](https://discuss.elastic.co/t/ingesting-json-data-samples-w-logstash/275185 "2021-06-07T18:18:09Z")\
**Posts on this page:** 15\
**Page:** 2

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 9, 2021, 12:33am UTC](https://discuss.elastic.co/t/ingesting-json-data-samples-w-logstash/275185/21 "2021-06-09T00:33:51Z")

</div>

> [@](#):
>
> [TRACE][filewatch.discoverer][main][ember\_training] discover\_files {:count=\>0}

It is finding no files that match the value of @path. Could be a permissions issue.

Also @file\_completed\_log\_path should be a file, not a directory.

---

<div class="post-metadata">

**Author:** ![pkward](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pkward/32/95305_2.png) [@pkward](https://discuss.elastic.co/u/pkward)\
**Post date:** [June 9, 2021, 12:36am UTC](https://discuss.elastic.co/t/ingesting-json-data-samples-w-logstash/275185/22 "2021-06-09T00:36:45Z")

</div>

Badger please don't be upset with me, but it was a misspelling of the file name. 🤦‍♂️

---

<div class="post-metadata">

**Author:** ![pkward](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pkward/32/95305_2.png) [@pkward](https://discuss.elastic.co/u/pkward)\
**Post date:** [June 9, 2021, 12:41am UTC](https://discuss.elastic.co/t/ingesting-json-data-samples-w-logstash/275185/23 "2021-06-09T00:41:49Z")

</div>

Do you know how I can fix this?

```auto
JSON parse error, original data now in message field {:error=>#<LogStash::Json::ParserError: Unexpected character (',' (code 44)): Expected space separating root-level values

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 9, 2021, 12:43am UTC](https://discuss.elastic.co/t/ingesting-json-data-samples-w-logstash/275185/24 "2021-06-09T00:43:00Z")

</div>

Look at the [message] field. Is it valid JSON?

---

<div class="post-metadata">

**Author:** ![pkward](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pkward/32/95305_2.png) [@pkward](https://discuss.elastic.co/u/pkward)\
**Post date:** [June 9, 2021, 12:54am UTC](https://discuss.elastic.co/t/ingesting-json-data-samples-w-logstash/275185/25 "2021-06-09T00:54:39Z")

</div>

There are @message within the JSON, but they aren't necessarily used as a field. It's apart of the string in the field.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 9, 2021, 1:04am UTC](https://discuss.elastic.co/t/ingesting-json-data-samples-w-logstash/275185/26 "2021-06-09T01:04:18Z")

</div>

> [@pkward](#):
>
> `original data now in message field`

What is in the [message] field?

---

<div class="post-metadata">

**Author:** ![pkward](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pkward/32/95305_2.png) [@pkward](https://discuss.elastic.co/u/pkward)\
**Post date:** [June 9, 2021, 1:10am UTC](https://discuss.elastic.co/t/ingesting-json-data-samples-w-logstash/275185/27 "2021-06-09T01:10:18Z")

</div>

When I grep for it this is what I find and I see the comma that the error points out.

```auto
cat train_features_0.json | grep '"message"'
      "message",

```

---

<div class="post-metadata">

**Author:** ![pkward](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pkward/32/95305_2.png) [@pkward](https://discuss.elastic.co/u/pkward)\
**Post date:** [June 9, 2021, 1:16am UTC](https://discuss.elastic.co/t/ingesting-json-data-samples-w-logstash/275185/28 "2021-06-09T01:16:15Z")

</div>

Good news, is that I only found one occurrence.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 9, 2021, 1:16am UTC](https://discuss.elastic.co/t/ingesting-json-data-samples-w-logstash/275185/29 "2021-06-09T01:16:54Z")

</div>

No, I meant on stdout. But what you posted there indicates the problem. A json codec does not keep accumulating lines until it has a complete JSON object, it parses each line as an object. What you need is a multiline code to combine all the lines for one object into a single event, then parse it using a json filter.

If you want to ingest the whole file as one event see [Merge Multiline json into single line json using codec multiline plugin - #2 by Badger](https://discuss.elastic.co/t/merge-multiline-json-into-single-line-json-using-codec-multiline-plugin/143175/2). If a file contains multiple objects then [this](https://discuss.elastic.co/t/multiline-json-file-not-ingested-with-logstash-to-elasticsearch/177144) might work.

---

<div class="post-metadata">

**Author:** ![pkward](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pkward/32/95305_2.png) [@pkward](https://discuss.elastic.co/u/pkward)\
**Post date:** [June 9, 2021, 2:44am UTC](https://discuss.elastic.co/t/ingesting-json-data-samples-w-logstash/275185/30 "2021-06-09T02:44:50Z")

</div>

I was able to finally get data in, but now I'm having issues with my license and elasticsearch can't be reached.

---

<div class="post-metadata">

**Author:** ![pkward](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pkward/32/95305_2.png) [@pkward](https://discuss.elastic.co/u/pkward)\
**Post date:** [June 9, 2021, 2:48am UTC](https://discuss.elastic.co/t/ingesting-json-data-samples-w-logstash/275185/31 "2021-06-09T02:48:06Z")

</div>

```auto
elasticsearch]$ uncaught exception in thread [main]
java.lang.IllegalStateException: failed to obtain node locks, tried [[/home/elastic/elasticsearch/data]] with lock id [0]; maybe these locations are not writable or multiple nodes were started without increasing [node.max_local_storage_nodes] (was [1])?
	at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:292)
	at org.elasticsearch.node.Node.<init>(Node.java:368)
	at org.elasticsearch.node.Node.<init>(Node.java:278)
	at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:217)
	at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:217)
	at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:397)
	at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159)
	at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150)
	at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:75)
	at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:116)
	at org.elasticsearch.cli.Command.main(Command.java:79)
	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115)
	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:81)
For complete error details, refer to the log at /home/elastic/elasticsearch/logs/dev.log

```

---

<div class="post-metadata">

**Author:** ![pkward](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pkward/32/95305_2.png) [@pkward](https://discuss.elastic.co/u/pkward)\
**Post date:** [June 9, 2021, 2:48am UTC](https://discuss.elastic.co/t/ingesting-json-data-samples-w-logstash/275185/32 "2021-06-09T02:48:54Z")

</div>

Whenever I make some progress, I get stumped.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 9, 2021, 3:00am UTC](https://discuss.elastic.co/t/ingesting-json-data-samples-w-logstash/275185/33 "2021-06-09T03:00:48Z")

</div>

I suggest you ask a new question. This one has covered way too many issues.

---

<div class="post-metadata">

**Author:** ![pkward](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pkward/32/95305_2.png) [@pkward](https://discuss.elastic.co/u/pkward)\
**Post date:** [June 9, 2021, 9:57pm UTC](https://discuss.elastic.co/t/ingesting-json-data-samples-w-logstash/275185/34 "2021-06-09T21:57:10Z")

</div>

I was able to finally get data in.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 7, 2021, 9:57pm UTC](https://discuss.elastic.co/t/ingesting-json-data-samples-w-logstash/275185/35 "2021-07-07T21:57:26Z")

</div>

Closed on July 7, 2021, 9:57pm UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

[Previous page](https://discuss.elastic.co/t/ingesting-json-data-samples-w-logstash/275185.md?page=1)