# Yet another logstash and json issue, unable to ingest a "basic" json

**URL:** <https://discuss.elastic.co/t/yet-another-logstash-and-json-issue-unable-to-ingest-a-basic-json/277991>\
**Category:** Logstash\
**Created:** [July 6, 2021, 7:36pm UTC](https://discuss.elastic.co/t/yet-another-logstash-and-json-issue-unable-to-ingest-a-basic-json/277991 "2021-07-06T19:36:50Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![cibernicola](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cibernicola/32/85621_2.png) [@cibernicola](https://discuss.elastic.co/u/cibernicola)\
**Post date:** [July 6, 2021, 7:36pm UTC](https://discuss.elastic.co/t/yet-another-logstash-and-json-issue-unable-to-ingest-a-basic-json/277991/1 "2021-07-06T19:36:50Z")

</div>

I have a dir with tons of subdirs with a json file like this:

```auto
 
{
    "id": "2234",
    "name": "Text",
    "url": "https://url.com",
    "param": "string",
    "last": "2021-07-06 20:05:49.458724",
    "url2": "url.com"
}

```

I've tried all things I'm able to search, actually I'm with this L config:

```auto
# Input section
input {
  file {
    type => "json"
	codec => multiline { pattern => "^Spalanzani" what => "previous" negate => true auto_flush_interval => 1 }

    path => "path/**/*.json"
    start_position => "beginning"
	sincedb_path => "path.log"
  }
}

#filter section
filter {
		  json {
			source => "message"
		  }
		mutate { 
		 remove_field => ["host", "@version", "type", "path", "tags", "@timestamp"]				
		}	
}

output {
  elasticsearch {
    hosts => ["http://localhost:9200"]
    index => "myIndex"
    user => "usr" 
    password => "mypwd"
  }
    stdout {codec => rubydebug}
}

```

Don't know te reason but I'm sure that some time ago I was able to ingest a json file much more complex simply setting up type and condec to "json".... With previous config I get a message field filled with all fields, without last }, Why?  
Any idea suggestion? Thanks.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [July 6, 2021, 8:28pm UTC](https://discuss.elastic.co/t/yet-another-logstash-and-json-issue-unable-to-ingest-a-basic-json/277991/2 "2021-07-06T20:28:30Z")

</div>

> [@cibernicola](#):
>
> With previous config I get a message field filled with all fields, without last }, Why?

Are you saying that with that example file you get

```auto
{ "id": "2234", "name": "Text", "url": "https://url.com", "param": "string", "last": "2021-07-06 20:05:49.458724", "url2": "url.com"

in the message field?
```

---

<div class="post-metadata">

**Author:** ![cibernicola](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cibernicola/32/85621_2.png) [@cibernicola](https://discuss.elastic.co/u/cibernicola)\
**Post date:** [July 6, 2021, 8:57pm UTC](https://discuss.elastic.co/t/yet-another-logstash-and-json-issue-unable-to-ingest-a-basic-json/277991/3 "2021-07-06T20:57:13Z")

</div>

Exactly!  
I can't get tje point ☹  
Why with this basic json E.L. can't ingest all data in each field inside the doc? What m I doing wrong?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [July 6, 2021, 9:35pm UTC](https://discuss.elastic.co/t/yet-another-logstash-and-json-issue-unable-to-ingest-a-basic-json/277991/4 "2021-07-06T21:35:34Z")

</div>

I do not see anything wrong with your configuration. You could try increasing auto\_flush\_interval to see if that makes any difference but I very much doubt that it will.

---

<div class="post-metadata">

**Author:** ![cibernicola](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cibernicola/32/85621_2.png) [@cibernicola](https://discuss.elastic.co/u/cibernicola)\
**Post date:** [July 7, 2021, 10:09am UTC](https://discuss.elastic.co/t/yet-another-logstash-and-json-issue-unable-to-ingest-a-basic-json/277991/5 "2021-07-07T10:09:56Z")

</div>

The "problem" is that it inserts all the fields from the json file into the "message" field instead of creating a separate field within the elasticsearch doc for each field in the json file.  
I have tried much more complex files that, once ingested by logstash to elasticsearch have had different fields. I don't know what changes from one to the other :S

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 4, 2021, 10:10am UTC](https://discuss.elastic.co/t/yet-another-logstash-and-json-issue-unable-to-ingest-a-basic-json/277991/6 "2021-08-04T10:10:19Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.