# How to break down a multiline entry with nested json - File input plugin

**URL:** <https://discuss.elastic.co/t/how-to-break-down-a-multiline-entry-with-nested-json-file-input-plugin/303245>\
**Category:** Logstash\
**Created:** [April 26, 2022, 9:28am UTC](https://discuss.elastic.co/t/how-to-break-down-a-multiline-entry-with-nested-json-file-input-plugin/303245 "2022-04-26T09:28:42Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![stillfreem](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stillfreem/32/85628_2.png) [@stillfreem](https://discuss.elastic.co/u/stillfreem)\
**Post date:** [April 26, 2022, 9:28am UTC](https://discuss.elastic.co/t/how-to-break-down-a-multiline-entry-with-nested-json-file-input-plugin/303245/1 "2022-04-26T09:28:42Z")

</div>

Hi All,

I have the following multiline log in json.  
My setup is that I read from a file (where my events are in embedded json) and the events gets broken down (each row is a treated as a single entry) wrongly.  
How can I tell logstash that one entry starts with a `{` and ends with a `}`?  
As you can see there are some `{}` in between but they are indented so if I can somehow tell to logstash to mind oif there are spaces before the `{`?  
I was thinking to use the `delimiter` properties of the file input plugin but this will still break down the entries?

```auto
{
     "metadata": {
         "customerIDString": "aa0a01be755244378d9aeff317ae3876",
         "offset": 43852,
         "eventType": "ScheduledReportNotificationEvent",
         "eventCreationTime": 1648080282000,
         "version": "1.0"
     },
     "event": {
         "UserUUID": "1fccffe5-8965-4889-9a9a-e020024aae1a",
         "UserID": "Sergei.Pogrebnyak@softwareag.com",
         "ExecutionID": "514f482c31624a408e683c9f9e452d25",
         "ReportID": "65b7b69e2e9e4d1fa946d0d1d5db14e4",
         "ReportName": "Container usage report",
         "ReportType": "dashboard",
         "Status": 2,
         "StatusMessage": "Dashboard not found"
     }
}

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 26, 2022, 4:59pm UTC](https://discuss.elastic.co/t/how-to-break-down-a-multiline-entry-with-nested-json-file-input-plugin/303245/2 "2022-04-26T16:59:34Z")

</div>

You can use a [multiline](https://www.elastic.co/guide/en/logstash/current/plugins-codecs-multiline.html) code to combine everything up to a ^} as a single event. You may need to use the auto\_flush\_interval option.

---

<div class="post-metadata">

**Author:** ![stillfreem](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stillfreem/32/85628_2.png) [@stillfreem](https://discuss.elastic.co/u/stillfreem)\
**Post date:** [April 26, 2022, 5:49pm UTC](https://discuss.elastic.co/t/how-to-break-down-a-multiline-entry-with-nested-json-file-input-plugin/303245/3 "2022-04-26T17:49:39Z")

</div>

it worked @Badger.  
However, could you please elaborate a bit on the auto\_flush option.  
Currently, my config works but now and then it pops up the following INFOs in the log file and nothing happens (The last three lines.)

```auto
[2022-04-26T18:35:47,914][INFO][logstash.javapipeline] Starting pipeline {:pipeline_id=>"crowdstrike", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>1000, "pipeline.sources"=>["/etc/logstash/conf.d/crowdstrike.conf"], :thread=>"#<Thread:0x7a198858@/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:54 run>"}
[2022-04-26T18:35:49,064][INFO][logstash.javapipeline] Pipeline Java execution initialization time {"seconds"=>1.15}
[2022-04-26T18:35:49,678][INFO][logstash.inputs.file] No sincedb_path set, generating one based on the "path" setting {:sincedb_path=>"/var/lib/logstash/plugins/inputs/file/.sincedb_ddb72ae539c9547189b77bcc700407f6", :path=>["/var/log/crowdstrike/falconhoseclient/output"]}
[2022-04-26T18:35:49,701][INFO][logstash.javapipeline] Pipeline started {"pipeline.id"=>"crowdstrike"}
[2022-04-26T18:35:49,766][INFO][filewatch.observingtail] START, creating Discoverer, Watch with file and sincedb collections
[2022-04-26T18:35:49,837][INFO][com.microsoft.azure.kusto.ingest.ResourceManager] Refreshing Ingestion Resources
[2022-04-26T19:35:49,837][INFO][com.microsoft.azure.kusto.ingest.ResourceManager] Refreshing Ingestion Auth Token
[2022-04-26T19:35:50,139][INFO][com.microsoft.azure.kusto.ingest.ResourceManager] Refreshing Ingestion Resources

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 26, 2022, 6:08pm UTC](https://discuss.elastic.co/t/how-to-break-down-a-multiline-entry-with-nested-json-file-input-plugin/303245/4 "2022-04-26T18:08:43Z")

</div>

I cannot speak to the com.microsoft.azure... messages. I have no idea what they are from or about.

> [@stillfreem](#):
>
> could you please elaborate a bit on the auto\_flush option.

If you have a file like

```
{
"someField": "a"
}
{
"anotherField": 1
}

```

and use a configuration of the multiline codec like

```
codec => multiline {
    pattern => "^{"
    negate => true
    what => "previous"
}

```

then it will combine a line that starts with { with every line until the next line that starts with {. That means it will only flush the `{ "someField": "a" }` event, because there is not a third { to cause the `{ "anotherField": 1 }` event to be flushed. auto flush will flush the second event based on a timeout.

---

<div class="post-metadata">

**Author:** ![stillfreem](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stillfreem/32/85628_2.png) [@stillfreem](https://discuss.elastic.co/u/stillfreem)\
**Post date:** [April 26, 2022, 6:28pm UTC](https://discuss.elastic.co/t/how-to-break-down-a-multiline-entry-with-nested-json-file-input-plugin/303245/5 "2022-04-26T18:28:03Z")

</div>

Thank you very much @Badger 🙂

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 24, 2022, 6:28pm UTC](https://discuss.elastic.co/t/how-to-break-down-a-multiline-entry-with-nested-json-file-input-plugin/303245/6 "2022-05-24T18:28:38Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.