# Ingest json file

**URL:** https://discuss.elastic.co/t/ingest-json-file/175652
**Category:** Logstash
**Created:** [April 5, 2019, 8:38pm UTC](https://discuss.elastic.co/t/ingest-json-file/175652 "2019-04-05T20:38:40Z")
**Posts on this page:** 8
**Page:** 1

<div class="post-metadata">

### Author: ![Miguel\_Plazas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/miguel_plazas/32/40038_2.png) [@Miguel\_Plazas](https://discuss.elastic.co/u/Miguel_Plazas)
#### Post date: [April 5, 2019, 8:38pm UTC](https://discuss.elastic.co/t/ingest-json-file/175652/1 "2019-04-05T20:38:40Z")

</div>

Currently I'm trying to ingest the following json file:

**my\_file.json**

```
{
  "key_1": {
    "key_1-a": "value_a",
    "key_1-b": "value_b"
  },
  "key_2": {
    "key_2-a": "value_a",
    "key_2-b": "value_b"
  }
}

```

**my\_config.conf**

```
input {
  file {

    codec => multiline
    {
        pattern => "^{"
        negate => true
        what => previous
    }

    path => "/etc/logstash/conf.d/my_file.json"
    start_position => "beginning"
    sincedb_path => "/dev/null"
    type => "json"
  }
}

filter {
  json {
    source => "message"
  }
}

output {
  stdout { }
}

```

}

Currently i'm getting following error when i hit **ctrl+c** :

> Error parsing json {:source=\>"message", :raw=\> ... " JSON content"  
> exception=\>#\<LogStash::Json::ParserError: Unexpected end-of-input: expected close marker for Object (start marker at [Source: (byte)"

And also some error tags:

```
    [0] "multiline",
    [1] "_jsonparsefailure"

```

Any idea to solve it?  
Thanks.

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [April 5, 2019, 9:48pm UTC](https://discuss.elastic.co/t/ingest-json-file/175652/2 "2019-04-05T21:48:08Z")

</div>

With the JSON that you show, that error does not occur. That suggests there is actually a problem with your JSON.

To consume an entire file as one event another option is to use a pattern that never matches and auto\_flush.

```
codec => multiline { pattern => "^Spalanzani" what => "previous" negate => true auto_flush_interval => 1 }
```

---

<div class="post-metadata">

### Author: ![Miguel\_Plazas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/miguel_plazas/32/40038_2.png) [@Miguel\_Plazas](https://discuss.elastic.co/u/Miguel_Plazas)
#### Post date: [April 5, 2019, 10:03pm UTC](https://discuss.elastic.co/t/ingest-json-file/175652/3 "2019-04-05T22:03:15Z")

</div>

> [@Badger](#):
>
> ^Spalanzani

Thanks. Currently i'm getting following error:

```
[2019-04-05T16:57:51,322][WARN][logstash.filters.json] Error parsing json {:source=>"message", :raw=>"{\n \"Efectivo y equivalentes de efectivo (Nota 3)\": {\n \"2016\": \"6,267,955\",\n \"2017\": \"8,514,586\"\n },\n \"Deudores, neto (Nota 4)\": {\n \"2016\": \"1,568,387\",\n \"2017\": \"72,531\"\n },\n \"Total activos corrientes\": {\n \"2016\": \"7,836,342\",\n \"2017\": \"8,587,117\"\n },\n \"Cuentas por pagar (Nota 6)\": {\n \"2016\": \"5,887,271\",\n \"2017\": \"3,297,923\"\n },\n \"Impuestos y administraciones públicas (Nota 14)\": {\n \"2016\": \"68,318\",\n \"2017\": \"21,432\"\n },\n \"Beneficio a empleados (Nota 7)\": {\n \"2016\": \"193,971\",\n \"2017\": \"186,035\"\n },\n \"Total pasivos\": {\n \"2016\": \"6,150,060\",\n \"2017\": \"3,505,390\"\n },\n \"Superávit del año\": {\n \"2016\": \"7,822,052\",\n \"2017\": \"4,218,621\"\n },\n \"Total activos netos\": {\n \"2016\": \"7,922,052\",\n \"2017\": \"5,081,727\"\n },\n \"Total pasivos y saldos de activos netos\": {\n \"2016\": \"14,072,112\",\n \"2017\": \"8,587,117\"\n },\n \"Tarjeta Profesional\": {\n \"2016\": \"61851-T Tarjeta\",\n \"2017\": \"Profbsional 110141-T\"\n }", :exception=>#<LogStash::Json::ParserError: Unexpected end-of-input: expected close marker for Object (start marker at [Source: (byte[])"{
  "Efectivo y equivalentes de efectivo (Nota 3)": {
    "2016": "6,267,955",
    "2017": "8,514,586"
  },
....
  },
  "Impuestos y administraciones publicas (Nota 14)": {
    "2016": "68,318",
    "2017": "21,432"
  },
  "Beneficio a empleados (Nota 7"[truncated 491 bytes]; line: 45, column: 995]>}

```

Why this could happend?

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [April 5, 2019, 10:44pm UTC](https://discuss.elastic.co/t/ingest-json-file/175652/4 "2019-04-05T22:44:42Z")

</div>

> [@Miguel\_Plazas](#):
>
> expected close marker for Object

If you look at your JSON it ends with

```auto
\"Profbsional 110141-T\"\n }`

```

That closes out "Tarjeta Profesional", but does not close the overall object. Your JSON is missing a trailing }

---

<div class="post-metadata">

### Author: ![Miguel\_Plazas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/miguel_plazas/32/40038_2.png) [@Miguel\_Plazas](https://discuss.elastic.co/u/Miguel_Plazas)
#### Post date: [April 5, 2019, 11:12pm UTC](https://discuss.elastic.co/t/ingest-json-file/175652/5 "2019-04-05T23:12:24Z")

</div>

Thanks for your help. But even using the following JSON with { and } start and ending still not working:

```
{
  "Efectivo y equivalentes de efectivo (Nota 3)": {
    "2016": "6,267,955",
    "2017": "8,514,586"
  },
  "Deudores, neto (Nota 4)": {
    "2016": "1,568,387",
    "2017": "72,531"
  },
  "Total activos corrientes": {
    "2016": "7,836,342",
    "2017": "8,587,117"
  },
  "Cuentas por pagar (Nota 6)": {
    "2016": "5,887,271",
    "2017": "3,297,923"
  },
  "Impuestos y administraciones públicas (Nota 14)": {
    "2016": "68,318",
    "2017": "21,432"
  },
  "Beneficio a empleados (Nota 7)": {
    "2016": "193,971",
    "2017": "186,035"
  },
  "Total pasivos": {
    "2016": "6,150,060",
    "2017": "3,505,390"
  },
  "Superávit del año": {
    "2016": "7,822,052",
    "2017": "4,218,621"
  },
  "Total activos netos": {
    "2016": "7,922,052",
    "2017": "5,081,727"
  },
  "Total pasivos y saldos de activos netos": {
    "2016": "14,072,112",
    "2017": "8,587,117"
  },
  "Tarjeta Profesional": {
    "2016": "61851-T Tarjeta",
    "2017": "Profbsional 110141-T"
  }
}

```

Same error:

```
[2019-04-05T18:13:52,075][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=>9600}
[2019-04-05T18:13:53,954][WARN][logstash.filters.json] Error parsing json {:source=>"message", :raw=>"{\n \"Efectivo y equivalentes de efectivo (Nota 3)\": {\n \"2016\": \"6,267,955\",\n \"2017\": \"8,514,586\"\n },\n \"Deudores, neto (Nota 4)\": {\n \"2016\": \"1,568,387\",\n \"2017\": \"72,531\"\n },\n \"Total activos corrientes\": {\n \"2016\": \"7,836,342\",\n \"2017\": \"8,587,117\"\n },\n \"Cuentas por pagar (Nota 6)\": {\n \"2016\": \"5,887,271\",\n \"2017\": \"3,297,923\"\n },\n \"Impuestos y administraciones públicas (Nota 14)\": {\n \"2016\": \"68,318\",\n \"2017\": \"21,432\"\n },\n \"Beneficio a empleados (Nota 7)\": {\n \"2016\": \"193,971\",\n \"2017\": \"186,035\"\n },\n \"Total pasivos\": {\n \"2016\": \"6,150,060\",\n \"2017\": \"3,505,390\"\n },\n \"Superávit del año\": {\n \"2016\": \"7,822,052\",\n \"2017\": \"4,218,621\"\n },\n \"Total activos netos\": {\n \"2016\": \"7,922,052\",\n \"2017\": \"5,081,727\"\n },\n \"Total pasivos y saldos de activos netos\": {\n \"2016\": \"14,072,112\",\n \"2017\": \"8,587,117\"\n },\n \"Tarjeta Profesional\": {\n \"2016\": \"61851-T Tarjeta\",\n \"2017\": \"Profbsional 110141-T\"\n }", :exception=>#<LogStash::Json::ParserError: Unexpected end-of-input: expected close marker for Object (start marker at [Source: (byte[])"{
  "Efectivo y equivalentes de efectivo (Nota 3)": {
    "2016": "6,267,955",
    "2017": "8,514,586"
  },
  "Deudores, neto (Nota 4)": {
    "2016": "1,568,387",
    "2017": "72,531"
  },
  "Total activos corrientes": {
    "2016": "7,836,342",
    "2017": "8,587,117"
  },
  "Cuentas por pagar (Nota 6)": {
    "2016": "5,887,271",
    "2017": "3,297,923"
  },
  "Impuestos y administraciones públicas (Nota 14)": {
    "2016": "68,318",
    "2017": "21,432"
  },
  "Beneficio a empleados (Nota 7"[truncated 491 bytes]; line: 1, column: 1])
 at [Source: (byte[])"{
  "Efectivo y equivalentes de efectivo (Nota 3)": {
    "2016": "6,267,955",
    "2017": "8,514,586"
  },
  "Deudores, neto (Nota 4)": {
    "2016": "1,568,387",
    "2017": "72,531"
  },
  "Total activos corrientes": {
    "2016": "7,836,342",
    "2017": "8,587,117"
  },
  "Cuentas por pagar (Nota 6)": {
    "2016": "5,887,271",
    "2017": "3,297,923"
  },
  "Impuestos y administraciones públicas (Nota 14)": {
    "2016": "68,318",
    "2017": "21,432"
  },
  "Beneficio a empleados (Nota 7"[truncated 491 bytes]; line: 45, column: 995]>}
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/awesome_print-1.7.0/lib/awesome_print/formatters/base_formatter.rb:31: warning: constant ::Fixnum is deprecated
{
       "message" => "{\n \"Efectivo y equivalentes de efectivo (Nota 3)\": {\n \"2016\": \"6,267,955\",\n \"2017\": \"8,514,586\"\n },\n \"Deudores, neto (Nota 4)\": {\n \"2016\": \"1,568,387\",\n \"2017\": \"72,531\"\n },\n \"Total activos corrientes\": {\n \"2016\": \"7,836,342\",\n \"2017\": \"8,587,117\"\n },\n \"Cuentas por pagar (Nota 6)\": {\n \"2016\": \"5,887,271\",\n \"2017\": \"3,297,923\"\n },\n \"Impuestos y administraciones públicas (Nota 14)\": {\n \"2016\": \"68,318\",\n \"2017\": \"21,432\"\n },\n \"Beneficio a empleados (Nota 7)\": {\n \"2016\": \"193,971\",\n \"2017\": \"186,035\"\n },\n \"Total pasivos\": {\n \"2016\": \"6,150,060\",\n \"2017\": \"3,505,390\"\n },\n \"Superávit del año\": {\n \"2016\": \"7,822,052\",\n \"2017\": \"4,218,621\"\n },\n \"Total activos netos\": {\n \"2016\": \"7,922,052\",\n \"2017\": \"5,081,727\"\n },\n \"Total pasivos y saldos de activos netos\": {\n \"2016\": \"14,072,112\",\n \"2017\": \"8,587,117\"\n },\n \"Tarjeta Profesional\": {\n \"2016\": \"61851-T Tarjeta\",\n \"2017\": \"Profbsional 110141-T\"\n }",
    "@timestamp" => 2019-04-05T23:13:53.751Z,
          "host" => "ofnanezn-AX370-Gaming-K5",
          "tags" => [
        [0] "multiline",
        [1] "_jsonparsefailure"
    ],
          "type" => "json",
          "path" => "/home/miguel/ocr/demo/finanzas_ocr1.json",
      "@version" => "1"
}
```

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [April 5, 2019, 11:48pm UTC](https://discuss.elastic.co/t/ingest-json-file/175652/6 "2019-04-05T23:48:00Z")

</div>

Both the error message and the "message" field that you are still missing the closing }

```
   "message" => "{\n [...] \"Tarjeta Profesional\": {\n \"2016\": [...] }",

```

What does your input codec look like now?

---

<div class="post-metadata">

### Author: ![Miguel\_Plazas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/miguel_plazas/32/40038_2.png) [@Miguel\_Plazas](https://discuss.elastic.co/u/Miguel_Plazas)
#### Post date: [April 6, 2019, 12:00am UTC](https://discuss.elastic.co/t/ingest-json-file/175652/7 "2019-04-06T00:00:38Z")

</div>

Thanks for your help.

I don't understand why the message is different from the original json file.

I just passing exactly like this:

```
{
  "Efectivo y equivalentes de efectivo (Nota 3)": {
    "2016": "6,267,955",
    "2017": "8,514,586"
  },
  "Deudores, neto (Nota 4)": {
    "2016": "1,568,387",
    "2017": "72,531"
  },
  "Total activos corrientes": {
    "2016": "7,836,342",
    "2017": "8,587,117"
  },
  "Cuentas por pagar (Nota 6)": {
    "2016": "5,887,271",
    "2017": "3,297,923"
  },
  "Impuestos y administraciones públicas (Nota 14)": {
    "2016": "68,318",
    "2017": "21,432"
  },
  "Beneficio a empleados (Nota 7)": {
    "2016": "193,971",
    "2017": "186,035"
  },
  "Total pasivos": {
    "2016": "6,150,060",
    "2017": "3,505,390"
  },
  "Superávit del año": {
    "2016": "7,822,052",
    "2017": "4,218,621"
  },
  "Total activos netos": {
    "2016": "7,922,052",
    "2017": "5,081,727"
  },
  "Total pasivos y saldos de activos netos": {
    "2016": "14,072,112",
    "2017": "8,587,117"
  },
  "Tarjeta Profesional": {
    "2016": "61851-T Tarjeta",
    "2017": "Profbsional 110141-T"
  }
}

```

This is my input codec:

```
input {
  file {

    codec => multiline
    {
        pattern => "^Spalanzani"
        negate => true
        what => previous
        auto_flush_interval => 1
    }

    path => "/home/miguel/ocr/demo/finanzas_ocr1.json"
    start_position => "beginning"
    sincedb_path => "/dev/null"
    type => "json"
  }
}

filter {
  json {
    source => "message"
  }
}

output {
  stdout {}
}
```

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [May 4, 2019, 12:00am UTC](https://discuss.elastic.co/t/ingest-json-file/175652/8 "2019-05-04T00:00:39Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.