# Parsing a Raw JSON file "Invalid Setting for json filter"

**URL:** <https://discuss.elastic.co/t/parsing-a-raw-json-file-invalid-setting-for-json-filter/280735>\
**Category:** Logstash\
**Created:** [August 8, 2021, 2:59pm UTC](https://discuss.elastic.co/t/parsing-a-raw-json-file-invalid-setting-for-json-filter/280735 "2021-08-08T14:59:04Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Wilks](https://avatars.discourse-cdn.com/v4/letter/w/f475e1/32.png) [@Wilks](https://discuss.elastic.co/u/Wilks)\
**Post date:** [August 8, 2021, 2:59pm UTC](https://discuss.elastic.co/t/parsing-a-raw-json-file-invalid-setting-for-json-filter/280735/1 "2021-08-08T14:59:04Z")

</div>

Hi,  
I am trying to parse a simple raw JSON file but I am getting an error about invalid setting for JSON filter. I am wondering if its the way my filter is setup and the fact that its nested json?

[ERROR] 2021-08-08 10:52:34.886 [Converge PipelineAction::Create] json - Invalid setting for json filter plugin:

filter {  
json {  
# This setting must be a string  
# Expected string, got ["TEST", "eventName", "date", "time", "ID"]  
source =\> ["TEST", "eventName", "date", "time", "ID"]  
...  
}  
}

The raw log looks like this:

```auto
{
  "TEST" : {
    "dataRec" : {
      "eventName" : "IN",
      "date" : "210721",
      "time" : "06:27:56",
      "ID" : "TEST ",
      "group" : "TEST0001",
      "job" : "QRTSDHL4",
      "terminal" : "ABC21QJ2",
      "Name" : "TESTPREPROD",
      "returnCode" : "00",
      "detailsReasonCode" : "TEST ",
      "userName" : "1234567,LAST,FIRST (ABCDEF) "
    }
  }
}

```

Filter looks like this

```auto
input {
  file{
  # type => "json"
  # codec => json { }
   path => "/home/test.txt"
   mode => "tail"
   start_position => "beginning"
  }
}

filter {
  json {
  source => "TEST"
  source => "eventName"
  source => "date"
  source => "time"
  source => "ID"
  source => "group"
     }
}
 
output {
stdout {} 
file { path => "/home/output.log" }
}

```

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [August 8, 2021, 3:05pm UTC](https://discuss.elastic.co/t/parsing-a-raw-json-file-invalid-setting-for-json-filter/280735/2 "2021-08-08T15:05:01Z")

</div>

The `json` filter only supports **one** field at a time, if your original message is a `json` document, then you just need to use:

```auto
json {
    source => "message"
} 

```

---

<div class="post-metadata">

**Author:** ![Wilks](https://avatars.discourse-cdn.com/v4/letter/w/f475e1/32.png) [@Wilks](https://discuss.elastic.co/u/Wilks)\
**Post date:** [August 8, 2021, 3:45pm UTC](https://discuss.elastic.co/t/parsing-a-raw-json-file-invalid-setting-for-json-filter/280735/3 "2021-08-08T15:45:31Z")

</div>

Thanks for the response. When I change it to that I get the following error:  
so filter:

```auto
filter {
  json {
    source => "message"
    
}     
}

```

Error

```auto
[WARN] 2021-08-08 11:39:16.919 [[main]>worker0] json - Error parsing json {:source=>"message", :raw=>"{", :exception=>#<LogStash::Json::ParserError: Unexpected end-of-input: expected close marker for Object (start marker at [Source: (byte[])"{"; line: 1, column: 1])
 at [Source: (byte[])"{"; line: 1, column: 3]>}
[WARN] 2021-08-08 11:39:16.920 [[main]>worker1] json - Error parsing json {:source=>"message", :raw=>" \"TEST\" : {", :exception=>#<LogStash::Json::ParserError: Unexpected character (':' (code 58)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
 at [Source: (byte[])" "TEST" : {"; line: 1, column: 11]>}
[WARN] 2021-08-08 11:39:16.923 [[main]>worker1] json - Error parsing json {:source=>"message", :raw=>" \"dataRec\" : {", :exception=>#<LogStash::Json::ParserError: Unexpected character (':' (code 58)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
 at [Source: (byte[])" "dataRec" : {"; line: 1, column: 16]>}
[WARN] 2021-08-08 11:39:16.924 [[main]>worker1] json - Error parsing json {:source=>"message", :raw=>" \"eventName\" : \"IN\",", :exception=>#<LogStash::Json::ParserError: Unexpected character (':' (code 58)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
 at [Source: (byte[])" "eventName" : "IN","; line: 1, column: 20]>}
[WARN] 2021-08-08 11:39:16.925 [[main]>worker1] json - Error parsing json {:source=>"message", :raw=>" \"date\" : \"210721\",", :exception=>#<LogStash::Json::ParserError: Unexpected character (':' (code 58)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
 at [Source: (byte[])" "date" : "210721","; line: 1, column: 15]>}
[WARN] 2021-08-08 11:39:16.926 [[main]>worker1] json - Error parsing json {:source=>"message", :raw=>" \"time\" : \"06:27:56\",", :exception=>#<LogStash::Json::ParserError: Unexpected character (':' (code 58)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
 at [Source: (byte[])" "time" : "06:27:56","; line: 1, column: 15]>}
[WARN] 2021-08-08 11:39:16.927 [[main]>worker1] json - Error parsing json {:source=>"message", :raw=>" \"accountID\" : \"TEST \",", :exception=>#<LogStash::Json::ParserError: Unexpected character (':' (code 58)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
 at [Source: (byte[])" "accountID" : "TEST ","; line: 1, column: 20]>}
[WARN] 2021-08-08 11:39:16.928 [[main]>worker1] json - Error parsing json {:source=>"message", :raw=>" \"department\" : \"TEST0001\",", :exception=>#<LogStash::Json::ParserError: Unexpected character (':' (code 58)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
 at [Source: (byte[])" "department" : "TEST0001","; line: 1, column: 21]>}
[WARN] 2021-08-08 11:39:16.929 [[main]>worker1] json - Error parsing json {:source=>"message", :raw=>" \"jobName\" : \"QRTSDHL4\",", :exception=>#<LogStash::Json::ParserError: Unexpected character (':' (code 58)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
 at [Source: (byte[])" "jobName" : "QRTSDHL4","; line: 1, column: 18]>}
[WARN] 2021-08-08 11:39:16.930 [[main]>worker1] json - Error parsing json {:source=>"message", :raw=>" \"terminalID\" : \"ABC21QJ2\",", :exception=>#<LogStash::Json::ParserError: Unexpected character (':' (code 58)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
 at [Source: (byte[])" "terminalID" : "ABC21QJ2","; line: 1, column: 21]>}
[WARN] 2021-08-08 11:39:16.931 [[main]>worker1] json - Error parsing json {:source=>"message", :raw=>" \"facilityName\" : \"TESTPREPROD\",", :exception=>#<LogStash::Json::ParserError: Unexpected character (':' (code 58)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
 at [Source: (byte[])" "facilityName" : "TESTPREPROD","; line: 1, column: 23]>}
[WARN] 2021-08-08 11:39:16.932 [[main]>worker1] json - Error parsing json {:source=>"message", :raw=>" \"returnCode\" : \"00\",", :exception=>#<LogStash::Json::ParserError: Unexpected character (':' (code 58)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
 at [Source: (byte[])" "returnCode" : "00","; line: 1, column: 21]>}
[WARN] 2021-08-08 11:39:16.933 [[main]>worker1] json - Error parsing json {:source=>"message", :raw=>" \"detailsReasonCode\" : \"TEST \",", :exception=>#<LogStash::Json::ParserError: Unexpected character (':' (code 58)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
 at [Source: (byte[])" "detailsReasonCode" : "TEST ","; line: 1, column: 28]>}
[WARN] 2021-08-08 11:39:16.934 [[main]>worker1] json - Error parsing json {:source=>"message", :raw=>" \"userName\" : \"1234567,LAST,FIRST (ABCDEF) \"", :exception=>#<LogStash::Json::ParserError: Unexpected character (':' (code 58)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
 at [Source: (byte[])" "userName" : "1234567,LAST,FIRST (ABCDEF) ""; line: 1, column: 19]>}
[WARN] 2021-08-08 11:39:16.937 [[main]>worker1] json - Error parsing json {:source=>"message", :raw=>" }", :exception=>#<LogStash::Json::ParserError: Unexpected close marker '}': expected ']' (for root starting at [Source: (byte[])" }"; line: 1, column: 0])
 at [Source: (byte[])" }"; line: 1, column: 6]>}
[WARN] 2021-08-08 11:39:16.954 [[main]>worker1] json - Error parsing json {:source=>"message", :raw=>" }", :exception=>#<LogStash::Json::ParserError: Unexpected close marker '}': expected ']' (for root starting at [Source: (byte[])" }"; line: 1, column: 0])
 at [Source: (byte[])" }"; line: 1, column: 4]>}
[WARN] 2021-08-08 11:39:16.957 [[main]>worker1] json - Error parsing json {:source=>"message", :raw=>"}", :exception=>#<LogStash::Json::ParserError: Unexpected close marker '}': expected ']' (for root starting at [Source: (byte[])"}"; line: 1, column: 0])
 at [Source: (byte[])"}"; line: 1, column: 2]>}
[WARN] 2021-08-08 11:39:16.959 [[main]>worker1] json - Parsed JSON object/hash requires a target configuration option {:source=>"message", :raw=>""}
{
    "@timestamp" => 2021-08-08T15:39:16.596Z,
          "tags" => [
        [0] "_jsonparsefailure"
    ],
      "@version" => "1",
          "host" => "testhost.ca",
          "path" => "/home/example/test2.log",
       "message" => "{"
}
[INFO] 2021-08-08 11:39:17.235 [[main]>worker0] file - Opening file {:path=>"/home/example/output.log"}
{
    "@timestamp" => 2021-08-08T15:39:16.622Z,
          "tags" => [
        [0] "_jsonparsefailure"
    ],
      "@version" => "1",
          "host" => "testhost.ca",
          "path" => "/home/example/test2.log",
       "message" => " \"TEST\" : {"
}
{
    "@timestamp" => 2021-08-08T15:39:16.623Z,
          "tags" => [
        [0] "_jsonparsefailure"
    ],
      "@version" => "1",
          "host" => "testhost.ca",
          "path" => "/home/example/test2.log",
       "message" => " \"dataRec\" : {"
}
{
    "@timestamp" => 2021-08-08T15:39:16.623Z,
          "tags" => [
        [0] "_jsonparsefailure"
    ],
      "@version" => "1",
          "host" => "testhost.ca",
          "path" => "/home/example/test2.log",
       "message" => " \"eventName\" : \"IN\","
}
{
    "@timestamp" => 2021-08-08T15:39:16.624Z,
          "tags" => [
        [0] "_jsonparsefailure"
    ],
      "@version" => "1",
          "host" => "testhost.ca",
          "path" => "/home/example/test2.log",
       "message" => " \"date\" : \"210721\","
}
{
    "@timestamp" => 2021-08-08T15:39:16.624Z,
          "tags" => [
        [0] "_jsonparsefailure"
    ],
      "@version" => "1",
          "host" => "testhost.ca",
          "path" => "/home/example/test2.log",
       "message" => " \"time\" : \"06:27:56\","
}
{
    "@timestamp" => 2021-08-08T15:39:16.624Z,
          "tags" => [
        [0] "_jsonparsefailure"
    ],
      "@version" => "1",
          "host" => "testhost.ca",
          "path" => "/home/example/test2.log",
       "message" => " \"accountID\" : \"TEST \","
}
{
    "@timestamp" => 2021-08-08T15:39:16.625Z,
          "tags" => [
        [0] "_jsonparsefailure"
    ],
      "@version" => "1",
          "host" => "testhost.ca",
          "path" => "/home/example/test2.log",
       "message" => " \"department\" : \"TEST0001\","
}
{
    "@timestamp" => 2021-08-08T15:39:16.625Z,
          "tags" => [
        [0] "_jsonparsefailure"
    ],
      "@version" => "1",
          "host" => "testhost.ca",
          "path" => "/home/example/test2.log",
       "message" => " \"jobName\" : \"QRTSDHL4\","
}
{
    "@timestamp" => 2021-08-08T15:39:16.625Z,
          "tags" => [
        [0] "_jsonparsefailure"
    ],
      "@version" => "1",
          "host" => "testhost.ca",
          "path" => "/home/example/test2.log",
       "message" => " \"terminalID\" : \"ABC21QJ2\","
}
{
    "@timestamp" => 2021-08-08T15:39:16.626Z,
          "tags" => [
        [0] "_jsonparsefailure"
    ],
      "@version" => "1",
          "host" => "testhost.ca",
          "path" => "/home/example/test2.log",
       "message" => " \"facilityName\" : \"TESTPREPROD\","
}

```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 8, 2021, 3:52pm UTC](https://discuss.elastic.co/t/parsing-a-raw-json-file-invalid-setting-for-json-filter/280735/4 "2021-08-08T15:52:18Z")

</div>

Your JSON is pretty-printed across multiple lines. You will need to use a multiline codec to combine all the lines of a single JSON object into one [message] field. See [here](https://discuss.elastic.co/t/parsing-array-of-json-objects-with-logstash-and-injesting-to-elastic/203197/2) and [here](https://discuss.elastic.co/t/unable-to-parse-json-array-object-in-logstash/89247/2).

---

<div class="post-metadata">

**Author:** ![Wilks](https://avatars.discourse-cdn.com/v4/letter/w/f475e1/32.png) [@Wilks](https://discuss.elastic.co/u/Wilks)\
**Post date:** [August 8, 2021, 7:38pm UTC](https://discuss.elastic.co/t/parsing-a-raw-json-file-invalid-setting-for-json-filter/280735/5 "2021-08-08T19:38:58Z")

</div>

thanks. that worked!!!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 5, 2021, 7:39pm UTC](https://discuss.elastic.co/t/parsing-a-raw-json-file-invalid-setting-for-json-filter/280735/6 "2021-09-05T19:39:12Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.