# Logstash code=\>"json" not parse complex json log as expected (please close this topic) **URL:** \ **Category:** Logstash\ **Created:** [January 20, 2019, 1:07pm UTC](https://discuss.elastic.co/t/logstash-code-json-not-parse-complex-json-log-as-expected-please-close-this-topic/164940 "2019-01-20T13:07:22Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![cheriemilk](https://avatars.discourse-cdn.com/v4/letter/c/c37758/32.png) [@cheriemilk](https://discuss.elastic.co/u/cheriemilk)\ **Post date:** [January 20, 2019, 1:07pm UTC](https://discuss.elastic.co/t/logstash-code-json-not-parse-complex-json-log-as-expected-please-close-this-topic/164940/1 "2019-01-20T13:07:22Z")
Hi team, I have below logstash.conf and json event. Most of the fields are parsed correctly, but for complex array types, it looks like that not parsed as expecte. For example, profile criteria and rating criteria are json object arrays and the actual parsed result is below. ![image](https://us1.discourse-cdn.com/elastic/original/3X/8/3/83bbda299fb5123fe928f88a332a6cd9500cbf4a.png) **But the expected parsed result is below, and the first level fields (searchOption, keywordCriteria, ratingcriteria or profilecriteria etc.) in body{} is dynamic and could be changed** "body.criteria.type": "text" "body.criteria.id": "country" "body.criteria.value": "USA" (here should be parsed as array) ... "body.ratingCriteria.Criterias": "2018 - 12 - 04", "2018 - 12 - 31" (here should be parsed as array, not string) \*\*1. logstash configuration \*\* input { file{ path =\> "C:/elkstack/elasticsearch-6.5.1/logs/app.log" start\_position =\> "beginning" sincedb\_path =\> "null" **codec =\> "json"** } } **2. json event f** ``` "body": { "Action": "TalentSearch", "searchOption": [{ "id": "allCriteriaOptional", "value": ["false"] }], "keywordCriteria": { "searchValue": "", "bgTypes": ["languages", "specialAssign", "insideWorkExperience", "memberships", "benefitselection", "outsideWorkExperience", "sysScoreCardDevelopmentObjectivesPortlet", "mobility", "documents", "compensation", "tsv2##mdftalentpoolcriteria", "promotability", "leadExperience", "awards", "googledocs", "varPayEmpHistData", "courses", "funcExperience", "education", "fsaelection", "certificates", "community", "preferredNextMove", "tsv2##jobinfo"] }, "userStatusCriteria": { "source": [{ "key": "includeInactive", "value": "true" }] }, "profileCriteria": [{ "id": "country", "type": "text", "value": ["USA"] }], "facetedCriteria": [{ "id": "department", "elements": [{ "value": "Marketing" }] }, { "id": "division", "elements": [] }, { "id": "location", "elements": [] }], "ratingCriteria": [{ "id": "sysOverallPotential", "type": "7", "name": "Potential - 3x3 Rating", "item": "null", "scaleId": "Potential", "scaleMin": "1", "scaleMax": "3", "criterias": [{ "id": "tsv2RatingValidFrom", "type": "date", "value": ["2018 - 12 - 04", "2018 - 12 - 31"] }, { "id": "tsv2RatingValidTo", "type": "date", "value": ["2018 - 12 - 31", "2018 - 12 - 31"] }, { "id": "tsv2RatingFromValue", "type": "prepopulate", "value": ["1.0"] }, { "id": "tsv2RatingEndValue", "type": "prepopulate", "value": ["2.0"] }] }] } ``` } ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [January 20, 2019, 5:49pm UTC](https://discuss.elastic.co/t/logstash-code-json-not-parse-complex-json-log-as-expected-please-close-this-topic/164940/2 "2019-01-20T17:49:47Z")
You JSON is not valid JSON. However, if I add {} around it and puts quotes around the dates in the value arrays inside ratingCriteria.criterias I get ``` "ratingCriteria" => [ [0] { "scaleMax" => "3", "scaleId" => "Potential", "scaleMin" => "1", "criterias" => [ [0] { "type" => "date", "id" => "tsv2RatingValidFrom", "value" => [ [0] "2018 - 12 - 04", [1] "2018 - 12 - 31" ] ``` As far as I can see, everything that should be an array is an array. ---
**Author:** ![cheriemilk](https://avatars.discourse-cdn.com/v4/letter/c/c37758/32.png) [@cheriemilk](https://discuss.elastic.co/u/cheriemilk)\ **Post date:** [January 21, 2019, 12:56am UTC](https://discuss.elastic.co/t/logstash-code-json-not-parse-complex-json-log-as-expected-please-close-this-topic/164940/3 "2019-01-21T00:56:04Z")
Hi Badger, Thank you ~ 1. There's quotes added actually for dates( Sorry , I pasted the wrong json version). What do you mean by add {} around it? like this?? "value": [{"2018 - 12 - 04", "2018 - 12 - 31"}] 2. I already got the result yesterday ,same as what you get. But the extract result looks like not flat enough, and still a json structure for profile criteria, rating criteria and faceted criteria. My expected parse result is "body.ratingCriteria.scaleMax": "3" "body.ratingCriteria.scaleId": "Potential" "body.ratingCriteria.scaleMin": "1" "body.ratingCriteria.criterias.type": "date" "body.ratingCriteria.criterias.id": "tsv2RatingValidFrom" "body.ratingCriteria.criterias.value": "2018 - 12 - 04", "2018 - 12 - 31" I googled yesterday that ruby filter might be a solution for such complex json and dynamic field scenario , but not familiar with ruby language. Do you have idea about it? or anyother alternatives I can use to meet my parse requirement? Thanks, Cherie Thanks, Cherie ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [January 21, 2019, 2:10am UTC](https://discuss.elastic.co/t/logstash-code-json-not-parse-complex-json-log-as-expected-please-close-this-topic/164940/4 "2019-01-21T02:10:12Z")
> [@cheriemilk](#): > > "body": { } When I say you need {} around that to be valid JSON I mean it needs to be ``` { "body": {...} } ``` I do not understand what you do not like about the way the JSON is parsed. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [February 20, 2019, 1:57am UTC](https://discuss.elastic.co/t/logstash-code-json-not-parse-complex-json-log-as-expected-please-close-this-topic/164940/7 "2019-02-20T01:57:06Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.