# Parse Array of JSON object

**URL:** <https://discuss.elastic.co/t/parse-array-of-json-object/333034>\
**Category:** Logstash\
**Created:** [May 10, 2023, 7:42am UTC](https://discuss.elastic.co/t/parse-array-of-json-object/333034 "2023-05-10T07:42:20Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Nurm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nurm/32/120844_2.png) [@Nurm](https://discuss.elastic.co/u/Nurm)\
**Post date:** [May 10, 2023, 7:42am UTC](https://discuss.elastic.co/t/parse-array-of-json-object/333034/1 "2023-05-10T07:42:20Z")

</div>

```auto
input {
    jdbc {
        jdbc_connection_string => "jdbc:postgresql://localhost:5432/db"
        jdbc_user => "user"
        jdbc_password => "pass"
        jdbc_driver_library => "/usr/share/logstash/lib/postgresql-42.5.4.jar"
        jdbc_driver_class => "org.postgresql.Driver"
        tracking_column => "last_updated"
        tracking_column_type => "timestamp"
        jdbc_default_timezone => "Asia/Almaty"
        use_column_value => true
        clean_run => true
        schedule => "*/5 * * * * *"
        statement => "SELECT t.id, t.last_updated, t.author_id,
                        (
                              SELECT jsonb_agg(jsonb_build_object(
                                'id', up.id,
                                'first_name', up.first_name,
                                'last_name', up.last_name,
                                'photo', up.photo
                              ))
                              FROM public.users AS up
                              WHERE up.id = ANY(t.assignee[1:3])
                          )::TEXT AS assignees_string
                      FROM forms.tasks AS t
                      LEFT JOIN forms.projects AS p ON p.id = t.project_id
                      LEFT JOIN forms.statuses AS s ON s.id = t.status
                      WHERE t.last_updated > :sql_last_value
                      ORDER BY last_updated ASC"
        last_run_metadata_path => "/mnt/.logstash_jdbc_last_run_test"
        type => "tasks"
    }

}

 filter {
    if [type] == "tasks" {
        ruby {
        code => "
          require 'json'
          begin
              assignees_json = JSON.parse(event.get('assignees_string').to_s || {})
              event.set('assignees', assignees_json)
          rescue Exception => e
                event.tag('invalid boundaries json')
          end
        "
        }
    }
}

filter {
  mutate {
    remove_field => ["@version", "@timestamp",, "assignees_string"]
  }
}

output {
        if [type] == 'tasks' {
                elasticsearch {
                    hosts => "127.0.0.1"
                    index => "test"
                    document_id => "%{id}"
                }
                stdout {codec=>"rubydebug"}
        }
}

```

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/0/f/0f560ca83375ad13d719d2dc728fa59dbcdf6e03.png)  
Can U help me to parse this plz?

---

<div class="post-metadata">

**Author:** ![Wolfram\_Haussig](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wolfram_haussig/32/70528_2.png) [@Wolfram\_Haussig](https://discuss.elastic.co/u/Wolfram_Haussig)\
**Post date:** [May 12, 2023, 4:53am UTC](https://discuss.elastic.co/t/parse-array-of-json-object/333034/2 "2023-05-12T04:53:28Z")

</div>

Hi,

Why are you using a ruby filter to parse JSON? Logstash has an extra [JSON filter](https://www.elastic.co/guide/en/logstash/current/plugins-filters-json.html#plugins-filters-json-source) for that.

```auto
 filter {
    if [type] == "tasks" {
        json {
          source => "assignees_string"
          target => "assignees"
        }
    }
}

```

Have you checked what the result of the SQL is and can you give us an example document (anonymize data if required before posting)?

Best regards  
Wolfram

---

<div class="post-metadata">

**Author:** ![Nurm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nurm/32/120844_2.png) [@Nurm](https://discuss.elastic.co/u/Nurm)\
**Post date:** [May 12, 2023, 6:56am UTC](https://discuss.elastic.co/t/parse-array-of-json-object/333034/3 "2023-05-12T06:56:26Z")

</div>

Hi, Thank You for your response, but...

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/1/3/13eb056fc223da8dbb4a7012d2c2a4b41715b351.png)

This is error happens in the console when I put your code

Code in logstash

```auto
input {
    jdbc {
        jdbc_connection_string => "jdbc:postgresql://localhost:5432/db"
        jdbc_user => "user"
        jdbc_password => "pass"
        jdbc_driver_library => "/usr/share/logstash/lib/postgresql-42.5.4.jar"
        jdbc_driver_class => "org.postgresql.Driver"
        tracking_column => "last_updated"
        tracking_column_type => "timestamp"
        jdbc_default_timezone => "UTC"
        use_column_value => true
        clean_run => true
        schedule => "*/5 * * * * *"
        statement => "SELECT t.id, t.last_updated,
				  (
                              SELECT jsonb_agg(jsonb_build_object(
                                'id', up.id,
                                'first_name', up.first_name,
                                'last_name', up.last_name,
                                'photo', up.photo
                              ))
                              FROM public.users AS up
                              WHERE up.id = ANY(t.assignee[1:3])
                          ) AS assignees_string
                      FROM forms.tasks AS t
		          LEFT JOIN forms.projects AS p ON p.id = t.project_id
                      LEFT JOIN forms.statuses AS s ON s.id = t.status
                      WHERE t.last_updated > :sql_last_value
                      ORDER BY t.last_updated ASC"
        last_run_metadata_path => "/mnt/.logstash_jdbc_last_run_test"
        tags => ["tasks"]
    }

}

 filter {
   if "tasks" in [tags] {
	json {
		source => "assignees_string"
		target => "assignees"
	}
   }
}

output {
	if 'tasks' in [tags] {

        	elasticsearch {
        	    hosts => "127.0.0.1"
        	    index => "test"
        	    document_id => "%{id}"
        	}

		stdout {codec=>"rubydebug"}

	}

    
}

```

Best regards  
Nurm

---

<div class="post-metadata">

**Author:** ![Wolfram\_Haussig](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wolfram_haussig/32/70528_2.png) [@Wolfram\_Haussig](https://discuss.elastic.co/u/Wolfram_Haussig)\
**Post date:** [May 12, 2023, 7:07am UTC](https://discuss.elastic.co/t/parse-array-of-json-object/333034/4 "2023-05-12T07:07:21Z")

</div>

Then I guess that this is the cause for your error: Logstash does not get a JSON string as expected, it gets a PGObject and Logstash cannot parse it.  
In your initiali pipeline you had `::TEXT` which is missing in the latest pipeline - I guess this should convert the object to the JSON string? Can you try it again with that?

---

<div class="post-metadata">

**Author:** ![Nurm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nurm/32/120844_2.png) [@Nurm](https://discuss.elastic.co/u/Nurm)\
**Post date:** [May 12, 2023, 7:10am UTC](https://discuss.elastic.co/t/parse-array-of-json-object/333034/5 "2023-05-12T07:10:55Z")

</div>

Yeah, I forgot this '::TEXT', Thank You very much!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 9, 2023, 7:11am UTC](https://discuss.elastic.co/t/parse-array-of-json-object/333034/6 "2023-06-09T07:11:25Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
