# How to Insert array of Json to Elasticsearch from Logstash

**URL:** https://discuss.elastic.co/t/how-to-insert-array-of-json-to-elasticsearch-from-logstash/132635
**Category:** Logstash
**Created:** [May 21, 2018, 9:20am UTC](https://discuss.elastic.co/t/how-to-insert-array-of-json-to-elasticsearch-from-logstash/132635 "2018-05-21T09:20:19Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![kartheek91](https://avatars.discourse-cdn.com/v4/letter/k/cc9497/32.png) [@kartheek91](https://discuss.elastic.co/u/kartheek91)
#### Post date: [May 21, 2018, 9:20am UTC](https://discuss.elastic.co/t/how-to-insert-array-of-json-to-elasticsearch-from-logstash/132635/1 "2018-05-21T09:20:19Z")

</div>

```
Hi all, 

I'm trying to import data from sql server to elasticsearch using logstash and please find attached conf file.I'm not able to add an array toPreformatted text the existing json.

```

This is the configuartion file:

```
input {
  jdbc {
    jdbc_driver_library => "c:\drivers\sqljdbc4" 
    jdbc_driver_class => "com.microsoft.sqlserver.jdbc.SQLServerDriver"
	 jdbc_connection_string => "jdbc:sqlserver://localhost\SAILS-DM29:1433;databasename=I9"

    jdbc_user => "sa"
    jdbc_password => "sails123"
    statement => "SELECT i9.Id AS i9FormId,
                  emp.AccountId, emp.FirstName, emp.LastName, emp.MiddleName, emp.MaidenName, emp.Alias,
                  emp.AddressId, emp.SSNEnc, emp.SSNHash, emp.SSNLast4, emp.Email, emp.Phone, emp.CreatedOn,
                  emp.ModifiedOn, emp.UserId, emp.EGuid, emp.LocationId, emp.OriginalHireDate, emp.MostRecentHireDate,
                  emp.TerminationDate, emp.DOB, emp.CitizenshipTypeId, emp.StoreId, emp.WOTCLocationId, emp.PayrollLocationId,
                  emp.UHRR, emp.ClientEmployeeId, emp.IsInvalidEmail, sd.DocListId, sd.I9FormId, sd.Id As supoortId
                  FROM Employee emp
                  INNER JOIN I9Form i9 ON emp.Id = i9.EmployeeId
                  LEFT JOIN SupportDoc sd ON sd.I9FormId = i9.Id
                  WHERE emp.Id = 1 "
		use_column_value => false
    tracking_column => "ModifiedOn"
		tracking_column_type => "timestamp"
    clean_run => false
  }
}
# The filter part of this file is commented out to indicate that it is
# optional.
filter {
    json_encode {
    add_tag => ["supportDoc"]
  }

  aggregate {
    task_id => "%{i9FormId}"
    code => "
     map['employee'] = {
        'id' => event.get('Id'), 
        'accountId' => event.get('emp.AccountId'),
        'firstName' => event.get('emp.FirstName'),
        'lastName' => event.get('emp.LastName'),
        'middleName' => event.get('emp.MiddleName'),
        'maidenName' => event.get('emp.MaidenName'),
        'alias' => event.get('emp.Alias'),
        'addressId' => event.get('emp.AddressId'),
        'sSNEnc' => event.get('emp.SSNEnc'),
        'sSNHash' => event.get('emp.SSNHash'),
        'sSNLast4' => event.get('emp.SSNLast4'),
        'email' => event.get('emp.Email'),
        'phone' => event.get('emp.Phone'),
        'createdOn' => event.get('emp.CreatedOn'),
        'modifiedOn' => event.get('emp.ModifiedOn'),
        'userId' => event.get('emp.UserId'),
        'eGuid' => event.get('emp.EGuid'),
        'locationId' => event.get('emp.LocationId'),
        'originalHireDate' => event.get('emp.OriginalHireDate'),
        'mostRecentHireDate' => event.get('emp.MostRecentHireDate'),
        'terminationDate' => event.get('emp.TerminationDate'),
        'dob' => event.get('emp.DOB'),
        'citizenShipTypeId' => event.get('emp.CitizenshipTypeId'),
        'storeId' => event.get('emp.StoreId'),
        'wotcLocationId' => event.get('emp.WOTCLocationId'),
        'payrollLocationId' => event.get('emp.PayrollLocationId'),
        'uhrr' => event.get('emp.UHRR'),
        'clientEmployeeId' => event.get('emp.ClientEmployeeId'),
        'isInvaliedEmail' => event.get('emp.IsInvalidEmail')
      }
       map['supportdocs'] ||= []
       map['supportdocs'] << {'i9FormId' => event.get('sd.I9FormId'),
                              'doclistid' => event.get('sd.DocListId')}
      event.cancel()
    "
    push_previous_map_as_event => true
    timeout => 10
  }
mutate {
#if needed remove/ delete fields 
#remove_field => ["ssnenc","ssnhash"] # if needed 
remove_field => ["support_docs_List"]

}
date {
     match => ["sql_last_value", "YYYY-MM-dd HH:mm:ss.SSS"] #2018-01-29 22:16:59.537
     timezone => "Etc/UTC"
    }
}
output {
  stdout { codec => rubydebug }
  elasticsearch{
	index => "i91"
  codec => "json"
	#action =>"update"	#if want to update existing index data based on ID column
	#ssl=>true # if node is on SSL
	hosts => ["localhost:9200"]
  manage_template => false
  document_type => "i9details"
	document_id => "%{sd.docListId}"
  doc_as_upsert => true
  action => "update"
	}
}
```

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [May 21, 2018, 2:02pm UTC](https://discuss.elastic.co/t/how-to-insert-array-of-json-to-elasticsearch-from-logstash/132635/2 "2018-05-21T14:02:57Z")

</div>

It is not clear from your posting what problem you want help with.

---

<div class="post-metadata">

### Author: ![kartheek91](https://avatars.discourse-cdn.com/v4/letter/k/cc9497/32.png) [@kartheek91](https://discuss.elastic.co/u/kartheek91)
#### Post date: [May 21, 2018, 4:24pm UTC](https://discuss.elastic.co/t/how-to-insert-array-of-json-to-elasticsearch-from-logstash/132635/3 "2018-05-21T16:24:28Z")

</div>

> [@kartheek91](#):
>
> supportdocs

supportdocs is an array of jsons and I'm not able to add to existing json

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [June 18, 2018, 4:24pm UTC](https://discuss.elastic.co/t/how-to-insert-array-of-json-to-elasticsearch-from-logstash/132635/4 "2018-06-18T16:24:33Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
