Convert from dynamodb json to regular json on ingest

Hey guys;

I'm ingesting DynamoDB stream data into Elastic using a function beat.

Thing is the data ingested is all DynamoDB JSON rather than regular JSON.

The AWS SDK has a method to do the conversion (see Module: AWS.DynamoDB.Converter — AWS SDK for JavaScript) but how might I accomplish this as part of a function beat.

var unmarshalled = AWS.DynamoDB.Converter.unmarshall({
  string: {S: 'foo'},
  list: {L: [{S: 'fizz'}, {S: 'buzz'}, {S: 'pop'}]},
  map: {
    M: {
      nestedMap: {
        M: {
          key: {S: 'value'}
  number: {N: '123'},
  nullValue: {NULL: true},
  boolValue: {BOOL: true}

Any pointers joyously appreciated.


Function Beat does not have this level of formatting control. Using Logstash would be the way to go here or write to the index directly in the desired format.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.