Convert from dynamodb json to regular json on ingest

Elastic Stack Beats
1

Hey guys;

I'm ingesting DynamoDB stream data into Elastic using a function beat.

Thing is the data ingested is all DynamoDB JSON rather than regular JSON.

The AWS SDK has a method to do the conversion (see Module: AWS.DynamoDB.Converter — AWS SDK for JavaScript) but how might I accomplish this as part of a function beat.

var unmarshalled = AWS.DynamoDB.Converter.unmarshall({
  string: {S: 'foo'},
  list: {L: [{S: 'fizz'}, {S: 'buzz'}, {S: 'pop'}]},
  map: {
    M: {
      nestedMap: {
        M: {
          key: {S: 'value'}
        }
      }
    }
  },
  number: {N: '123'},
  nullValue: {NULL: true},
  boolValue: {BOOL: true}
});

Any pointers joyously appreciated.

Cheers

2

Function Beat does not have this level of formatting control. Using Logstash would be the way to go here or write to the index directly in the desired format.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.