Filter to verify json fields type

[Amir_Gilad]

Hi,

We're currently using internal pipeline to store our raw data and want to move to Logstash.
The inputs are jsons and the json filter actually works quite well in parsing them.
The problem is that currently we have a local check that verifies each field according to a pre-defined hard-coded list.
I couldn't find an easy way to do it using the config file (besides using a huge nested if chain - we have ~100 fields per json).

The jsons don't necessarily contain the same fields exactly, but the "field2type list" contains all fields types.

Will be glad to get any smart ideas to deal with issue.

Thanks!

[magnusbaeck (Magnus Bäck)]

The problem is that currently we have a local check that verifies each field according to a pre-defined hard-coded list.

What, exactly, does this mean? Are you verifying that each field has the expected type?

I suggest you formalize the JSON messages using JSON Schema and write a custom plugin to validate events against your schemas.

[Amir_Gilad]

Thanks for the quick reply!

What, exactly, does this mean? Are you verifying that each field has the expected type?
Yep - that's exactly what we do.

Is there any exiting plug that does something like that?

[magnusbaeck (Magnus Bäck)]

No, I'm pretty sure there isn't any plugin for this already.

[Amir_Gilad]

OK - Thanks a lot for the help!