How can I use ECS with custom JSON input?

[janoliver (Jan Oliver Oelerich)]

Hi there!

I have a server that collects metrics (such as CPU, memory, etc.) from other servers and I'd like to send that already collected data via logstash to elastic. At the moment, I have a TCP input with JSON encoding set up, to which I send a JSON array with each element being the metrics of one of the servers:

[
    {"message": "...", "host": {"name": "my_host1"}, ...},
    {"message": "...", "host": {"name": "my_host2"}, ...},
    ...
]

How do I have to shape the JSON objects, so that the hosts and their metrics appear in Kibana's Infrastructure App (i.e., similar to if they were collected by metricbeat on the individual hosts). Running additional services such as metricbeat on all the hosts is currently not an option for me.

I found the ECS schema, but system metrics are not defined there. I tried using something similar to metricbeat, but the hosts don't appear in the infrastructure app's overview.

{
	"message": "metric",
	"system": {
		"cpu": {
			"user": {
				"pct": ...
			},
			"cores": 8,
		},
		"memory": {
			"actual": {
				"used": {
					"pct": ...
				}
			}
		},
		"load": {
			"5": ...
		}
	},
	"host": {
		"name": "..."
	}
}

How do I best shape my data?

Thank you!