Filter documents if a field exists and report on the value of that field

[Matt_Simcox]

Hi,

I have documents that may or may not contain certain fields in ES. I need to create a visualisation in kibana that restricts to only documents that:

1. Contain a particular field.
2. Report on the value of that field over time.

An example of the document is below:

The fields or relevance are:

"PURCHASE": 1,
"LOADED": 3,
"BILLS": 2,

The values of these fields over time change per document, but the fields might not always be present.

I've found the exists:PURCHASE style syntax so I can restrict only to PURCHASE, however it returns how many documents the PURCHASE tag appears in not the value.

If possible I'd also like to combine these into a line graph to compare the changing values of these fields over time.

I'm struggling to find an example of how to do this.

{
"_index": "index-2017.02.16",
"_type": "restart_log",
"_id": "AVpY1KMb8AbTx8cdxH9I",
"_score": null,
"_source": {
"offset": 812,
"input_type": "log",
"source": "C:\logs\restarts.log",
"message": "2017-02-17_09:18:20.587,BILLS=2,PURCHASE=1,LOADED=3",
"type": "switch_restart_log",
"tags": [
"beats_input_codec_plain_applied"
],
"PURCHASE": 1,
"LOADED": 3,
"BILLS": 2,
"@timestamp": "2017-02-16T22:18:20.587Z",
"@version": "1",
"beat": {
"hostname": "sydwpayapp04",
"name": "sydwpayapp04",
"version": "5.0.1"
},
"host": "sydwpayapp04",
"poolStats": "BILLS=2,PURCHASE=1,LOADED=3",
"time": "2017-02-17_09:18:20.587"
},
"fields": {
"@timestamp": [
1487283500587
]
},
"sort": [
1487283500587
]
}

Regards,

Matt

[Matt_Simcox]

In typical me fashion I work it out 10 minutes after posting. Just in case it helps anyone out there:

I filtered with:

exists:BILLPAY OR exists:LOADMONEY OR exists:PURCHASE

which restricted down to only the documents I needed and then added the three metrics to the Y axis on a line graph visualisation.

[weltenwort (Felix Stürmer)]

Thanks for sharing your solution, @Matt_Simcox! This might be useful to other users that have the same question.