Count within field

[treknado]

Hello,

i have documents with multiline fields in my.field and need to count the number of received filenames.

example:
document 1:

my.field: { "filename": "test.json", "type" : "File"}

document 2:
my.field: { "filename": "test2.json", "type" : "File"}, { "filename": "test3.json", "type" : "File"}

with my kibana skills, i can only count the number of documents --> 2
But i want the number filenames within my document:

document1 (test.json) + document2 (test2.json, test3.json) --> 3

How can i handle it ?

I already read articles about scripted fields. How can i use them here ?

best regards
treknado

[Fabio-sama (Fabio)]

Hi there,

best thing IMO would be to add an additional field to the document with the number of filenames for that document. You'd end up with documents like:

{
  "my_field":[
    {
      "filename": "test2.json", 
      "type" : "File"
    },
    {
      "filename": "test3.json", 
      "type" : "File"
    }
  ],
  "number_of_filenames": 2
}

How are you ingesting the doc into ES? If you can manipulate the ingestion (logstash, python ...) just add the additional field there.

Otherwise, you can use an ingest pipeline and compute it at ingestion time.

Then, it'd be sufficient to make a sum aggregation over that new number_of_filenames field.

[treknado]

thanks, that is exactly what I'm looking for.

i am ingesting the documents via logstash. How can I do that in logstash in a simple way ?
or should i create a new thread in the Logstash section ?

[Fabio-sama (Fabio)]

It might be more appropriate to open a thread in the Logstash section. Tag me there if you can and I'll answer there to your question

If I don't see anything I'll post here a possible solution to your problem.

P.S. is that my_field field always a list of objects? And is filename present in each of them? I'm asking if you have to make any check on the single objects or if you blindly get the size of the list.