Can't figure out how to wildcard fields

[Rhysyrhys (Rhys Davies)]

Hi, I have my Celery workers reporting their status to Kibana via logstash. When I get a snapshot the document Kibana receives is similar to this truncated doc:

{
  "tasks_total_processed": 2,
  "worker_count": 2,
  "worker_load_average": 0.92,
  "workers": {
    "celery4": {
      "stats": {
        "processes": 1,
        ...
      }
    },
  "celery5": {
    "stats": {
      "processes": 9,
      ....
    }
  }
}
}

I want to write a time series visualization of the number of processes for each of my celery workers and for a new a line to be added when I add a new celery worker automatically.

In other tools I might've written something that counts workers.*.stats.processes and gotten a line for celery4, celery5 and celeryN and so forth. How do I achieve this in Kibana??

[mikecote (Mike)]

Hi @Rhysyrhys,

If you change the data structure to something like

{
   "@timestamp": { "type": "date" },
   "name": { "type": "keyword" },
   "processes": { "type": "number" }
}

you will be able to have a line per worker automatically. This would also require a document per worker instead of a single document containing all the values. The top fields tasks_total_processed, worker_count and worker_load_average can also be calculated within the visualizations if necessary.

You can then use timelion with an expression like:

.es(index=mypattern*,split=name.keyword:10)

[Rhysyrhys (Rhys Davies)]

Hi Mike,

Thanks for the reply, I figured the shape thing out and have nicely working graphs now!

Best,

Rhys