# Creating data table. Table of done jobs

**URL:** <https://discuss.elastic.co/t/creating-data-table-table-of-done-jobs/186498>\
**Category:** Kibana\
**Created:** [June 19, 2019, 2:44pm UTC](https://discuss.elastic.co/t/creating-data-table-table-of-done-jobs/186498 "2019-06-19T14:44:29Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![optimistic](https://avatars.discourse-cdn.com/v4/letter/o/f6c823/32.png) [@optimistic](https://discuss.elastic.co/u/optimistic)\
**Post date:** [June 19, 2019, 2:44pm UTC](https://discuss.elastic.co/t/creating-data-table-table-of-done-jobs/186498/1 "2019-06-19T14:44:29Z")

</div>

I need to create table in Kibana.  
I have such logic of my app:  
User-refresh event creates, I have id of this event.  
Then all services should make some changes and send me info that they done a job.

So, I have such data (logs) in Elasticsearch:

> user-refresh id=21  
> user-refresh id=21, service1  
> user-refresh id=21, service2  
> user-refresh id=21, service3  
> user-refresh id=22  
> user-refresh id=22, service1  
> user-refresh id=22, service2  
> user-refresh id=23  
> user-refresh id=23, service1  
> user-refresh id=23, service3

I need to know which service did a job, and which is not.

If I have massage from a serviceN, it means service did a job - `v`  
if I don't have any massage from service with `user-refresh id` - service did not do a job - `x`

**How to create such a table?**  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/b/e/be2ab3c9c8faf8cd6f6c2d4508104d05e962da83.png)  
Thank you very much.

UPD. I can also change the output of logs, my main goal is to make such table.

---

<div class="post-metadata">

**Author:** ![lukas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lukas/32/6812_2.png) [@lukas](https://discuss.elastic.co/u/lukas)\
**Post date:** [June 19, 2019, 8:51pm UTC](https://discuss.elastic.co/t/creating-data-table-table-of-done-jobs/186498/2 "2019-06-19T20:51:39Z")

</div>

You won't likely be able to accomplish this using the "data table" feature in Kibana, but you might be able to using Canvas.

Could you actually provide examples of the JSON of the docs inside ES? Thanks!

---

<div class="post-metadata">

**Author:** ![optimistic](https://avatars.discourse-cdn.com/v4/letter/o/f6c823/32.png) [@optimistic](https://discuss.elastic.co/u/optimistic)\
**Post date:** [June 20, 2019, 7:17am UTC](https://discuss.elastic.co/t/creating-data-table-table-of-done-jobs/186498/3 "2019-06-20T07:17:55Z")

</div>

@lukas thanks for answer.

Yes, now I see that such a table don't possible to build with **Data table**.

This is a test JSON from _user-refresh_ event:

> {  
> "\_index": "logstash-2019.06.18-000001",  
> "\_type": "\_doc",  
> "\_id": "DECuc2sBt0eN0q1ZwlYQ",  
> "\_version": 1,  
> "\_score": null,  
> "\_source": {  
> "code": "user-refresh-test",  
> "schedule": null,  
> "@timestamp": "2019-06-20T06:59:21.128Z",  
> "@version": "1",  
> "creator": "notification-api",  
> "ip": null,  
> "id": 80,  
> "type": "rabbitmq"  
> },  
> "fields": {  
> "@timestamp": [  
> "2019-06-20T06:59:21.128Z"  
> ]  
> },  
> "sort": [  
> 1561013961128  
> ]  
> }

we just need event **"code": "user-refresh-test"** and it's id **"id": 80**

And this reply from _service1_ that job is done:

> {  
> "\_index": "logstash-2019.06.18-000001",  
> "\_type": "\_doc",  
> "\_id": "c0Cvc2sBt0eN0q1ZUFYC",  
> "\_version": 1,  
> "\_score": null,  
> "\_source": {  
> "code": "user-service1-test",  
> "schedule": null,  
> "@timestamp": "2019-06-20T06:59:57.465Z",  
> "@version": "1",  
> "creator": "notification-api",  
> "ip": null,  
> "id": 80,  
> "service1": true,  
> "type": "rabbitmq"  
> },  
> "fields": {  
> "@timestamp": [  
> "2019-06-20T06:59:57.465Z"  
> ]  
> },  
> "sort": [  
> 1561013997465  
> ]  
> }

Here we need approve that job is done **"service1": true** and id of a job **"id": 80**

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/8/6/86f52465f2e14cc41f1308440960135fd19520a8.png)  
Is it possible to create such table in **Visual builder**?  
Or I need to take a deeper look into **Canvas**?

---

<div class="post-metadata">

**Author:** ![optimistic](https://avatars.discourse-cdn.com/v4/letter/o/f6c823/32.png) [@optimistic](https://discuss.elastic.co/u/optimistic)\
**Post date:** [June 20, 2019, 1:52pm UTC](https://discuss.elastic.co/t/creating-data-table-table-of-done-jobs/186498/4 "2019-06-20T13:52:12Z")

</div>

I changed logic of logs aggregation.  
I configured my Logstash pipeline to change/update Elasticsearch document if its already have a documents with given **id**.

> output {  
> elasticsearch {  
> hosts =\> "elasticsearch:9200"  
> user =\> elastic  
> password =\> xx  
> index =\> "logstash-%{+YYYY.MM.dd}"  
> document\_id =\> "%{id}" #this field is main  
> doc\_as\_upsert =\> true  
> action =\> "update"

So my first service send event:

> 'user-refresh-test', id: 115

Than other service send:

> 'user-refresh-test', id: 115, serviceN1: 1

and it's update a first elasticsearch document.  
So now I have everything in one document and it is easy to build any visualization.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/0/5/0519051047432406e697d144667b99b138693cc9.png)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 18, 2019, 1:52pm UTC](https://discuss.elastic.co/t/creating-data-table-table-of-done-jobs/186498/5 "2019-07-18T13:52:18Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.