# Distributed Tracing logging and Integrating with Kibana

**URL:** <https://discuss.elastic.co/t/distributed-tracing-logging-and-integrating-with-kibana/17030>\
**Category:** Elasticsearch\
**Created:** [April 15, 2014, 5:16pm UTC](https://discuss.elastic.co/t/distributed-tracing-logging-and-integrating-with-kibana/17030 "2014-04-15T17:16:36Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Bhavesh\_Mistry](https://avatars.discourse-cdn.com/v4/letter/b/67e7ee/32.png) [@Bhavesh\_Mistry](https://discuss.elastic.co/u/Bhavesh_Mistry)\
**Post date:** [April 15, 2014, 5:16pm UTC](https://discuss.elastic.co/t/distributed-tracing-logging-and-integrating-with-kibana/17030/1 "2014-04-15T17:16:36Z")

</div>

I have implemented a Distributed Transaction Logging library with Tree like  
Structure as mention in Google Dapper(  
[http://research.google.com/pubs/pub36356.html](http://research.google.com/pubs/pub36356.html)) and eBay CAL Transaction  
Logging Framework(  
[http://devopsdotcom.files.wordpress.com/2012/11/screen-shot-2012-11-11-at-10-06-39-am.png](http://devopsdotcom.files.wordpress.com/2012/11/screen-shot-2012-11-11-at-10-06-39-am.png)).

The following is log format.

_Log Format_

TIMESTAMP HOSTNAME DATACENTER ENVIRONMENT EVENT\_GUID PARENT\_GUID TRACE\_GUID APPLICATION\_ID TREE\_LEVEL TRANSACTION\_TYPE TRANSACTION\_NAME STATUS\_CODE DURATION(in ms) PAYLOAD(key1=value2,key2=value2)

_GUID HEX NUMBER FORMAT_

MURMER\_HASH(HOSTNAME + DATACENTER +  
ENVIRONMENT)-JVM\_THREAD\_ID-(TIME\_STAMP+Atomic Counter)

The log format is like Relational Database Tables.  
Here is list of search query we will be doing:

Also I would like to know how I can index elastic search payload data so  
user specify some expression like in payload (duration \> 1000) then,  
Elastic Search will bring all the loglines that satisfy condition. Also, I  
would like to index Payload as Name=Value pair so user can query  
(key3=value2 or key4 = _exception_) some sort of regular expression. Please  
let me know if this can be achieved. Any help pointer would be great..  
Please note the Payload can be any string name or value(can be integer, or  
decimal or string).

Payload search example:

// (Duration \> 1000) and (key1 \> 10 or key1 \< 100) etc

Finally I would be integrating this with Kibana UI so we can have nice UI  
for searching and sorting etc.

I am new to Elastic Search and any pointer will be great. We will be using  
this in walmartlabs with around 100 machine elastic search cluster and  
~6000 FE servers logging data into Kafka and we will index from the Kafka  
into Elastic Search. Any pointer will be great.

Thanks,

Bhavesh

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/79f04627-29ac-42aa-9170-35c22ef76ffa%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/79f04627-29ac-42aa-9170-35c22ef76ffa%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 1:35am UTC](https://discuss.elastic.co/t/distributed-tracing-logging-and-integrating-with-kibana/17030/2 "2017-07-06T01:35:42Z")

</div>