Data inconsistency in timelion graphs


(Amandeep Singh) #1

I'm trying to plot a trend based(sum of cost for each day) and apply a split over the data based upon class id.


.es(index="egm-predicted*",split="class_id:5",metric="sum:cogs_pred",timefield="order_date").legend(position="ne",columns=4).label("class: $1", "^.* > class_id:(\S+) > .*"),

but when i do normal line plot the result vary

The timelion one is not selecting the classes in correct order, Though on increasing the split number the top classes in timelion changes

The question i have is inability to find why the top classes are varing


(Nathan Reese) #2

For the timelion config .es(index=logstash*,timefield=@timestamp,split=machine.os.keyword:2,metric=sum:bytes), the elasticsearch request contains the following aggs parameter

{
  "aggs": {
   "q": {
    "meta": {
     "type": "split"
    },
    "filters": {
     "filters": {
      "*": {
       "query_string": {
        "query": "*"
       }
      }
     }
    },
    "aggs": {
     "machine.os.keyword": {
      "meta": {
       "type": "split"
      },
      "terms": {
       "field": "machine.os.keyword",
       "size": 2
      },
      "aggs": {
       "time_buckets": {
        "meta": {
         "type": "time_buckets"
        },
        "date_histogram": {
         "field": "@timestamp",
         "interval": "1s",
         "time_zone": "America/Denver",
         "extended_bounds": {
          "min": 1527273824235,
          "max": 1527274724235
         },
         "min_doc_count": 0
        },
        "aggs": {
         "sum(bytes)": {
          "sum": {
           "field": "bytes"
          }
         }
        }
       }
      }
     }
    }
   }
  },
}

Getting the timelion request is hacky. I do it by adding the line console.log(JSON.stringify(body, null, ' ')); here

For a line visualization, the elasticsearch request contains the following aggs parameter

{
  "aggs": {
    "3": {
      "terms": {
        "field": "machine.os.keyword",
        "size": 2,
        "order": {
          "1": "desc"
        }
      },
      "aggs": {
        "1": {
          "sum": {
            "field": "bytes"
          }
        },
        "2": {
          "date_histogram": {
            "field": "@timestamp",
            "interval": "30s",
            "time_zone": "America/Denver",
            "min_doc_count": 1
          },
          "aggs": {
            "1": {
              "sum": {
                "field": "bytes"
              }
            }
          }
        }
      }
    }
  },
}

The difference between the two is that the line graph visualization provides the order param for the terms aggregation while the timelion aggregation does not. The order is ordering buckets by the metric value - in this case means they are ordered by top sum. When order is not provided - as is the case with timelion, then the buckets are ordered by count.


(Amandeep Singh) #3

can we add provide the order timelion one ?


(Nathan Reese) #4

no. I would recommend opening an enhancement request here


(system) #5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.