Timelion request error ---> [parsing exception]

Anne_Kim · October 29, 2019, 2:49pm

Hi there,

I'm trying to visualize and compare a daily unique count of "player_id" from 2 different indices (session_data and gameplay_stat) using timelion.
The request just for the first index vizualization looks like:
.es(interval=1d, index=session_data, metric=cardinality:player_id).label('Unique Players Logged')

And there is nothing, just a legend label has changed

Then I'm adding an expression for the 2nd index:
.es(index=gameplay_stat, metric=cardinality:game.player).label('Unique Players Played')

and got 2 errors

What's wrong with it?

christophilus · October 29, 2019, 6:13pm

It's hard to say without seeing the errors. It looks like they didn't post properly to Discuss. Could you repost them?

Anne_Kim · October 30, 2019, 7:57am

The errors screenshot is in "hidden details", so it is:
UPdsyqM

flash1293 · October 31, 2019, 8:41am

Did you put a comma between the two expressions? Testing locally

.es(interval=1d, index=session_data, metric=cardinality:player_id).label('Unique Players Logged') .es(index=gameplay_stat, metric=cardinality:game.player).label('Unique Players Played')

gave me the error you posted, but

.es(interval=1d, index=session_data, metric=cardinality:player_id).label('Unique Players Logged'), .es(index=gameplay_stat, metric=cardinality:game.player).label('Unique Players Played')

works fine for me.

Anne_Kim · October 31, 2019, 8:52am

I've just copied it, there were no any errors, but it has shown a zero-result (for 15 weeks!):

When the data is vizualized as metrics, everything works fine (the same result for 15 weeks):

flash1293 · October 31, 2019, 1:26pm

I'm just guessing, but is your index actually called session_data or should it be session_data* because there is some suffix (like the date) in the index name? Another reason could be the field names. If the * in the index name isn't the issue, could you paste the mapping of your indices and a few sample documents here?

Anne_Kim · October 31, 2019, 2:14pm

Well, actually yesterday I've created an index pattern called "session_data*", but it's already deleted and is not used here, it's just a coincidence.

So, here is a JSON view of an element ("session_data" index)

{
  "_index": "session_data",
  "_type": "data",
  "_id": "h9vmIW4BSOdvtuXlxyAc",
  "_version": 1,
  "_score": null,
  "_source": {
    "id": 27932,
    "player_id": 450,
    "player_login": "Bruno Buccelati",
    "ram_amount": "5634",
    "gpu_model": "Adreno (TM) 630",
    "cpu_model": "ARM64 FP ASIMD AES",
    "start_at": "2019-10-31T11:10:53.000+00:00",
    "end_at": "2019-10-31T11:35:45.000+00:00",
    "duration": 1492,
    "is_tester": true,
    "device": "Xiaomi POCOPHONE F1",
    "ip": "......"
  },
  "fields": {
    "end_at": [
      "2019-10-31T11:35:45.000Z"
    ],
    "start_at": [
      "2019-10-31T11:10:53.000Z"
    ],
    "day_of_month": [
      31
    ],
    "hour_of_day": [
      11
    ],
    "day_of_week": [
      4
    ]
  },
  "sort": [
    1572520253000
  ]
}

Its screenshot:

And some other elements of session_data index:

Oct 31, 2019 @ 13:48:22.000	id:27,949 player_id:474 player_login:Magore ram_amount:16313 gpu_model:Radeon (TM) RX 470 Graphics cpu_model:Intel(R) Core(TM) i5-3470 CPU @ 3.20GHz start_at:Oct 31, 2019 @ 13:48:22.000 end_at:Oct 31, 2019 @ 13:50:12.000 duration:0:01:50 is_tester:true device:MS-7808 (MSI) ip:....... _id:ltvmIW4BSOdvtuXlyCCW _type:data _index:session_data _score: - day_of_month:31 hour_of_day:11 day_of_week:4

Oct 31, 2019 @ 13:47:41.000	id:27,948 player_id:474 player_login:Magore ram_amount:16313 gpu_model:Radeon (TM) RX 470 Graphics cpu_model:Intel(R) Core(TM) i5-3470 CPU @ 3.20GHz start_at:Oct 31, 2019 @ 13:47:41.000 end_at:Oct 31, 2019 @ 13:50:13.000 duration:0:02:32 is_tester:true device:MS-7808 (MSI) ip:....... _id:ldvmIW4BSOdvtuXlyCCK _type:data _index:session_data _score: - day_of_month:31 hour_of_day:11 day_of_week:4

Oct 31, 2019 @ 13:47:15.000	id:27,947 player_id:446 player_login:HaxuJluBaIO ram_amount:5634 gpu_model:Adreno (TM) 630 cpu_model:ARM64 FP ASIMD AES start_at:Oct 31, 2019 @ 13:47:15.000 end_at:Oct 31, 2019 @ 13:54:18.000 duration:0:07:03 is_tester:true device:Xiaomi POCO F1 ip:........ _id:lNvmIW4BSOdvtuXlyCCB _type:data _index:session_data _score: - day_of_month:31 hour_of_day:11 day_of_week:4

Here is a JSON view of an element ("gameplay_stat" index)

{
  "_index": "gameplay_stat",
  "_type": "data",
  "_id": "hNsdIm4BSOdvtuXl2yH7",
  "_version": 1,
  "_score": null,
  "_source": {
    "game": {
      "player": 450,
      "player_login": "Bruno Buccelati",
      "distance": 0,
      "avg_lifetime": 71
    },
    "room": {
      "id": 14172,
      "efficiency": 185,
      "platform": "Android 5.0+",
      "player_amount": 10,
      "build": "0.2.3.1011",
      "squad_size": 4,
      "map_name": "empty41",
      "game_mode": "CaptureThePoint",
      "created_at": "2019-10-31T12:57:06.000+00:00"
    },
    "kills": 7,
    "place": 1,
    "assist": 3,
    "death": 1,
    "character_key": 1,
    "character_name": "Fire",
    "character_lvl": 0,
    "is_tester": true,
    "leaver": null,
    "room_length": 151,
    "is_win": true
  },
  "fields": {
    "room.created_at": [
      "2019-10-31T12:57:06.000Z"
    ]
  },
  "sort": [
    1572526626000
  ]
}

Its screenshot:

And some other elements of gameplay_stat index:

Oct 31, 2019 @ 14:56:38.000	game.player:464 game.player_login:Hate game.distance:0 game.avg_lifetime:43 room.id:14,171 room.efficiency:97 room.platform:Android 5.0+ room.player_amount:10 room.build:0.2.3.1011 room.squad_size:4 room.map_name:empty43 room.game_mode:CaptureThePoint room.created_at:Oct 31, 2019 @ 14:56:38.000 kills:8 place:1 assist:3 death:3 character_key:3 character_name:Lightning character_lvl:0 is_tester:true leaver: - room_length:139 is_win:false _id:ftsdIm4BSOdvtuXl2SHD _type:data _index:gameplay_stat _score: -
	
Oct 31, 2019 @ 14:54:12.000	game.player:451 game.player_login:Shiro Panos game.distance:0 game.avg_lifetime:70 room.id:14,167 room.efficiency:149 room.platform:Android 5.0+ room.player_amount:6 room.build:0.2.3.1009 room.squad_size:4 room.map_name:empty41 room.game_mode:CaptureThePoint room.created_at:Oct 31, 2019 @ 14:54:12.000 kills:10 place:1 assist:3 death:2 character_key:0 character_name:Frost character_lvl:0 is_tester:true leaver: - room_length:180 is_win:true _id:eNsdIm4BSOdvtuXl2CES _type:data _index:gameplay_stat _score: -
    	
Oct 31, 2019 @ 14:50:49.000	game.player:464 game.player_login:Hate game.distance:0 game.avg_lifetime:24 room.id:14,165 room.efficiency:145 room.platform:Android 5.0+ room.player_amount:10 room.build:0.2.3.1011 room.squad_size:4 room.map_name:empty43 room.game_mode:SkullGrab room.created_at:Oct 31, 2019 @ 14:50:49.000 kills:8 place:2 assist:7 death:6 character_key:2 character_name:Earth character_lvl:0 is_tester:true leaver: - room_length:184 is_win:false _id:ctsdIm4BSOdvtuXl1SHa _type:data _index:gameplay_stat _score: -

flash1293 · October 31, 2019, 2:45pm

Could you post the result of this queries: GET session_data/_mapping and GET gameplay_stat/_mapping ? The source itself looks fine in general but maybe there is an issue in how the data is indexed by Elasticsearch.

Anne_Kim · October 31, 2019, 3:31pm

Here they are:
{"session_data":{"mappings":{"properties":{"cpu_model":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}},"device":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}},"duration":{"type":"long"},"end_at":{"type":"date"},"gpu_model":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}},"id":{"type":"long"},"ip":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}},"is_tester":{"type":"boolean"},"player_id":{"type":"long"},"player_login":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}},"ram_amount":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}},"start_at":{"type":"date"}}}}}

and

{"gameplay_stat":{"mappings":{"properties":{"assist":{"type":"long"},"character_id":{"type":"long"},"character_key":{"type":"long"},"character_lvl":{"type":"long"},"character_name":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}},"death":{"type":"long"},"game":{"properties":{"avg_lifetime":{"type":"long"},"distance":{"type":"long"},"player":{"type":"long"},"player_login":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}}}},"is_tester":{"type":"boolean"},"is_win":{"type":"boolean"},"kills":{"type":"long"},"leaver":{"type":"boolean"},"place":{"type":"long"},"room":{"properties":{"build":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}},"created_at":{"type":"date"},"efficiency":{"type":"long"},"game_mode":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}},"id":{"type":"long"},"map_name":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}},"platform":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}},"player_amount":{"type":"long"},"squad_size":{"type":"long"}}},"room_length":{"type":"long"}}}}}

flash1293 · November 1, 2019, 9:26am

Thanks, that's helpful.

So your mapping is looking fine but it seems like there is no @timefield field - this is used by default to aggregate over time in Timelion. If you want to use a different field for the time axis (as in this example), you have to specify it as the timefield parameter in the .es() function:

.es(interval=1d, timefield='start_at', index=session_data, metric='cardinality:player_id').label('Unique Players Logged')

Another detail: It's recommended to put the metric into quotes (as in the example above) to avoid conflicts with reserved characters in the field name.

See also https://www.elastic.co/guide/en/kibana/current/timelion.html about more details.

Let me know whether that solves your problem.

Anne_Kim · November 1, 2019, 9:47am

Omg, it works. Thank you!

The last question: on the page you've mentionned I did not found how to change the line mode: now it's stepped. How to do this?

flash1293 · November 1, 2019, 9:56am

You should be able to change it by appending .lines(steps=false) to the expression

.es(interval=1d, timefield='start_at', index=session_data, metric='cardinality:player_id').label('Unique Players Logged').lines(steps=false)

But I'm unsure why this is happending, not using steps should be the default. @timroes do you know why this is happening?

Anne_Kim · November 1, 2019, 10:05am

Nope. I've tried .lines(steps=false), .lines(steps=0) and .lines(steps=null).
And nothing has changed

Anne_Kim · November 1, 2019, 12:31pm

2 hours later everything became ok (I've just edited the vizualization). May be it was a temporary bug.

Anne_Kim · November 1, 2019, 3:28pm

UPD. Lines become stepped if the interval is specified in the expression.
E.g., .es(interval=4h, timefield='start_at', index=session_data, metric='count:player_id')
If the interval is specified not in the expression (see screenshot), lines become straight.

system · November 29, 2019, 3:28pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.