Data Table troubles

Elastic Stack Kibana
1

I am having an issue building a data table visualization. I have data that contains a username, and an activity. I need a report for management that shows how many times a given user did each activity, in this case they only want to see three out the hundreds of possible activities.

I built a data table using the username, and a count of three filters. Everything seems ok but the second and third filters always show a count of zero. What ever order I put the filters in I get the actual count for the first filter, and zero for the next two.

I am not sure what I am doing wrong here. Any ideas?

Thanks!

2

Hi, can you give an example of the data you are working with, and a screenshot of how your current data table visualization is set up?

3

I finally have log entries mask well enough to post them here. Below are the three log lines in question. Assume that the users name, ip address, etc change but the ServiceAccess is static, as well as the operation. Also assume that we have proper Grok filters breaking this data up.

The ask is for a report that shows this:

USER101
Search: 194
Save: 65
Submit: 15

USER102
Search: 494
Save: 156
Submit: 87

There are millions of transactions, and just under 50,000 users so I opted for a datatable. So far this is not working out. The visualization is very simple:

Buckets:
Split rows:
  Agg: Terms
  Field: User.keyword
  Order by: Alpha
  Order: Asc Size: 50000
Split rows:
  Sub Agg: Filters
  Filter1: operation:*Authorization*
Split rows:
  Sub Agg: Filters
  Filter1: operation:submitAuthorizationRequest
Split rows:
  Sub Agg: Filters
  Filter1: operation:saveAuthorizationRequest
Split rows:
  Sub Agg: Filters
  Filter1: operation:getAuthorizationPodQueryBuilder

That's it. The metric is the default, Count.

That operation is pulled from the end of the string in the log line:

ServiceAccess: com.MyApp.services.XX.impl.AuthorizationTxService.saveAuthorizationRequest

Which ever filter is placed first in that sub agg shows actual values, while the remaining filters will always be zero. Move a filter that shows zero to the top and it will have actual counts and everything else goes to zero.

I have many request like this to provide reports from this data on a 'Today' time window which mean millions of records, 50,000 users, and countless combinations of transactions. For this report we are dealing with three out of hundreds of possible operations. It tends to timeout, or just outright fail. So if there is a better way to handle this I am all ears.

Thanks for your help!

Log line examples are below:

2019-06-27 11:18:47,490 INFO  [com.MyApp.aspect.ServicesLayerExceptionLogger] sessionId=7C055B40D058C5441C4C8DEDB4AD9421 memberId=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX screenName=AuthorizationRequest user=USER101 ipAddress=1.2.3.4 reqID=1061896 destination=AuthorizationRequestScreenService operation=save ServiceAccess: com.MyApp.services.XX.impl.AuthorizationTxService.saveAuthorizationRequest completed in 59 ms

2019-06-27 11:20:51,895 INFO  [com.MyApp.aspect.ServicesLayerExceptionLogger] sessionId=A7372834B6A2B6D6ED73E01B650F4F44 memberId=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX screenName=AuthorizationRequest user=USER101 ipAddress=1.2.3.4 reqID=123527 destination=AuthorizationRequestScreenService operation=submit ServiceAccess: com.MyApp.services.XX.impl.AuthorizationTxService.submitAuthorizationRequest completed in 465 ms

2019-06-27 11:21:34,423 INFO  [com.MyApp.aspect.ServicesLayerExceptionLogger] sessionId=3A94B34A42A28E0A07DA6D1865F29627 memberId= screenName= user=USER101 ipAddress=1.2.3.4 reqID=1102705 destination=authorizationPodScreenService operation=search ServiceAccess: com.MyApp.services.XX.impl.AuthorizationTxService.getAuthorizationPodQueryBuilder completed in 0 ms
4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.