Information about users that will become paying

[boksimon]

Hi,

We are indexing events performed by our users in an effort to create a better service. Each event is indexed into elasticsearch in real time, and we have a couple of dashboards visualizing the common actions. An example of one of these events in elastic could be something like this:

{
  [...]
  "_source": {
    "indexTime": 1601286327342,
    "eventType": "PAGE_FLIP",
    "userId": "123",
    "userCreatedAt": 1536096932688,
    "userLastTransactionAt": 0,
    "bookId": "abc",
    "pageNumber": 2,
    [...]
  },
  "fields": {
    "daysSinceSignup": [
      12
    ],
    "indexTime": [
      "2020-09-28T09:45:27.342Z"
    ],
    "daysUntilTestPeriodEnds": [
      2
    ],
    "userCreatedAt": [
      "2020-09-08T21:35:32.688Z"
    ],
    "userLastTransactionAt": [
      "1970-01-01T00:00:00.000Z"
    ]
  }
}

One of these events is to register a card (i.e. not use only the free version, but start paying us). This is obviously very important for us to track, but I would also like to visualize the the difference in actions leading up to someone starting a subscription.

Do I need to reindex all previous (often > 1000) events for a user when he/she subscripbes to be able to create visualizations in kibana of this?

Regards,
Simon

[warkolm (Mark Walkom)]

You have, what we refer to as, entity centric indexing. Where everything this built around your userID field.

The alternate approach is event based, where each action creates a new event and you can then do this sort of reporting based on that.

[boksimon]

This sounds interesting. We're small enough to change our indexing setup if it makes sense. Do you have any documentation or pointers on how this would work?

[warkolm (Mark Walkom)]

It's an activity/click stream approach, where each action is logged as an event in Elasticsearch. It's no different to a web/email log in its implementation (if that's easier to imagine).

You can still take that stream and then turn it into an entity centric approach, as additional indices. That way you get the best of both approaches.