Building Elasticsearch Project

[duck]

Hi, I'm building a metrics and logging user actions system for my project(an online game). I've just started to learning Elasticsearch. Therefore, there are some questions in my mind. I'm planning to use logging almost everything but logs will be small(max. 10 fields). What kind of thing do I need to build this system?

Questions

1. Should I use only one cluster and node for now? (I'm planning to have 1,000 unique users per day.)
2. What kind of system requirements do I need? (Like 100GB SSD/HDD, RAM for Elasticsearch cluster/node etc.)

Example Schemes

Scheme 1

Index: transcation

timestamp: (Millisecond)
user-id: (UUID)
product-id: 1
amount: 5
price: 1500
seller: (UUID)

Scheme 2

Index: action

timestamp: (Millisecond)
user-id: (UUID)
category: FRIEND
action: REQUEST
friend-id: (UUID)

Scheme 3

Index: action

timestamp: (Millisecond)
user-id: (UUID)
category: FRIEND
action: ADD
friend-id: (UUID)

[warkolm (Mark Walkom)]

1. That sounds reasonable. You can also use Elastic Cloud to remove the need to manage the nodes, and it lets you easily scale when you need.
2. That depends on how long you want to retain the data for

[duck]

Thanks for the information. It really helped me!

We are trying to store user logs as long as possible to analyze past and today. That means, it'll probably stay forever. Do we have to make automation for nodes? Like when they reach their storage limit, the system will create another empty node etc.

[warkolm (Mark Walkom)]

Auto scaling on Elastic Cloud will be available soon

Otherwise it's DIY.