Elasticsearch Architecture

Elastic Stack Elasticsearch
1

Hi,
I use elasticsearch 6.
I have 2 types of document, x and y.
From y I will have 50,000 documents, and from x I will have 500 millions.
The two type of documents have different fields.
I have 2 question:

  1. what is the recommendation, to use two types of index, or one type and add field type to the document that help distinguish between the types.
    2)If document of type x will index in one type of index and type y in another type of index, there is a difference between the next two options?:
    a. Devide the two types of indexes between the same two nodes, in node A will be shards from index x and shards from index y, and in node B also will be shards from x and from y.
    b. In node A will be shards from index x, and in node B will be shards from index y.

There is a difference in terms of performance?

Thanks.

2

  1. separate indexes (newer versions won't even have types)

  2. yes, one index is considerably bigger than the other since it has more data. So it's better to distribute it on all nodes.

3

Hi,
1)Why in .kibana you use the visualisation and index-pattern in the same index, why you didn't separate indexes.
2)The question is, do we need to distributed the two indexes in the same servers, or one index in 2 servers and the other type of index in another 2 servers?

4

If index is very small and very high query rate you can keep shard_num=1 and increase replication. Otherwise it's best to have shard_num=x where x is complex and depends on many factors, you should research on docs.

5

Thank you for your response, but I know how to decide, how much shards I need, what I don't know is what the effect of store two indexes , one very big and one small on the same machines, it's good or its better to store them in separate machines?

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.