Docs about sharding and scatter/gather

Elastic Stack Elasticsearch
1

Hi,

I can't seem to find the documentation about how ES shards documents
(and how one can control that), is this available anywhere?

Also, I'm looking for something that describes how ES avoids
scattering search requests to all nodes and how it knows how to
limit it only to a limited set of nodes, but also can't find that in
the docs.

Could you please point me to the relevant docs so I can RTFM?

Thanks,
Otis

2

Maybe that could help a bit

http://elasticsearch.karmi.cz/guide/reference/api/search/search-type.html

if not ask via irc:freenode

On 31 Mrz., 23:51, Otis otis.gospodne...@gmail.com wrote:

Hi,

I can't seem to find the documentation about how ES shards documents
(and how one can control that), is this available anywhere?

Also, I'm looking for something that describes how ES avoids
scattering search requests to all nodes and how it knows how to
limit it only to a limited set of nodes, but also can't find that in
the docs.

Could you please point me to the relevant docs so I can RTFM?

Thanks,
Otis

3

When you search against an index, the search will be executed across all shards. If you want to have fine grained control, you can use routing. When indexing / deleting you can provide an addition routing value, and when searching, you can provide a routing value(s) as well.

When using a routing value when indexing / deleting, its value is used instead of the id to decide which shard it will end up on (hashed).

When using routing value(s) when searching, the shard(s) those routing value(s) will match on will be searched.

-shay.banon
On Thursday, March 31, 2011 at 11:51 PM, Otis wrote:

Hi,

I can't seem to find the documentation about how ES shards documents
(and how one can control that), is this available anywhere?

Also, I'm looking for something that describes how ES avoids
scattering search requests to all nodes and how it knows how to
limit it only to a limited set of nodes, but also can't find that in
the docs.

Could you please point me to the relevant docs so I can RTFM?

Thanks,
Otis

4

Hi,

Yeah, routing is what I was looking for. Is
http://elasticsearch.karmi.cz/guide/reference/mapping/routing-field.html
the best/only doc for it or is there some other piece of documentation
where I can get the details?

Thanks,
Otis

On Mar 31, 9:25 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

When you search against an index, the search will be executed across all shards. If you want to have fine grained control, you can use routing. When indexing / deleting you can provide an addition routing value, and when searching, you can provide a routing value(s) as well.

When using a routing value when indexing / deleting, its value is used instead of the id to decide which shard it will end up on (hashed).

When using routing value(s) when searching, the shard(s) those routing value(s) will match on will be searched.

-shay.banon

On Thursday, March 31, 2011 at 11:51 PM, Otis wrote:

Hi,

I can't seem to find the documentation about how ES shards documents
(and how one can control that), is this available anywhere?

Also, I'm looking for something that describes how ES avoids
scattering search requests to all nodes and how it knows how to
limit it only to a limited set of nodes, but also can't find that in
the docs.

Could you please point me to the relevant docs so I can RTFM?

Thanks,
Otis

5

The routing field allows to specify you want to get the routing from a specific field in the document. But, a better approach is to provide the routing when indexing a doc explicitly.

Check the routing sections for each API, for example: Elasticsearch Platform — Find real-time answers at scale | Elastic, Elasticsearch Platform — Find real-time answers at scale | Elastic.

On Friday, April 1, 2011 at 8:50 AM, Otis wrote:

Hi,

Yeah, routing is what I was looking for. Is
http://elasticsearch.karmi.cz/guide/reference/mapping/routing-field.html
the best/only doc for it or is there some other piece of documentation
where I can get the details?

Thanks,
Otis

On Mar 31, 9:25 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

When you search against an index, the search will be executed across all shards. If you want to have fine grained control, you can use routing. When indexing / deleting you can provide an addition routing value, and when searching, you can provide a routing value(s) as well.

When using a routing value when indexing / deleting, its value is used instead of the id to decide which shard it will end up on (hashed).

When using routing value(s) when searching, the shard(s) those routing value(s) will match on will be searched.

-shay.banon

On Thursday, March 31, 2011 at 11:51 PM, Otis wrote:

Hi,

I can't seem to find the documentation about how ES shards documents
(and how one can control that), is this available anywhere?

Also, I'm looking for something that describes how ES avoids
scattering search requests to all nodes and how it knows how to
limit it only to a limited set of nodes, but also can't find that in
the docs.

Could you please point me to the relevant docs so I can RTFM?

Thanks,
Otis