# Split data into into two sets (test/train)
**URL:** https://discuss.elastic.co/t/split-data-into-into-two-sets-test-train/273236
**Category:** Elasticsearch
**Created:** [May 18, 2021, 6:52am UTC](https://discuss.elastic.co/t/split-data-into-into-two-sets-test-train/273236 "2021-05-18T06:52:39Z")
**Posts on this page:** 5
**Page:** 1
### Author:  [@eliase](https://discuss.elastic.co/u/eliase)
#### Post date: [May 18, 2021, 6:52am UTC](https://discuss.elastic.co/t/split-data-into-into-two-sets-test-train/273236/1 "2021-05-18T06:52:39Z")
Hi,
for machine learning validation I would like to split the data in elastic search into two sets.
The [random score](https://www.elastic.co/guide/en/elasticsearch/reference/7.13/query-dsl-function-score-query.html#function-random) in combination with the size parameter would give me one set but how can I obtain all other documents? Is there an easy way to do that?
---
### Author:  [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [May 18, 2021, 7:16am UTC](https://discuss.elastic.co/t/split-data-into-into-two-sets-test-train/273236/2 "2021-05-18T07:16:10Z")
If you use multiple shards to index the data it’s already randomly distributed across shards and you can use search [routing](https://www.elastic.co/guide/en/elasticsearch/reference/master/search-shard-routing.html#search-routing) to query only one of several shards.
---
### Author:  [@eliase](https://discuss.elastic.co/u/eliase)
#### Post date: [May 18, 2021, 7:26am UTC](https://discuss.elastic.co/t/split-data-into-into-two-sets-test-train/273236/3 "2021-05-18T07:26:02Z")
Thanks for the quick reply. So far, it's only in one shard.
Even if there were multiple shards, that doesn't seem to be very flexible solution in regards to getting a specified split ratio like 20% / 80%
---
### Author:  [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [May 18, 2021, 9:01am UTC](https://discuss.elastic.co/t/split-data-into-into-two-sets-test-train/273236/4 "2021-05-18T09:01:49Z")
The routing function is a common hash modulo N function where N is the number of shards.
You could apply the same logic to querying the ID of documents with a script query e.g. something like:
```auto
{
"query": {
"script": {
"script": " doc['_id'].value.hashCode()%2 == 0"
}
}
}
```
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [June 15, 2021, 9:02am UTC](https://discuss.elastic.co/t/split-data-into-into-two-sets-test-train/273236/5 "2021-06-15T09:02:23Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.