ElasticSearch Benchmark


(Jianjun Hu) #1

Hi all,

How to do benchmark of the ElasticSearch? Thanks!

Best,
Jianjun

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d3f3f50c-53de-4ec1-a5aa-2f867d1791bb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Leslie Hawthorn-2) #2

Hi Jianjun,

On Thu, Apr 10, 2014 at 3:02 PM, Jianjun Hu 4nexushu@gmail.com wrote:

Hi all,

How to do benchmark of the ElasticSearch? Thanks!

The answer to this question is "it depends." It would be very helpful to
describe a bit more about what you are working on so people can give a
better answer to you. :slight_smile:

Best,
LH

--
Leslie Hawthorn
Community Manager
http://elasticsearch.com

Other Places to Find Me:
Freenode: lh
Twitter: @lhawthorn
Skype: mebelh
Voice: +31 20 794 7300

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAB5RFS8y1oOQk7ehyrOnxDbf0KkL793suTnZtXKXZN1mtc3SBg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Jianjun Hu) #3

Hi Leslie,

Thanks for your reply.

We have some data in MySQL database and some PDF documents. We want to
index them and let our user to use. So we decide to use the ElasticSearch.
Before using the ElsaticSearch, we want to know the performance of the
ElasticSearch, so we want to do benchmark of the ElasticSearch. However, we
don't know how to do. I'm not sure if I express my question clearly.

Best,
Jianjun

On Thursday, April 10, 2014 9:11:37 AM UTC-4, Leslie Hawthorn wrote:

Hi Jianjun,

On Thu, Apr 10, 2014 at 3:02 PM, Jianjun Hu <4nex...@gmail.com<javascript:>

wrote:

Hi all,

How to do benchmark of the ElasticSearch? Thanks!

The answer to this question is "it depends." It would be very helpful to
describe a bit more about what you are working on so people can give a
better answer to you. :slight_smile:

Best,
LH

--
Leslie Hawthorn
Community Manager
http://elasticsearch.com

Other Places to Find Me:
Freenode: lh
Twitter: @lhawthorn
Skype: mebelh
Voice: +31 20 794 7300

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/b6aa8d62-842a-4d2e-b184-0ef40530d47e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Leslie Hawthorn-2) #4

Hi Jianjun,

Thank you for the additional details. I think we will need some more
information but I will let someone who is better skilled at benchmarking
Elasticsearch ask you about it.

Welcome to the mailing list! We are happy to have you here and to help.

Cheers,
LH

On Thu, Apr 10, 2014 at 3:42 PM, Jianjun Hu 4nexushu@gmail.com wrote:

Hi Leslie,

Thanks for your reply.

We have some data in MySQL database and some PDF documents. We want to
index them and let our user to use. So we decide to use the ElasticSearch.
Before using the ElsaticSearch, we want to know the performance of the
ElasticSearch, so we want to do benchmark of the ElasticSearch. However, we
don't know how to do. I'm not sure if I express my question clearly.

Best,
Jianjun

On Thursday, April 10, 2014 9:11:37 AM UTC-4, Leslie Hawthorn wrote:

Hi Jianjun,

On Thu, Apr 10, 2014 at 3:02 PM, Jianjun Hu 4nex...@gmail.com wrote:

Hi all,

How to do benchmark of the ElasticSearch? Thanks!

The answer to this question is "it depends." It would be very helpful to
describe a bit more about what you are working on so people can give a
better answer to you. :slight_smile:

Best,
LH

--
Leslie Hawthorn
Community Manager
http://elasticsearch.com

Other Places to Find Me:
Freenode: lh
Twitter: @lhawthorn
Skype: mebelh
Voice: +31 20 794 7300

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/b6aa8d62-842a-4d2e-b184-0ef40530d47e%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/b6aa8d62-842a-4d2e-b184-0ef40530d47e%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Leslie Hawthorn
Community Manager
http://elasticsearch.com

Other Places to Find Me:
Freenode: lh
Twitter: @lhawthorn
Skype: mebelh
Voice: +31 20 794 7300

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAB5RFS-_2SRbUkc_yVU4W_fPPq4efXPKnB%2Bnmh%2B2cveKoq_BVQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Mark Walkom) #5

The best way to know is to test it yourself.
It's very dependant on your hardware, your settings and the data that you
are indexing.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 10 April 2014 23:49, Leslie Hawthorn
leslie.hawthorn@elasticsearch.comwrote:

Hi Jianjun,

Thank you for the additional details. I think we will need some more
information but I will let someone who is better skilled at benchmarking
Elasticsearch ask you about it.

Welcome to the mailing list! We are happy to have you here and to help.

Cheers,
LH

On Thu, Apr 10, 2014 at 3:42 PM, Jianjun Hu 4nexushu@gmail.com wrote:

Hi Leslie,

Thanks for your reply.

We have some data in MySQL database and some PDF documents. We want to
index them and let our user to use. So we decide to use the ElasticSearch.
Before using the ElsaticSearch, we want to know the performance of the
ElasticSearch, so we want to do benchmark of the ElasticSearch. However, we
don't know how to do. I'm not sure if I express my question clearly.

Best,
Jianjun

On Thursday, April 10, 2014 9:11:37 AM UTC-4, Leslie Hawthorn wrote:

Hi Jianjun,

On Thu, Apr 10, 2014 at 3:02 PM, Jianjun Hu 4nex...@gmail.com wrote:

Hi all,

How to do benchmark of the ElasticSearch? Thanks!

The answer to this question is "it depends." It would be very helpful
to describe a bit more about what you are working on so people can give a
better answer to you. :slight_smile:

Best,
LH

--
Leslie Hawthorn
Community Manager
http://elasticsearch.com

Other Places to Find Me:
Freenode: lh
Twitter: @lhawthorn
Skype: mebelh
Voice: +31 20 794 7300

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/b6aa8d62-842a-4d2e-b184-0ef40530d47e%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/b6aa8d62-842a-4d2e-b184-0ef40530d47e%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Leslie Hawthorn
Community Manager
http://elasticsearch.com

Other Places to Find Me:
Freenode: lh
Twitter: @lhawthorn
Skype: mebelh
Voice: +31 20 794 7300

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAB5RFS-_2SRbUkc_yVU4W_fPPq4efXPKnB%2Bnmh%2B2cveKoq_BVQ%40mail.gmail.comhttps://groups.google.com/d/msgid/elasticsearch/CAB5RFS-_2SRbUkc_yVU4W_fPPq4efXPKnB%2Bnmh%2B2cveKoq_BVQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624YnV%3DSSTejFa9tS-EX1LJ0skEyH3zj7fUTNfA5txY_gKw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Jianjun Hu) #6

Am I in the mailling list now?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/83af0f2c-fb4d-4741-8662-47631bfc95e5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Jianjun Hu) #7

Could you please give me some more detailed suggestions on how to test the
elasticsearch? thanks.

On Thursday, April 10, 2014 10:13:49 AM UTC-4, Mark Walkom wrote:

The best way to know is to test it yourself.
It's very dependant on your hardware, your settings and the data that you
are indexing.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: ma...@campaignmonitor.com <javascript:>
web: www.campaignmonitor.com

On 10 April 2014 23:49, Leslie Hawthorn <leslie....@elasticsearch.com<javascript:>

wrote:

Hi Jianjun,

Thank you for the additional details. I think we will need some more
information but I will let someone who is better skilled at benchmarking
Elasticsearch ask you about it.

Welcome to the mailing list! We are happy to have you here and to help.

Cheers,
LH

On Thu, Apr 10, 2014 at 3:42 PM, Jianjun Hu <4nex...@gmail.com<javascript:>

wrote:

Hi Leslie,

Thanks for your reply.

We have some data in MySQL database and some PDF documents. We want to
index them and let our user to use. So we decide to use the ElasticSearch.
Before using the ElsaticSearch, we want to know the performance of the
ElasticSearch, so we want to do benchmark of the ElasticSearch. However, we
don't know how to do. I'm not sure if I express my question clearly.

Best,
Jianjun

On Thursday, April 10, 2014 9:11:37 AM UTC-4, Leslie Hawthorn wrote:

Hi Jianjun,

On Thu, Apr 10, 2014 at 3:02 PM, Jianjun Hu 4nex...@gmail.com wrote:

Hi all,

How to do benchmark of the ElasticSearch? Thanks!

The answer to this question is "it depends." It would be very helpful
to describe a bit more about what you are working on so people can give a
better answer to you. :slight_smile:

Best,
LH

--
Leslie Hawthorn
Community Manager
http://elasticsearch.com

Other Places to Find Me:
Freenode: lh
Twitter: @lhawthorn
Skype: mebelh
Voice: +31 20 794 7300

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/b6aa8d62-842a-4d2e-b184-0ef40530d47e%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/b6aa8d62-842a-4d2e-b184-0ef40530d47e%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Leslie Hawthorn
Community Manager
http://elasticsearch.com

Other Places to Find Me:
Freenode: lh
Twitter: @lhawthorn
Skype: mebelh
Voice: +31 20 794 7300

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAB5RFS-_2SRbUkc_yVU4W_fPPq4efXPKnB%2Bnmh%2B2cveKoq_BVQ%40mail.gmail.comhttps://groups.google.com/d/msgid/elasticsearch/CAB5RFS-_2SRbUkc_yVU4W_fPPq4efXPKnB%2Bnmh%2B2cveKoq_BVQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d8f2dd67-c113-4965-8f4c-7002c2053a1d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Rafał Kuć) #8

Hello!

Depends on what you want to test. If you want to test indexing - start from scratch, see how many documents you can index with the clean Elasticsearch cluster, when you start to see slowdowns. Try seeing what the bottleneck is if any. In general, such tests depend on how much data you have and how much indexing you are planning to do.

If you want to test querying performance - start with some setup that you assume should be OK. Index your data and start testing, for example by sending production like queries with JMeter. That will give you information on how the cluster behaves in such setup. Then you can change the deployment and tune. For example adding replicas, re-sharding and having more/less shards, depending on the data and its volume.

During all tests, you should be monitoring your cluster. You can use anything available, although for performance testing anything will be good, even the command line tools that will help you understand what is happening on the operating system level, how garbage collector is performing.

--

Regards,

Rafał Kuć

Performance Monitoring * Log Analytics * Search Analytics

Solr & Elasticsearch Support * http://sematext.com/

Could you please give me some more detailed suggestions on how to test the elasticsearch? thanks.

On Thursday, April 10, 2014 10:13:49 AM UTC-4, Mark Walkom wrote:

The best way to know is to test it yourself.

It's very dependant on your hardware, your settings and the data that you are indexing.

Regards,

Mark Walkom

Infrastructure Engineer

Campaign Monitor

email: ma...@campaignmonitor.com

web: www.campaignmonitor.com

On 10 April 2014 23:49, Leslie Hawthorn <leslie....@elasticsearch.com> wrote:

Hi Jianjun,

Thank you for the additional details. I think we will need some more information but I will let someone who is better skilled at benchmarking Elasticsearch ask you about it.

Welcome to the mailing list! We are happy to have you here and to help.

Cheers,

LH

On Thu, Apr 10, 2014 at 3:42 PM, Jianjun Hu <4nex...@gmail.com> wrote:

Hi Leslie,

Thanks for your reply.

We have some data in MySQL database and some PDF documents. We want to index them and let our user to use. So we decide to use the ElasticSearch. Before using the ElsaticSearch, we want to know the performance of the ElasticSearch, so we want to do benchmark of the ElasticSearch. However, we don't know how to do. I'm not sure if I express my question clearly.

Best,

Jianjun

On Thursday, April 10, 2014 9:11:37 AM UTC-4, Leslie Hawthorn wrote:

Hi Jianjun,

On Thu, Apr 10, 2014 at 3:02 PM, Jianjun Hu <4nex...@gmail.com> wrote:

Hi all,

How to do benchmark of the ElasticSearch? Thanks!

The answer to this question is "it depends." It would be very helpful to describe a bit more about what you are working on so people can give a better answer to you. :slight_smile:

Best,

LH

--

Leslie Hawthorn

Community Manager

http://elasticsearch.com

Other Places to Find Me:

Freenode: lh

Twitter: @lhawthorn

Skype: mebelh

Voice: +31 20 794 7300

--

You received this message because you are subscribed to the Google Groups "elasticsearch" group.

To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/b6aa8d62-842a-4d2e-b184-0ef40530d47e%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Leslie Hawthorn

Community Manager

http://elasticsearch.com

Other Places to Find Me:

Freenode: lh

Twitter: @lhawthorn

Skype: mebelh

Voice: +31 20 794 7300

--

You received this message because you are subscribed to the Google Groups "elasticsearch" group.

To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAB5RFS-_2SRbUkc_yVU4W_fPPq4efXPKnB%2Bnmh%2B2cveKoq_BVQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "elasticsearch" group.

To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d8f2dd67-c113-4965-8f4c-7002c2053a1d%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.


(Jianjun Hu) #9

Hi Rafał,

Can't the JMeter be used to test the indexing performance?

Can I adding replicas and re-sharding on my virtual machine? The OS of my
PC is Windows 7. I use VirtualBox Debian for testing ElasticSearch.

Which tool is suitable for monitoring the cluster, OS and the garbage?

Regards,

Jianjun

On Thursday, April 10, 2014 10:57:42 AM UTC-4, Rafał Kuć wrote:

Hello!

Depends on what you want to test. If you want to test indexing - start
from scratch, see how many documents you can index with the clean
Elasticsearch cluster, when you start to see slowdowns. Try seeing what the
bottleneck is if any. In general, such tests depend on how much data you
have and how much indexing you are planning to do.

If you want to test querying performance - start with some setup that you
assume should be OK. Index your data and start testing, for example by
sending production like queries with JMeter. That will give you information
on how the cluster behaves in such setup. Then you can change the
deployment and tune. For example adding replicas, re-sharding and having
more/less shards, depending on the data and its volume.

During all tests, you should be monitoring your cluster. You can use
anything available, although for performance testing anything will be good,
even the command line tools that will help you understand what is happening
on the operating system level, how garbage collector is performing.

*-- Regards, Rafał Kuć Performance Monitoring * Log Analytics * Search
Analytics Solr & Elasticsearch Support * *http://sematext.com/

Could you please give me some more detailed suggestions on how to test
the elasticsearch? thanks.

On Thursday, April 10, 2014 10:13:49 AM UTC-4, Mark Walkom wrote:
The best way to know is to test it yourself.
It's very dependant on your hardware, your settings and the data that you
are indexing.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: ma...@campaignmonitor.com
web: www.campaignmonitor.com

On 10 April 2014 23:49, Leslie Hawthorn leslie....@elasticsearch.com
wrote:
Hi Jianjun,

Thank you for the additional details. I think we will need some more
information but I will let someone who is better skilled at benchmarking
Elasticsearch ask you about it.

Welcome to the mailing list! We are happy to have you here and to help.

Cheers,
LH

On Thu, Apr 10, 2014 at 3:42 PM, Jianjun Hu 4nex...@gmail.com wrote:
Hi Leslie,

Thanks for your reply.

We have some data in MySQL database and some PDF documents. We want to
index them and let our user to use. So we decide to use the ElasticSearch.
Before using the ElsaticSearch, we want to know the performance of the
ElasticSearch, so we want to do benchmark of the ElasticSearch. However, we
don't know how to do. I'm not sure if I express my question clearly.

Best,
Jianjun

On Thursday, April 10, 2014 9:11:37 AM UTC-4, Leslie Hawthorn wrote:
Hi Jianjun,

On Thu, Apr 10, 2014 at 3:02 PM, Jianjun Hu 4nex...@gmail.com wrote:
Hi all,

How to do benchmark of the ElasticSearch? Thanks!

The answer to this question is "it depends." It would be very helpful to
describe a bit more about what you are working on so people can give a
better answer to you. :slight_smile:

Best,
LH

--
Leslie Hawthorn
Community Manager
http://elasticsearch.com

Other Places to Find Me:
Freenode: lh
Twitter: @lhawthorn
Skype: mebelh
Voice: +31 20 794 7300

You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/b6aa8d62-842a-4d2e-b184-0ef40530d47e%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/b6aa8d62-842a-4d2e-b184-0ef40530d47e%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Leslie Hawthorn
Community Manager
http://elasticsearch.com

Other Places to Find Me:
Freenode: lh
Twitter: @lhawthorn
Skype: mebelh
Voice: +31 20 794 7300

You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAB5RFS-_2SRbUkc_yVU4W_fPPq4efXPKnB%2Bnmh%2B2cveKoq_BVQ%40mail.gmail.comhttps://groups.google.com/d/msgid/elasticsearch/CAB5RFS-_2SRbUkc_yVU4W_fPPq4efXPKnB%2Bnmh%2B2cveKoq_BVQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/d8f2dd67-c113-4965-8f4c-7002c2053a1d%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/d8f2dd67-c113-4965-8f4c-7002c2053a1d%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/91d41ad8-b322-46a3-b027-7efe2de7d60e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Jianjun Hu) #10

I want to use some third party tools to test the indexing/querying
performance of the ElasticSearch. I think maybe it is easy.

Regards,

Jianjun

On Thursday, April 10, 2014 10:13:49 AM UTC-4, Mark Walkom wrote:

The best way to know is to test it yourself.
It's very dependant on your hardware, your settings and the data that you
are indexing.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: ma...@campaignmonitor.com <javascript:>
web: www.campaignmonitor.com

On 10 April 2014 23:49, Leslie Hawthorn <leslie....@elasticsearch.com<javascript:>

wrote:

Hi Jianjun,

Thank you for the additional details. I think we will need some more
information but I will let someone who is better skilled at benchmarking
Elasticsearch ask you about it.

Welcome to the mailing list! We are happy to have you here and to help.

Cheers,
LH

On Thu, Apr 10, 2014 at 3:42 PM, Jianjun Hu <4nex...@gmail.com<javascript:>

wrote:

Hi Leslie,

Thanks for your reply.

We have some data in MySQL database and some PDF documents. We want to
index them and let our user to use. So we decide to use the ElasticSearch.
Before using the ElsaticSearch, we want to know the performance of the
ElasticSearch, so we want to do benchmark of the ElasticSearch. However, we
don't know how to do. I'm not sure if I express my question clearly.

Best,
Jianjun

On Thursday, April 10, 2014 9:11:37 AM UTC-4, Leslie Hawthorn wrote:

Hi Jianjun,

On Thu, Apr 10, 2014 at 3:02 PM, Jianjun Hu 4nex...@gmail.com wrote:

Hi all,

How to do benchmark of the ElasticSearch? Thanks!

The answer to this question is "it depends." It would be very helpful
to describe a bit more about what you are working on so people can give a
better answer to you. :slight_smile:

Best,
LH

--
Leslie Hawthorn
Community Manager
http://elasticsearch.com

Other Places to Find Me:
Freenode: lh
Twitter: @lhawthorn
Skype: mebelh
Voice: +31 20 794 7300

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/b6aa8d62-842a-4d2e-b184-0ef40530d47e%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/b6aa8d62-842a-4d2e-b184-0ef40530d47e%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Leslie Hawthorn
Community Manager
http://elasticsearch.com

Other Places to Find Me:
Freenode: lh
Twitter: @lhawthorn
Skype: mebelh
Voice: +31 20 794 7300

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAB5RFS-_2SRbUkc_yVU4W_fPPq4efXPKnB%2Bnmh%2B2cveKoq_BVQ%40mail.gmail.comhttps://groups.google.com/d/msgid/elasticsearch/CAB5RFS-_2SRbUkc_yVU4W_fPPq4efXPKnB%2Bnmh%2B2cveKoq_BVQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/3fca7764-0bde-4503-9776-fe15315ad2bd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Nik Everett) #11

I benchmark Elasticsearch and my application together by replaying searches
from my old search system against the one I'm deploying. I've benchmarked
indexing in the past just by jamming data in really fast. My experience
has been that if I find Elasticsearch is slow it is because I need to tweek
some setting or change the instructions I send to it.

Example 1:
I sped up rebuilding my index from scratch by 2.5x by turning off replicas
while rebuilding the index and setting the refresh interval to -1 during
the process. Adding the replicas later is much faster and causes much less
load on the servers.

Example 2:
The plain highlighter is very slow at fuzzy queries so I don't use it for
them.

Example 3:
The phrase suggester was slow for large queries if I set the maximum number
of errors to correct to a big number (5). When I set it to a small number
(2) it was much faster.

Stuff like that.

Nik

On Fri, Apr 11, 2014 at 8:54 AM, Jianjun Hu 4nexushu@gmail.com wrote:

Hi Rafał,

Can't the JMeter be used to test the indexing performance?

Can I adding replicas and re-sharding on my virtual machine? The OS of my
PC is Windows 7. I use VirtualBox Debian for testing ElasticSearch.

Which tool is suitable for monitoring the cluster, OS and the garbage?

Regards,

Jianjun

On Thursday, April 10, 2014 10:57:42 AM UTC-4, Rafał Kuć wrote:

Hello!

Depends on what you want to test. If you want to test indexing - start
from scratch, see how many documents you can index with the clean
Elasticsearch cluster, when you start to see slowdowns. Try seeing what the
bottleneck is if any. In general, such tests depend on how much data you
have and how much indexing you are planning to do.

If you want to test querying performance - start with some setup that you
assume should be OK. Index your data and start testing, for example by
sending production like queries with JMeter. That will give you information
on how the cluster behaves in such setup. Then you can change the
deployment and tune. For example adding replicas, re-sharding and having
more/less shards, depending on the data and its volume.

During all tests, you should be monitoring your cluster. You can use
anything available, although for performance testing anything will be good,
even the command line tools that will help you understand what is happening
on the operating system level, how garbage collector is performing.

*-- Regards, Rafał Kuć Performance Monitoring * Log Analytics * Search
Analytics Solr & Elasticsearch Support * *http://sematext.com/

Could you please give me some more detailed suggestions on how to test
the elasticsearch? thanks.

On Thursday, April 10, 2014 10:13:49 AM UTC-4, Mark Walkom wrote:
The best way to know is to test it yourself.
It's very dependant on your hardware, your settings and the data that you
are indexing.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: ma...@campaignmonitor.com
web: www.campaignmonitor.com

On 10 April 2014 23:49, Leslie Hawthorn leslie....@elasticsearch.com
wrote:
Hi Jianjun,

Thank you for the additional details. I think we will need some more
information but I will let someone who is better skilled at benchmarking
Elasticsearch ask you about it.

Welcome to the mailing list! We are happy to have you here and to help.

Cheers,
LH

On Thu, Apr 10, 2014 at 3:42 PM, Jianjun Hu 4nex...@gmail.com wrote:
Hi Leslie,

Thanks for your reply.

We have some data in MySQL database and some PDF documents. We want to
index them and let our user to use. So we decide to use the ElasticSearch.
Before using the ElsaticSearch, we want to know the performance of the
ElasticSearch, so we want to do benchmark of the ElasticSearch. However, we
don't know how to do. I'm not sure if I express my question clearly.

Best,
Jianjun

On Thursday, April 10, 2014 9:11:37 AM UTC-4, Leslie Hawthorn wrote:
Hi Jianjun,

On Thu, Apr 10, 2014 at 3:02 PM, Jianjun Hu 4nex...@gmail.com wrote:
Hi all,

How to do benchmark of the ElasticSearch? Thanks!

The answer to this question is "it depends." It would be very helpful to
describe a bit more about what you are working on so people can give a
better answer to you. :slight_smile:

Best,
LH

--
Leslie Hawthorn
Community Manager
http://elasticsearch.com

Other Places to Find Me:
Freenode: lh
Twitter: @lhawthorn
Skype: mebelh
Voice: +31 20 794 7300

You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/b6aa8d62-842a-4d2e-b184-0ef40530d47e%
40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/b6aa8d62-842a-4d2e-b184-0ef40530d47e%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Leslie Hawthorn
Community Manager
http://elasticsearch.com

Other Places to Find Me:
Freenode: lh
Twitter: @lhawthorn
Skype: mebelh
Voice: +31 20 794 7300

You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/CAB5RFS-_2SRbUkc_yVU4W_fPPq4efXPKnB%
2Bnmh%2B2cveKoq_BVQ%40mail.gmail.comhttps://groups.google.com/d/msgid/elasticsearch/CAB5RFS-_2SRbUkc_yVU4W_fPPq4efXPKnB%2Bnmh%2B2cveKoq_BVQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com
.

To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/d8f2dd67-
c113-4965-8f4c-7002c2053a1d%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/d8f2dd67-c113-4965-8f4c-7002c2053a1d%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/91d41ad8-b322-46a3-b027-7efe2de7d60e%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/91d41ad8-b322-46a3-b027-7efe2de7d60e%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPmjWd3FX_OjK%2Bwn%3DQ9fn%2BvpBR4bAg_QGhGf%2BVjfzyEmfJ3EaA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(system) #12