Leave content in mySQL and use ElasticSearch only for Index

Elastic Stack Elasticsearch
1

Hello,

I want to use Elasticsearch or only indexing and searching E-Mails. We want
to store the meta-info within Elasticsearch, keeping the content/body of
every Mail in an mySQL database. So Elasticsearch shall have a reference to
the mail body.

Is that possible and how?

Regards
Michael

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/928f30a8-31fd-456c-a527-e2e51ebd69f4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

Have a look at the JDBC plugin. With that plugin, you can push metadata
from MySQL to Elasticsearch.

Jörg

On Wed, Aug 6, 2014 at 1:21 PM, aseknoppik@gmail.com wrote:

Hello,

I want to use Elasticsearch or only indexing and searching E-Mails. We
want to store the meta-info within Elasticsearch, keeping the content/body
of every Mail in an mySQL database. So Elasticsearch shall have a reference
to the mail body.

Is that possible and how?

Regards
Michael

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/928f30a8-31fd-456c-a527-e2e51ebd69f4%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/928f30a8-31fd-456c-a527-e2e51ebd69f4%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGP6w%2BxS%2BQLoBR9r2q6tz_iz1rez1zAE%2B3j56WgFnSTKQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

3

Using this plugin would lead to a migration from mysql data into
Elasticsearch.

So let me reformulate my question:

My infrastructure is like this:

client---->Elasticsearch
|
|
-------->mySQL

So I have a client which generates an index and some metadata for a
mail(header and body). The mail is stored in mySQL. And the client-side
generated index and metadata is stored in Elasticsearch.

The reason is because I have > 1 TB of mail content every day. This content
shall still be written to mySQL. Elasticsearch shall keep only the index.
Is that possible? And how?

Regards
Michael

Am Mittwoch, 6. August 2014 13:21:09 UTC+2 schrieb asekn...@gmail.com:

Hello,

I want to use Elasticsearch or only indexing and searching E-Mails. We
want to store the meta-info within Elasticsearch, keeping the content/body
of every Mail in an mySQL database. So Elasticsearch shall have a reference
to the mail body.

Is that possible and how?

Regards
Michael

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/bb04102a-9aca-469d-8145-d5e644481d39%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

4

What I don't understand is why you generate an index and want to store it
in elasticsearch. You could use the plugin as Jörg suggested, transfer you
data to elasticsearch, set index:true for the fields you want and set
store:false in the mapping. This way you get an index build by
elasticsearch, can search on it, get the id as result and the data is not
stored (except metadata, if you set it to be stored). See
Elasticsearch Platform — Find real-time answers at scale | Elastic.

Cheers,
Andrej

Am Mittwoch, 6. August 2014 15:34:11 UTC+2 schrieb asekn...@gmail.com:

Using this plugin would lead to a migration from mysql data into
Elasticsearch.

So let me reformulate my question:

My infrastructure is like this:

client---->Elasticsearch
|
|
-------->mySQL

So I have a client which generates an index and some metadata for a
mail(header and body). The mail is stored in mySQL. And the client-side
generated index and metadata is stored in Elasticsearch.

The reason is because I have > 1 TB of mail content every day. This
content shall still be written to mySQL. Elasticsearch shall keep only the
index. Is that possible? And how?

Regards
Michael

Am Mittwoch, 6. August 2014 13:21:09 UTC+2 schrieb asekn...@gmail.com:

Hello,

I want to use Elasticsearch or only indexing and searching E-Mails. We
want to store the meta-info within Elasticsearch, keeping the content/body
of every Mail in an mySQL database. So Elasticsearch shall have a reference
to the mail body.

Is that possible and how?

Regards
Michael

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d1ffc059-abad-4b11-8179-35ed3c077cbf%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

5

JDBC plugin is not for migration.

It can be configured to select the data from the RDBMS you want. You can
fetch the metadata fields and index them into Elasticsearch with a simple
SQL select statement.

Jörg

On Wed, Aug 6, 2014 at 3:48 PM, Andrej Rosenheinrich <
andrej.rosenheinrich@unister.de> wrote:

What I don't understand is why you generate an index and want to store it
in elasticsearch. You could use the plugin as Jörg suggested, transfer you
data to elasticsearch, set index:true for the fields you want and set
store:false in the mapping. This way you get an index build by
elasticsearch, can search on it, get the id as result and the data is not
stored (except metadata, if you set it to be stored). See
Elasticsearch Platform — Find real-time answers at scale | Elastic
.

Cheers,
Andrej

Am Mittwoch, 6. August 2014 15:34:11 UTC+2 schrieb asekn...@gmail.com:

Using this plugin would lead to a migration from mysql data into
Elasticsearch.

So let me reformulate my question:

My infrastructure is like this:

client---->Elasticsearch
|
|
-------->mySQL

So I have a client which generates an index and some metadata for a
mail(header and body). The mail is stored in mySQL. And the client-side
generated index and metadata is stored in Elasticsearch.

The reason is because I have > 1 TB of mail content every day. This
content shall still be written to mySQL. Elasticsearch shall keep only the
index. Is that possible? And how?

Regards
Michael

Am Mittwoch, 6. August 2014 13:21:09 UTC+2 schrieb asekn...@gmail.com:

Hello,

I want to use Elasticsearch or only indexing and searching E-Mails. We
want to store the meta-info within Elasticsearch, keeping the content/body
of every Mail in an mySQL database. So Elasticsearch shall have a reference
to the mail body.

Is that possible and how?

Regards
Michael

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/d1ffc059-abad-4b11-8179-35ed3c077cbf%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/d1ffc059-abad-4b11-8179-35ed3c077cbf%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFRoWCW1ea-ZXr%2B9VfLzFDSvtenP8xHX35Tyi6Q6zDwSw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.