Best practice architecture using ES

Elastic Stack Elasticsearch
1

I have simple web app containing the client side(Angular JS) and the server
layer(Spring,Spring MVC)

I am taking the simplest case of searching when a user have single search
input. behind the scenes I assume the JS will be send search request to the
server layers and wait for response.

I am interested how do you suggest to structure the layers on the server
end (containing the ES requests) using Controllers, DTO'S, DAO'S,
Async/sync requests, etc....

Any simple examples warmly welcomed.

Thanks,
ray.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/4ac48b59-3bbc-4b11-8c4b-847668bd6c47%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

Hi,

ill try to focus my question. Maybe that will gain any replies.

I am looking for a simple java application which using ES. I want to learn
best practices of the layers structured. DTO's mapping/services/json/dto
mappings..... I just wanna create(right) the logic layer between the client
and the ES server

Any simple project to look at?

Thanks.

On Sunday, July 13, 2014 12:04:27 PM UTC+3, rayman wrote:

I have simple web app containing the client side(Angular JS) and the
server layer(Spring,Spring MVC)

I am taking the simplest case of searching when a user have single search
input. behind the scenes I assume the JS will be send search request to the
server layers and wait for response.

I am interested how do you suggest to structure the layers on the server
end (containing the ES requests) using Controllers, DTO'S, DAO'S,
Async/sync requests, etc....

Any simple examples warmly welcomed.

Thanks,
ray.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/4ab9ee9f-90d9-4c85-9fdd-2bc67ce5ccb5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

3

I'm afraid that's still too broad a question

The communication between your SPA and Controller is up to you to decide.
Then you can structure a query manually quite easily using the QueryBuilder
Java classes from the elasticsearch jar. Mappings will depends on your data
structure and query types.

One advice I could tell you is to try and avoid introducing too much
friction, like duplicating the model too many times (DTO, DAO etc). If you
can use the same structure for display as you use for indexing in ES, use
that.

HTH

--

Itamar Syn-Hershko
http://code972.com | @synhershko https://twitter.com/synhershko
Freelance Developer & Consultant
Author of RavenDB in Action http://manning.com/synhershko/

On Mon, Jul 14, 2014 at 5:20 PM, rayman idan.frid@gmail.com wrote:

Hi,

ill try to focus my question. Maybe that will gain any replies.

I am looking for a simple java application which using ES. I want to learn
best practices of the layers structured. DTO's mapping/services/json/dto
mappings..... I just wanna create(right) the logic layer between the client
and the ES server

Any simple project to look at?

Thanks.

On Sunday, July 13, 2014 12:04:27 PM UTC+3, rayman wrote:

I have simple web app containing the client side(Angular JS) and the
server layer(Spring,Spring MVC)

I am taking the simplest case of searching when a user have single search
input. behind the scenes I assume the JS will be send search request to the
server layers and wait for response.

I am interested how do you suggest to structure the layers on the server
end (containing the ES requests) using Controllers, DTO'S, DAO'S,
Async/sync requests, etc....

Any simple examples warmly welcomed.

Thanks,
ray.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/4ab9ee9f-90d9-4c85-9fdd-2bc67ce5ccb5%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/4ab9ee9f-90d9-4c85-9fdd-2bc67ce5ccb5%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAHTr4Zs74rcLfQxTwDTeafoLGoQ2__%2BmP_-2oYYT1_gKMqy_7A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

4

Ray

Kind of a broad question. I assume you want to provide search to an
existing Web app with MVC backend and Angular front-end.

Just my 2c - but we are developing in Meteor and node js - and I would
suggest shaking free of your MVC mind-set.

I would have the data model push updates/inserts to the ES index using the
API and have angular js issue client side search requests to Elastic Search

  • this is the architecture Kibana uses - where the client talks directly to
    ES.

HTH
Danny

On Sunday, July 13, 2014 12:04:27 PM UTC+3, rayman wrote:

I have simple web app containing the client side(Angular JS) and the
server layer(Spring,Spring MVC)

I am taking the simplest case of searching when a user have single search
input. behind the scenes I assume the JS will be send search request to the
server layers and wait for response.

I am interested how do you suggest to structure the layers on the server
end (containing the ES requests) using Controllers, DTO'S, DAO'S,
Async/sync requests, etc....

Any simple examples warmly welcomed.

Thanks,
ray.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/7e93b884-0fc0-40f6-816c-e417d3e88cb2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

5

I would strongly suggest against that. Never expose ES to the public,
always put it behind a server facade. To get a glimpse of what you are
exposing yourself, see this recent blog post

--

Itamar Syn-Hershko
http://code972.com | @synhershko https://twitter.com/synhershko
Freelance Developer & Consultant
Author of RavenDB in Action http://manning.com/synhershko/

On Mon, Jul 14, 2014 at 6:07 PM, Danny Lieberman dl@software.co.il wrote:

Ray

Kind of a broad question. I assume you want to provide search to an
existing Web app with MVC backend and Angular front-end.

Just my 2c - but we are developing in Meteor and node js - and I would
suggest shaking free of your MVC mind-set.

I would have the data model push updates/inserts to the ES index using the
API and have angular js issue client side search requests to Elastic Search

  • this is the architecture Kibana uses - where the client talks directly to
    ES.

HTH
Danny

On Sunday, July 13, 2014 12:04:27 PM UTC+3, rayman wrote:

I have simple web app containing the client side(Angular JS) and the
server layer(Spring,Spring MVC)

I am taking the simplest case of searching when a user have single search
input. behind the scenes I assume the JS will be send search request to the
server layers and wait for response.

I am interested how do you suggest to structure the layers on the server
end (containing the ES requests) using Controllers, DTO'S, DAO'S,
Async/sync requests, etc....

Any simple examples warmly welcomed.

Thanks,
ray.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/7e93b884-0fc0-40f6-816c-e417d3e88cb2%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/7e93b884-0fc0-40f6-816c-e417d3e88cb2%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZuUnkHJi%3D6AoYtXVo5KOJ8hvE_9%2B9H7-_PTa2GX%2BWXp8w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

6

Itamar

You are 100% right!

Of course - one would not open your ES server to public but put it behind a
firewall or proxy like nginx.

That is a Very Bad Idea.

I was (perish the thought!!) not suggesting that.

I was suggesting that the approach of client code talking directly to a
firewalled/proxied search service is more productive than the client
sending an HTTP request to a server side View, that routes it it to a
server side Controller that accesses a Model and then bubbles back
responses to the Angular JS client.

Danny

On Sunday, July 13, 2014 12:04:27 PM UTC+3, rayman wrote:

I have simple web app containing the client side(Angular JS) and the
server layer(Spring,Spring MVC)

I am taking the simplest case of searching when a user have single search
input. behind the scenes I assume the JS will be send search request to the
server layers and wait for response.

I am interested how do you suggest to structure the layers on the server
end (containing the ES requests) using Controllers, DTO'S, DAO'S,
Async/sync requests, etc....

Any simple examples warmly welcomed.

Thanks,
ray.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/c6ad21af-9a31-4fcc-95e8-320555ef7c85%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

7

Except from the fact those search commands will still not be verified and
could compromise your system (by executing PUT and DELETE verbs or
scripts), or simply leak information

In most scenarios, you should definitely not just proxy client requests but
verify them or build them from scratch in your server side facade

--

Itamar Syn-Hershko
http://code972.com | @synhershko https://twitter.com/synhershko
Freelance Developer & Consultant
Author of RavenDB in Action http://manning.com/synhershko/

On Mon, Jul 14, 2014 at 6:21 PM, Danny Lieberman dl@software.co.il wrote:

Itamar

You are 100% right!

Of course - one would not open your ES server to public but put it behind
a firewall or proxy like nginx.

That is a Very Bad Idea.

I was (perish the thought!!) not suggesting that.

I was suggesting that the approach of client code talking directly to a
firewalled/proxied search service is more productive than the client
sending an HTTP request to a server side View, that routes it it to a
server side Controller that accesses a Model and then bubbles back
responses to the Angular JS client.

Danny

On Sunday, July 13, 2014 12:04:27 PM UTC+3, rayman wrote:

I have simple web app containing the client side(Angular JS) and the
server layer(Spring,Spring MVC)

I am taking the simplest case of searching when a user have single search
input. behind the scenes I assume the JS will be send search request to the
server layers and wait for response.

I am interested how do you suggest to structure the layers on the server
end (containing the ES requests) using Controllers, DTO'S, DAO'S,
Async/sync requests, etc....

Any simple examples warmly welcomed.

Thanks,
ray.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/c6ad21af-9a31-4fcc-95e8-320555ef7c85%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/c6ad21af-9a31-4fcc-95e8-320555ef7c85%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZtAYTnZGUJZ81-VsyWTNJbYwTPA5QeQ_UmJ_Tt6jpEjEQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

8

@Itamar,

"..like duplicating the model too many times "

I am talking about best practice layers that in the future will help me to
maintenance the code in case of changing.. (pretty similar to GenericDAO
that we used to have with Hibernate..).

take the most simple scenario when I am creating ESService which Indexing
and querying the ES nodes.

So I thought about this:

Layer 1: Spring MVC Controller(Will "speak" with angular/JS/any other rest
requests)
Layer 2: ESService(Holding ESDao, Do some json manipulations, etc.. before
serving back to the controller)
Layer 3: ESDao to the actually executions and retrieve the raw results from
ES

"use the same structure for display as you use for indexing in ES"

What do you think? Any thing I should cut out?

@Danny,

I was suggesting that the approach of client code talking directly to a
firewalled/proxied search service is more productive than the client
sending an HTTP request to a server side

If I go this way than the logic will be duplicated within each client. what
about decoupling ??

Thank for your response,
ray.

On Sunday, July 13, 2014 12:04:27 PM UTC+3, rayman wrote:

I have simple web app containing the client side(Angular JS) and the
server layer(Spring,Spring MVC)

I am taking the simplest case of searching when a user have single search
input. behind the scenes I assume the JS will be send search request to the
server layers and wait for response.

I am interested how do you suggest to structure the layers on the server
end (containing the ES requests) using Controllers, DTO'S, DAO'S,
Async/sync requests, etc....

Any simple examples warmly welcomed.

Thanks,
ray.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/73d391bc-b6b0-4768-a819-e80757f86b77%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

9

Ray -

Specifically relating to the search aspect not to business logic between
the client and server.

There is no added value having a search box and sending a search request to
server layers and waiting for a response when the client can send the query
directly to the ES server and get a request directly (The ES server sits
behind a reverse proxy like nginx -
see GitHub - sous-chefs/elasticsearch: Development repository for the elasticsearch cookbook and you have
configured appropriate controls like acl, ssl as required for your web app.

Using the ES API your single search box is trivial to implement, faster,
more robust and easier to debug and you save all the fooling around on the
server side

The back end Model/controller would update the ES index whenever a data
entity was inserted/updated.

HTH
Danny
PS - I'm amused that this thread is all Israeli :wink:

On Sunday, July 13, 2014 12:04:27 PM UTC+3, rayman wrote:

I have simple web app containing the client side(Angular JS) and the
server layer(Spring,Spring MVC)

I am taking the simplest case of searching when a user have single search
input. behind the scenes I assume the JS will be send search request to the
server layers and wait for response.

I am interested how do you suggest to structure the layers on the server
end (containing the ES requests) using Controllers, DTO'S, DAO'S,
Async/sync requests, etc....

Any simple examples warmly welcomed.

Thanks,
ray.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/71395ec1-63ad-4b45-bac6-5078d95b53b5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

10

On Mon, Jul 14, 2014 at 4:25 PM, Danny Lieberman dl@software.co.il wrote:

There is no added value having a search box and sending a search request
to server layers and waiting for a response when the client can send the
query directly to the ES server and get a request directly (The ES server
sits behind a reverse proxy like nginx - see
GitHub - sous-chefs/elasticsearch: Development repository for the elasticsearch cookbook and you have
configured appropriate controls like acl, ssl as required for your web app.

I wouldn't trust a reverse proxy to block requests for super expensive
operations query operations like using the plain highlighter on fuzzy
matches. I'm in the keep your application between Elasticsearch and your
users camp.

Nik

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPmjWd30BBNutOaqFucrR4TcxA0Y1SX%2BGfAXU37fn898QDbQ9A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

11

Nik

Good discussion - excellent point.

Maybe I'm missing something but it seems to me this is why Elasticsearch
has paging and why Ray is running Angular JS in the client so that he can
enforce some business rules to throttle the amount of data requested and
returned by a search and paginate the search results.
Elasticsearch Platform — Find real-time answers at scale | Elastic

No ?

Danny

On Sunday, July 13, 2014 12:04:27 PM UTC+3, rayman wrote:

I have simple web app containing the client side(Angular JS) and the
server layer(Spring,Spring MVC)

I am taking the simplest case of searching when a user have single search
input. behind the scenes I assume the JS will be send search request to the
server layers and wait for response.

I am interested how do you suggest to structure the layers on the server
end (containing the ES requests) using Controllers, DTO'S, DAO'S,
Async/sync requests, etc....

Any simple examples warmly welcomed.

Thanks,
ray.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/dc91afcf-206a-4f2e-aa8e-c2b931177114%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

12

I'm looking at this from the perspective of a script kiddy trying to make
your life hard - if you open up Elasticsearch someone will notice that
queries are going through and then start hammering you with requests for
pages of ten billion records each. Or something. I'm just saying that
trying to filter out the expensive operations from a proxy is going to
ultimately leave something exposed accidentally. Better to have the
application in the way. I suppose a simple option is to dig out some json
validation stuff and make it super tight. That might be enough.

On Mon, Jul 14, 2014 at 4:44 PM, Danny Lieberman dl@software.co.il wrote:

Nik

Good discussion - excellent point.

Maybe I'm missing something but it seems to me this is why Elasticsearch
has paging and why Ray is running Angular JS in the client so that he can
enforce some business rules to throttle the amount of data requested and
returned by a search and paginate the search results.
Elasticsearch Platform — Find real-time answers at scale | Elastic

No ?

Danny

On Sunday, July 13, 2014 12:04:27 PM UTC+3, rayman wrote:

I have simple web app containing the client side(Angular JS) and the
server layer(Spring,Spring MVC)

I am taking the simplest case of searching when a user have single search
input. behind the scenes I assume the JS will be send search request to the
server layers and wait for response.

I am interested how do you suggest to structure the layers on the server
end (containing the ES requests) using Controllers, DTO'S, DAO'S,
Async/sync requests, etc....

Any simple examples warmly welcomed.

Thanks,
ray.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/dc91afcf-206a-4f2e-aa8e-c2b931177114%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/dc91afcf-206a-4f2e-aa8e-c2b931177114%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPmjWd38bkGF8yDcw3J-S1_UGUeP%3DuJUc-bMmDv-D7at1dMcUw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

13

@Danny,
Nope. Well maybe that could be a future reason. But I was looking for best
practice implementation.

thinking:

  1. Should we have middle tier between ES and the client.

2.(Which is more important) how did you structure it(in case you did). That
way ill be able to learn from you, experienced es users who already built
es projects on production.

P.S this thread is no longer only israeli;P

2014-07-14 23:44 GMT+03:00 Danny Lieberman dl@software.co.il:

Nik

Good discussion - excellent point.

Maybe I'm missing something but it seems to me this is why Elasticsearch
has paging and why Ray is running Angular JS in the client so that he can
enforce some business rules to throttle the amount of data requested and
returned by a search and paginate the search results.
Elasticsearch Platform — Find real-time answers at scale | Elastic

No ?

Danny

On Sunday, July 13, 2014 12:04:27 PM UTC+3, rayman wrote:

I have simple web app containing the client side(Angular JS) and the
server layer(Spring,Spring MVC)

I am taking the simplest case of searching when a user have single search
input. behind the scenes I assume the JS will be send search request to the
server layers and wait for response.

I am interested how do you suggest to structure the layers on the server
end (containing the ES requests) using Controllers, DTO'S, DAO'S,
Async/sync requests, etc....

Any simple examples warmly welcomed.

Thanks,
ray.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/PUs1Ma6a8c4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/dc91afcf-206a-4f2e-aa8e-c2b931177114%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/dc91afcf-206a-4f2e-aa8e-c2b931177114%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAK1_pJAgYftP%2Bhe9_HAaHt2tzND4b17sY2yiZcDsX02fg5MBCA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

14

Nikolas, How did you structure your middle tier?(Java? Node? some details
on the layers inside..)

2014-07-14 23:48 GMT+03:00 Nikolas Everett nik9000@gmail.com:

I'm looking at this from the perspective of a script kiddy trying to make
your life hard - if you open up Elasticsearch someone will notice that
queries are going through and then start hammering you with requests for
pages of ten billion records each. Or something. I'm just saying that
trying to filter out the expensive operations from a proxy is going to
ultimately leave something exposed accidentally. Better to have the
application in the way. I suppose a simple option is to dig out some json
validation stuff and make it super tight. That might be enough.

On Mon, Jul 14, 2014 at 4:44 PM, Danny Lieberman dl@software.co.il
wrote:

Nik

Good discussion - excellent point.

Maybe I'm missing something but it seems to me this is why Elasticsearch
has paging and why Ray is running Angular JS in the client so that he can
enforce some business rules to throttle the amount of data requested and
returned by a search and paginate the search results.
Elasticsearch Platform — Find real-time answers at scale | Elastic

No ?

Danny

On Sunday, July 13, 2014 12:04:27 PM UTC+3, rayman wrote:

I have simple web app containing the client side(Angular JS) and the
server layer(Spring,Spring MVC)

I am taking the simplest case of searching when a user have single
search input. behind the scenes I assume the JS will be send search request
to the server layers and wait for response.

I am interested how do you suggest to structure the layers on the server
end (containing the ES requests) using Controllers, DTO'S, DAO'S,
Async/sync requests, etc....

Any simple examples warmly welcomed.

Thanks,
ray.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/dc91afcf-206a-4f2e-aa8e-c2b931177114%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/dc91afcf-206a-4f2e-aa8e-c2b931177114%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/PUs1Ma6a8c4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAPmjWd38bkGF8yDcw3J-S1_UGUeP%3DuJUc-bMmDv-D7at1dMcUw%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAPmjWd38bkGF8yDcw3J-S1_UGUeP%3DuJUc-bMmDv-D7at1dMcUw%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAK1_pJCj0AsFzbewke7BxK1paT%3DzpH9emG%2BkxGAqYWXdNFQqxQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

15

You could look here:
http://git.wikimedia.org/tree/mediawiki%2Fextensions%2FCirrusSearch.git/master/
Its a long way from perfect but it is open source. Its PHP so the general
organization is a bit different then you'd get in Java. But it does work.

Nik

On Mon, Jul 14, 2014 at 4:49 PM, Idan Fridman idan.frid@gmail.com wrote:

@Danny,
Nope. Well maybe that could be a future reason. But I was looking for best
practice implementation.

thinking:

  1. Should we have middle tier between ES and the client.

2.(Which is more important) how did you structure it(in case you did).
That way ill be able to learn from you, experienced es users who already
built es projects on production.

P.S this thread is no longer only israeli;P

2014-07-14 23:44 GMT+03:00 Danny Lieberman dl@software.co.il:

Nik

Good discussion - excellent point.

Maybe I'm missing something but it seems to me this is why Elasticsearch
has paging and why Ray is running Angular JS in the client so that he can
enforce some business rules to throttle the amount of data requested and
returned by a search and paginate the search results.
Elasticsearch Platform — Find real-time answers at scale | Elastic

No ?

Danny

On Sunday, July 13, 2014 12:04:27 PM UTC+3, rayman wrote:

I have simple web app containing the client side(Angular JS) and the
server layer(Spring,Spring MVC)

I am taking the simplest case of searching when a user have single
search input. behind the scenes I assume the JS will be send search request
to the server layers and wait for response.

I am interested how do you suggest to structure the layers on the server
end (containing the ES requests) using Controllers, DTO'S, DAO'S,
Async/sync requests, etc....

Any simple examples warmly welcomed.

Thanks,
ray.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/PUs1Ma6a8c4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/dc91afcf-206a-4f2e-aa8e-c2b931177114%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/dc91afcf-206a-4f2e-aa8e-c2b931177114%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAK1_pJAgYftP%2Bhe9_HAaHt2tzND4b17sY2yiZcDsX02fg5MBCA%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAK1_pJAgYftP%2Bhe9_HAaHt2tzND4b17sY2yiZcDsX02fg5MBCA%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPmjWd2OAwx2EncMDgmZa986ROX%3DZ6xvQ%3D_v99ebVpKVsjRmfg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

16

Nik

Excellent point.

However - consider that sending search requests to a MVC back end service
doesn't mitigate client-side Javascript denial of service threat scenario -
it just means that the attacker is now attacking the back end and not the
proxy.

Idan's MVC application is by definition more vulnerable and less robust
than nginx

nginx provides HTTP request flood protection which would mitigate your
script kiddy threat scenario.

So - if we want, we can deploy 2 security countermeasures:

  1. nginx reverse proxy with HTTP flood protection
  2. a bit of code running in node.js that takes the search request and
    sanitizes it before passing the request to the ES. This would not impose an
    additional burden on application code and would separate the duties of
    security and business logic.

Of course if your Web app is getting 5 requests/month - this might be
overkill :wink:

Your mileage will vary...

Regards
Danny

On Sunday, July 13, 2014 12:04:27 PM UTC+3, rayman wrote:

I have simple web app containing the client side(Angular JS) and the
server layer(Spring,Spring MVC)

I am taking the simplest case of searching when a user have single search
input. behind the scenes I assume the JS will be send search request to the
server layers and wait for response.

I am interested how do you suggest to structure the layers on the server
end (containing the ES requests) using Controllers, DTO'S, DAO'S,
Async/sync requests, etc....

Any simple examples warmly welcomed.

Thanks,
ray.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/cfacc329-705c-4992-bd7a-64643d9f451a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

17

Danny,

a bit of code running in node.js

That piece of code is is my java mid layer.

On Tuesday, July 15, 2014 8:37:03 AM UTC+3, Danny Lieberman wrote:

Nik

Excellent point.

However - consider that sending search requests to a MVC back end service
doesn't mitigate client-side Javascript denial of service threat scenario -
it just means that the attacker is now attacking the back end and not the
proxy.

Idan's MVC application is by definition more vulnerable and less robust
than nginx

nginx provides HTTP request flood protection which would mitigate your
script kiddy threat scenario.

So - if we want, we can deploy 2 security countermeasures:

  1. nginx reverse proxy with HTTP flood protection
  2. a bit of code running in node.js that takes the search request and
    sanitizes it before passing the request to the ES. This would not impose an
    additional burden on application code and would separate the duties of
    security and business logic.

Of course if your Web app is getting 5 requests/month - this might be
overkill :wink:

Your mileage will vary...

Regards
Danny

On Sunday, July 13, 2014 12:04:27 PM UTC+3, rayman wrote:

I have simple web app containing the client side(Angular JS) and the
server layer(Spring,Spring MVC)

I am taking the simplest case of searching when a user have single search
input. behind the scenes I assume the JS will be send search request to the
server layers and wait for response.

I am interested how do you suggest to structure the layers on the server
end (containing the ES requests) using Controllers, DTO'S, DAO'S,
Async/sync requests, etc....

Any simple examples warmly welcomed.

Thanks,
ray.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a9c0cbb1-f971-46e2-894a-3e8e19111c0a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

18

Thanks, Will do.

On Monday, July 14, 2014 11:57:14 PM UTC+3, Nikolas Everett wrote:

You could look here:
Query: Active Repositories
Its a long way from perfect but it is open source. Its PHP so the general
organization is a bit different then you'd get in Java. But it does work.

Nik

On Mon, Jul 14, 2014 at 4:49 PM, Idan Fridman <idan...@gmail.com
<javascript:>> wrote:

@Danny,
Nope. Well maybe that could be a future reason. But I was looking for
best practice implementation.

thinking:

  1. Should we have middle tier between ES and the client.

2.(Which is more important) how did you structure it(in case you did).
That way ill be able to learn from you, experienced es users who already
built es projects on production.

P.S this thread is no longer only israeli;P

2014-07-14 23:44 GMT+03:00 Danny Lieberman <d...@software.co.il
<javascript:>>:

Nik

Good discussion - excellent point.

Maybe I'm missing something but it seems to me this is why Elasticsearch
has paging and why Ray is running Angular JS in the client so that he can
enforce some business rules to throttle the amount of data requested and
returned by a search and paginate the search results.
Elasticsearch Platform — Find real-time answers at scale | Elastic

No ?

Danny

On Sunday, July 13, 2014 12:04:27 PM UTC+3, rayman wrote:

I have simple web app containing the client side(Angular JS) and the
server layer(Spring,Spring MVC)

I am taking the simplest case of searching when a user have single
search input. behind the scenes I assume the JS will be send search request
to the server layers and wait for response.

I am interested how do you suggest to structure the layers on the
server end (containing the ES requests) using Controllers, DTO'S, DAO'S,
Async/sync requests, etc....

Any simple examples warmly welcomed.

Thanks,
ray.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/PUs1Ma6a8c4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/dc91afcf-206a-4f2e-aa8e-c2b931177114%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/dc91afcf-206a-4f2e-aa8e-c2b931177114%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAK1_pJAgYftP%2Bhe9_HAaHt2tzND4b17sY2yiZcDsX02fg5MBCA%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAK1_pJAgYftP%2Bhe9_HAaHt2tzND4b17sY2yiZcDsX02fg5MBCA%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/441ccfdb-60bd-4bef-8a20-08f7ede7d8d3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

19

Thanks for your replies guys.

On Sunday, July 13, 2014 12:04:27 PM UTC+3, Idan wrote:

I have simple web app containing the client side(Angular JS) and the
server layer(Spring,Spring MVC)

I am taking the simplest case of searching when a user have single search
input. behind the scenes I assume the JS will be send search request to the
server layers and wait for response.

I am interested how do you suggest to structure the layers on the server
end (containing the ES requests) using Controllers, DTO'S, DAO'S,
Async/sync requests, etc....

Any simple examples warmly welcomed.

Thanks,
ray.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ed9e3a58-0673-482b-bca7-20a5254050b1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.