Starting with ES

evil80 · November 28, 2012, 3:17pm

hi,

I'm new to ElasticSearch I'm just trying to build a little web application
to understand how it works.
I have some questions, if somebody would like to give me an answer :

I'm using a .NET client for ES and I used it to index new object in
the ES Server, but in real life I have documents, so I have to write
specific code to wrap this document in .NET object or I can index them
automatically ?
- If I can index them automatically how can I control what fields
  will be built ?
Can someone explain me what Facets are what are their benefits ?
Imagine I start indexing two different type of document and I want to
let the user search against some field like : branch code, date, prices,
and so on. How is the best way to start building something like that ?
Can I let ES store its data in HDFS (Hadoop) and using it as the only
storage ?

I've understood it's a real powerful framework for text searching but I
need to understand how I can design my application to get good results.

Thanks a lot!

--

Atharva_Patel · November 28, 2012, 4:09pm

On Wednesday, 28 November 2012 20:47:14 UTC+5:30, evil80 wrote:

hi,

I'm new to Elasticsearch I'm just trying to build a little web application
to understand how it works.
I have some questions, if somebody would like to give me an answer :

I'm using a .NET client for ES and I used it to index new object in
the ES Server, but in real life I have documents, so I have to write
specific code to wrap this document in .NET object or I can index them
automatically ?

If I can index them automatically how can I control what fields
will be built ?

Can someone explain me what Facets are what are their benefits ?

Imagine I start indexing two different type of document and I want
to let the user search against some field like : branch code, date, prices,
and so on. How is the best way to start building something like that ?

Can I let ES store its data in HDFS (Hadoop) and using it as the
only storage ?

I've understood it's a real powerful framework for text searching but I
need to understand how I can design my application to get good results.

Thanks a lot!

--

Atharva_Patel · November 28, 2012, 4:33pm

ES has Java and Groovy API. It's also popular for it's RESTful API. So
you can use either of them to pass your documents to the ES. Fields won't
be built automatically. You need to specify them in the document object
that you construct for indexing. You can also specify the type of each
field using the mappingshttp://www.elasticsearch.org/guide/reference/api/admin-indices-put-mapping.html
and core typeshttp://www.elasticsearch.org/guide/reference/mapping/core-types.html
.
As per I can conceive from reading the documentation, Facets are great
to get aggregated data on your field values over a set of documents. This
set can be entire document set or just some of them matching with your
query.
Once you have defined your mappings for each document type you want to
store, you can make query on all those different types of documents using
search query in ES. The REST call looks something like this:
curl -XGET 'http://localhost:9200///_search' -d

You can leave the "/" in this uri if you want to search
across different types of documents in same index name.

curl -XGET 'http://localhost:9200//_search' -d
Here document type refers to differently structured documents and not
file types like .pdf, .txt etc. They can be indexed using attachment mapper
plugin and passing them in base64 encoded string form and surrounded by
quotest as field value in json.
You can store your original file's data in any File system you like.
As per general suggestions by people, Hadoop deployment should be used when
you are planning to have very large amount of data (probably in Terra
Bytes). There also scalable distributed file systems available like Gridfs
by Mongodb which you can use to store your files. Elasticsearch will take
care of storing index data itself so you don't have to worry about that.

I hope it helped. : ) Just a tip, next time try to ask each question in
separate post to get the answers quickly by participants in FOSS forums. :
) Good luck!

On Wednesday, 28 November 2012 20:47:14 UTC+5:30, evil80 wrote:

hi,

I'm new to Elasticsearch I'm just trying to build a little web application
to understand how it works.
I have some questions, if somebody would like to give me an answer :

I'm using a .NET client for ES and I used it to index new object in
the ES Server, but in real life I have documents, so I have to write
specific code to wrap this document in .NET object or I can index them
automatically ?

If I can index them automatically how can I control what fields
will be built ?

Can someone explain me what Facets are what are their benefits ?

Imagine I start indexing two different type of document and I want
to let the user search against some field like : branch code, date, prices,
and so on. How is the best way to start building something like that ?

Can I let ES store its data in HDFS (Hadoop) and using it as the
only storage ?

I've understood it's a real powerful framework for text searching but I
need to understand how I can design my application to get good results.

Thanks a lot!

--

evil80 · November 29, 2012, 10:17pm

Hi,

really thanks for your answer.
I just want to know :

So if I use a .NET client for ES I have to read the data I want to
index in a .NET object and then load i into ES with the desired fields ?
I've read that Faces allows to filter the result of a query, but in a
web scenario after I bind the first result query to a control (let say a
table) if I want to filter it with the Facets I have to re-run my query ?
What is the common architecture use ? I think it's not a good idea to store
the result of the query in the user session

I will write next questions in separate post as you suggest.

Thanks a lot again!
Il giorno mercoledì 28 novembre 2012 17:33:02 UTC+1, Atharva Patel ha
scritto:

ES has Java and Groovy API. It's also popular for it's RESTful API.
So you can use either of them to pass your documents to the ES. Fields
won't be built automatically. You need to specify them in the document
object that you construct for indexing. You can also specify the type of
each field using the mappingshttp://www.elasticsearch.org/guide/reference/api/admin-indices-put-mapping.html
and core typeshttp://www.elasticsearch.org/guide/reference/mapping/core-types.html
.

As per I can conceive from reading the documentation, Facets are
great to get aggregated data on your field values over a set of documents.
This set can be entire document set or just some of them matching with your
query.

Once you have defined your mappings for each document type you want
to store, you can make query on all those different types of documents
using search query in ES. The REST call looks something like this:

curl -XGET 'http://localhost:9200///_search'
-d

You can leave the "/" in this uri if you want to search
across different types of documents in same index name.

curl -XGET 'http://localhost:9200//_search' -d

Here document type refers to differently structured documents and
not file types like .pdf, .txt etc. They can be indexed using attachment
mapper plugin and passing them in base64 encoded string form and surrounded
by quotest as field value in json.

You can store your original file's data in any File system you like.
As per general suggestions by people, Hadoop deployment should be used when
you are planning to have very large amount of data (probably in Terra
Bytes). There also scalable distributed file systems available like Gridfs
by Mongodb which you can use to store your files. Elasticsearch will take
care of storing index data itself so you don't have to worry about that.

I hope it helped. : ) Just a tip, next time try to ask each question in
separate post to get the answers quickly by participants in FOSS forums. :
) Good luck!

On Wednesday, 28 November 2012 20:47:14 UTC+5:30, evil80 wrote:

hi,

I'm new to Elasticsearch I'm just trying to build a little web
application to understand how it works.
I have some questions, if somebody would like to give me an answer :

I'm using a .NET client for ES and I used it to index new object in
the ES Server, but in real life I have documents, so I have to write
specific code to wrap this document in .NET object or I can index them
automatically ?

If I can index them automatically how can I control what fields
will be built ?

Can someone explain me what Facets are what are their benefits ?

Imagine I start indexing two different type of document and I want
to let the user search against some field like : branch code, date, prices,
and so on. How is the best way to start building something like that ?

Can I let ES store its data in HDFS (Hadoop) and using it as the
only storage ?

I've understood it's a real powerful framework for text searching but I
need to understand how I can design my application to get good results.

Thanks a lot!

--

Atharva_Patel · November 30, 2012, 5:22am

Seems like there is no API for .Net to work with ES. But you can start
working by using REST api. So basic work-flow for you can be like this:

Look at the JSON api by ES for the task that you want to achieve with
ES like updating mapping, indexing, searching etc.
Decide the message structure that you would like to pass as JSON in
the URI requests from your code to ES.
Prepare objects in your code with fields matching with those in your
desired JSON message.
Use .Net specific Object to JSON converters and dynamically generate
JSON strings from those objects
Make appropriate HTTP request with method type set as PUT, POST, GET,
DELETE and pass the required JSON string in the request body.
You will receive HTTP response from ES. Check the error codes and
response body etc. to verify the success of your desired operation.

For getting better understanding of using Facets, better ask it as a
separate question and explain your use case with an example there! : )

Good day ! : )

On Friday, 30 November 2012 03:47:31 UTC+5:30, evil80 wrote:

Hi,

really thanks for your answer.
I just want to know :

So if I use a .NET client for ES I have to read the data I want to
index in a .NET object and then load i into ES with the desired fields ?

I've read that Faces allows to filter the result of a query, but in
a web scenario after I bind the first result query to a control (let say a
table) if I want to filter it with the Facets I have to re-run my query ?
What is the common architecture use ? I think it's not a good idea to store
the result of the query in the user session

I will write next questions in separate post as you suggest.

Thanks a lot again!
Il giorno mercoledì 28 novembre 2012 17:33:02 UTC+1, Atharva Patel ha
scritto:

ES has Java and Groovy API. It's also popular for it's RESTful API.
So you can use either of them to pass your documents to the ES. Fields
won't be built automatically. You need to specify them in the document
object that you construct for indexing. You can also specify the type of
each field using the mappingshttp://www.elasticsearch.org/guide/reference/api/admin-indices-put-mapping.html
and core typeshttp://www.elasticsearch.org/guide/reference/mapping/core-types.html
.

As per I can conceive from reading the documentation, Facets are
great to get aggregated data on your field values over a set of documents.
This set can be entire document set or just some of them matching with your
query.

Once you have defined your mappings for each document type you want
to store, you can make query on all those different types of documents
using search query in ES. The REST call looks something like this:

curl -XGET 'http://localhost:9200/
//_search' -d

You can leave the "/" in this uri if you want to search
across different types of documents in same index name.

curl -XGET 'http://localhost:9200//_search' -d

Here document type refers to differently structured documents and
not file types like .pdf, .txt etc. They can be indexed using attachment
mapper plugin and passing them in base64 encoded string form and surrounded
by quotest as field value in json.

You can store your original file's data in any File system you
like. As per general suggestions by people, Hadoop deployment should be
used when you are planning to have very large amount of data (probably in
Terra Bytes). There also scalable distributed file systems available like
Gridfs by Mongodb which you can use to store your files. Elasticsearch will
take care of storing index data itself so you don't have to worry about
that.

I hope it helped. : ) Just a tip, next time try to ask each question in
separate post to get the answers quickly by participants in FOSS forums. :
) Good luck!

On Wednesday, 28 November 2012 20:47:14 UTC+5:30, evil80 wrote:

hi,

I'm new to Elasticsearch I'm just trying to build a little web
application to understand how it works.
I have some questions, if somebody would like to give me an answer :

I'm using a .NET client for ES and I used it to index new object
in the ES Server, but in real life I have documents, so I have to write
specific code to wrap this document in .NET object or I can index them
automatically ?

If I can index them automatically how can I control what fields
will be built ?

Can someone explain me what Facets are what are their benefits ?

Imagine I start indexing two different type of document and I want
to let the user search against some field like : branch code, date, prices,
and so on. How is the best way to start building something like that ?

Can I let ES store its data in HDFS (Hadoop) and using it as the
only storage ?

I've understood it's a real powerful framework for text searching but I
need to understand how I can design my application to get good results.

Thanks a lot!

--

Atharva_Patel · November 30, 2012, 6:30am

My mistake for saying there are no APIs for .NET. There are few pointed out
here:

On Friday, 30 November 2012 10:52:27 UTC+5:30, Atharva Patel wrote:

Seems like there is no API for .Net to work with ES. But you can start
working by using REST api. So basic work-flow for you can be like this:

Look at the JSON api by ES for the task that you want to achieve
with ES like updating mapping, indexing, searching etc.

Decide the message structure that you would like to pass as JSON in
the URI requests from your code to ES.

Prepare objects in your code with fields matching with those in
your desired JSON message.

Use .Net specific Object to JSON converters and dynamically
generate JSON strings from those objects

Make appropriate HTTP request with method type set as PUT, POST,
GET, DELETE and pass the required JSON string in the request body.

You will receive HTTP response from ES. Check the error codes and
response body etc. to verify the success of your desired operation.

For getting better understanding of using Facets, better ask it as a
separate question and explain your use case with an example there! : )

Good day ! : )

On Friday, 30 November 2012 03:47:31 UTC+5:30, evil80 wrote:

Hi,

really thanks for your answer.
I just want to know :

So if I use a .NET client for ES I have to read the data I want to
index in a .NET object and then load i into ES with the desired fields ?

I've read that Faces allows to filter the result of a query, but in
a web scenario after I bind the first result query to a control (let say a
table) if I want to filter it with the Facets I have to re-run my query ?
What is the common architecture use ? I think it's not a good idea to store
the result of the query in the user session

I will write next questions in separate post as you suggest.

Thanks a lot again!
Il giorno mercoledì 28 novembre 2012 17:33:02 UTC+1, Atharva Patel ha
scritto:

ES has Java and Groovy API. It's also popular for it's RESTful
API. So you can use either of them to pass your documents to the ES. Fields
won't be built automatically. You need to specify them in the document
object that you construct for indexing. You can also specify the type of
each field using the mappingshttp://www.elasticsearch.org/guide/reference/api/admin-indices-put-mapping.html
and core typeshttp://www.elasticsearch.org/guide/reference/mapping/core-types.html
.

As per I can conceive from reading the documentation, Facets are
great to get aggregated data on your field values over a set of documents.
This set can be entire document set or just some of them matching with your
query.

Once you have defined your mappings for each document type you
want to store, you can make query on all those different types of documents
using search query in ES. The REST call looks something like this:

curl -XGET 'http://localhost:9200/
//_search' -d

You can leave the "/" in this uri if you want to
search across different types of documents in same index name.

curl -XGET 'http://localhost:9200//_search' -d

Here document type refers to differently structured documents and
not file types like .pdf, .txt etc. They can be indexed using attachment
mapper plugin and passing them in base64 encoded string form and surrounded
by quotest as field value in json.

You can store your original file's data in any File system you
like. As per general suggestions by people, Hadoop deployment should be
used when you are planning to have very large amount of data (probably in
Terra Bytes). There also scalable distributed file systems available like
Gridfs by Mongodb which you can use to store your files. Elasticsearch will
take care of storing index data itself so you don't have to worry about
that.

I hope it helped. : ) Just a tip, next time try to ask each question in
separate post to get the answers quickly by participants in FOSS forums. :
) Good luck!

On Wednesday, 28 November 2012 20:47:14 UTC+5:30, evil80 wrote:

hi,

I'm new to Elasticsearch I'm just trying to build a little web
application to understand how it works.
I have some questions, if somebody would like to give me an answer :

I'm using a .NET client for ES and I used it to index new object
in the ES Server, but in real life I have documents, so I have to write
specific code to wrap this document in .NET object or I can index them
automatically ?

If I can index them automatically how can I control what
fields will be built ?

Can someone explain me what Facets are what are their benefits ?

Imagine I start indexing two different type of document and I
want to let the user search against some field like : branch code, date,
prices, and so on. How is the best way to start building something like
that ?

Can I let ES store its data in HDFS (Hadoop) and using it as the
only storage ?

I've understood it's a real powerful framework for text searching but I
need to understand how I can design my application to get good results.

Thanks a lot!

--

evil80 · November 30, 2012, 8:17am

Hi,

I've found this : GitHub - elastic/elasticsearch-net: This strongly-typed, client library enables working with Elasticsearch. It is the official client maintained and supported by Elastic.
So I think I don't need to interact with JSon.

I'm going to open a specific post for Facets.

Thanks!

Il giorno venerdì 30 novembre 2012 06:22:27 UTC+1, Atharva Patel ha scritto:

Seems like there is no API for .Net to work with ES. But you can start
working by using REST api. So basic work-flow for you can be like this:

Look at the JSON api by ES for the task that you want to achieve
with ES like updating mapping, indexing, searching etc.

Decide the message structure that you would like to pass as JSON in
the URI requests from your code to ES.

Prepare objects in your code with fields matching with those in
your desired JSON message.

Use .Net specific Object to JSON converters and dynamically
generate JSON strings from those objects

Make appropriate HTTP request with method type set as PUT, POST,
GET, DELETE and pass the required JSON string in the request body.

You will receive HTTP response from ES. Check the error codes and
response body etc. to verify the success of your desired operation.

For getting better understanding of using Facets, better ask it as a
separate question and explain your use case with an example there! : )

Good day ! : )

On Friday, 30 November 2012 03:47:31 UTC+5:30, evil80 wrote:

Hi,

really thanks for your answer.
I just want to know :

So if I use a .NET client for ES I have to read the data I want to
index in a .NET object and then load i into ES with the desired fields ?

I've read that Faces allows to filter the result of a query, but in
a web scenario after I bind the first result query to a control (let say a
table) if I want to filter it with the Facets I have to re-run my query ?
What is the common architecture use ? I think it's not a good idea to store
the result of the query in the user session

I will write next questions in separate post as you suggest.

Thanks a lot again!
Il giorno mercoledì 28 novembre 2012 17:33:02 UTC+1, Atharva Patel ha
scritto:

ES has Java and Groovy API. It's also popular for it's RESTful
API. So you can use either of them to pass your documents to the ES. Fields
won't be built automatically. You need to specify them in the document
object that you construct for indexing. You can also specify the type of
each field using the mappingshttp://www.elasticsearch.org/guide/reference/api/admin-indices-put-mapping.html
and core typeshttp://www.elasticsearch.org/guide/reference/mapping/core-types.html
.

As per I can conceive from reading the documentation, Facets are
great to get aggregated data on your field values over a set of documents.
This set can be entire document set or just some of them matching with your
query.

Once you have defined your mappings for each document type you
want to store, you can make query on all those different types of documents
using search query in ES. The REST call looks something like this:

curl -XGET 'http://localhost:9200/
//_search' -d

You can leave the "/" in this uri if you want to
search across different types of documents in same index name.

curl -XGET 'http://localhost:9200//_search' -d

Here document type refers to differently structured documents and
not file types like .pdf, .txt etc. They can be indexed using attachment
mapper plugin and passing them in base64 encoded string form and surrounded
by quotest as field value in json.

You can store your original file's data in any File system you
like. As per general suggestions by people, Hadoop deployment should be
used when you are planning to have very large amount of data (probably in
Terra Bytes). There also scalable distributed file systems available like
Gridfs by Mongodb which you can use to store your files. Elasticsearch will
take care of storing index data itself so you don't have to worry about
that.

I hope it helped. : ) Just a tip, next time try to ask each question in
separate post to get the answers quickly by participants in FOSS forums. :
) Good luck!

On Wednesday, 28 November 2012 20:47:14 UTC+5:30, evil80 wrote:

hi,

I'm new to Elasticsearch I'm just trying to build a little web
application to understand how it works.
I have some questions, if somebody would like to give me an answer :

I'm using a .NET client for ES and I used it to index new object
in the ES Server, but in real life I have documents, so I have to write
specific code to wrap this document in .NET object or I can index them
automatically ?

If I can index them automatically how can I control what
fields will be built ?

Can someone explain me what Facets are what are their benefits ?

Imagine I start indexing two different type of document and I
want to let the user search against some field like : branch code, date,
prices, and so on. How is the best way to start building something like
that ?

Can I let ES store its data in HDFS (Hadoop) and using it as the
only storage ?

I've understood it's a real powerful framework for text searching but I
need to understand how I can design my application to get good results.

Thanks a lot!

--

Topic		Replies	Views
Hadoop / Elasticsearch functionality Elasticsearch es-hadoop	20	3226	July 6, 2017
How is Hadoop and ES typically used? Elasticsearch es-hadoop	8	1713	July 6, 2017
First post, some questions Elasticsearch	2	339	July 6, 2017
Questions from a newbie Elasticsearch	15	403	July 6, 2017
Use of ElasticSearch Mapreduce Elasticsearch	1	267	July 6, 2017

Starting with ES

Related topics