Starting with ES

hi,

I'm new to ElasticSearch I'm just trying to build a little web application
to understand how it works.
I have some questions, if somebody would like to give me an answer :

  • I'm using a .NET client for ES and I used it to index new object in
    the ES Server, but in real life I have documents, so I have to write
    specific code to wrap this document in .NET object or I can index them
    automatically ?
    • If I can index them automatically how can I control what fields
      will be built ?
  • Can someone explain me what Facets are what are their benefits ?
  • Imagine I start indexing two different type of document and I want to
    let the user search against some field like : branch code, date, prices,
    and so on. How is the best way to start building something like that ?
  • Can I let ES store its data in HDFS (Hadoop) and using it as the only
    storage ?

I've understood it's a real powerful framework for text searching but I
need to understand how I can design my application to get good results.

Thanks a lot!

--

On Wednesday, 28 November 2012 20:47:14 UTC+5:30, evil80 wrote:

hi,

I'm new to Elasticsearch I'm just trying to build a little web application
to understand how it works.
I have some questions, if somebody would like to give me an answer :

  • I'm using a .NET client for ES and I used it to index new object in
    the ES Server, but in real life I have documents, so I have to write
    specific code to wrap this document in .NET object or I can index them
    automatically ?
    • If I can index them automatically how can I control what fields
      will be built ?
  • Can someone explain me what Facets are what are their benefits ?
  • Imagine I start indexing two different type of document and I want
    to let the user search against some field like : branch code, date, prices,
    and so on. How is the best way to start building something like that ?
  • Can I let ES store its data in HDFS (Hadoop) and using it as the
    only storage ?

I've understood it's a real powerful framework for text searching but I
need to understand how I can design my application to get good results.

Thanks a lot!

--

  • ES has Java and Groovy API. It's also popular for it's RESTful API. So
    you can use either of them to pass your documents to the ES. Fields won't
    be built automatically. You need to specify them in the document object
    that you construct for indexing. You can also specify the type of each
    field using the mappingshttp://www.elasticsearch.org/guide/reference/api/admin-indices-put-mapping.html
    and core typeshttp://www.elasticsearch.org/guide/reference/mapping/core-types.html
    .
  • As per I can conceive from reading the documentation, Facets are great
    to get aggregated data on your field values over a set of documents. This
    set can be entire document set or just some of them matching with your
    query.
  • Once you have defined your mappings for each document type you want to
    store, you can make query on all those different types of documents using
    search query in ES. The REST call looks something like this:
  • curl -XGET 'http://localhost:9200///_search' -d

You can leave the "/" in this uri if you want to search
across different types of documents in same index name.

  • curl -XGET 'http://localhost:9200//_search' -d

  • Here document type refers to differently structured documents and not
    file types like .pdf, .txt etc. They can be indexed using attachment mapper
    plugin and passing them in base64 encoded string form and surrounded by
    quotest as field value in json.

  • You can store your original file's data in any File system you like.
    As per general suggestions by people, Hadoop deployment should be used when
    you are planning to have very large amount of data (probably in Terra
    Bytes). There also scalable distributed file systems available like Gridfs
    by Mongodb which you can use to store your files. Elasticsearch will take
    care of storing index data itself so you don't have to worry about that.

I hope it helped. : ) Just a tip, next time try to ask each question in
separate post to get the answers quickly by participants in FOSS forums. :
) Good luck!

On Wednesday, 28 November 2012 20:47:14 UTC+5:30, evil80 wrote:

hi,

I'm new to Elasticsearch I'm just trying to build a little web application
to understand how it works.
I have some questions, if somebody would like to give me an answer :

  • I'm using a .NET client for ES and I used it to index new object in
    the ES Server, but in real life I have documents, so I have to write
    specific code to wrap this document in .NET object or I can index them
    automatically ?
    • If I can index them automatically how can I control what fields
      will be built ?
  • Can someone explain me what Facets are what are their benefits ?
  • Imagine I start indexing two different type of document and I want
    to let the user search against some field like : branch code, date, prices,
    and so on. How is the best way to start building something like that ?
  • Can I let ES store its data in HDFS (Hadoop) and using it as the
    only storage ?

I've understood it's a real powerful framework for text searching but I
need to understand how I can design my application to get good results.

Thanks a lot!

--

Hi,

really thanks for your answer.
I just want to know :

  • So if I use a .NET client for ES I have to read the data I want to
    index in a .NET object and then load i into ES with the desired fields ?
  • I've read that Faces allows to filter the result of a query, but in a
    web scenario after I bind the first result query to a control (let say a
    table) if I want to filter it with the Facets I have to re-run my query ?
    What is the common architecture use ? I think it's not a good idea to store
    the result of the query in the user session

I will write next questions in separate post as you suggest.

Thanks a lot again!
Il giorno mercoledì 28 novembre 2012 17:33:02 UTC+1, Atharva Patel ha
scritto:

  • ES has Java and Groovy API. It's also popular for it's RESTful API.
    So you can use either of them to pass your documents to the ES. Fields
    won't be built automatically. You need to specify them in the document
    object that you construct for indexing. You can also specify the type of
    each field using the mappingshttp://www.elasticsearch.org/guide/reference/api/admin-indices-put-mapping.html
    and core typeshttp://www.elasticsearch.org/guide/reference/mapping/core-types.html
    .
  • As per I can conceive from reading the documentation, Facets are
    great to get aggregated data on your field values over a set of documents.
    This set can be entire document set or just some of them matching with your
    query.
  • Once you have defined your mappings for each document type you want
    to store, you can make query on all those different types of documents
    using search query in ES. The REST call looks something like this:
  • curl -XGET 'http://localhost:9200///_search'
    -d

You can leave the "/" in this uri if you want to search
across different types of documents in same index name.

  • curl -XGET 'http://localhost:9200//_search' -d

  • Here document type refers to differently structured documents and
    not file types like .pdf, .txt etc. They can be indexed using attachment
    mapper plugin and passing them in base64 encoded string form and surrounded
    by quotest as field value in json.

  • You can store your original file's data in any File system you like.
    As per general suggestions by people, Hadoop deployment should be used when
    you are planning to have very large amount of data (probably in Terra
    Bytes). There also scalable distributed file systems available like Gridfs
    by Mongodb which you can use to store your files. Elasticsearch will take
    care of storing index data itself so you don't have to worry about that.

I hope it helped. : ) Just a tip, next time try to ask each question in
separate post to get the answers quickly by participants in FOSS forums. :
) Good luck!

On Wednesday, 28 November 2012 20:47:14 UTC+5:30, evil80 wrote:

hi,

I'm new to Elasticsearch I'm just trying to build a little web
application to understand how it works.
I have some questions, if somebody would like to give me an answer :

  • I'm using a .NET client for ES and I used it to index new object in
    the ES Server, but in real life I have documents, so I have to write
    specific code to wrap this document in .NET object or I can index them
    automatically ?
    • If I can index them automatically how can I control what fields
      will be built ?
  • Can someone explain me what Facets are what are their benefits ?
  • Imagine I start indexing two different type of document and I want
    to let the user search against some field like : branch code, date, prices,
    and so on. How is the best way to start building something like that ?
  • Can I let ES store its data in HDFS (Hadoop) and using it as the
    only storage ?

I've understood it's a real powerful framework for text searching but I
need to understand how I can design my application to get good results.

Thanks a lot!

--

Seems like there is no API for .Net to work with ES. But you can start
working by using REST api. So basic work-flow for you can be like this:

  1. Look at the JSON api by ES for the task that you want to achieve with
    ES like updating mapping, indexing, searching etc.
  2. Decide the message structure that you would like to pass as JSON in
    the URI requests from your code to ES.
  3. Prepare objects in your code with fields matching with those in your
    desired JSON message.
  4. Use .Net specific Object to JSON converters and dynamically generate
    JSON strings from those objects
  5. Make appropriate HTTP request with method type set as PUT, POST, GET,
    DELETE and pass the required JSON string in the request body.
  6. You will receive HTTP response from ES. Check the error codes and
    response body etc. to verify the success of your desired operation.

For getting better understanding of using Facets, better ask it as a
separate question and explain your use case with an example there! : )

Good day ! : )

On Friday, 30 November 2012 03:47:31 UTC+5:30, evil80 wrote:

Hi,

really thanks for your answer.
I just want to know :

  • So if I use a .NET client for ES I have to read the data I want to
    index in a .NET object and then load i into ES with the desired fields ?
  • I've read that Faces allows to filter the result of a query, but in
    a web scenario after I bind the first result query to a control (let say a
    table) if I want to filter it with the Facets I have to re-run my query ?
    What is the common architecture use ? I think it's not a good idea to store
    the result of the query in the user session

I will write next questions in separate post as you suggest.

Thanks a lot again!
Il giorno mercoledì 28 novembre 2012 17:33:02 UTC+1, Atharva Patel ha
scritto:

  • ES has Java and Groovy API. It's also popular for it's RESTful API.
    So you can use either of them to pass your documents to the ES. Fields
    won't be built automatically. You need to specify them in the document
    object that you construct for indexing. You can also specify the type of
    each field using the mappingshttp://www.elasticsearch.org/guide/reference/api/admin-indices-put-mapping.html
    and core typeshttp://www.elasticsearch.org/guide/reference/mapping/core-types.html
    .
  • As per I can conceive from reading the documentation, Facets are
    great to get aggregated data on your field values over a set of documents.
    This set can be entire document set or just some of them matching with your
    query.
  • Once you have defined your mappings for each document type you want
    to store, you can make query on all those different types of documents
    using search query in ES. The REST call looks something like this:
  • curl -XGET 'http://localhost:9200/
    //_search' -d

You can leave the "/" in this uri if you want to search
across different types of documents in same index name.

  • curl -XGET 'http://localhost:9200//_search' -d

  • Here document type refers to differently structured documents and
    not file types like .pdf, .txt etc. They can be indexed using attachment
    mapper plugin and passing them in base64 encoded string form and surrounded
    by quotest as field value in json.

  • You can store your original file's data in any File system you
    like. As per general suggestions by people, Hadoop deployment should be
    used when you are planning to have very large amount of data (probably in
    Terra Bytes). There also scalable distributed file systems available like
    Gridfs by Mongodb which you can use to store your files. Elasticsearch will
    take care of storing index data itself so you don't have to worry about
    that.

I hope it helped. : ) Just a tip, next time try to ask each question in
separate post to get the answers quickly by participants in FOSS forums. :
) Good luck!

On Wednesday, 28 November 2012 20:47:14 UTC+5:30, evil80 wrote:

hi,

I'm new to Elasticsearch I'm just trying to build a little web
application to understand how it works.
I have some questions, if somebody would like to give me an answer :

  • I'm using a .NET client for ES and I used it to index new object
    in the ES Server, but in real life I have documents, so I have to write
    specific code to wrap this document in .NET object or I can index them
    automatically ?
    • If I can index them automatically how can I control what fields
      will be built ?
  • Can someone explain me what Facets are what are their benefits ?
  • Imagine I start indexing two different type of document and I want
    to let the user search against some field like : branch code, date, prices,
    and so on. How is the best way to start building something like that ?
  • Can I let ES store its data in HDFS (Hadoop) and using it as the
    only storage ?

I've understood it's a real powerful framework for text searching but I
need to understand how I can design my application to get good results.

Thanks a lot!

--

My mistake for saying there are no APIs for .NET. There are few pointed out
here:

On Friday, 30 November 2012 10:52:27 UTC+5:30, Atharva Patel wrote:

Seems like there is no API for .Net to work with ES. But you can start
working by using REST api. So basic work-flow for you can be like this:

  1. Look at the JSON api by ES for the task that you want to achieve
    with ES like updating mapping, indexing, searching etc.
  2. Decide the message structure that you would like to pass as JSON in
    the URI requests from your code to ES.
  3. Prepare objects in your code with fields matching with those in
    your desired JSON message.
  4. Use .Net specific Object to JSON converters and dynamically
    generate JSON strings from those objects
  5. Make appropriate HTTP request with method type set as PUT, POST,
    GET, DELETE and pass the required JSON string in the request body.
  6. You will receive HTTP response from ES. Check the error codes and
    response body etc. to verify the success of your desired operation.

For getting better understanding of using Facets, better ask it as a
separate question and explain your use case with an example there! : )

Good day ! : )

On Friday, 30 November 2012 03:47:31 UTC+5:30, evil80 wrote:

Hi,

really thanks for your answer.
I just want to know :

  • So if I use a .NET client for ES I have to read the data I want to
    index in a .NET object and then load i into ES with the desired fields ?
  • I've read that Faces allows to filter the result of a query, but in
    a web scenario after I bind the first result query to a control (let say a
    table) if I want to filter it with the Facets I have to re-run my query ?
    What is the common architecture use ? I think it's not a good idea to store
    the result of the query in the user session

I will write next questions in separate post as you suggest.

Thanks a lot again!
Il giorno mercoledì 28 novembre 2012 17:33:02 UTC+1, Atharva Patel ha
scritto:

  • ES has Java and Groovy API. It's also popular for it's RESTful
    API. So you can use either of them to pass your documents to the ES. Fields
    won't be built automatically. You need to specify them in the document
    object that you construct for indexing. You can also specify the type of
    each field using the mappingshttp://www.elasticsearch.org/guide/reference/api/admin-indices-put-mapping.html
    and core typeshttp://www.elasticsearch.org/guide/reference/mapping/core-types.html
    .
  • As per I can conceive from reading the documentation, Facets are
    great to get aggregated data on your field values over a set of documents.
    This set can be entire document set or just some of them matching with your
    query.
  • Once you have defined your mappings for each document type you
    want to store, you can make query on all those different types of documents
    using search query in ES. The REST call looks something like this:
  • curl -XGET 'http://localhost:9200/
    //_search' -d

You can leave the "/" in this uri if you want to
search across different types of documents in same index name.

  • curl -XGET 'http://localhost:9200//_search' -d

  • Here document type refers to differently structured documents and
    not file types like .pdf, .txt etc. They can be indexed using attachment
    mapper plugin and passing them in base64 encoded string form and surrounded
    by quotest as field value in json.

  • You can store your original file's data in any File system you
    like. As per general suggestions by people, Hadoop deployment should be
    used when you are planning to have very large amount of data (probably in
    Terra Bytes). There also scalable distributed file systems available like
    Gridfs by Mongodb which you can use to store your files. Elasticsearch will
    take care of storing index data itself so you don't have to worry about
    that.

I hope it helped. : ) Just a tip, next time try to ask each question in
separate post to get the answers quickly by participants in FOSS forums. :
) Good luck!

On Wednesday, 28 November 2012 20:47:14 UTC+5:30, evil80 wrote:

hi,

I'm new to Elasticsearch I'm just trying to build a little web
application to understand how it works.
I have some questions, if somebody would like to give me an answer :

  • I'm using a .NET client for ES and I used it to index new object
    in the ES Server, but in real life I have documents, so I have to write
    specific code to wrap this document in .NET object or I can index them
    automatically ?
    • If I can index them automatically how can I control what
      fields will be built ?
  • Can someone explain me what Facets are what are their benefits ?
  • Imagine I start indexing two different type of document and I
    want to let the user search against some field like : branch code, date,
    prices, and so on. How is the best way to start building something like
    that ?
  • Can I let ES store its data in HDFS (Hadoop) and using it as the
    only storage ?

I've understood it's a real powerful framework for text searching but I
need to understand how I can design my application to get good results.

Thanks a lot!

--

Hi,

I've found this : GitHub - elastic/elasticsearch-net: This strongly-typed, client library enables working with Elasticsearch. It is the official client maintained and supported by Elastic.
So I think I don't need to interact with JSon.

I'm going to open a specific post for Facets.

Thanks!

Il giorno venerdì 30 novembre 2012 06:22:27 UTC+1, Atharva Patel ha scritto:

Seems like there is no API for .Net to work with ES. But you can start
working by using REST api. So basic work-flow for you can be like this:

  1. Look at the JSON api by ES for the task that you want to achieve
    with ES like updating mapping, indexing, searching etc.
  2. Decide the message structure that you would like to pass as JSON in
    the URI requests from your code to ES.
  3. Prepare objects in your code with fields matching with those in
    your desired JSON message.
  4. Use .Net specific Object to JSON converters and dynamically
    generate JSON strings from those objects
  5. Make appropriate HTTP request with method type set as PUT, POST,
    GET, DELETE and pass the required JSON string in the request body.
  6. You will receive HTTP response from ES. Check the error codes and
    response body etc. to verify the success of your desired operation.

For getting better understanding of using Facets, better ask it as a
separate question and explain your use case with an example there! : )

Good day ! : )

On Friday, 30 November 2012 03:47:31 UTC+5:30, evil80 wrote:

Hi,

really thanks for your answer.
I just want to know :

  • So if I use a .NET client for ES I have to read the data I want to
    index in a .NET object and then load i into ES with the desired fields ?
  • I've read that Faces allows to filter the result of a query, but in
    a web scenario after I bind the first result query to a control (let say a
    table) if I want to filter it with the Facets I have to re-run my query ?
    What is the common architecture use ? I think it's not a good idea to store
    the result of the query in the user session

I will write next questions in separate post as you suggest.

Thanks a lot again!
Il giorno mercoledì 28 novembre 2012 17:33:02 UTC+1, Atharva Patel ha
scritto:

  • ES has Java and Groovy API. It's also popular for it's RESTful
    API. So you can use either of them to pass your documents to the ES. Fields
    won't be built automatically. You need to specify them in the document
    object that you construct for indexing. You can also specify the type of
    each field using the mappingshttp://www.elasticsearch.org/guide/reference/api/admin-indices-put-mapping.html
    and core typeshttp://www.elasticsearch.org/guide/reference/mapping/core-types.html
    .
  • As per I can conceive from reading the documentation, Facets are
    great to get aggregated data on your field values over a set of documents.
    This set can be entire document set or just some of them matching with your
    query.
  • Once you have defined your mappings for each document type you
    want to store, you can make query on all those different types of documents
    using search query in ES. The REST call looks something like this:
  • curl -XGET 'http://localhost:9200/
    //_search' -d

You can leave the "/" in this uri if you want to
search across different types of documents in same index name.

  • curl -XGET 'http://localhost:9200//_search' -d

  • Here document type refers to differently structured documents and
    not file types like .pdf, .txt etc. They can be indexed using attachment
    mapper plugin and passing them in base64 encoded string form and surrounded
    by quotest as field value in json.

  • You can store your original file's data in any File system you
    like. As per general suggestions by people, Hadoop deployment should be
    used when you are planning to have very large amount of data (probably in
    Terra Bytes). There also scalable distributed file systems available like
    Gridfs by Mongodb which you can use to store your files. Elasticsearch will
    take care of storing index data itself so you don't have to worry about
    that.

I hope it helped. : ) Just a tip, next time try to ask each question in
separate post to get the answers quickly by participants in FOSS forums. :
) Good luck!

On Wednesday, 28 November 2012 20:47:14 UTC+5:30, evil80 wrote:

hi,

I'm new to Elasticsearch I'm just trying to build a little web
application to understand how it works.
I have some questions, if somebody would like to give me an answer :

  • I'm using a .NET client for ES and I used it to index new object
    in the ES Server, but in real life I have documents, so I have to write
    specific code to wrap this document in .NET object or I can index them
    automatically ?
    • If I can index them automatically how can I control what
      fields will be built ?
  • Can someone explain me what Facets are what are their benefits ?
  • Imagine I start indexing two different type of document and I
    want to let the user search against some field like : branch code, date,
    prices, and so on. How is the best way to start building something like
    that ?
  • Can I let ES store its data in HDFS (Hadoop) and using it as the
    only storage ?

I've understood it's a real powerful framework for text searching but I
need to understand how I can design my application to get good results.

Thanks a lot!

--