Case sensitivity in ES

Elastic Stack Elasticsearch
1

Can someone refer me to a good documentation where it shows how to
enable the ES case sensitivity. I need to integrate it with my Java
client where I only issue queries.

Thanks,
Karine

2

Case sensitivity relates to the type of analysis used on a field. If
you use the standard analyzer, then all the terms are lowercased. Make
sure you are using the same analyzer for both indexing and querying.
If you lowercase during indexing, but not during querying, then you
will have a mismatch.

The standard, stop and simple analyzers lowercase terms. The
whitespace and keyword analyzers do not. All these principals derive
from Lucene, so you can look up in Lucene how
analyzers/tokenizers/filters work.

Cheers,

Ivan

On Wed, May 30, 2012 at 11:12 AM, Kara karine.marabyan@aero.org wrote:

Can someone refer me to a good documentation where it shows how to
enable the ES case sensitivity. I need to integrate it with my Java
client where I only issue queries.

Thanks,
Karine

3

I used the default ES settings and it is the lowercased. I would like
it be able to search for both case sensitive and insensitive. How can
I configure ES to do that?

Thanks,
Karine

On May 30, 11:25 am, Ivan Brusic i...@brusic.com wrote:

Case sensitivity relates to the type of analysis used on a field. If
you use the standard analyzer, then all the terms are lowercased. Make
sure you are using the same analyzer for both indexing and querying.
If you lowercase during indexing, but not during querying, then you
will have a mismatch.

The standard, stop and simple analyzers lowercase terms. The
whitespace and keyword analyzers do not. All these principals derive
from Lucene, so you can look up in Lucene how
analyzers/tokenizers/filters work.

Cheers,

Ivan

On Wed, May 30, 2012 at 11:12 AM, Kara karine.marab...@aero.org wrote:

Can someone refer me to a good documentation where it shows how to
enable the ES case sensitivity. I need to integrate it with my Java
client where I only issue queries.

Thanks,
Karine

1 Like
4

I am sorry - the query is case insensitive. It ignores the case. I
would like to have case sensitive queries too. The user interface has
the option to check or uncheck the checkbox to ignore the case. Here
is the code I have now:

SearchRequestBuilder builder = client.prepareSearch("mongoindex");
QueryStringQueryBuilder qb = QueryBuilders.queryString("bag*");
builder.setQuery(qb);
SearchResponse response = builder.execute().actionGet();
SearchHit docs = response.getHits().getHits();
System.out.println("size is " + docs.length);
client.close();

On May 30, 3:41 pm, Kara karine.marab...@aero.org wrote:

I used the default ES settings and it is the lowercased. I would like
it be able to search for both case sensitive and insensitive. How can
I configure ES to do that?

Thanks,
Karine

On May 30, 11:25 am, Ivan Brusic i...@brusic.com wrote:

Case sensitivity relates to the type of analysis used on a field. If
you use the standard analyzer, then all the terms are lowercased. Make
sure you are using the same analyzer for both indexing and querying.
If you lowercase during indexing, but not during querying, then you
will have a mismatch.

The standard, stop and simple analyzers lowercase terms. The
whitespace and keyword analyzers do not. All these principals derive
from Lucene, so you can look up in Lucene how
analyzers/tokenizers/filters work.

Cheers,

Ivan

On Wed, May 30, 2012 at 11:12 AM, Kara karine.marab...@aero.org wrote:

Can someone refer me to a good documentation where it shows how to
enable the ES case sensitivity. I need to integrate it with my Java
client where I only issue queries.

Thanks,
Karine

5

You can use different mappings for each field, using multifield : Elasticsearch Platform — Find real-time answers at scale | Elastic

But, even if I don't really understand your need, I think I will create two index.
The first contains all docs with standard mapping. The other one with case sensitive mapping.

Just because, if you have many fields for each doc, using multifield for all would be a bit long to setup.

HTH
David

Le 31 mai 2012 à 03:45, Kara karine.marabyan@aero.org a écrit :

I am sorry - the query is case insensitive. It ignores the case. I
would like to have case sensitive queries too. The user interface has
the option to check or uncheck the checkbox to ignore the case. Here
is the code I have now:

SearchRequestBuilder builder = client.prepareSearch("mongoindex");
QueryStringQueryBuilder qb = QueryBuilders.queryString("bag*");
builder.setQuery(qb);
SearchResponse response = builder.execute().actionGet();
SearchHit docs = response.getHits().getHits();
System.out.println("size is " + docs.length);
client.close();

On May 30, 3:41 pm, Kara karine.marab...@aero.org wrote:

I used the default ES settings and it is the lowercased. I would like
it be able to search for both case sensitive and insensitive. How can
I configure ES to do that?

Thanks,
Karine

On May 30, 11:25 am, Ivan Brusic i...@brusic.com wrote:

Case sensitivity relates to the type of analysis used on a field. If
you use the standard analyzer, then all the terms are lowercased. Make
sure you are using the same analyzer for both indexing and querying.
If you lowercase during indexing, but not during querying, then you
will have a mismatch.

The standard, stop and simple analyzers lowercase terms. The
whitespace and keyword analyzers do not. All these principals derive
from Lucene, so you can look up in Lucene how
analyzers/tokenizers/filters work.

Cheers,

Ivan

On Wed, May 30, 2012 at 11:12 AM, Kara karine.marab...@aero.org wrote:

Can someone refer me to a good documentation where it shows how to
enable the ES case sensitivity. I need to integrate it with my Java
client where I only issue queries.

Thanks,
Karine

6

Thanks David!

How can I create case sensitive mapping? Is it the elasticsearch.yml
that needs to be configured for it?

On May 30, 10:01 pm, David Pilato da...@pilato.fr wrote:

You can use different mappings for each field, using multifield :Elasticsearch Platform — Find real-time answers at scale | Elastic...

But, even if I don't really understand your need, I think I will create two index.
The first contains all docs with standard mapping. The other one with case sensitive mapping.

Just because, if you have many fields for each doc, using multifield for all would be a bit long to setup.

HTH
David

Le 31 mai 2012 à 03:45, Kara karine.marab...@aero.org a écrit :

I am sorry - the query is case insensitive. It ignores the case. I
would like to have case sensitive queries too. The user interface has
the option to check or uncheck the checkbox to ignore the case. Here
is the code I have now:

SearchRequestBuilder builder = client.prepareSearch("mongoindex");
QueryStringQueryBuilder qb = QueryBuilders.queryString("bag*");
builder.setQuery(qb);
SearchResponse response = builder.execute().actionGet();
SearchHit docs = response.getHits().getHits();
System.out.println("size is " + docs.length);
client.close();

On May 30, 3:41 pm, Kara karine.marab...@aero.org wrote:

I used the default ES settings and it is the lowercased. I would like
it be able to search for both case sensitive and insensitive. How can
I configure ES to do that?

Thanks,
Karine

On May 30, 11:25 am, Ivan Brusic i...@brusic.com wrote:

Case sensitivity relates to the type of analysis used on a field. If
you use the standard analyzer, then all the terms are lowercased. Make
sure you are using the same analyzer for both indexing and querying.
If you lowercase during indexing, but not during querying, then you
will have a mismatch.

The standard, stop and simple analyzers lowercase terms. The
whitespace and keyword analyzers do not. All these principals derive
from Lucene, so you can look up in Lucene how
analyzers/tokenizers/filters work.

Cheers,

Ivan

On Wed, May 30, 2012 at 11:12 AM, Kara karine.marab...@aero.org wrote:

Can someone refer me to a good documentation where it shows how to
enable the ES case sensitivity. I need to integrate it with my Java
client where I only issue queries.

Thanks,
Karine

7

Let's say that you want to define a case insensitive for twitter users:
$ curl -XPUT http://localhost:9200/twitter/user/_mapping -d '{
"user" : {
"properties" : {
"name" : { "type" : "string" }
}
}
}'

And now, you want to create the sensitive index:
$ curl -XPOST localhost:9200/twitter_ins -d '{
"analysis" : {
"analyzer" : {
"case_sensitive" : {
"type" : "custom",
"tokenizer" : "keyword"
}
}
},
"mappings" : {
"user" : {
"properties" : {
"name" : { "type" : "string", "analyzer" : "case_sensitive"
}
}
}
}
}'

I didn't test it but I think that it should work.

If you want to have only one index and use multifield:
$ curl -XPOST localhost:9200/twitter_ins -d '{
"analysis" : {
"analyzer" : {
"case_sensitive" : {
"type" : "custom",
"tokenizer" : "keyword"
}
}
}
}'

$ curl -XPUT http://localhost:9200/twitter/user/_mapping -d '{
"user" : {
"properties" : {
"name" : {
"type" : "multi_field",
"fields" : {
"name" : {"type" : "string" },
"sensitive" : {"type" : "string", "analyzer" :
"case_sensitive" }
}
}
}
}
}'

Then, you will have to search in field name or in field name.sensitive

See here :

HTH
David

-----Message d'origine-----
De : elasticsearch@googlegroups.com
[mailto:elasticsearch@googlegroups.com] De la part de Kara
Envoyé : jeudi 31 mai 2012 16:55
À : elasticsearch
Objet : Re: case sensitivity in ES

Thanks David!

How can I create case sensitive mapping? Is it the elasticsearch.yml
that needs to be configured for it?

On May 30, 10:01 pm, David Pilato da...@pilato.fr wrote:

You can use different mappings for each field, using multifield
:Elasticsearch Platform — Find real-time answers at scale | Elastic
type...

But, even if I don't really understand your need, I think I will
create two index.
The first contains all docs with standard mapping. The other one with
case sensitive mapping.

Just because, if you have many fields for each doc, using multifield
for all would be a bit long to setup.

HTH
David

Le 31 mai 2012 à 03:45, Kara karine.marab...@aero.org a écrit :

I am sorry - the query is case insensitive. It ignores the case. I
would like to have case sensitive queries too. The user interface
has the option to check or uncheck the checkbox to ignore the case.
Here is the code I have now:

SearchRequestBuilder builder = client.prepareSearch("mongoindex");
QueryStringQueryBuilder qb =
QueryBuilders.queryString("bag*");
builder.setQuery(qb);
SearchResponse response = builder.execute().actionGet();
SearchHit docs = response.getHits().getHits();
System.out.println("size is " + docs.length);
client.close();

On May 30, 3:41 pm, Kara karine.marab...@aero.org wrote:

I used the default ES settings and it is the lowercased. I would
like it be able to search for both case sensitive and insensitive.
How can I configure ES to do that?

Thanks,
Karine

On May 30, 11:25 am, Ivan Brusic i...@brusic.com wrote:

Case sensitivity relates to the type of analysis used on a field.
If you use the standard analyzer, then all the terms are
lowercased. Make sure you are using the same analyzer for both
indexing and querying.
If you lowercase during indexing, but not during querying, then
you will have a mismatch.

The standard, stop and simple analyzers lowercase terms. The
whitespace and keyword analyzers do not. All these principals
derive from Lucene, so you can look up in Lucene how
analyzers/tokenizers/filters work.

Cheers,

Ivan

On Wed, May 30, 2012 at 11:12 AM, Kara karine.marab...@aero.org
wrote:

Can someone refer me to a good documentation where it shows how
to enable the ES case sensitivity. I need to integrate it with
my
Java client where I only issue queries.

Thanks,
Karine

1 Like
8

David nicely illustrates two different approaches. I personally prefer
to use the multi-field approach, but everything is us- case dependent.
But ultimately you cannot use a single field to support both types of
queries. Either use different indices or different fields.

David used the keyword analyzer to maintain case sensitivity, but
there are other analyzers such as the WhitespaceAnalyzer that allow do
not lowercase fields. Use the analysis API to see how each analyzer
works.

--
Ivan

On Thu, May 31, 2012 at 1:15 PM, David Pilato david@pilato.fr wrote:

Let's say that you want to define a case insensitive for twitter users:
$ curl -XPUT http://localhost:9200/twitter/user/_mapping -d '{
"user" : {
"properties" : {
"name" : { "type" : "string" }
}
}
}'

And now, you want to create the sensitive index:
$ curl -XPOST localhost:9200/twitter_ins -d '{
"analysis" : {
"analyzer" : {
"case_sensitive" : {
"type" : "custom",
"tokenizer" : "keyword"
}
}
},
"mappings" : {
"user" : {
"properties" : {
"name" : { "type" : "string", "analyzer" : "case_sensitive"
}
}
}
}
}'

I didn't test it but I think that it should work.

If you want to have only one index and use multifield:
$ curl -XPOST localhost:9200/twitter_ins -d '{
"analysis" : {
"analyzer" : {
"case_sensitive" : {
"type" : "custom",
"tokenizer" : "keyword"
}
}
}
}'

$ curl -XPUT http://localhost:9200/twitter/user/_mapping -d '{
"user" : {
"properties" : {
"name" : {
"type" : "multi_field",
"fields" : {
"name" : {"type" : "string" },
"sensitive" : {"type" : "string", "analyzer" :
"case_sensitive" }
}
}
}
}
}'

Then, you will have to search in field name or in field name.sensitive

See here :
Elasticsearch Platform — Find real-time answers at scale | Elastic

HTH
David

-----Message d'origine-----
De : elasticsearch@googlegroups.com
[mailto:elasticsearch@googlegroups.com] De la part de Kara
Envoyé : jeudi 31 mai 2012 16:55
À : elasticsearch
Objet : Re: case sensitivity in ES

Thanks David!

How can I create case sensitive mapping? Is it the elasticsearch.yml
that needs to be configured for it?

On May 30, 10:01 pm, David Pilato da...@pilato.fr wrote:

You can use different mappings for each field, using multifield
:Elasticsearch Platform — Find real-time answers at scale | Elastic
type...

But, even if I don't really understand your need, I think I will
create two index.
The first contains all docs with standard mapping. The other one with
case sensitive mapping.

Just because, if you have many fields for each doc, using multifield
for all would be a bit long to setup.

HTH
David

Le 31 mai 2012 à 03:45, Kara karine.marab...@aero.org a écrit :

I am sorry - the query is case insensitive. It ignores the case. I
would like to have case sensitive queries too. The user interface
has the option to check or uncheck the checkbox to ignore the case.
Here is the code I have now:

SearchRequestBuilder builder = client.prepareSearch("mongoindex");
QueryStringQueryBuilder qb =
QueryBuilders.queryString("bag*");
builder.setQuery(qb);
SearchResponse response = builder.execute().actionGet();
SearchHit docs = response.getHits().getHits();
System.out.println("size is " + docs.length);
client.close();

On May 30, 3:41 pm, Kara karine.marab...@aero.org wrote:

I used the default ES settings and it is the lowercased. I would
like it be able to search for both case sensitive and insensitive.
How can I configure ES to do that?

Thanks,
Karine

On May 30, 11:25 am, Ivan Brusic i...@brusic.com wrote:

Case sensitivity relates to the type of analysis used on a field.
If you use the standard analyzer, then all the terms are
lowercased. Make sure you are using the same analyzer for both
indexing and querying.
If you lowercase during indexing, but not during querying, then
you will have a mismatch.

The standard, stop and simple analyzers lowercase terms. The
whitespace and keyword analyzers do not. All these principals
derive from Lucene, so you can look up in Lucene how
analyzers/tokenizers/filters work.

Cheers,

Ivan

On Wed, May 30, 2012 at 11:12 AM, Kara karine.marab...@aero.org
wrote:

Can someone refer me to a good documentation where it shows how
to enable the ES case sensitivity. I need to integrate it with
my
Java client where I only issue queries.

Thanks,
Karine

9

You can search case sensitive by this :

public static SearchResponse search(String indexName,String indexType,String
key,String value , Client client) {

    QueryFilterBuilder qb =

FilterBuilders.queryFilter(QueryBuilders.queryString(value+"*"));

    return client.prepareSearch(indexName)
            .setTypes(indexType)
            .setSearchType(SearchType.DFS_QUERY_AND_FETCH)

.setQuery(QueryBuilders.filteredQuery(QueryBuilders.prefixQuery(key, value),
qb)).execute()
.actionGet();
}

--
View this message in context: http://elasticsearch-users.115913.n3.nabble.com/case-sensitivity-in-ES-tp4018579p4063834.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/1411359439874-4063834.post%40n3.nabble.com.
For more options, visit https://groups.google.com/d/optout.

10

You can search case sensitive by this :

public static SearchResponse search(String indexName,String indexType,String key,String value , Client client) {

    QueryFilterBuilder qb =   FilterBuilders.queryFilter(QueryBuilders.queryString(value+"*")); 
    
    return client.prepareSearch(indexName) 
            .setTypes(indexType) 
            .setSearchType(SearchType.DFS_QUERY_AND_FETCH) 
            .setQuery(QueryBuilders.filteredQuery(QueryBuilders.prefixQuery(key, value), qb)).execute() 
            .actionGet(); 
}
11

You can search case sensitive by this :

public static SearchResponse search(String indexName,String indexType,String key,String value , Client client) {

    QueryFilterBuilder qb =   FilterBuilders.queryFilter(QueryBuilders.queryString(value+"*")); 
    
    return client.prepareSearch(indexName) 
            .setTypes(indexType) 
            .setSearchType(SearchType.DFS_QUERY_AND_FETCH) 
            .setQuery(QueryBuilders.filteredQuery(QueryBuilders.prefixQuery(key, value), qb)).execute() 
            .actionGet(); 
}