Issue with my template creation

Elastic Stack Elasticsearch
1

This used to work great when I went through pyes and was creating index.
But when I try convert it into template PUT, I see the following error.
There seems to be some issue with the "search_analyzer" too. Any idea, what
is wrong with this?

Thanks in Advance
-pk

Error:
*
*
{"error":"ElasticSearchIllegalArgumentException[Malformed mappings section
for type [index_analyzer], should include an inner object describing the
mapping]","status":400}

Template:
*
*
curl -XPUT localhost:9200/template/foo -d '{
"template" : "foo*",
"settings" : { "number_of_shards": 1,
"analysis": {
"filter": {
"mynGram" : {
"type" : "nGram",
"min_gram": 1,
"max_gram": 20
}
},
"tokenizer" : {
"email_tokenizer" : {
"type" : "pattern",
"pattern" :
"[^@:/\.\!=\-\w\p{L}\d]+"
}
},
"analyzer": {
"a1" : {
"type" : "custom",
"tokenizer":"email_tokenizer",
"filter" : ["lowercase", "mynGram"]
}
}
}
},
"mappings" : {
"index_analyzer" : "a1",
"search_analyzer" : "whitespace",
"properties" : {
"message" : {
"type" : "string",
"store": "yes"
}
}
}
}'

--

2

Elasticsearch expect to see a type name inside mappings:

    "mappings" : {

  •         "doc" : {*
                  "index_analyzer"  : "a1",
                  "search_analyzer" : "whitespace",
                  "properties" : {
                      "message" : {
                          "type" : "string",
                          "store": "yes"
                      }
                  }

  •                }*
          }

On Thursday, November 15, 2012 8:41:40 PM UTC-5, Praveen Kariyanahalli
wrote:

This used to work great when I went through pyes and was creating index.
But when I try convert it into template PUT, I see the following error.
There seems to be some issue with the "search_analyzer" too. Any idea, what
is wrong with this?

Thanks in Advance
-pk

Error:
*
*
{"error":"ElasticSearchIllegalArgumentException[Malformed mappings section
for type [index_analyzer], should include an inner object describing the
mapping]","status":400}

Template:
*
*
curl -XPUT localhost:9200/template/foo -d '{
"template" : "foo*",
"settings" : { "number_of_shards": 1,
"analysis": {
"filter": {
"mynGram" : {
"type" : "nGram",
"min_gram": 1,
"max_gram": 20
}
},
"tokenizer" : {
"email_tokenizer" : {
"type" : "pattern",
"pattern" :
"[^@:/\.\!=\-\w\p{L}\d]+"
}
},
"analyzer": {
"a1" : {
"type" : "custom",
"tokenizer":"email_tokenizer",
"filter" : ["lowercase", "mynGram"]
}
}
}
},
"mappings" : {
"index_analyzer" : "a1",
"search_analyzer" : "whitespace",
"properties" : {
"message" : {
"type" : "string",
"store": "yes"
}
}
}
}'

--

3

Thanks .. that helped.

I am facing another problem now. Here is the new template. I added
date_formats. It accepts. When I try insert a document .. it complains the
following.

Insert Document:

curl -XPOST 'http://127.0.0.1:9200/foo_2012-11-15/foo_doc?pretty=1' -d ' {
"message":"Insignificant message", "created_timestamp" :
"2012-11-15T10:10:10"}'
*
*
Error:

{
"error" : "MapperParsingException[mapping [elastica_logs]]; nested:
IllegalArgumentException[Illegal pattern component: T]; ",
"status" : 400
}

Template:

curl -XPUT localhost:9200/template/foo -d '{
"template" : "foo",
"settings" : { "number_of_shards": 1,
"analysis": {
"filter": {
"mynGram" : {
"type" : "nGram",
"min_gram": 1,
"max_gram": 20
}
},
"tokenizer" : {
"email_tokenizer" : {
"type" : "pattern",
"pattern" :
"[^@\:\.\-/=\-\w\p{L}\d]+"
}
},
"analyzer": {
"a1" : {
"type" : "custom",
"tokenizer":"email_tokenizer",
"filter" : ["lowercase", "mynGram"]
}
}
}
},
"mappings" : {
"foo_doc" : {
"index_analyzer" : "a1",
"search_analyzer" : "whitespace",
* "date_formats" : ["yyyy-MM-dd'T'HH:mm:ss"],
"properties" : {
"created_timestamp" : {
"index" : "not_analyzed",
"type" : "date",
"store" : "yes"
},
"message" : {
"index": "not_analyzed",
"type" : "string",
"store": "yes"
}
}
}
}
}'

--

4

You need to escape apostrophes when you are using curl. Otherwise your date
format is sent to es as "yyyy-MM-ddTHH:mm:ss" Try this:

"date_formats" : ["yyyy-MM-dd'''T'''HH:mm:ss"],

On Friday, November 16, 2012 4:36:15 PM UTC-5, Praveen Kariyanahalli wrote:

Thanks .. that helped.

I am facing another problem now. Here is the new template. I added
date_formats. It accepts. When I try insert a document .. it complains the
following.

Insert Document:

curl -XPOST 'http://127.0.0.1:9200/foo_2012-11-15/foo_doc?pretty=1' -d '
{ "message":"Insignificant message", "created_timestamp" :
"2012-11-15T10:10:10"}'
*
*
Error:

{
"error" : "MapperParsingException[mapping [elastica_logs]]; nested:
IllegalArgumentException[Illegal pattern component: T]; ",
"status" : 400
}

Template:

curl -XPUT localhost:9200/template/foo -d '{
"template" : "foo",
"settings" : { "number_of_shards": 1,
"analysis": {
"filter": {
"mynGram" : {
"type" : "nGram",
"min_gram": 1,
"max_gram": 20
}
},
"tokenizer" : {
"email_tokenizer" : {
"type" : "pattern",
"pattern" :
"[^@\:\.\-/=\-\w\p{L}\d]+"
}
},
"analyzer": {
"a1" : {
"type" : "custom",
"tokenizer":"email_tokenizer",
"filter" : ["lowercase", "mynGram"]
}
}
}
},
"mappings" : {
"foo_doc" : {
"index_analyzer" : "a1",
"search_analyzer" : "whitespace",
* "date_formats" : ["yyyy-MM-dd'T'HH:mm:ss"],
"properties" : {
"created_timestamp" : {
"index" : "not_analyzed",
"type" : "date",
"store" : "yes"
},
"message" : {
"index": "not_analyzed",
"type" : "string",
"store": "yes"
}
}
}
}
}'

--

5

When you are using curl, you need to escape apostrophes. Otherwise your
date format is sent to es as "yyyy-MM-ddTHH:mm:ss" Try this:

"date_formats" : ["yyyy-MM-dd'''T'''HH:mm:ss"],

On Friday, November 16, 2012 4:36:15 PM UTC-5, Praveen Kariyanahalli wrote:

Thanks .. that helped.

I am facing another problem now. Here is the new template. I added
date_formats. It accepts. When I try insert a document .. it complains the
following.

Insert Document:

curl -XPOST 'http://127.0.0.1:9200/foo_2012-11-15/foo_doc?pretty=1' -d '
{ "message":"Insignificant message", "created_timestamp" :
"2012-11-15T10:10:10"}'
*
*
Error:

{
"error" : "MapperParsingException[mapping [elastica_logs]]; nested:
IllegalArgumentException[Illegal pattern component: T]; ",
"status" : 400
}

Template:

curl -XPUT localhost:9200/template/foo -d '{
"template" : "foo",
"settings" : { "number_of_shards": 1,
"analysis": {
"filter": {
"mynGram" : {
"type" : "nGram",
"min_gram": 1,
"max_gram": 20
}
},
"tokenizer" : {
"email_tokenizer" : {
"type" : "pattern",
"pattern" :
"[^@\:\.\-/=\-\w\p{L}\d]+"
}
},
"analyzer": {
"a1" : {
"type" : "custom",
"tokenizer":"email_tokenizer",
"filter" : ["lowercase", "mynGram"]
}
}
}
},
"mappings" : {
"foo_doc" : {
"index_analyzer" : "a1",
"search_analyzer" : "whitespace",
* "date_formats" : ["yyyy-MM-dd'T'HH:mm:ss"],
"properties" : {
"created_timestamp" : {
"index" : "not_analyzed",
"type" : "date",
"store" : "yes"
},
"message" : {
"index": "not_analyzed",
"type" : "string",
"store": "yes"
}
}
}
}
}'

--

6

Hello Praveen,

If you use curl from the command-line, you'll probably have to escape
the quotes, like:

"date_formats" : ["yyyy-MM-dd'"'T'"'HH:mm:ss"]

So instead of: single-quote-T-single-quote -> which will translate
into the string T, since you have single quotes at the beginning and
the end of your data. So the single quotes there will only end the
quoted string started first, add a T, then begin a new quoted string

You can put: single-quote-double-quote-single-quote-T-single-quote-double-quote-single-quote
-> which will translate to 'T', which is what you want. That's because
the first single quote ends the first part of your JSON, then you
start a double-quoted string which contains 'T', then you use
single-quotes again to continue your JSON.

That's a lot of quotes :slight_smile: Hope it helps, though.

Best regards,
Radu

http://sematext.com/ -- Elasticsearch -- Solr -- Lucene

On Fri, Nov 16, 2012 at 11:36 PM, Praveen Kariyanahalli
praveen.kariyanahalli@gmail.com wrote:

Thanks .. that helped.

I am facing another problem now. Here is the new template. I added
date_formats. It accepts. When I try insert a document .. it complains the
following.

Insert Document:

curl -XPOST 'http://127.0.0.1:9200/foo_2012-11-15/foo_doc?pretty=1' -d ' {
"message":"Insignificant message", "created_timestamp" :
"2012-11-15T10:10:10"}'

Error:

{
"error" : "MapperParsingException[mapping [elastica_logs]]; nested:
IllegalArgumentException[Illegal pattern component: T]; ",
"status" : 400
}

Template:

curl -XPUT localhost:9200/template/foo -d '{
"template" : "foo*",
"settings" : { "number_of_shards": 1,
"analysis": {
"filter": {
"mynGram" : {
"type" : "nGram",
"min_gram": 1,
"max_gram": 20
}
},
"tokenizer" : {
"email_tokenizer" : {
"type" : "pattern",
"pattern" :
"[^@\:\.\-/=\-\w\p{L}\d]+"
}
},
"analyzer": {
"a1" : {
"type" : "custom",
"tokenizer":"email_tokenizer",
"filter" : ["lowercase", "mynGram"]
}
}
}
},
"mappings" : {
"foo_doc" : {
"index_analyzer" : "a1",
"search_analyzer" : "whitespace",
"date_formats" : ["yyyy-MM-dd'T'HH:mm:ss"],
"properties" : {
"created_timestamp" : {
"index" : "not_analyzed",
"type" : "date",
"store" : "yes"
},
"message" : {
"index": "not_analyzed",
"type" : "string",
"store": "yes"
}
}
}
}
}'

--

--

7

That did the trick. Thanks Igor and Radu.

Regards
-pk

On Sat, Nov 17, 2012 at 6:37 AM, Radu Gheorghe
radu.gheorghe@sematext.comwrote:

Hello Praveen,

If you use curl from the command-line, you'll probably have to escape
the quotes, like:

"date_formats" : ["yyyy-MM-dd'"'T'"'HH:mm:ss"]

So instead of: single-quote-T-single-quote -> which will translate
into the string T, since you have single quotes at the beginning and
the end of your data. So the single quotes there will only end the
quoted string started first, add a T, then begin a new quoted string

You can put:
single-quote-double-quote-single-quote-T-single-quote-double-quote-single-quote
-> which will translate to 'T', which is what you want. That's because
the first single quote ends the first part of your JSON, then you
start a double-quoted string which contains 'T', then you use
single-quotes again to continue your JSON.

That's a lot of quotes :slight_smile: Hope it helps, though.

Best regards,
Radu

http://sematext.com/ -- Elasticsearch -- Solr -- Lucene

On Fri, Nov 16, 2012 at 11:36 PM, Praveen Kariyanahalli
praveen.kariyanahalli@gmail.com wrote:

Thanks .. that helped.

I am facing another problem now. Here is the new template. I added
date_formats. It accepts. When I try insert a document .. it complains
the
following.

Insert Document:

curl -XPOST 'http://127.0.0.1:9200/foo_2012-11-15/foo_doc?pretty=1' -d
' {
"message":"Insignificant message", "created_timestamp" :
"2012-11-15T10:10:10"}'

Error:

{
"error" : "MapperParsingException[mapping [elastica_logs]]; nested:
IllegalArgumentException[Illegal pattern component: T]; ",
"status" : 400
}

Template:

curl -XPUT localhost:9200/template/foo -d '{
"template" : "foo*",
"settings" : { "number_of_shards": 1,
"analysis": {
"filter": {
"mynGram" : {
"type" : "nGram",
"min_gram": 1,
"max_gram": 20
}
},
"tokenizer" : {
"email_tokenizer" : {
"type" : "pattern",
"pattern" :
"[^@\:\.\-/=\-\w\p{L}\d]+"
}
},
"analyzer": {
"a1" : {
"type" : "custom",
"tokenizer":"email_tokenizer",
"filter" : ["lowercase", "mynGram"]
}
}
}
},
"mappings" : {
"foo_doc" : {
"index_analyzer" : "a1",
"search_analyzer" : "whitespace",
"date_formats" : ["yyyy-MM-dd'T'HH:mm:ss"],
"properties" : {
"created_timestamp" : {
"index" : "not_analyzed",
"type" : "date",
"store" : "yes"
},
"message" : {
"index": "not_analyzed",
"type" : "string",
"store": "yes"
}
}
}
}
}'

--

--

--