Unmapped, searchable JSON field

Elastic Stack Elasticsearch
1

I'd like to be able to store JSON in a field of a document that is
searchable as a string, but unmapped so that I can store any structure
within this field without conflict.

I've experimented with different mappings, none of which allow me to
perform a text search on the field and my setup makes it cumbersome to
rapidly modify settings to find the one that works.

I assume this is possible and I've just been unable to discover the
correct combination of settings. If anyone can help me out I'd
appreciate it.

Thanks

2

You mean within a json field, you will have a json value encoded into that
string? If so, whats the problem with it?

On Mon, Jan 16, 2012 at 7:13 PM, marcuslongmuir marcuslongmuir@me.comwrote:

I'd like to be able to store JSON in a field of a document that is
searchable as a string, but unmapped so that I can store any structure
within this field without conflict.

I've experimented with different mappings, none of which allow me to
perform a text search on the field and my setup makes it cumbersome to
rapidly modify settings to find the one that works.

I assume this is possible and I've just been unable to discover the
correct combination of settings. If anyone can help me out I'd
appreciate it.

Thanks

3

This is the functionality I would like to achieve, but without
explicitly setting the JSON structure as a string in the document. I
tried setting the mapping type to "String", but I was unable to search
on the field.

On Jan 17, 9:41 am, Shay Banon kim...@gmail.com wrote:

You mean within a json field, you will have a json value encoded into that
string? If so, whats the problem with it?

On Mon, Jan 16, 2012 at 7:13 PM, marcuslongmuir marcuslongm...@me.comwrote:

I'd like to be able to store JSON in a field of a document that is
searchable as a string, but unmapped so that I can store any structure
within this field without conflict.

I've experimented with different mappings, none of which allow me to
perform a text search on the field and my setup makes it cumbersome to
rapidly modify settings to find the one that works.

I assume this is possible and I've just been unable to discover the
correct combination of settings. If anyone can help me out I'd
appreciate it.

Thanks

4

On Tue, 2012-01-17 at 19:18 -0800, marcuslongmuir wrote:

This is the functionality I would like to achieve, but without
explicitly setting the JSON structure as a string in the document. I
tried setting the mapping type to "String", but I was unable to search
on the field.

The mapping type should be 'string', not 'String'.

Here is an example: Index a doc which contains a JSON-encoded string · GitHub

I index a doc which contains a JSON encoded string (so in the curl
statement, that string is double-encoded).

This automatically creates the 'json' field and sets its type to
'string', and analyzes its contents with the default analyzer.

Then I demonstrate a search on the contents of the JSON, and finally
show all the terms stored in the 'json' field, indicating how the string
has been analyzed.

clint

5

Thanks for the solution. I would definitely be using this method were
it not for a rather difficult limitation.

Unfortunately without having at least some redundancy I can't do this
as I need to be able to return the document straight from a search
result without the overhead of parsing it.

Is there no mapping setup that would allow me to simply set the
indexing of the field to a string and have the structure processed the
same was as if it were double encoded as you demonstrate in your gist?

Thanks

On Jan 18, 4:10 am, Clinton Gormley cl...@traveljury.com wrote:

On Tue, 2012-01-17 at 19:18 -0800, marcuslongmuir wrote:

This is the functionality I would like to achieve, but without
explicitly setting theJSONstructure as astringin the document. I
tried setting the mapping type to "String", but I was unable to search
on the field.

The mapping type should be 'string', not 'String'.

Here is an example:Index a doc which contains a JSON-encoded string · GitHub

I index a doc which contains aJSONencodedstring(so in the curl
statement, thatstringis double-encoded).

This automatically creates the 'json' field and sets its type to
'string', and analyzes its contents with the default analyzer.

Then I demonstrate a search on the contents of theJSON, and finally
show all the terms stored in the 'json' field, indicating how thestring
has been analyzed.

clint

6

No, there isn't one.

On Fri, Jan 20, 2012 at 5:13 AM, marcuslongmuir marcuslongmuir@me.comwrote:

Thanks for the solution. I would definitely be using this method were
it not for a rather difficult limitation.

Unfortunately without having at least some redundancy I can't do this
as I need to be able to return the document straight from a search
result without the overhead of parsing it.

Is there no mapping setup that would allow me to simply set the
indexing of the field to a string and have the structure processed the
same was as if it were double encoded as you demonstrate in your gist?

Thanks

On Jan 18, 4:10 am, Clinton Gormley cl...@traveljury.com wrote:

On Tue, 2012-01-17 at 19:18 -0800, marcuslongmuir wrote:

This is the functionality I would like to achieve, but without
explicitly setting theJSONstructure as astringin the document. I
tried setting the mapping type to "String", but I was unable to search
on the field.

The mapping type should be 'string', not 'String'.

Here is an example:Index a doc which contains a JSON-encoded string · GitHub

I index a doc which contains aJSONencodedstring(so in the curl
statement, thatstringis double-encoded).

This automatically creates the 'json' field and sets its type to
'string', and analyzes its contents with the default analyzer.

Then I demonstrate a search on the contents of theJSON, and finally
show all the terms stored in the 'json' field, indicating how thestring
has been analyzed.

clint

7

Ok. Thanks for the solution. I'll have to put the encoding in the
application layer, not too much of an overhead.

On Jan 20, 3:05 pm, Shay Banon kim...@gmail.com wrote:

No, there isn't one.

On Fri, Jan 20, 2012 at 5:13 AM, marcuslongmuir marcuslongm...@me.comwrote:

Thanks for the solution. I would definitely be using this method were
it not for a rather difficult limitation.

Unfortunately without having at least some redundancy I can't do this
as I need to be able to return the document straight from a search
result without the overhead of parsing it.

Is there no mapping setup that would allow me to simply set the
indexing of the field to a string and have the structure processed the
same was as if it were double encoded as you demonstrate in your gist?

Thanks

On Jan 18, 4:10 am, Clinton Gormley cl...@traveljury.com wrote:

On Tue, 2012-01-17 at 19:18 -0800, marcuslongmuir wrote:

This is the functionality I would like to achieve, but without
explicitly setting theJSONstructure as astringin the document. I
tried setting the mapping type to "String", but I was unable to search
on the field.

The mapping type should be 'string', not 'String'.

Here is an example:Index a doc which contains a JSON-encoded string · GitHub

I index a doc which contains aJSONencodedstring(so in the curl
statement, thatstringis double-encoded).

This automatically creates the 'json' field and sets its type to
'string', and analyzes its contents with the default analyzer.

Then I demonstrate a search on the contents of theJSON, and finally
show all the terms stored in the 'json' field, indicating how thestring
has been analyzed.

clint