Retrieving fields from _source that don't have mappings?

Elastic Stack Elasticsearch
1

Hey,
I'm currently using 0.17.2 release. At some point in the past, in
order to pull fields from the _source without pulling back the entire
_source item, it was required to use script fields. So to set the
script field, it looked like this:

	for (String field : params.returnFields) {
		String fieldLower = field.toLowerCase();

		// This script is in the default mvel
		// This script will pull the values we need from the source field,

without
// needing to pull all the content over the wire. Documents with
large amounts
// of full text content benefit from this.
Map<String, Object> searchParams = new HashMap<String, Object>();
searchParams.put("fieldName", fieldLower);
searchBuilder.addScriptField(fieldLower, "_source[fieldName]",
searchParams);
}

And getting the value back looked like this:

public String getReturnValue(String Name, SearchHit hit) {
	String value = (String)hit.getFields().get(Name).value();
	if (value == null)
		value = "";

	return value;
}

I noticed that this no longer seemed to be the case and moved to doing
things like this:

	for (String field : params.returnFields) {
		String fieldLower = field.toLowerCase();
		searchBuilder.addFields(fieldLower);
	}

public String getReturnValue(String Name, SearchHit hit) {
	String value;
	SearchHitField returnField = hit.getFields().get(Name);
	if (returnField == null) {
		value = "";
	} else {
		value = (String)returnField.value();
		if (value == null)
			value = "";
	}

	return value;
}

This worked fine for any fields that mapping were enabled for.
However, for fields that are retrievable only, I have not been setting
any mappings and just adding them to the _source. However, it appears
that fields must have mappings enabled for the second method to work.

I can get things to work by adding mappings that define the
retrievable only fields as store=no and index=no, which is reasonable,
but was curious if the second method was possible without adding the
mappings for each retrievable field.

Thanks!
Paul

2

You can still use script fields, and specify something like
_source.obj1.obj2. Or, you can also specify it in the fields section, though
for fields without mappings, you should prefix it with _source in fields as
well.

On Wed, Sep 7, 2011 at 1:50 AM, ppearcy ppearcy@gmail.com wrote:

Hey,
I'm currently using 0.17.2 release. At some point in the past, in
order to pull fields from the _source without pulling back the entire
_source item, it was required to use script fields. So to set the
script field, it looked like this:

           for (String field : params.returnFields) {
                   String fieldLower = field.toLowerCase();

                   // This script is in the default mvel
                   // This script will pull the values we need from the

source field,
without
// needing to pull all the content over the wire.
Documents with
large amounts
// of full text content benefit from this.
Map<String, Object> searchParams = new
HashMap<String, Object>();
searchParams.put("fieldName", fieldLower);
searchBuilder.addScriptField(fieldLower,
"_source[fieldName]",
searchParams);
}

And getting the value back looked like this:

   public String getReturnValue(String Name, SearchHit hit) {
           String value = (String)hit.getFields().get(Name).value();
           if (value == null)
                   value = "";

           return value;
   }

I noticed that this no longer seemed to be the case and moved to doing
things like this:

           for (String field : params.returnFields) {
                   String fieldLower = field.toLowerCase();
                   searchBuilder.addFields(fieldLower);
           }

   public String getReturnValue(String Name, SearchHit hit) {
           String value;
           SearchHitField returnField = hit.getFields().get(Name);
           if (returnField == null) {
                   value = "";
           } else {
                   value = (String)returnField.value();
                   if (value == null)
                           value = "";
           }

           return value;
   }

This worked fine for any fields that mapping were enabled for.
However, for fields that are retrievable only, I have not been setting
any mappings and just adding them to the _source. However, it appears
that fields must have mappings enabled for the second method to work.

I can get things to work by adding mappings that define the
retrievable only fields as store=no and index=no, which is reasonable,
but was curious if the second method was possible without adding the
mappings for each retrievable field.

Thanks!
Paul

3

Sweet, that works great. Thank you!

On Sep 7, 1:11 am, Shay Banon kim...@gmail.com wrote:

You can still use script fields, and specify something like
_source.obj1.obj2. Or, you can also specify it in the fields section, though
for fields without mappings, you should prefix it with _source in fields as
well.

On Wed, Sep 7, 2011 at 1:50 AM, ppearcy ppea...@gmail.com wrote:

Hey,
I'm currently using 0.17.2 release. At some point in the past, in
order to pull fields from the _source without pulling back the entire
_source item, it was required to use script fields. So to set the
script field, it looked like this:

           for (String field : params.returnFields) {
                   String fieldLower = field.toLowerCase();

                   // This script is in the default mvel
                   // This script will pull the values we need from the

source field,
without
// needing to pull all the content over the wire.
Documents with
large amounts
// of full text content benefit from this.
Map<String, Object> searchParams = new
HashMap<String, Object>();
searchParams.put("fieldName", fieldLower);
searchBuilder.addScriptField(fieldLower,
"_source[fieldName]",
searchParams);
}

And getting the value back looked like this:

   public String getReturnValue(String Name, SearchHit hit) {
           String value = (String)hit.getFields().get(Name).value();
           if (value == null)
                   value = "";

           return value;
   }

I noticed that this no longer seemed to be the case and moved to doing
things like this:

           for (String field : params.returnFields) {
                   String fieldLower = field.toLowerCase();
                   searchBuilder.addFields(fieldLower);
           }

   public String getReturnValue(String Name, SearchHit hit) {
           String value;
           SearchHitField returnField = hit.getFields().get(Name);
           if (returnField == null) {
                   value = "";
           } else {
                   value = (String)returnField.value();
                   if (value == null)
                           value = "";
           }

           return value;
   }

This worked fine for any fields that mapping were enabled for.
However, for fields that are retrievable only, I have not been setting
any mappings and just adding them to the _source. However, it appears
that fields must have mappings enabled for the second method to work.

I can get things to work by adding mappings that define the
retrievable only fields as store=no and index=no, which is reasonable,
but was curious if the second method was possible without adding the
mappings for each retrievable field.

Thanks!
Paul