Binary fields do not support searching es 6.0.0-rc1

I've followed the instructions given by rjernst at the below link but I'm getting "Binary fields do not support searching" error.

Index Mapping:

{
  "testindex" : {
    "mappings" : {
      "testtype" : {
        "date_detection" : false,
        "properties" : {
          "body" : {
            "type" : "binary"
          }
        }
      }
    }
  }
}

The class looks like below. I've tried to put debugging points into the newinstance method but it's never hit.

import java.io.IOException;
import java.io.UncheckedIOException;
import java.util.Map;
import java.util.function.Function;

//import org.apache.log4j.Logger;
import org.apache.lucene.index.BinaryDocValues;
import org.apache.lucene.index.LeafReaderContext;
import org.elasticsearch.common.Nullable;
import org.elasticsearch.script.ExecutableScript;
import org.elasticsearch.script.SearchScript;
import org.elasticsearch.search.lookup.SearchLookup;
import org.elasticsearch.script.ScriptContext;
import org.elasticsearch.script.ScriptEngine;
import org.apache.lucene.index.PostingsEnum;
import org.apache.lucene.index.Term;
import org.apache.lucene.store.ByteArrayDataInput;
import org.apache.lucene.util.BytesRef;

public class MyExpertScriptEngine implements ScriptEngine {
	//final static Logger logger = Logger.getLogger(MyExpertScriptEngine.class);
	
    @Override
    public String getType() {
        return "expert_scripts";
    }

    @Override
    public <T> T compile(String scriptName, String scriptSource, ScriptContext<T> context, Map<String, String> params) {
        if (context.equals(SearchScript.CONTEXT) == false) {
            throw new IllegalArgumentException(getType() + " scripts cannot be used for context [" + context.name + "]");
        }
        // we use the script "source" as the script identifier
        if ("pure_df".equals(scriptSource)) {
        	//logger.debug("hello world");
        	System.out.println("hello world");
            SearchScript.Factory factory = (p, lookup) -> new SearchScript.LeafFactory() {
                final String field;
                final String term;
                boolean success;
                {
                    if (p.containsKey("field") == false) {
                        throw new IllegalArgumentException("Missing parameter [field]");
                    }
                    if (p.containsKey("term") == false) {
                        throw new IllegalArgumentException("Missing parameter [term]");
                    }
                    field = p.get("field").toString();
                    term = p.get("term").toString();
                }

                @Override
                public SearchScript newInstance(LeafReaderContext context) throws IOException {
                    //PostingsEnum postings = context.reader().postings(new Term(field, term));
                    BinaryDocValues binaryDocValues = context.reader().getBinaryDocValues(field);
                    
                    /*if (postings == null) {
                        // the field and/or term don't exist in this segment, so always return 0
                        return new SearchScript(p, lookup, context) {
                            @Override
                            public double runAsDouble() {
                                return 0.0d;
                            }
                        };
                    }*/
                    return new SearchScript(p, lookup, context) {
                        int currentDocid = -1;
                        @Override
                        public void setDocument(int docid) {
                            // advance has undefined behavior calling with a docid <= its current docid
                            /*if (postings.docID() < docid) {
                                try {
                                    postings.advance(docid);
                                } catch (IOException e) {
                                    throw new UncheckedIOException(e);
                                }
                            }*/
                        	try {
                        		success = binaryDocValues.advanceExact(docid);
							} catch (IOException e) {
								// TODO Auto-generated catch block
								e.printStackTrace();
							}
                            //currentDocid = docid;
                        }
                        @Override
                        public double runAsDouble() {
                            /*if (postings.docID() != currentDocid) {
                                // advance moved past the current doc, so this doc has no occurrences of the term
                                return 0.0d;
                            }
                            try {
                                return postings.freq();
                            } catch (IOException e) {
                                throw new UncheckedIOException(e);
                            }*/
                        	if(success) {
                        		try {
									BytesRef bytesRef = binaryDocValues.binaryValue();
									ByteArrayDataInput dataInput = new ByteArrayDataInput(bytesRef.bytes);
									int i =dataInput.readVInt();
									System.out.println(i);
								} catch (IOException e) {
									// TODO Auto-generated catch block
									e.printStackTrace();
								}
                        	}
                        	return 0.0d;
                        }
                    };
                }

                @Override
                public boolean needs_score() {
                    return false;
                }
            };
            return context.factoryClazz.cast(factory);
        }
        throw new IllegalArgumentException("Unknown script name " + scriptSource);
    }

    @Override
    public void close() {
        // optionally close resources
    }
}

Your code snippet formatting seems to be messed up, making it difficult to tell what is happening. You also appear to be missing the end of the compile method. Can you fix that so we can see the entire method/class?

Sorry for the bad format, corrected original post as above.

Another question is how does p and lookup get initialized in this line?
SearchScript.Factory factory = (p, lookup) -> new SearchScript.LeafFactory()

Sorry for the delayed response here, I missed the update. (p, lookup) -> is a lambda in java. SearchScript.Factory is an interface with a single method taking two parameters, which are p and lookup.

My main concern is on searching on binary fields, can you provide more insights on that (already formatted code above as you requested)?

Let me rephrase my question on (p,lookup). I know how lambda works, what I want to know is how they are initialized. You might want to point me to the source code somewhere.

Binary fields aren't really meant for searching. Their purpose is mainly to allow binary blobs to be returned in search results. I only say this to caution your expectations on gaining 10x performance.

Lambda's aren't "initialized", they are called. It is just a function that returns a new SearchScript.LeafFactory instance. That interface will be called once per lucene segment to create a SearchScript specific to that segment. If you are really curious about the inner workings, you probably want to look at ScriptScoreFunction.getLeafScoreFunction.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.