How to fetch field content from elasticsearch using java API

Elastic Stack Elasticsearch
1

Hie team,
I am trying to fetch content of "message" field which is of type binary,but it returns null.
In elasticsearch data is stored as below

{
  "_index": "twitter",
  "_type": "_doc",
  "_id": "56",
  "_version": 1,
  "_seq_no": 0,
  "_primary_term": 1,
  "found": true,
  "_source": {
    "message": "JVBERi0xLjcNCg0KNCAwIG9iag0KPDwNCi9FIDYwNDQyDQovSCBbIDM......"
  }
}

My java code looks as below

GetRequest getRequest = new GetRequest("twitter","_doc", "56");
getRequest.fetchSourceContext(FetchSourceContext.FETCH_SOURCE);
getRequest.storedFields("message");
GetResponse getResponse = client().get(getRequest, RequestOptions.DEFAULT);
String message = getResponse.getField("message").getValue();
System.out.println("msg\t"+message);

It throws null pointer exception

When I am trying to execute via postman http://localhost:9200/twitter/_doc/56?stored_fields . It is just returning

{
    "_index": "twitter",
    "_type": "_doc",
    "_id": "56",
    "_version": 1,
    "_seq_no": 0,
    "_primary_term": 1,
    "found": true
}

How to fetch content of message ?

2

What is the mapping?

3

Sorry I didn't get you.
You mean how did I map messaage?
Below is what I did

	String filePath="C:\\Users\\xyz.pdf";
	String encodedfile = null;
	
	File file = new File(filePath);
	try {
	    FileInputStream fileInputStreamReader = new FileInputStream(file);
	    byte[] bytes = new byte[(int) file.length()];
	    fileInputStreamReader.read(bytes);
	    encodedfile = new String(Base64.getEncoder().encodeToString(bytes));
	} catch (IOException e) {
	    e.printStackTrace();
	}
	
	System.out.println(encodedfile);
	//creating index
	CreateIndexRequest request = new CreateIndexRequest("twitter");
	CreateIndexResponse createIndexResponse = client().indices().create(request, RequestOptions.DEFAULT);
    System.out.print(createIndexResponse);
    
	
  
    //mapping properties
    PutMappingRequest request2 = new PutMappingRequest("twitter");
    
    request2.source(
	        "{\n" +
	        "  \"properties\": {\n" +
	        "    \"message\": {\n" +
	        "      \"type\": \"binary\"\n" +
	        "    }\n" +
	        "  }\n" +
	        "}", 
	        XContentType.JSON);
    AcknowledgedResponse putMappingResponse = client().indices().putMapping(request2, RequestOptions.DEFAULT);
   

	IndexRequest request3 = new IndexRequest("twitter","_doc","56"); 
	 request3.source("{\n" + "  \"message\": " +"\""+ encodedfile +"\""+ "}", XContentType.JSON);
	
	IndexResponse createIndexResponse1= client().index(request3, RequestOptions.DEFAULT);
4

Could you provide a full recreation script as described in About the Elasticsearch category. It will help to better understand what you are doing. Please, try to keep the example as simple as possible.

A full reproduction script will help readers to understand, reproduce and if needed fix your problem. It will also most likely help to get a faster answer.

Please provide it as a Kibana script which can be run in the Dev Console not as Java code. Once we will have a working script, we can translate it to java.

5

Here is what I have done
1)created an index named "twitter" and provided properties

PUT twitter
{
  "mappings": {
    "properties": {
      "message": {
        "type": "binary"
      }
    }
  }
}

OUTPUT:-

{
    "acknowledged": true,
    "shards_acknowledged": true,
    "index": "twitter"
}

2)Passing the data to message field

PUT twitter/_doc/56
{
  "message": "JVBERi0xLjcNCg0KNCAwIG9iag0KPDwNCi9FIDYwNDQyDQovSCBbIDM......" 
}

OUTPUT:-

{
    "_index": "twitter",
    "_type": "_doc",
    "_id": "56",
    "_version": 1,
    "result": "created",
    "_shards": {
        "total": 2,
        "successful": 1,
        "failed": 0
    },
    "_seq_no": 1,
    "_primary_term": 1
}

To retrieve the data

GET twitter/_doc/56

OUTPUT:-

{
  "_index": "twitter",
  "_type": "_doc",
  "_id": "56",
  "_version": 1,
  "_seq_no": 0,
  "_primary_term": 1,
  "found": true,
  "_source": {
    "message": "JVBERi0xLjcNCg0KNCAwIG9iag0KPDwNCi9FIDYwNDQyDQovSCBbIDM......"
  }
}

Everything is going smooth.Now i would to extract the content of message field using java by using GET API

GetRequest getRequest = new GetRequest("twitter","_doc", "56");
getRequest.fetchSourceContext(FetchSourceContext.FETCH_SOURCE);
getRequest.storedFields("message");
GetResponse getResponse = client().get(getRequest, RequestOptions.DEFAULT);
String message = getResponse.getField("message").getValue();

But here it doesn't fetch the data under message field. It returns null.
I even tried using but it also doesn't return anything.

"GetResponse response = client.prepareGet("twitter", "_doc","56").setFields("message").execute().actionGet();"

Please I really need help on this.

6

You did not try to run:

GET twitter/_doc/56?stored_fields=message

Otherwise, you'd have seen that it does not produce any field.

So here is the fix:

DELETE twitter
PUT twitter
{
  "mappings": {
    "properties": {
      "message": {
        "type": "binary",
        "store": true
      }
    }
  }
}
PUT twitter/_doc/56
{
  "message": "e1xydGYxXGFuc2kNCkxvcmVtIGlwc3VtIGRvbG9yIHNpdCBhbWV0DQpccGFyIH0=" 
}
GET twitter/_doc/56?stored_fields=message

This gives:

{
  "_index" : "twitter",
  "_type" : "_doc",
  "_id" : "56",
  "_version" : 1,
  "_seq_no" : 0,
  "_primary_term" : 1,
  "found" : true,
  "fields" : {
    "message" : [
      "e1xydGYxXGFuc2kNCkxvcmVtIGlwc3VtIGRvbG9yIHNpdCBhbWV0DQpccGFyIH0="
    ]
  }
}

As you can see in the documentation (Binary field type | Elasticsearch Guide [8.11] | Elastic), store is not enabled by default:

parameter description
doc_values Should the field be stored on disk in a column-stride fashion, so that it can later be used for sorting, aggregations, or scripting? Accepts true or false (default).
store Whether the field value should be stored and retrievable separately from the _source field. Accepts true or false (default).

So this can not produce any content unless you fix the mapping:

getRequest.storedFields("message");
7

Thanku very very much.It got resolved :slight_smile:

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.