# How to index and store pdf file in elastic search using spring boot?

**URL:** <https://discuss.elastic.co/t/how-to-index-and-store-pdf-file-in-elastic-search-using-spring-boot/220289>\
**Category:** Elasticsearch\
**Created:** [February 21, 2020, 4:04am UTC](https://discuss.elastic.co/t/how-to-index-and-store-pdf-file-in-elastic-search-using-spring-boot/220289 "2020-02-21T04:04:02Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![samanvi\_sweety](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/samanvi_sweety/32/56190_2.png) [@samanvi\_sweety](https://discuss.elastic.co/u/samanvi_sweety)\
**Post date:** [February 21, 2020, 4:04am UTC](https://discuss.elastic.co/t/how-to-index-and-store-pdf-file-in-elastic-search-using-spring-boot/220289/1 "2020-02-21T04:04:02Z")

</div>

I am new to elastic search. I would like to know how to index and store the pdf files in elastic search using spring boot microservices

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [February 21, 2020, 6:28am UTC](https://discuss.elastic.co/t/how-to-index-and-store-pdf-file-in-elastic-search-using-spring-boot/220289/2 "2020-02-21T06:28:55Z")

</div>

You can use the ingest attachment plugin to parse your PDF documents at index time and extract the meaningful information.

You will need for that to send the binary file as a BASE64 String.

---

<div class="post-metadata">

**Author:** ![samanvi\_sweety](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/samanvi_sweety/32/56190_2.png) [@samanvi\_sweety](https://discuss.elastic.co/u/samanvi_sweety)\
**Post date:** [February 21, 2020, 8:37am UTC](https://discuss.elastic.co/t/how-to-index-and-store-pdf-file-in-elastic-search-using-spring-boot/220289/3 "2020-02-21T08:37:03Z")

</div>

Hie David,  
I tried using ingest plugin

-------This is the function which is called in postman inorder to upload pdf file in es---------

```auto
@PostMapping("/upload")
 public String upload()
 {
	 
	String filePath = "C://x.pdf";
	String encodedfile = null;
	RestHighLevelClient restHighLevelClient = null;
	File file = new File(filePath);
	try {
	    FileInputStream fileInputStreamReader = new FileInputStream(file);
	    byte[] bytes = new byte[(int) file.length()];
	    fileInputStreamReader.read(bytes);
	    encodedfile = new String(Base64.getEncoder().encodeToString(bytes));
	} catch (IOException e) {
	    e.printStackTrace();
	}
	try {
	    if (restHighLevelClient != null) {
	        restHighLevelClient.close();
	    }
	} catch (final Exception e) {
	    System.out.println("Error closing ElasticSearch client: ");
	}
	try {
	    restHighLevelClient = new RestHighLevelClient(RestClient.builder(new HttpHost("localhost", 9200, "http"),
	            new HttpHost("localhost", 9201, "http")));
	} catch (Exception e) {
	    System.out.println(e.getMessage());
	}
	
	
	Map<String, Object> jsonMap = new HashMap<>();
	jsonMap.put("Name", "samanvi");
	jsonMap.put("postDate", new Date());
	jsonMap.put("hra", encodedfile);
 

	IndexRequest request = new IndexRequest("index","pdf","56")
			.index("index")
			.source("field",jsonMap)
	        .setPipeline("samanvi");
	System.out.println("pipeline"+request.getPipeline());
	System.out.println("index"+request.index());
	try {
	    IndexResponse response = restHighLevelClient.index(request,RequestOptions.DEFAULT);
	} catch(ElasticsearchException | IOException e) {
	    if (((ElasticsearchException) e).status() == RestStatus.CONFLICT) {
	    }	
 }
	return "uploaded";
 }

```

As per the above code the pipeline should get created with index ="index" but It isn't created

but then i executed the below one

```auto
PUT _ingest/pipeline/samanvi
{
{"description" : "Extract attachment information",
  "processors" : [
    {
      "attachment" : {
        "field" : "hra"
      }
    }
  ]
}
}

```

and the got the acknowledgement true and pipeline got created  
Is there anything wrong with my code?

And when I am executing the get call  
[http://localhost:9200/index/pdf/56?pipeline=samanvi](http://localhost:9200/index/pdf/56?pipeline=samanvi)

I am unable to fetch any details  
and I am getting the following error

```auto
{
    "error": {
        "root_cause": [
            {
                "type": "illegal_argument_exception",
                "reason": "request [/index/pdf/56] contains unrecognized parameter: [pipeline]"
            }
        ],
        "type": "illegal_argument_exception",
        "reason": "request [/index/pdf/56] contains unrecognized parameter: [pipeline]"
    },
    "status": 400
}

```

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [February 21, 2020, 10:06am UTC](https://discuss.elastic.co/t/how-to-index-and-store-pdf-file-in-elastic-search-using-spring-boot/220289/4 "2020-02-21T10:06:17Z")

</div>

Please format your code, logs or configuration files using `</>` icon as explained in [this guide](https://discuss.elastic.co/t/about-the-elasticsearch-category/21) and not the citation button. It will make your post more readable.

Or use markdown style like:

````
```
CODE
```

````

This is the icon to use if you are not using markdown format:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/7/e/7e6e239431ec2d71cbf1beef741f2e93e7cc762c.jpg)

There's a live preview panel for exactly this reasons.

Lots of people read these forums, and many of them will simply skip over a post that is difficult to read, because it's just too large an investment of their time to try and follow a wall of badly formatted text.  
If your goal is to get an answer to your questions, it's in your interest to make it as easy to read and understand as possible.  
Please update your post.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [February 21, 2020, 10:07am UTC](https://discuss.elastic.co/t/how-to-index-and-store-pdf-file-in-elastic-search-using-spring-boot/220289/5 "2020-02-21T10:07:11Z")

</div>

> [@samanvi\_sweety](#):
>
> And when I am executing the get call  
> [http://localhost:9200/index/pdf/56?pipeline=samanvi](http://localhost:9200/index/pdf/56?pipeline=samanvi)

It does not work with `GET`.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [February 21, 2020, 10:07am UTC](https://discuss.elastic.co/t/how-to-index-and-store-pdf-file-in-elastic-search-using-spring-boot/220289/6 "2020-02-21T10:07:49Z")

</div>

BTW you should use `_doc` as the type instead of `pdf`.

---

<div class="post-metadata">

**Author:** ![samanvi\_sweety](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/samanvi_sweety/32/56190_2.png) [@samanvi\_sweety](https://discuss.elastic.co/u/samanvi_sweety)\
**Post date:** [February 21, 2020, 11:08am UTC](https://discuss.elastic.co/t/how-to-index-and-store-pdf-file-in-elastic-search-using-spring-boot/220289/7 "2020-02-21T11:08:01Z")

</div>

I used \_doc in my upload() function.  
How to check whether the pipeline is created and file is uploaded ?

GET call  
[http://localhost:9200/\_ingest/pipeline/samanvi](http://localhost:9200/_ingest/pipeline/samanvi)  
returns that pipeline with id [samanvi] does not exist

---

<div class="post-metadata">

**Author:** ![samanvi\_sweety](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/samanvi_sweety/32/56190_2.png) [@samanvi\_sweety](https://discuss.elastic.co/u/samanvi_sweety)\
**Post date:** [February 21, 2020, 11:22am UTC](https://discuss.elastic.co/t/how-to-index-and-store-pdf-file-in-elastic-search-using-spring-boot/220289/8 "2020-02-21T11:22:23Z")

</div>

okay

---

<div class="post-metadata">

**Author:** ![samanvi\_sweety](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/samanvi_sweety/32/56190_2.png) [@samanvi\_sweety](https://discuss.elastic.co/u/samanvi_sweety)\
**Post date:** [February 25, 2020, 3:35am UTC](https://discuss.elastic.co/t/how-to-index-and-store-pdf-file-in-elastic-search-using-spring-boot/220289/9 "2020-02-25T03:35:07Z")

</div>

Hi,  
Can you please look into to my above problem.  
The pipleine is not getting created .  
How to make sure that pdf is saved in Elastic search

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [February 25, 2020, 8:23am UTC](https://discuss.elastic.co/t/how-to-index-and-store-pdf-file-in-elastic-search-using-spring-boot/220289/10 "2020-02-25T08:23:18Z")

</div>

I have no idea what the current status of your problem is.

Could you provide a full recreation script as described in [About the Elasticsearch category](https://discuss.elastic.co/t/about-the-elasticsearch-category/21). It will help to better understand what you are doing. Please, try to keep the example as simple as possible.

A full reproduction script will help readers to understand, reproduce and if needed fix your problem. It will also most likely help to get a faster answer.

Once we will have a fully functional script, we will be able to move on the Java code.

---

<div class="post-metadata">

**Author:** ![samanvi\_sweety](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/samanvi_sweety/32/56190_2.png) [@samanvi\_sweety](https://discuss.elastic.co/u/samanvi_sweety)\
**Post date:** [March 5, 2020, 6:09am UTC](https://discuss.elastic.co/t/how-to-index-and-store-pdf-file-in-elastic-search-using-spring-boot/220289/13 "2020-03-05T06:09:27Z")

</div>

My task is to store the pdf files in elastic search using ingest plugin and spring boot.

Below mentioned function is to store the pdf files in ES

```auto
@PostMapping("/upload")
 public String upload()
 {
	 
	String filePath = "C://x.pdf";
	String encodedfile = null;
	RestHighLevelClient restHighLevelClient = null;
	File file = new File(filePath);
	try {
	    FileInputStream fileInputStreamReader = new FileInputStream(file);
	    byte[] bytes = new byte[(int) file.length()];
	    fileInputStreamReader.read(bytes);
	    encodedfile = new String(Base64.getEncoder().encodeToString(bytes));
	} catch (IOException e) {
	    e.printStackTrace();
	}
	try {
	    if (restHighLevelClient != null) {
	        restHighLevelClient.close();
	    }
	} catch (final Exception e) {
	    System.out.println("Error closing ElasticSearch client: ");
	}
	try {
	    restHighLevelClient = new RestHighLevelClient(RestClient.builder(new HttpHost("localhost", 9200, "http"),
	            new HttpHost("localhost", 9201, "http")));
	} catch (Exception e) {
	    System.out.println(e.getMessage());
	}
	
	
	Map<String, Object> jsonMap = new HashMap<>();
	jsonMap.put("Name", "samanvi");
	jsonMap.put("postDate", new Date());
	jsonMap.put("hra", encodedfile);
 

	IndexRequest request = new IndexRequest("index","_doc","56")
			.index("index")
			.source("field",jsonMap)
	        .setPipeline("samanvi");
	System.out.println("pipeline"+request.getPipeline());
	System.out.println("index"+request.index());
	try {
	    IndexResponse response = restHighLevelClient.index(request,RequestOptions.DEFAULT);
	} catch(ElasticsearchException | IOException e) {
	    if (((ElasticsearchException) e).status() == RestStatus.CONFLICT) {
	    }	
 }
	return "uploaded";
 }

```

So as per the above code when I hit the post url in postman it has to create the pipeline with name samanvi and store in ES

But GET call  
[http://localhost:9200/\_ingest/pipeline/samanvi](http://localhost:9200/_ingest/pipeline/samanvi)  
returns that pipeline with id [samanvi] does not exist

My doubts?  
1)I would like to know why pipeline is not created  
2) how to check whether the pdf is stored in ES if pipeline is created

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [March 5, 2020, 9:45am UTC](https://discuss.elastic.co/t/how-to-index-and-store-pdf-file-in-elastic-search-using-spring-boot/220289/14 "2020-03-05T09:45:11Z")

</div>

How did you define the pipeline?

Could you run from Kibana dev console:

```auto
GET /_ingest/samanvi

```

And share the output here?

---

<div class="post-metadata">

**Author:** ![samanvi\_sweety](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/samanvi_sweety/32/56190_2.png) [@samanvi\_sweety](https://discuss.elastic.co/u/samanvi_sweety)\
**Post date:** [March 6, 2020, 3:47am UTC](https://discuss.elastic.co/t/how-to-index-and-store-pdf-file-in-elastic-search-using-spring-boot/220289/15 "2020-03-06T03:47:50Z")

</div>

This is how I defined the pipeline in my function

```auto
IndexRequest request = new IndexRequest("index","_doc","56")
			.index("index")
			.source("field",jsonMap)
	        .setPipeline("samanvi");

```

When I am running the below command from kibana console, the output is

```auto
GET/_ingest/samanvi 

```

```auto
{
  "error": "Incorrect HTTP method for uri [/_ingest/samanvi?pretty] and method [GET], allowed: [POST]",
  "status": 405
}

```

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [March 6, 2020, 7:39am UTC](https://discuss.elastic.co/t/how-to-index-and-store-pdf-file-in-elastic-search-using-spring-boot/220289/16 "2020-03-06T07:39:39Z")

</div>

You did not create an ingest pipeline.  
If you don't need an ingest pipeline (probably you don't know what it is), just remove `.setPipeline("samanvi")` from your code.

---

<div class="post-metadata">

**Author:** ![samanvi\_sweety](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/samanvi_sweety/32/56190_2.png) [@samanvi\_sweety](https://discuss.elastic.co/u/samanvi_sweety)\
**Post date:** [March 6, 2020, 8:59am UTC](https://discuss.elastic.co/t/how-to-index-and-store-pdf-file-in-elastic-search-using-spring-boot/220289/17 "2020-03-06T08:59:19Z")

</div>

I would like to know how to create ingest pipeline by using the function.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [March 6, 2020, 9:07am UTC](https://discuss.elastic.co/t/how-to-index-and-store-pdf-file-in-elastic-search-using-spring-boot/220289/18 "2020-03-06T09:07:12Z")

</div>

An ingest pipeline which does what?

---

<div class="post-metadata">

**Author:** ![samanvi\_sweety](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/samanvi_sweety/32/56190_2.png) [@samanvi\_sweety](https://discuss.elastic.co/u/samanvi_sweety)\
**Post date:** [March 10, 2020, 3:21am UTC](https://discuss.elastic.co/t/how-to-index-and-store-pdf-file-in-elastic-search-using-spring-boot/220289/19 "2020-03-10T03:21:55Z")

</div>

> [@dadoonet](#):
>
> ingest pipeline which does what

An ingest pipeline which store pdf files

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [March 10, 2020, 6:10am UTC](https://discuss.elastic.co/t/how-to-index-and-store-pdf-file-in-elastic-search-using-spring-boot/220289/20 "2020-03-10T06:10:09Z")

</div>

Did you create it?

---

<div class="post-metadata">

**Author:** ![samanvi\_sweety](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/samanvi_sweety/32/56190_2.png) [@samanvi\_sweety](https://discuss.elastic.co/u/samanvi_sweety)\
**Post date:** [March 10, 2020, 7:00am UTC](https://discuss.elastic.co/t/how-to-index-and-store-pdf-file-in-elastic-search-using-spring-boot/220289/21 "2020-03-10T07:00:23Z")

</div>

> [@dadoonet](#):
>
> ingest attachment plugin to parse your PDF documents at index time and extract the meaningful information.

No That is the problem I am facing with. I would like to know how to create pipeline using java so that i can store the pdfs

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [March 10, 2020, 7:16am UTC](https://discuss.elastic.co/t/how-to-index-and-store-pdf-file-in-elastic-search-using-spring-boot/220289/22 "2020-03-10T07:16:29Z")

</div>

Did you read the documentation here? [https://www.elastic.co/guide/en/elasticsearch/plugins/7.6/ingest-attachment.html](https://www.elastic.co/guide/en/elasticsearch/plugins/7.6/ingest-attachment.html)

[Next page](https://discuss.elastic.co/t/how-to-index-and-store-pdf-file-in-elastic-search-using-spring-boot/220289.md?page=2)