How to retrieve all document form a content source of Workplace search

Elastic Search
1

Hello
I am new to Elasticsearch.
Yesterday I created content source and inserted few documents in workplace search.

Now I am trying to retrieve all the documents from that content source. What is the api that I need to use.
I tried couple and everything is erring out please help.

2

You need to use the search API. Search API Reference | Workplace Search Guide [7.15] | Elastic

But I don't think there's a way to extract all the documents. What would be the use case for this?

3

I was trying to show a demo to my team about workplace search today.

As part of it I wanted to show them all the documents that I inserted in to my content source.

so I was trying to find a way to retrieve all the documents in a custom content source that I created similar to select * from a table.

4

Why not just using the standard user interface? I think that'd help to run such a demo.

5

True, I was thinking about the same.
but I read the we can retrieve with index like
curl -X GET "https://{YOUR_SERVER}:9200/{YOUR_INDEX}_search" -H 'Content-Type: application/json' -d

but how do I find index of my content source

6

You should not use directly Elasticsearch when you are using workplace search.

But have a look at this API:

GET /_cat/indices?v

One of the internal indices is the one you are looking for. Check by the number of documents to find its name. Can't recall from the top of my head.

7

Worked...... I was able to get all my documents along with many more information.

I completely agree with you, that I should not use Elasticsearch. My use case is to get familiar with workplace search and evaluate whether we can incorporate in our organization.

So there is no way in workplace search at present to get info of my documents? If I want to update some document and If I don't remember its unique id then what are my options?

8

Good question, @Mymail_Cpn.

  1. One option is to provide the documents with meaningful IDs when you index them. Then you can use the Get document by ID API to retrieve them.
  2. Another option is to use the Search API as @dadoonet originally suggested. With an empty query term, this will return the first 100 documents, and with pagination you can use it to "list" up to 10k documents. Note that since this is a "search", documents that tie in score may be returned in different orders, depending on which shard is queried under the hood.
  3. Be patient. :slight_smile: You're not the first to request a List All Documents API, and we are working to introduce one. However the desire to return results in a consistent order and allow you to scan though an arbitrarily large dataset has some fun and complex technical considerations, which is why it wasn't added originally. Hopefully this will be available in a release soon.

I hope this is helpful!

9

Thank you for providing me with some options. And relieved to hear that I am not the first person to ask.
1. I tried your first option to retrieve a single document by its Id. It worked wonderfully.
2. About the second option search API,
POST /api/ws/v1/search
I get below error :
{ "error" : "no handler found for uri [/api/ws/v1/search?pretty=true] and method [POST]" }

10

That error looks like you may have run it from Kibana dev tools? You can't currently use the Kibana dev tools to interact with Enterprise Search APIs - you'll need to use something like CUrl. Like:

curl -X POST http://localhost:3002/api/ws/v1/search \
-H "Authorization: Bearer $ACCESS_TOKEN" \
-H "Content-Type: application/json" \
-d '{
  "query": ""
}'

Note that the ACCESS_TOKEN here is retrieved by going through an OAuth flow. See: Building Custom Search Experiences | Workplace Search Guide [7.16] | Elastic

11

Yes, you were right I was running it through Kibana dev tools.
Thank you for providing me detail instruction it was very helpful.

12

Good news, @Mymail_Cpn! I can now share that the List Documents API is coming soon! While not yet released, we have put the documentation for it out to the public: see Listing Documents.
This typically indicates that the feature will be available in the next release. So stay tuned!

13

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.