Can i use NLP (Question answer) on structured data?

Elastic Search
1

We are using Elasticsearch database. We are planning to provide global search with lot of filters. Our data is mostly structured and there are built in relationships. Having many filters on global search can create usability problems.

Is it possible to perform NLP tasks like question/answer or chat bot on structured data? I believe this will help us to get rid of filters and user will be able to search through questions.

1 Like
2

Hi Ashit,

Thank you for your question! There is ongoing development in the (tech-preview) inference API, which will probably have more task types added in the future. There's also built-in NLP support, which also supports question answering models. Does this help?

3

Hi Tim,
Thanks for prompt response.

I was wondering if it is possible to do following:
User will ask questions like in UI -
How many students are suffering from covid 19?
My index doesn't store values in vector form but I would still like to convert above question into proper elastic query and return result to user.

4

we have some examples of how you would do this within LangChain.

In this example notebook, you provide a schema for your index and it will transform the question into an BM25 only query.

Behind the scenes, LangChain is providing a prompt to transform the query into a structured query, based on the schema and then translating that structured query into a BM25 query. You could do this process manually, without LangChain.

5

Hi Joseph,
This is really useful. I will try this out.

Could you please elaborate more on your comment - we can translate into structured query manually without LangChain

General Question - Any recommendation on security measures?

6

This article may help give you an understanding of whats happening in self querying Self-querying | 🦜️🔗 Langchain

You could do this yourself without needing LangChain, just means more code to write :). The prompt is:

  • providing an instruction to transform the given query into a structured query and giving examples of this.
  • The prompt also provides a schema of the fields available, so the LLM knows what attributes it could use for filtering / searching on

This article may help give context also How to use Elasticsearch to prompt ChatGPT with natural language | Elastic Blog

On security, depends on what security requirements you have. You would need to sanitise the inputs just like you do in any production application.

Joe

7

Thank you !
Appreciate your help

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.