Hi Team,
Currently i am working on a POC wherein I am using Elasticsearch for vector search. I am getting good results for all the queries but when I tried to change the query a bit like changing the subject in query I still get same response, which i feel is bad because contextually it is incorrect.
Problem Statement: In certain instances, the elasticsearch is able to identify and extract information that is semantically accurate and directly relevant to the specific terms or concepts in the query. This means that, at a surface level, the system is able to "understand" the question and retrieve content that appears to answer it correctly.
However, the issue arises when the query specifically references "Company-A," or any specialized term or organization, for which the retrieved answers, though semantically valid, fail to address the underlying context or intent of the question. In other words, while the system may fetch responses that are factually correct or contextually related to the individual keywords in the query, the answers may not be relevant in the specific context of Company-A or the type of information the user is seeking.
Query
How is company-A able to create inclusivity in their organisation ?
Chunk
Company-A: Company-B is one of the few large European insurers with four women on its management board achieving near parity among men and women on the five-member panel. What steps has Company-B taken to achieve this and what can other insurers learn from its approach? Stephane: I'm very proud to work for Company-B a truly diverse company in many aspects. There is space for everyone because we actively decided to address this and ensure we get more diversity in leadership roles. Going further down the ladder to give women a more level playing field for wherever they want to go in their careers we are asking more questions about what they need allowing more flexible work models and rethinking our talent models and leadership formats. I sponsor a series of internal webinars featuring women in senior tech roles because this space is often seen as",
Note:
With semantic search, the following chunk is identified as the best match. This would be relevant if the question were “How is Company-B able to create inclusivity in their organization?” However, the chunk comes from an interview where someone from Company-A is interviewing Stephane from Company-B. Therefore, the statements refer to Company-B, not Company-A.
Ideal response should not contain this chunk when question is asked for company-A.
It would be great help if anyone can give some ideas to deal with such a problem.