How to add an analyzer that can remove duplicate tokens from the analyzed field?

I want to add a token filter in my custom analyzer to remove duplicate tokens in the text field while matching.

Eg: Input: "random petrol pump, beside random2 petrol pump"
After analyzer: ["random", "petrol", "pump", "beside", "random2"]

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.