Filter by attribute possible values dictionary

Hello ES team et all,

We're looking to replace our simple search (C# Entity Framework calls) with a more advanced search function. We have been using the EF solution for anything doing internal lookups for our eCommerce solution (products, categories, dynamic attributes, etc).

I've got a provider started with ElasticSearch (we use a plugin model so separate clients can have different workflows without affecting each other) and already have functioning category filters, price ranges, and some simple field searches. The part that I am getting stuck on is dynamic attribute filters. I can get it to properly filter one dynamic attribute, but when I add a second (that should match a single product) I get no results.

The concept is this: The end user will select one or more attribute values, like Color: [Red, Green] and this will come back to the search as a Dictionary<string,string[]>. After validating that the dictionary and/or it's values aren't null, I need to filter the attributes which are originally a Dictionary<string,SerializableAttributeObject>. SerializableAttributeObject has a schema like { int ID, string Key, object Value, string Group, string UofM ... }. For the purposes of indexing, I've converted this dictionary to a simple array of IndexableSerializableAttributeObject which changes the Value type to string and drops a few unused properties.

The provider essentially started with the example solution with NuGet packages and I've converted it to our document models so I could get up and running quickly. Other than a couple hitches, that example worked wonders to teach me how to start writing to an index and search against it which was awesome.

Mapping

...product...
.Nested<IndexableSerializableAttributeObject>(n => n
    .Name(p => p.Attributes.First())
    .AutoMap()
    .Properties(props => props
        .Text(t => t.Name(a => a.ID).Fields(fs => fs.Text(ss => ss.Name("raw")).Keyword(ss => ss.Name("keyword"))))
        .Text(t => t.Name(a => a.Key).Analyzer("attribute-key-analyzer").Fields(fs => fs.Text(ss => ss.Name("raw").Analyzer("attribute-key-keyword")).Keyword(ss => ss.Name("keyword"))))
        .Text(t => t.Name(a => a.Group).Fields(fs => fs.Text(ss => ss.Name("raw")).Keyword(ss => ss.Name("keyword"))))
        .Text(t => t.Name(a => a.UofM).Fields(fs => fs.Text(ss => ss.Name("raw")).Keyword(ss => ss.Name("keyword"))))
        .Text(t => t.Name(a => a.Value).Fields(fs => fs.Text(ss => ss.Name("raw")).Keyword(ss => ss.Name("keyword"))))
    )
)

Query part (after the FunctionScore and And's to it like the example with authors)

&& +q.Nested(n => n.Name("product-attributes-any")
    .Boost(1.1)
    .InnerHits(i => i.Explain())
    .Path(p => p.Attributes)
    .Query(nq => +nq
        .Bool(AttributeValuesBoolQueryDescriptor)
    )
    .IgnoreUnmapped()
)
...
private BoolQueryDescriptor<IndexableProductModel> AttributeValuesBoolQueryDescriptor(BoolQueryDescriptor<IndexableProductModel> b)
{
    if (Form.Attributes == null || Form.Attributes.Values.Where(x => x != null).SelectMany(x => x).All(string.IsNullOrWhiteSpace)) { return b; }
    var filters = new List<Func<QueryContainerDescriptor<IndexableProductModel>, QueryContainer>>();
    foreach (var attr in Form.Attributes.Where(x => !string.IsNullOrWhiteSpace(x.Key) && x.Value != null && !x.Value.All(string.IsNullOrWhiteSpace)))
    {
        filters.Add(m => m.Term(p => p.Attributes.First().Key.Suffix("keyword"), attr.Key) && m.Terms(t => t.Field(p => p.Attributes.First().Value.Suffix("keyword")).Terms(attr.Value)));
        //filters.Add(m => m.Terms(t => t.Field(p => p.Attributes.First().Value.Suffix("keyword")).Terms(attr.Value)));
    }
    return b.Must(filters);
}

I've tried the above with the commented line instead of the ...&& m.Terms... with the line before it and it's functionally the same.

There are 2 attributes I'm testing with, if I can get these working together I'm sure I can propagate with any data and have it work.

Here is how I pass in the Attributes allowed (filter to these) values dictionary:

Attributes = new Dictionary<string, string[]> { { "Price", new[] { "108.32" } }, { "WebUrl", new[] { "p/845" } } }

There is 1 product with a matching price attribute and 5 with the WebUrl value, of the 5 WebUrl matches, 1 is the one with the Price value match too. Individually, I can get the 1 or 5 results respectively, but together I should get only the one product that has both attributes with these values.

This is just garbage sample data we have, so they wouldn't mean anything in the real world, so long as the concept works, that's what I care about.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

It took a while to circle back to this issue in our system and we finally came to a resolution. We hope this helps other users of ES looking to resolve similar points.

C# Nest Query partial:

#region Attributes
&& +q.Nested(n => n.Name("product-attributes-any")
                   .Boost(1.1)
                   .InnerHits(i => i.Explain())
     .Path(p => p.Attributes)
     .Query(nq => +nq.Bool(AttributeValuesBoolQueryDescriptor))
     .IgnoreUnmapped()
)
#endregion

C# Func
(The lines are broken out more to better fit this page than originally written)

private BoolQueryDescriptor<IndexableProductModel> AttributeValuesBoolQueryDescriptor(
    BoolQueryDescriptor<IndexableProductModel> b)
{
    if (Form.Attributes == null
        || Form.Attributes.Values
               .Where(x => x != null)
               .SelectMany(x => x)
               .All(string.IsNullOrWhiteSpace))
    { return b; }
    // 100% to match all requirements, use lower percentages to allow "fuzzy" matching
    b = b.MinimumShouldMatch(MinimumShouldMatch.Percentage(1.00d));
    return Form.Attributes
        // Skip all blanks/nulls
        .Where(x => !string.IsNullOrWhiteSpace(x.Key)
                 && x.Value != null
                 && !x.Value.All(string.IsNullOrWhiteSpace))
        .Aggregate(b, (current, attr) => current
            .Should(m =>
                 m.Term(p => p.Attributes.First().Key.Suffix("keyword"), attr.Key)
                 && m.Terms(t => t.Field(p => p.Attributes.First().Value
                                  .Suffix("keyword")).Terms(attr.Value))));
}
1 Like