Store and query generic attributes/fields

Elastic Stack Elasticsearch
1

We have a software that allows custom attributes for products.
Example:

   "products" : [
    {
    	"id" : "222222",
    	"sku" : "12345",
    	"name" : "Fancy notebook",
    	"category" : "computer",
    	...
    	"attributes": {
    		"Power_consumption" : "220 W",
    		"Power_requirements" : "AC 100-240V@50-60Hz",
    		"Processor_model" : "Intel Xeon",
    		"Power_supply" : "700 W",
    		"Networking_features" : "10 Gigabit Ethernet",
    		"Ethernet_LAN_(RJ-45)_ports_quantity" : "2",
    		"Weight" : {
    			"unit" : "g",
    			"value" : 2800
    		},
    		"Certification" : "FCC, CE, C-Tick, KCC, VCCI, CCC, RoHS, WEEE",
    		"USB_2_0_ports_quantity" : "2",
    		"Hard_disk_interface" : "Serial ATA"
    		...
    	}
    },
    "product" : {
    	"id" : "111111",
    	"sku" : "23456",
    	"name" : "Network cable",
    	"category" : "accessories",
    	...
    	"attributes": {
    		"Connector_2" : "RJ45",
    		"Connector_1" : "RJ45",
    		"Material" : "ABS synthetics",
    		"Weight" : {
    			"unit" : "g",
    			"value" : 40
    		},
    		"Colour_of_product" : "Black",
    		"Cable_length" : {
    			"unit" : "m",
    			"value" : "1.5 m"
    		}
    		...
    	}
    },
...
]

I have tried to index 2800 of our demo products strait forward. After increasing the index.mapping.total_fields.limit to 7000 the process finished without errors.
7000 total fields doesn't feel right and with >500 000 products this value will be much higher.
Is there a better way to store and search such data structure? This problem looks like very common for online shops and product information systems, so I would expect a solution. But as far I didn't find one. May be I am blind :slight_smile:
All solution, hints suggestions are welcome.

2

I would use an array of nested attributes like : key, unit & value

{
  "attributes": [
    {
      "key": "Power_consumption",
      "value": "220 W"
    },
    {
      "key": "Power_requirements",
      "value": "AC 100-240V@50-60Hz"
    },
    {
      "key": "Weight",
      "unit": "g",
      "value": 2800
    }
  ]
}
3

This only works if all values have the same type. I can convert all values to text, with the drawback of losing type information and functionality (e.g. query a long value that is lesser or greater, dates before, ...).

4

Yes of course :slight_smile:

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.