# 0.90.RC2 postings format **URL:** \ **Category:** Elasticsearch\ **Created:** [April 13, 2013, 9:34pm UTC](https://discuss.elastic.co/t/0-90-rc2-postings-format/11572 "2013-04-13T21:34:17Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Rafal\_Kuc\_3](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rafal_kuc_3/32/799_2.png) [@Rafal\_Kuc\_3](https://discuss.elastic.co/u/Rafal_Kuc_3)\ **Post date:** [April 13, 2013, 9:34pm UTC](https://discuss.elastic.co/t/0-90-rc2-postings-format/11572/1 "2013-04-13T21:34:17Z")
Hello! I've got a question about the postings format. When reading the documentation we can see that there is a bloom posting format type. However when trying to use it ElasticSearch throws an exception, for example: curl -XPOST 'localhost:9200/posts' -d '{ "settings" : { "index" : { "codec" : { "postings\_format" : { "custom" : { "type" : "bloom", "delegate" : "pulsing" } } } } }, "mappings" : { "post" : { "properties" : { "id" : { "type" : "long", "store" : "yes", "precision\_step" : "0", "postings\_format" : "custom" }, "name" : { "type" : "string", "store" : "yes", "index" : "analyzed" }, "contents" : { "type" : "string", "store" : "no", "index" : "analyzed" } } } } }' And the exception is as follows: {"error":"IndexCreationException[[posts] failed to create index]; nested: NoClassSettingsException[Failed to load class setting [type] with value [bloom]]; nested: ClassNotFoundException[org.elasticsearch.index.codec.postingsformat.bloom.BloomPostingsFormatProvider]; ", "status":500} According to the code there we are allowed to use the bloom\_default or bloom\_pulsing types, but not the bloom itself (at least as the pre-configured ones). And of course when configuring the id field with one of the mentioned postings format it works without any problem, which can be seen in the mappings: { "posts" : { "post" : { "properties" : { "contents" : { "type" : "string" }, "id" : { "type" : "long", "store" : true, "postings\_format" : "bloom\_pulsing", "precision\_step" : 2147483647 }, "name" : { "type" : "string", "store" : true } } } } } Am I missing something when it comes to the bloom type ? I'm using 0.90.RC2. Thanks for the answer. -- Regards, Rafał Kuć Sematext :: [http://sematext.com/](http://sematext.com/) :: Solr - Lucene - Nutch - ElasticSearch -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [April 14, 2013, 10:33am UTC](https://discuss.elastic.co/t/0-90-rc2-postings-format/11572/2 "2013-04-14T10:33:38Z")
The bloom codec needs to wrap another codec. Using "bloom" means "maintain a bloom filter in memory" but doesn't specify how the data should be stored on disk. > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. That said, it could throw a better error message > > > If the \`postings\_format\` is specified as just \`bloom\`, then it throws a class-no…t-found exception. Instead, would be better to have a message like "The bloom codec cannot be used by itself. Use bloom\_default or bloom\_pulsing instead" On Sat, Apr 13, 2013 at 11:34 PM, Rafał Kuć [r.kuc@solr.pl](mailto:r.kuc@solr.pl) wrote: > Hello! > > I've got a question about the postings format. When reading the > documentation we can see that there is a bloom posting format type. > However when trying to use it Elasticsearch throws an exception, for > example: > > curl -XPOST 'localhost:9200/posts' -d '{ > "settings" : { > "index" : { > "codec" : { > "postings\_format" : { > "custom" : { > "type" : "bloom", > "delegate" : "pulsing" > } > } > } > } > }, > "mappings" : { > "post" : { > "properties" : { > "id" : { "type" : "long", "store" : "yes", "precision\_step" : "0", > "postings\_format" : "custom" }, > "name" : { "type" : "string", "store" : "yes", "index" : "analyzed" }, > "contents" : { "type" : "string", "store" : "no", "index" : "analyzed" > } > } > } > } > }' > > And the exception is as follows: > {"error":"IndexCreationException[[posts] failed to create index]; nested: > NoClassSettingsException[Failed to load class setting > [type] with value [bloom]]; nested: > ClassNotFoundException[org.elasticsearch.index.codec.postingsformat.bloom.BloomPostingsFormatProvider]; > ", > "status":500} > > According to the code there we are allowed to use the bloom\_default or > bloom\_pulsing types, but not the bloom itself (at least as the > pre-configured ones). And of course when configuring the id field with > one of the mentioned postings format it works without any problem, > which can be seen in the mappings: > > { > "posts" : { > "post" : { > "properties" : { > "contents" : { > "type" : "string" > }, > "id" : { > "type" : "long", > "store" : true, > "postings\_format" : "bloom\_pulsing", > "precision\_step" : 2147483647 > }, > "name" : { > "type" : "string", > "store" : true > } > } > } > } > } > > Am I missing something when it comes to the bloom type ? I'm using > 0.90.RC2. Thanks for the answer. > > -- > Regards, > Rafał Kuć > Sematext :: [http://sematext.com/](http://sematext.com/) :: Solr - Lucene - Nutch - Elasticsearch > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Rafal\_Kuc\_3](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rafal_kuc_3/32/799_2.png) [@Rafal\_Kuc\_3](https://discuss.elastic.co/u/Rafal_Kuc_3)\ **Post date:** [April 14, 2013, 12:00pm UTC](https://discuss.elastic.co/t/0-90-rc2-postings-format/11572/3 "2013-04-14T12:00:17Z")
Thanks Clinton, I'm aware of the bloom\_pulsing and bloom\_default postings formats. I was wondering if I'm missing something after looking at the docs at [http://www.elasticsearch.org/guide/reference/index-modules/codec/,](http://www.elasticsearch.org/guide/reference/index-modules/codec/) because of the type name "bloom". I thought one can use the "bloom" type when defining a custom postings format and set the appropriate delegate, to for example pulsing or default. But now, I've got another question. Is it possible to use a custom bloom filter based codec, like the bloom\_default or bloom\_pulsing ? For example, the following request: curl -XPOST 'localhost:9200/posts/' -d '{ "settings" : { "index" : { "codec" : { ``` "postings_format" : { "custom" : { "type" : "bloom_default", "delegate" : "default", "ffp" : "10k=0.01,1m=0.03" } } ``` } } }, "mappings" : { "post" : { "properties" : { ``` "id" : { "type" : "long", "store" : "yes", "precision_step" : "0", "postings_format" : "custom" }, "name" : { "type" : "string", "store" : "yes", "index" : "analyzed" }, "contents" : { "type" : "string", "store" : "no", "index" : "analyzed" } ``` } } } }' Gives the following exception: {"error":"IndexCreationException[[posts] failed to create index]; nested: NoClassSettingsException[Failed to load class setting [type] with value [bloom\_default]]; nested: ClassNotFoundException[org.elasticsearch.index.codec.postingsformat.bloomdefault.BloomDefaultPostingsFormatProvider]; ","status":500} _--_ Regards, Rafał Kuć Sematext :: [http://sematext.com/](http://sematext.com/) _:: Solr - Lucene - Nutch - ElasticSearch_ The bloom codec needs to wrap another codec. Using "bloom" means "maintain a bloom filter in memory" but doesn't specify how the data should be stored on disk. [http://www.elasticsearch.org/guide/reference/index-modules/codec/](http://www.elasticsearch.org/guide/reference/index-modules/codec/) That said, it could throw a better error message [https://github.com/elasticsearch/elasticsearch/issues/2893](https://github.com/elasticsearch/elasticsearch/issues/2893) On Sat, Apr 13, 2013 at 11:34 PM, Rafał Kuć \<[r.kuc@solr.pl](mailto:r.kuc@solr.pl)\> wrote: Hello! I've got a question about the postings format. When reading the documentation we can see that there is a bloom posting format type. However when trying to use it ElasticSearch throws an exception, for example: curl -XPOST 'localhost:9200/posts' -d '{ "settings" : { "index" : { "codec" : { ``` "postings_format" : { "custom" : { "type" : "bloom", "delegate" : "pulsing" } } ``` } } }, "mappings" : { "post" : { "properties" : { ``` "id" : { "type" : "long", "store" : "yes", "precision_step" : "0", "postings_format" : "custom" }, "name" : { "type" : "string", "store" : "yes", "index" : "analyzed" }, "contents" : { "type" : "string", "store" : "no", "index" : "analyzed" } ``` } } } }' And the exception is as follows: {"error":"IndexCreationException[[posts] failed to create index]; nested: NoClassSettingsException[Failed to load class setting [type] with value [bloom]]; nested: ClassNotFoundException[org.elasticsearch.index.codec.postingsformat.bloom.BloomPostingsFormatProvider]; ", "status":500} According to the code there we are allowed to use the bloom\_default or bloom\_pulsing types, but not the bloom itself (at least as the pre-configured ones). And of course when configuring the id field with one of the mentioned postings format it works without any problem, which can be seen in the mappings: { "posts" : { ``` "post" : { "properties" : { "contents" : { "type" : "string" }, "id" : { "type" : "long", "store" : true, "postings_format" : "bloom_pulsing", "precision_step" : 2147483647 }, "name" : { "type" : "string", "store" : true } } } ``` } } Am I missing something when it comes to the bloom type ? I'm using 0.90.RC2. Thanks for the answer. -- Regards, Rafał Kuć Sematext :: [http://sematext.com/](http://sematext.com/) :: Solr - Lucene - Nutch - ElasticSearch -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch%2Bunsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Jerome\_Gagnon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jerome_gagnon/32/2178_2.png) [@Jerome\_Gagnon](https://discuss.elastic.co/u/Jerome_Gagnon)\ **Post date:** [April 19, 2013, 1:55pm UTC](https://discuss.elastic.co/t/0-90-rc2-postings-format/11572/4 "2013-04-19T13:55:49Z")
I'm experiencing the same issue, so any updates would be appreciated, thanks ! On Sunday, April 14, 2013 8:00:17 AM UTC-4, Rafał Kuć wrote: > Thanks Clinton, > > I'm aware of the bloom\_pulsing and bloom\_default postings formats. I was > wondering if I'm missing something after looking at the docs at > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/codec/,)[http://www.elasticsearch.org/guide/reference/index-modules/codec/](http://www.elasticsearch.org/guide/reference/index-modules/codec/) because > of the type name "bloom". I thought one can use the "bloom" type when > defining a custom postings format and set the appropriate delegate, to for > example pulsing or default. > > But now, I've got another question. Is it possible to use a custom bloom > filter based codec, like the bloom\_default or bloom\_pulsing ? > > For example, the following request: > > curl -XPOST 'localhost:9200/posts/' -d '{ > "settings" : { > "index" : { > "codec" : { > "postings\_format" : { > "custom" : { > "type" : "bloom\_default", > "delegate" : "default", > "ffp" : "10k=0.01,1m=0.03" > } > } > } > } > }, > "mappings" : { > "post" : { > "properties" : { > "id" : { "type" : "long", "store" : "yes", "precision\_step" : "0", > "postings\_format" : "custom" }, > "name" : { "type" : "string", "store" : "yes", "index" : "analyzed" }, > "contents" : { "type" : "string", "store" : "no", "index" : "analyzed" > } > } > } > } > }' > > Gives the following exception: > > {"error":"IndexCreationException[[posts] failed to create index]; nested: > NoClassSettingsException[Failed to load class setting [type] with value > [bloom\_default]]; nested: > ClassNotFoundException[org.elasticsearch.index.codec.postingsformat.bloomdefault.BloomDefaultPostingsFormatProvider]; > ","status":500} > > \*-- > Regards, > Rafał Kuć > Sematext :: _[http://sematext.com/](http://sematext.com/)_ :: Solr - Lucene - Nutch - > Elasticsearch > > - > > The bloom codec needs to wrap another codec. Using "bloom" means > "maintain a bloom filter in memory" but doesn't specify how the data should > be stored on disk. > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/codec/) > > That said, it could throw a better error message > > [Better error message when using bloom codec · Issue #2893 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/2893) > > On Sat, Apr 13, 2013 at 11:34 PM, Rafał Kuć \ > > > wrote: > > Hello! > > I've got a question about the postings format. When reading the > documentation we can see that there is a bloom posting format type. > However when trying to use it Elasticsearch throws an exception, for > example: > > curl -XPOST 'localhost:9200/posts' -d '{ > "settings" : { > "index" : { > "codec" : { > "postings\_format" : { > "custom" : { > "type" : "bloom", > "delegate" : "pulsing" > } > } > } > } > }, > "mappings" : { > "post" : { > "properties" : { > "id" : { "type" : "long", "store" : "yes", "precision\_step" : "0", > "postings\_format" : "custom" }, > "name" : { "type" : "string", "store" : "yes", "index" : "analyzed" }, > "contents" : { "type" : "string", "store" : "no", "index" : "analyzed" > } > } > } > } > }' > > And the exception is as follows: > {"error":"IndexCreationException[[posts] failed to create index]; nested: > NoClassSettingsException[Failed to load class setting > [type] with value [bloom]]; nested: > ClassNotFoundException[org.elasticsearch.index.codec.postingsformat.bloom.BloomPostingsFormatProvider]; > ", > "status":500} > > According to the code there we are allowed to use the bloom\_default or > bloom\_pulsing types, but not the bloom itself (at least as the > pre-configured ones). And of course when configuring the id field with > one of the mentioned postings format it works without any problem, > which can be seen in the mappings: > > { > "posts" : { > "post" : { > "properties" : { > "contents" : { > "type" : "string" > }, > "id" : { > "type" : "long", > "store" : true, > "postings\_format" : "bloom\_pulsing", > "precision\_step" : 2147483647 > }, > "name" : { > "type" : "string", > "store" : true > } > } > } > } > } > > Am I missing something when it comes to the bloom type ? I'm using > 0.90.RC2. Thanks for the answer. > > -- > Regards, > Rafał Kuć > Sematext :: [http://sematext.com/](http://sematext.com/) :: Solr - Lucene - Nutch - Elasticsearch > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Jerome\_Gagnon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jerome_gagnon/32/2178_2.png) [@Jerome\_Gagnon](https://discuss.elastic.co/u/Jerome_Gagnon)\ **Post date:** [April 19, 2013, 1:57pm UTC](https://discuss.elastic.co/t/0-90-rc2-postings-format/11572/5 "2013-04-19T13:57:50Z")
Just found... the class on Elasticsearch source is BloomFilterPostingsFormatProvider, so the type to user is "bloom\_filter". The documentation should be updated accordingly. On Friday, April 19, 2013 9:55:49 AM UTC-4, Jérôme Gagnon wrote: > I'm experiencing the same issue, so any updates would be appreciated, > thanks ! > > On Sunday, April 14, 2013 8:00:17 AM UTC-4, Rafał Kuć wrote: > > > Thanks Clinton, > > > > I'm aware of the bloom\_pulsing and bloom\_default postings formats. I was > > wondering if I'm missing something after looking at the docs at > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/codec/,)[http://www.elasticsearch.org/guide/reference/index-modules/codec/](http://www.elasticsearch.org/guide/reference/index-modules/codec/) because > > of the type name "bloom". I thought one can use the "bloom" type when > > defining a custom postings format and set the appropriate delegate, to for > > example pulsing or default. > > > > But now, I've got another question. Is it possible to use a custom bloom > > filter based codec, like the bloom\_default or bloom\_pulsing ? > > > > For example, the following request: > > > > curl -XPOST 'localhost:9200/posts/' -d '{ > > "settings" : { > > "index" : { > > "codec" : { > > "postings\_format" : { > > "custom" : { > > "type" : "bloom\_default", > > "delegate" : "default", > > "ffp" : "10k=0.01,1m=0.03" > > } > > } > > } > > } > > }, > > "mappings" : { > > "post" : { > > "properties" : { > > "id" : { "type" : "long", "store" : "yes", "precision\_step" : "0", > > "postings\_format" : "custom" }, > > "name" : { "type" : "string", "store" : "yes", "index" : "analyzed" }, > > "contents" : { "type" : "string", "store" : "no", "index" : > > "analyzed" } > > } > > } > > } > > }' > > > > Gives the following exception: > > > > {"error":"IndexCreationException[[posts] failed to create index]; nested: > > NoClassSettingsException[Failed to load class setting [type] with value > > [bloom\_default]]; nested: > > ClassNotFoundException[org.elasticsearch.index.codec.postingsformat.bloomdefault.BloomDefaultPostingsFormatProvider]; > > ","status":500} > > > > \*-- > > Regards, > > Rafał Kuć > > Sematext :: _[http://sematext.com/](http://sematext.com/)_ :: Solr - Lucene - Nutch - > > Elasticsearch > > > > - > > > > The bloom codec needs to wrap another codec. Using "bloom" means > > "maintain a bloom filter in memory" but doesn't specify how the data should > > be stored on disk. > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/codec/) > > > > That said, it could throw a better error message > > > > [Better error message when using bloom codec · Issue #2893 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/2893) > > > > On Sat, Apr 13, 2013 at 11:34 PM, Rafał Kuć [r....@solr.pl](mailto:r....@solr.pl) wrote: > > Hello! > > > > I've got a question about the postings format. When reading the > > documentation we can see that there is a bloom posting format type. > > However when trying to use it Elasticsearch throws an exception, for > > example: > > > > curl -XPOST 'localhost:9200/posts' -d '{ > > "settings" : { > > "index" : { > > "codec" : { > > "postings\_format" : { > > "custom" : { > > "type" : "bloom", > > "delegate" : "pulsing" > > } > > } > > } > > } > > }, > > "mappings" : { > > "post" : { > > "properties" : { > > "id" : { "type" : "long", "store" : "yes", "precision\_step" : "0", > > "postings\_format" : "custom" }, > > "name" : { "type" : "string", "store" : "yes", "index" : "analyzed" }, > > "contents" : { "type" : "string", "store" : "no", "index" : > > "analyzed" } > > } > > } > > } > > }' > > > > And the exception is as follows: > > {"error":"IndexCreationException[[posts] failed to create index]; nested: > > NoClassSettingsException[Failed to load class setting > > [type] with value [bloom]]; nested: > > ClassNotFoundException[org.elasticsearch.index.codec.postingsformat.bloom.BloomPostingsFormatProvider]; > > ", > > "status":500} > > > > According to the code there we are allowed to use the bloom\_default or > > bloom\_pulsing types, but not the bloom itself (at least as the > > pre-configured ones). And of course when configuring the id field with > > one of the mentioned postings format it works without any problem, > > which can be seen in the mappings: > > > > { > > "posts" : { > > "post" : { > > "properties" : { > > "contents" : { > > "type" : "string" > > }, > > "id" : { > > "type" : "long", > > "store" : true, > > "postings\_format" : "bloom\_pulsing", > > "precision\_step" : 2147483647 > > }, > > "name" : { > > "type" : "string", > > "store" : true > > } > > } > > } > > } > > } > > > > Am I missing something when it comes to the bloom type ? I'm using > > 0.90.RC2. Thanks for the answer. > > > > -- > > Regards, > > Rafał Kuć > > Sematext :: [http://sematext.com/](http://sematext.com/) :: Solr - Lucene - Nutch - > > Elasticsearch > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Jerome\_Gagnon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jerome_gagnon/32/2178_2.png) [@Jerome\_Gagnon](https://discuss.elastic.co/u/Jerome_Gagnon)\ **Post date:** [April 19, 2013, 6:00pm UTC](https://discuss.elastic.co/t/0-90-rc2-postings-format/11572/6 "2013-04-19T18:00:41Z")
Hey Rafal, While we are on it; Is it just me or the description of pulsing and bloom filter codec on this page; [http://elasticsearchserverbook.com/elasticsearch-0-90-using-codecs/](http://elasticsearchserverbook.com/elasticsearch-0-90-using-codecs/) saying that it's appropriate for LOW cardinality field is wrong ? Pulsing and bloom are good for id lookup, so that's high cardinality field. An id field is normally the highest possible cardinality ? Maybe I'm wrong, but if so I would like to know what I don't understand 🙂 Jerome On Friday, April 19, 2013 9:57:50 AM UTC-4, Jérôme Gagnon wrote: > Just found... the class on Elasticsearch source is > BloomFilterPostingsFormatProvider, so the type to user is "bloom\_filter". > > The documentation should be updated accordingly. > > On Friday, April 19, 2013 9:55:49 AM UTC-4, Jérôme Gagnon wrote: > > > I'm experiencing the same issue, so any updates would be appreciated, > > thanks ! > > > > On Sunday, April 14, 2013 8:00:17 AM UTC-4, Rafał Kuć wrote: > > > > > Thanks Clinton, > > > > > > I'm aware of the bloom\_pulsing and bloom\_default postings formats. I was > > > wondering if I'm missing something after looking at the docs at > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/codec/,)[http://www.elasticsearch.org/guide/reference/index-modules/codec/](http://www.elasticsearch.org/guide/reference/index-modules/codec/) because > > > of the type name "bloom". I thought one can use the "bloom" type when > > > defining a custom postings format and set the appropriate delegate, to for > > > example pulsing or default. > > > > > > But now, I've got another question. Is it possible to use a custom bloom > > > filter based codec, like the bloom\_default or bloom\_pulsing ? > > > > > > For example, the following request: > > > > > > curl -XPOST 'localhost:9200/posts/' -d '{ > > > "settings" : { > > > "index" : { > > > "codec" : { > > > "postings\_format" : { > > > "custom" : { > > > "type" : "bloom\_default", > > > "delegate" : "default", > > > "ffp" : "10k=0.01,1m=0.03" > > > } > > > } > > > } > > > } > > > }, > > > "mappings" : { > > > "post" : { > > > "properties" : { > > > "id" : { "type" : "long", "store" : "yes", "precision\_step" : "0", > > > "postings\_format" : "custom" }, > > > "name" : { "type" : "string", "store" : "yes", "index" : "analyzed" > > > }, > > > "contents" : { "type" : "string", "store" : "no", "index" : > > > "analyzed" } > > > } > > > } > > > } > > > }' > > > > > > Gives the following exception: > > > > > > {"error":"IndexCreationException[[posts] failed to create index]; > > > nested: NoClassSettingsException[Failed to load class setting [type] with > > > value [bloom\_default]]; nested: > > > ClassNotFoundException[org.elasticsearch.index.codec.postingsformat.bloomdefault.BloomDefaultPostingsFormatProvider]; > > > ","status":500} > > > > > > \*-- > > > Regards, > > > Rafał Kuć > > > Sematext :: _[http://sematext.com/](http://sematext.com/)_ :: Solr - Lucene - Nutch - > > > Elasticsearch > > > > > > - > > > > > > The bloom codec needs to wrap another codec. Using "bloom" means > > > "maintain a bloom filter in memory" but doesn't specify how the data should > > > be stored on disk. > > > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/codec/) > > > > > > That said, it could throw a better error message > > > > > > [Better error message when using bloom codec · Issue #2893 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/2893) > > > > > > On Sat, Apr 13, 2013 at 11:34 PM, Rafał Kuć [r....@solr.pl](mailto:r....@solr.pl) wrote: > > > Hello! > > > > > > I've got a question about the postings format. When reading the > > > documentation we can see that there is a bloom posting format type. > > > However when trying to use it Elasticsearch throws an exception, for > > > example: > > > > > > curl -XPOST 'localhost:9200/posts' -d '{ > > > "settings" : { > > > "index" : { > > > "codec" : { > > > "postings\_format" : { > > > "custom" : { > > > "type" : "bloom", > > > "delegate" : "pulsing" > > > } > > > } > > > } > > > } > > > }, > > > "mappings" : { > > > "post" : { > > > "properties" : { > > > "id" : { "type" : "long", "store" : "yes", "precision\_step" : "0", > > > "postings\_format" : "custom" }, > > > "name" : { "type" : "string", "store" : "yes", "index" : "analyzed" > > > }, > > > "contents" : { "type" : "string", "store" : "no", "index" : > > > "analyzed" } > > > } > > > } > > > } > > > }' > > > > > > And the exception is as follows: > > > {"error":"IndexCreationException[[posts] failed to create index]; > > > nested: NoClassSettingsException[Failed to load class setting > > > [type] with value [bloom]]; nested: > > > ClassNotFoundException[org.elasticsearch.index.codec.postingsformat.bloom.BloomPostingsFormatProvider]; > > > ", > > > "status":500} > > > > > > According to the code there we are allowed to use the bloom\_default or > > > bloom\_pulsing types, but not the bloom itself (at least as the > > > pre-configured ones). And of course when configuring the id field with > > > one of the mentioned postings format it works without any problem, > > > which can be seen in the mappings: > > > > > > { > > > "posts" : { > > > "post" : { > > > "properties" : { > > > "contents" : { > > > "type" : "string" > > > }, > > > "id" : { > > > "type" : "long", > > > "store" : true, > > > "postings\_format" : "bloom\_pulsing", > > > "precision\_step" : 2147483647 > > > }, > > > "name" : { > > > "type" : "string", > > > "store" : true > > > } > > > } > > > } > > > } > > > } > > > > > > Am I missing something when it comes to the bloom type ? I'm using > > > 0.90.RC2. Thanks for the answer. > > > > > > -- > > > Regards, > > > Rafał Kuć > > > Sematext :: [http://sematext.com/](http://sematext.com/) :: Solr - Lucene - Nutch - > > > Elasticsearch > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Rafal\_Kuc\_3](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rafal_kuc_3/32/799_2.png) [@Rafal\_Kuc\_3](https://discuss.elastic.co/u/Rafal_Kuc_3)\ **Post date:** [April 19, 2013, 6:08pm UTC](https://discuss.elastic.co/t/0-90-rc2-postings-format/11572/7 "2013-04-19T18:08:23Z")
Hello Jérôme! Yes you are right, it should be for HIGH cardinality fields, so LOW frequent terms. I've updated the post - very big thanks for pointing that out ! _--_ Regards, Rafał Kuć Sematext :: [http://sematext.com/](http://sematext.com/) _:: Solr - Lucene - Nutch - ElasticSearch_ Hey Rafal, While we are on it; Is it just me or the description of pulsing and bloom filter codec on this page; [http://elasticsearchserverbook.com/elasticsearch-0-90-using-codecs/](http://elasticsearchserverbook.com/elasticsearch-0-90-using-codecs/) saying that it's appropriate for LOW cardinality field is wrong ? Pulsing and bloom are good for id lookup, so that's high cardinality field. An id field is normally the highest possible cardinality ? Maybe I'm wrong, but if so I would like to know what I don't understand 🙂 Jerome On Friday, April 19, 2013 9:57:50 AM UTC-4, Jérôme Gagnon wrote: Just found... the class on ElasticSearch source is BloomFilterPostingsFormatProvider, so the type to user is "bloom\_filter". The documentation should be updated accordingly. On Friday, April 19, 2013 9:55:49 AM UTC-4, Jérôme Gagnon wrote: I'm experiencing the same issue, so any updates would be appreciated, thanks ! On Sunday, April 14, 2013 8:00:17 AM UTC-4, Rafał Kuć wrote: Thanks Clinton, I'm aware of the bloom\_pulsing and bloom\_default postings formats. I was wondering if I'm missing something after looking at the docs at [http://www.elasticsearch.](http://www.elasticsearch.org/guide/reference/index-modules/codec/)[org/guide/reference/index-](http://www.elasticsearch.org/guide/reference/index-modules/codec/)[modules/codec/,](http://www.elasticsearch.org/guide/reference/index-modules/codec/) because of the type name "bloom". I thought one can use the "bloom" type when defining a custom postings format and set the appropriate delegate, to for example pulsing or default. But now, I've got another question. Is it possible to use a custom bloom filter based codec, like the bloom\_default or bloom\_pulsing ? For example, the following request: curl -XPOST 'localhost:9200/posts/' -d '{ "settings" : { "index" : { "codec" : { ``` "postings_format" : { "custom" : { "type" : "bloom_default", "delegate" : "default", "ffp" : "10k=0.01,1m=0.03" } } ``` } } }, "mappings" : { "post" : { "properties" : { ``` "id" : { "type" : "long", "store" : "yes", "precision_step" : "0", "postings_format" : "custom" }, "name" : { "type" : "string", "store" : "yes", "index" : "analyzed" }, "contents" : { "type" : "string", "store" : "no", "index" : "analyzed" } ``` } } } }' Gives the following exception: {"error":"IndexCreationException[[posts] failed to create index]; nested: NoClassSettingsException[Failed to load class setting [type] with value [bloom\_default]]; nested: ClassNotFoundException[org.elasticsearch.index.codec.postingsformat.bloomdefault.BloomDefaultPostingsFormatProvider]; ","status":500} _--_ Regards, Rafał Kuć Sematext :: [http://sematext.com/](http://sematext.com/) _:: Solr - Lucene - Nutch - ElasticSearch_ The bloom codec needs to wrap another codec. Using "bloom" means "maintain a bloom filter in memory" but doesn't specify how the data should be stored on disk. [http://www.elasticsearch.org/](http://www.elasticsearch.org/guide/reference/index-modules/codec/)[guide/reference/index-modules/](http://www.elasticsearch.org/guide/reference/index-modules/codec/)[codec/](http://www.elasticsearch.org/guide/reference/index-modules/codec/) That said, it could throw a better error message [https://github.com/](https://github.com/elasticsearch/elasticsearch/issues/2893)[elasticsearch/elasticsearch/](https://github.com/elasticsearch/elasticsearch/issues/2893)[issues/2893](https://github.com/elasticsearch/elasticsearch/issues/2893) On Sat, Apr 13, 2013 at 11:34 PM, Rafał Kuć \ wrote: Hello! I've got a question about the postings format. When reading the documentation we can see that there is a bloom posting format type. However when trying to use it ElasticSearch throws an exception, for example: curl -XPOST 'localhost:9200/posts' -d '{ "settings" : { "index" : { "codec" : { ``` "postings_format" : { "custom" : { "type" : "bloom", "delegate" : "pulsing" } } ``` } } }, "mappings" : { "post" : { "properties" : { ``` "id" : { "type" : "long", "store" : "yes", "precision_step" : "0", "postings_format" : "custom" }, "name" : { "type" : "string", "store" : "yes", "index" : "analyzed" }, "contents" : { "type" : "string", "store" : "no", "index" : "analyzed" } ``` } } } }' And the exception is as follows: {"error":"IndexCreationException[[posts] failed to create index]; nested: NoClassSettingsException[Failed to load class setting [type] with value [bloom]]; nested: ClassNotFoundException[org.elasticsearch.index.codec.postingsformat.bloom.BloomPostingsFormatProvider]; ", "status":500} According to the code there we are allowed to use the bloom\_default or bloom\_pulsing types, but not the bloom itself (at least as the pre-configured ones). And of course when configuring the id field with one of the mentioned postings format it works without any problem, which can be seen in the mappings: { "posts" : { ``` "post" : { "properties" : { "contents" : { "type" : "string" }, "id" : { "type" : "long", "store" : true, "postings_format" : "bloom_pulsing", "precision_step" : 2147483647 }, "name" : { "type" : "string", "store" : true } } } ``` } } Am I missing something when it comes to the bloom type ? I'm using 0.90.RC2. Thanks for the answer. -- Regards, Rafał Kuć Sematext :: [http://sematext.com/](http://sematext.com/) :: Solr - Lucene - Nutch - ElasticSearch -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). For more options, visit [https://groups.google.](https://groups.google.com/groups/opt_out)[com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). For more options, visit [https://groups.google.](https://groups.google.com/groups/opt_out)[com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:40am UTC](https://discuss.elastic.co/t/0-90-rc2-postings-format/11572/8 "2017-07-06T02:40:20Z")