# Elasticsearch + attachment plugin + Kibana + couchbase **URL:** \ **Category:** Elasticsearch\ **Created:** [February 12, 2015, 1:59pm UTC](https://discuss.elastic.co/t/elasticsearch-attachment-plugin-kibana-couchbase/22133 "2015-02-12T13:59:55Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![Nadav\_Hashimshony](https://avatars.discourse-cdn.com/v4/letter/n/a88e57/32.png) [@Nadav\_Hashimshony](https://discuss.elastic.co/u/Nadav_Hashimshony)\ **Post date:** [February 12, 2015, 1:59pm UTC](https://discuss.elastic.co/t/elasticsearch-attachment-plugin-kibana-couchbase/22133/1 "2015-02-12T13:59:55Z")
Hi, I'm new to the group, hope ill find what i need and share my experience as i go along.. im using ES with the attachment-plugin in order to store and search files. when i set the mapping right and insert the file data in a Base64 manner I'm able to query my data via Kibana. my problem is this. if i create the index + mapping in ES, then insert the data to Couchbase and use XDRC to replicate it to ES, i can't query the Data with Kibana. it looks like the mapping of the index created in ES doesn't index well the data it gets from Couchbase. has anyone encounter such an issue? Thanks You Nadav. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/8092eaf5-0ef8-4249-8e5d-acff8281a81a%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/8092eaf5-0ef8-4249-8e5d-acff8281a81a%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Itamar\_Syn\_Hershko](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/itamar_syn_hershko/32/725_2.png) [@Itamar\_Syn\_Hershko](https://discuss.elastic.co/u/Itamar_Syn_Hershko)\ **Post date:** [February 12, 2015, 2:02pm UTC](https://discuss.elastic.co/t/elasticsearch-attachment-plugin-kibana-couchbase/22133/2 "2015-02-12T14:02:52Z")
The XDCR plugin indexes the data using an envelope document. Long story short, make sure you use the latest XDCR plugin as older ones are missing lots of important functions, and use templates and dynamic templates with proper field paths for this to work correctly [http://code972.com/blog/2015/02/80-elasticsearch-one-tip-a-day-managing-index-mappings-like-a-pro](http://code972.com/blog/2015/02/80-elasticsearch-one-tip-a-day-managing-index-mappings-like-a-pro) [http://code972.com/blog/2015/02/81-elasticsearch-one-tip-a-day-using-dynamic-templates-to-avoid-rigorous-mappings](http://code972.com/blog/2015/02/81-elasticsearch-one-tip-a-day-using-dynamic-templates-to-avoid-rigorous-mappings) -- Itamar Syn-Hershko [http://code972.com](http://code972.com) | @synhershko [https://twitter.com/synhershko](https://twitter.com/synhershko) Freelance Developer & Consultant [Lucene.NET](http://Lucene.NET) committer and PMC member On Thu, Feb 12, 2015 at 3:59 PM, Nadav Hashimshony [nadavh@gmail.com](mailto:nadavh@gmail.com) wrote: > Hi, > > I'm new to the group, hope ill find what i need and share my experience as > i go along.. > > im using ES with the attachment-plugin in order to store and search files. > when i set the mapping right and insert the file data in a Base64 manner > I'm able to query my data via Kibana. > > my problem is this. > > if i create the index + mapping in ES, then insert the data to Couchbase > and use XDRC to replicate it to ES, i can't query the Data with Kibana. > it looks like the mapping of the index created in ES doesn't index well > the data it gets from Couchbase. > > has anyone encounter such an issue? > > Thanks You > > Nadav. > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/8092eaf5-0ef8-4249-8e5d-acff8281a81a%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/8092eaf5-0ef8-4249-8e5d-acff8281a81a%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/8092eaf5-0ef8-4249-8e5d-acff8281a81a%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/8092eaf5-0ef8-4249-8e5d-acff8281a81a%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAHTr4Zs5OwXJe9aT1pPNu9vuooXO10Z3Mx7xc8CJh77EN9s%3DCQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAHTr4Zs5OwXJe9aT1pPNu9vuooXO10Z3Mx7xc8CJh77EN9s%3DCQ%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Nadav\_Hashimshony](https://avatars.discourse-cdn.com/v4/letter/n/a88e57/32.png) [@Nadav\_Hashimshony](https://discuss.elastic.co/u/Nadav_Hashimshony)\ **Post date:** [February 12, 2015, 2:59pm UTC](https://discuss.elastic.co/t/elasticsearch-attachment-plugin-kibana-couchbase/22133/3 "2015-02-12T14:59:47Z")
Thanks you for the response i am using mapping, i created the following index PUT /storage/files/\_mapping { "files": { "properties": { "file": { "type": "attachment", "path": "full", "fields": { "content\_type": { "type": "string", "store": true } } } } } } when i insert data via ES and query it, all is fine. the problem is when data is inserted to Couchbase.. Nadav On Thursday, February 12, 2015 at 4:03:01 PM UTC+2, Itamar Syn-Hershko wrote: > The XDCR plugin indexes the data using an envelope document. Long story > short, make sure you use the latest XDCR plugin as older ones are missing > lots of important functions, and use templates and dynamic templates with > proper field paths for this to work correctly > > [Elasticsearch One Tip a Day: Managing Index Mappings Like a Pro](http://code972.com/blog/2015/02/80-elasticsearch-one-tip-a-day-managing-index-mappings-like-a-pro) > > [Elasticsearch One Tip a Day: Using Dynamic Templates To Avoid Rigorous Mappings](http://code972.com/blog/2015/02/81-elasticsearch-one-tip-a-day-using-dynamic-templates-to-avoid-rigorous-mappings) > > -- > > Itamar Syn-Hershko > [http://code972.com](http://code972.com) | @synhershko [https://twitter.com/synhershko](https://twitter.com/synhershko) > Freelance Developer & Consultant > [Lucene.NET](http://Lucene.NET) committer and PMC member > > On Thu, Feb 12, 2015 at 3:59 PM, Nadav Hashimshony \<[nad...@gmail.com](mailto:nad...@gmail.com) > \\> wrote: > > > Hi, > > > > I'm new to the group, hope ill find what i need and share my experience > > as i go along.. > > > > im using ES with the attachment-plugin in order to store and search files. > > when i set the mapping right and insert the file data in a Base64 manner > > I'm able to query my data via Kibana. > > > > my problem is this. > > > > if i create the index + mapping in ES, then insert the data to Couchbase > > and use XDRC to replicate it to ES, i can't query the Data with Kibana. > > it looks like the mapping of the index created in ES doesn't index well > > the data it gets from Couchbase. > > > > has anyone encounter such an issue? > > > > Thanks You > > > > Nadav. > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/8092eaf5-0ef8-4249-8e5d-acff8281a81a%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/8092eaf5-0ef8-4249-8e5d-acff8281a81a%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/8092eaf5-0ef8-4249-8e5d-acff8281a81a%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/8092eaf5-0ef8-4249-8e5d-acff8281a81a%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/1d9c8ce5-116f-40cc-a5e3-6ebe47191850%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/1d9c8ce5-116f-40cc-a5e3-6ebe47191850%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Itamar\_Syn\_Hershko](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/itamar_syn_hershko/32/725_2.png) [@Itamar\_Syn\_Hershko](https://discuss.elastic.co/u/Itamar_Syn_Hershko)\ **Post date:** [February 12, 2015, 3:04pm UTC](https://discuss.elastic.co/t/elasticsearch-attachment-plugin-kibana-couchbase/22133/4 "2015-02-12T15:04:12Z")
Like I said, you need the mapping to "catch" before the XDCR plugin begins the replication - so you need to put a template with this mapping that will override XDCR's -- Itamar Syn-Hershko [http://code972.com](http://code972.com) | @synhershko [https://twitter.com/synhershko](https://twitter.com/synhershko) Freelance Developer & Consultant [Lucene.NET](http://Lucene.NET) committer and PMC member On Thu, Feb 12, 2015 at 4:59 PM, Nadav Hashimshony [nadavh@gmail.com](mailto:nadavh@gmail.com) wrote: > Thanks you for the response > > i am using mapping, i created the following index > PUT /storage/files/\_mapping > { > "files": { > "properties": { > "file": { > "type": "attachment", > "path": "full", > "fields": { > "content\_type": { > "type": "string", > "store": true > } > } > } > } > } > } > > when i insert data via ES and query it, all is fine. > the problem is when data is inserted to Couchbase.. > > Nadav > > On Thursday, February 12, 2015 at 4:03:01 PM UTC+2, Itamar Syn-Hershko > wrote: > > > The XDCR plugin indexes the data using an envelope document. Long story > > short, make sure you use the latest XDCR plugin as older ones are missing > > lots of important functions, and use templates and dynamic templates with > > proper field paths for this to work correctly > > > > [http://code972.com/blog/2015/02/80-elasticsearch-one-tip-a-](http://code972.com/blog/2015/02/80-elasticsearch-one-tip-a-) > > day-managing-index-mappings-like-a-pro > > [http://code972.com/blog/2015/02/81-elasticsearch-one-tip-a-](http://code972.com/blog/2015/02/81-elasticsearch-one-tip-a-) > > day-using-dynamic-templates-to-avoid-rigorous-mappings > > > > -- > > > > Itamar Syn-Hershko > > [http://code972.com](http://code972.com) | @synhershko [https://twitter.com/synhershko](https://twitter.com/synhershko) > > Freelance Developer & Consultant > > [Lucene.NET](http://Lucene.NET) committer and PMC member > > > > On Thu, Feb 12, 2015 at 3:59 PM, Nadav Hashimshony [nad...@gmail.com](mailto:nad...@gmail.com) > > wrote: > > > > > Hi, > > > > > > I'm new to the group, hope ill find what i need and share my experience > > > as i go along.. > > > > > > im using ES with the attachment-plugin in order to store and search > > > files. > > > when i set the mapping right and insert the file data in a Base64 manner > > > I'm able to query my data via Kibana. > > > > > > my problem is this. > > > > > > if i create the index + mapping in ES, then insert the data to Couchbase > > > and use XDRC to replicate it to ES, i can't query the Data with Kibana. > > > it looks like the mapping of the index created in ES doesn't index well > > > the data it gets from Couchbase. > > > > > > has anyone encounter such an issue? > > > > > > Thanks You > > > > > > Nadav. > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/) > > > msgid/elasticsearch/8092eaf5-0ef8-4249-8e5d-acff8281a81a% > > > [40googlegroups.com](http://40googlegroups.com) > > > [https://groups.google.com/d/msgid/elasticsearch/8092eaf5-0ef8-4249-8e5d-acff8281a81a%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/8092eaf5-0ef8-4249-8e5d-acff8281a81a%40googlegroups.com?utm_medium=email&utm_source=footer) > > > . > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/1d9c8ce5-116f-40cc-a5e3-6ebe47191850%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/1d9c8ce5-116f-40cc-a5e3-6ebe47191850%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/1d9c8ce5-116f-40cc-a5e3-6ebe47191850%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/1d9c8ce5-116f-40cc-a5e3-6ebe47191850%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAHTr4Zuh7HHK8XmdznuHnw7E01ffXV8BC-49D70ekMc1-YhQCA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAHTr4Zuh7HHK8XmdznuHnw7E01ffXV8BC-49D70ekMc1-YhQCA%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Nadav\_Hashimshony](https://avatars.discourse-cdn.com/v4/letter/n/a88e57/32.png) [@Nadav\_Hashimshony](https://discuss.elastic.co/u/Nadav_Hashimshony)\ **Post date:** [February 12, 2015, 3:12pm UTC](https://discuss.elastic.co/t/elasticsearch-attachment-plugin-kibana-couchbase/22133/5 "2015-02-12T15:12:18Z")
ok, just to be clear. the steps i did was as followed: 1. create the index with the mapping. 2. define the XDCR to replicate my bucket with the index in ES. 3. insert data to couchbase. 4. try to query with kibana What you suggest is to Add another BEFORE step 1: 0. create a template to include my mapping. 1. crate the index in ES and so on... did i get it right? Thanks. Nadav. On Thursday, February 12, 2015 at 5:04:24 PM UTC+2, Itamar Syn-Hershko wrote: > Like I said, you need the mapping to "catch" before the XDCR plugin begins > the replication - so you need to put a template with this mapping that will > override XDCR's > > -- > > Itamar Syn-Hershko > [http://code972.com](http://code972.com) | @synhershko [https://twitter.com/synhershko](https://twitter.com/synhershko) > Freelance Developer & Consultant > [Lucene.NET](http://Lucene.NET) committer and PMC member > > On Thu, Feb 12, 2015 at 4:59 PM, Nadav Hashimshony \<[nad...@gmail.com](mailto:nad...@gmail.com) > \\> wrote: > > > Thanks you for the response > > > > i am using mapping, i created the following index > > PUT /storage/files/\_mapping > > { > > "files": { > > "properties": { > > "file": { > > "type": "attachment", > > "path": "full", > > "fields": { > > "content\_type": { > > "type": "string", > > "store": true > > } > > } > > } > > } > > } > > } > > > > when i insert data via ES and query it, all is fine. > > the problem is when data is inserted to Couchbase.. > > > > Nadav > > > > On Thursday, February 12, 2015 at 4:03:01 PM UTC+2, Itamar Syn-Hershko > > wrote: > > > > > The XDCR plugin indexes the data using an envelope document. Long story > > > short, make sure you use the latest XDCR plugin as older ones are missing > > > lots of important functions, and use templates and dynamic templates with > > > proper field paths for this to work correctly > > > > > > [http://code972.com/blog/2015/02/80-elasticsearch-one-tip-a-](http://code972.com/blog/2015/02/80-elasticsearch-one-tip-a-) > > > day-managing-index-mappings-like-a-pro > > > [http://code972.com/blog/2015/02/81-elasticsearch-one-tip-a-](http://code972.com/blog/2015/02/81-elasticsearch-one-tip-a-) > > > day-using-dynamic-templates-to-avoid-rigorous-mappings > > > > > > -- > > > > > > Itamar Syn-Hershko > > > [http://code972.com](http://code972.com) | @synhershko [https://twitter.com/synhershko](https://twitter.com/synhershko) > > > Freelance Developer & Consultant > > > [Lucene.NET](http://Lucene.NET) committer and PMC member > > > > > > On Thu, Feb 12, 2015 at 3:59 PM, Nadav Hashimshony [nad...@gmail.com](mailto:nad...@gmail.com) > > > wrote: > > > > > > > Hi, > > > > > > > > I'm new to the group, hope ill find what i need and share my experience > > > > as i go along.. > > > > > > > > im using ES with the attachment-plugin in order to store and search > > > > files. > > > > when i set the mapping right and insert the file data in a Base64 > > > > manner I'm able to query my data via Kibana. > > > > > > > > my problem is this. > > > > > > > > if i create the index + mapping in ES, then insert the data to > > > > Couchbase and use XDRC to replicate it to ES, i can't query the Data with > > > > Kibana. > > > > it looks like the mapping of the index created in ES doesn't index well > > > > the data it gets from Couchbase. > > > > > > > > has anyone encounter such an issue? > > > > > > > > Thanks You > > > > > > > > Nadav. > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/) > > > > msgid/elasticsearch/8092eaf5-0ef8-4249-8e5d-acff8281a81a% > > > > [40googlegroups.com](http://40googlegroups.com) > > > > [https://groups.google.com/d/msgid/elasticsearch/8092eaf5-0ef8-4249-8e5d-acff8281a81a%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/8092eaf5-0ef8-4249-8e5d-acff8281a81a%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > -- > > > You received this message because you are subscribed to the Google Groups > > > "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send an > > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/1d9c8ce5-116f-40cc-a5e3-6ebe47191850%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/1d9c8ce5-116f-40cc-a5e3-6ebe47191850%40googlegroups.com) > > > [https://groups.google.com/d/msgid/elasticsearch/1d9c8ce5-116f-40cc-a5e3-6ebe47191850%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/1d9c8ce5-116f-40cc-a5e3-6ebe47191850%40googlegroups.com?utm_medium=email&utm_source=footer) > > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/8628ef07-2e10-423a-9de0-13ebaa37a0e8%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/8628ef07-2e10-423a-9de0-13ebaa37a0e8%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Itamar\_Syn\_Hershko](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/itamar_syn_hershko/32/725_2.png) [@Itamar\_Syn\_Hershko](https://discuss.elastic.co/u/Itamar_Syn_Hershko)\ **Post date:** [February 12, 2015, 3:14pm UTC](https://discuss.elastic.co/t/elasticsearch-attachment-plugin-kibana-couchbase/22133/6 "2015-02-12T15:14:33Z")
Yes. Just make sure the template reflects the actual document structure - as I said XDCR wraps your document in an envelope document -- Itamar Syn-Hershko [http://code972.com](http://code972.com) | @synhershko [https://twitter.com/synhershko](https://twitter.com/synhershko) Freelance Developer & Consultant [Lucene.NET](http://Lucene.NET) committer and PMC member On Thu, Feb 12, 2015 at 5:12 PM, Nadav Hashimshony [nadavh@gmail.com](mailto:nadavh@gmail.com) wrote: > ok, just to be clear. > > the steps i did was as followed: > > 1. create the index with the mapping. > 2. define the XDCR to replicate my bucket with the index in ES. > 3. insert data to couchbase. > 4. try to query with kibana > > What you suggest is to Add another BEFORE step 1: > 0. create a template to include my mapping. > > 1. crate the index in ES > and so on... > > did i get it right? > > Thanks. > Nadav. > > On Thursday, February 12, 2015 at 5:04:24 PM UTC+2, Itamar Syn-Hershko > wrote: > > > Like I said, you need the mapping to "catch" before the XDCR plugin > > begins the replication - so you need to put a template with this mapping > > that will override XDCR's > > > > -- > > > > Itamar Syn-Hershko > > [http://code972.com](http://code972.com) | @synhershko [https://twitter.com/synhershko](https://twitter.com/synhershko) > > Freelance Developer & Consultant > > [Lucene.NET](http://Lucene.NET) committer and PMC member > > > > On Thu, Feb 12, 2015 at 4:59 PM, Nadav Hashimshony [nad...@gmail.com](mailto:nad...@gmail.com) > > wrote: > > > > > Thanks you for the response > > > > > > i am using mapping, i created the following index > > > PUT /storage/files/\_mapping > > > { > > > "files": { > > > "properties": { > > > "file": { > > > "type": "attachment", > > > "path": "full", > > > "fields": { > > > "content\_type": { > > > "type": "string", > > > "store": true > > > } > > > } > > > } > > > } > > > } > > > } > > > > > > when i insert data via ES and query it, all is fine. > > > the problem is when data is inserted to Couchbase.. > > > > > > Nadav > > > > > > On Thursday, February 12, 2015 at 4:03:01 PM UTC+2, Itamar Syn-Hershko > > > wrote: > > > > > > > The XDCR plugin indexes the data using an envelope document. Long story > > > > short, make sure you use the latest XDCR plugin as older ones are missing > > > > lots of important functions, and use templates and dynamic templates with > > > > proper field paths for this to work correctly > > > > > > > > [http://code972.com/blog/2015/02/80-elasticsearch-one-tip-a-d](http://code972.com/blog/2015/02/80-elasticsearch-one-tip-a-d) > > > > ay-managing-index-mappings-like-a-pro > > > > [http://code972.com/blog/2015/02/81-elasticsearch-one-tip-a-d](http://code972.com/blog/2015/02/81-elasticsearch-one-tip-a-d) > > > > ay-using-dynamic-templates-to-avoid-rigorous-mappings > > > > > > > > -- > > > > > > > > Itamar Syn-Hershko > > > > [http://code972.com](http://code972.com) | @synhershko [https://twitter.com/synhershko](https://twitter.com/synhershko) > > > > Freelance Developer & Consultant > > > > [Lucene.NET](http://Lucene.NET) committer and PMC member > > > > > > > > On Thu, Feb 12, 2015 at 3:59 PM, Nadav Hashimshony [nad...@gmail.com](mailto:nad...@gmail.com) > > > > wrote: > > > > > > > > > Hi, > > > > > > > > > > I'm new to the group, hope ill find what i need and share my > > > > > experience as i go along.. > > > > > > > > > > im using ES with the attachment-plugin in order to store and search > > > > > files. > > > > > when i set the mapping right and insert the file data in a Base64 > > > > > manner I'm able to query my data via Kibana. > > > > > > > > > > my problem is this. > > > > > > > > > > if i create the index + mapping in ES, then insert the data to > > > > > Couchbase and use XDRC to replicate it to ES, i can't query the Data with > > > > > Kibana. > > > > > it looks like the mapping of the index created in ES doesn't index > > > > > well the data it gets from Couchbase. > > > > > > > > > > has anyone encounter such an issue? > > > > > > > > > > Thanks You > > > > > > > > > > Nadav. > > > > > > > > > > -- > > > > > You received this message because you are subscribed to the Google > > > > > Groups "elasticsearch" group. > > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/) > > > > > msgid/elasticsearch/8092eaf5-0ef8-4249-8e5d-acff8281a81a%40goo > > > > > [glegroups.com](http://glegroups.com) > > > > > [https://groups.google.com/d/msgid/elasticsearch/8092eaf5-0ef8-4249-8e5d-acff8281a81a%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/8092eaf5-0ef8-4249-8e5d-acff8281a81a%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > > . > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/) > > > > msgid/elasticsearch/1d9c8ce5-116f-40cc-a5e3-6ebe47191850% > > > > [40googlegroups.com](http://40googlegroups.com) > > > > [https://groups.google.com/d/msgid/elasticsearch/1d9c8ce5-116f-40cc-a5e3-6ebe47191850%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/1d9c8ce5-116f-40cc-a5e3-6ebe47191850%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > . > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/8628ef07-2e10-423a-9de0-13ebaa37a0e8%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/8628ef07-2e10-423a-9de0-13ebaa37a0e8%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/8628ef07-2e10-423a-9de0-13ebaa37a0e8%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/8628ef07-2e10-423a-9de0-13ebaa37a0e8%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAHTr4Zuoj0kZmmY%2BeS\_mH54nBp39HUSGDETJemEU4kdFLcjdCg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAHTr4Zuoj0kZmmY%2BeS_mH54nBp39HUSGDETJemEU4kdFLcjdCg%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Nadav\_Hashimshony](https://avatars.discourse-cdn.com/v4/letter/n/a88e57/32.png) [@Nadav\_Hashimshony](https://discuss.elastic.co/u/Nadav_Hashimshony)\ **Post date:** [February 12, 2015, 3:18pm UTC](https://discuss.elastic.co/t/elasticsearch-attachment-plugin-kibana-couchbase/22133/7 "2015-02-12T15:18:20Z")
ok, ill try. this "envelope document", is it something i need to be concerned about when I'm querying via Kibana? On Thursday, February 12, 2015 at 5:14:39 PM UTC+2, Itamar Syn-Hershko wrote: > Yes. Just make sure the template reflects the actual document structure - > as I said XDCR wraps your document in an envelope document > > -- > > Itamar Syn-Hershko > [http://code972.com](http://code972.com) | @synhershko [https://twitter.com/synhershko](https://twitter.com/synhershko) > Freelance Developer & Consultant > [Lucene.NET](http://Lucene.NET) committer and PMC member > > On Thu, Feb 12, 2015 at 5:12 PM, Nadav Hashimshony \<[nad...@gmail.com](mailto:nad...@gmail.com) > \\> wrote: > > > ok, just to be clear. > > > > the steps i did was as followed: > > > > 1. create the index with the mapping. > > 2. define the XDCR to replicate my bucket with the index in ES. > > 3. insert data to couchbase. > > 4. try to query with kibana > > > > What you suggest is to Add another BEFORE step 1: > > 0. create a template to include my mapping. > > > > 1. crate the index in ES > > and so on... > > > > did i get it right? > > > > Thanks. > > Nadav. > > > > On Thursday, February 12, 2015 at 5:04:24 PM UTC+2, Itamar Syn-Hershko > > wrote: > > > > > Like I said, you need the mapping to "catch" before the XDCR plugin > > > begins the replication - so you need to put a template with this mapping > > > that will override XDCR's > > > > > > -- > > > > > > Itamar Syn-Hershko > > > [http://code972.com](http://code972.com) | @synhershko [https://twitter.com/synhershko](https://twitter.com/synhershko) > > > Freelance Developer & Consultant > > > [Lucene.NET](http://Lucene.NET) committer and PMC member > > > > > > On Thu, Feb 12, 2015 at 4:59 PM, Nadav Hashimshony [nad...@gmail.com](mailto:nad...@gmail.com) > > > wrote: > > > > > > > Thanks you for the response > > > > > > > > i am using mapping, i created the following index > > > > PUT /storage/files/\_mapping > > > > { > > > > "files": { > > > > "properties": { > > > > "file": { > > > > "type": "attachment", > > > > "path": "full", > > > > "fields": { > > > > "content\_type": { > > > > "type": "string", > > > > "store": true > > > > } > > > > } > > > > } > > > > } > > > > } > > > > } > > > > > > > > when i insert data via ES and query it, all is fine. > > > > the problem is when data is inserted to Couchbase.. > > > > > > > > Nadav > > > > > > > > On Thursday, February 12, 2015 at 4:03:01 PM UTC+2, Itamar Syn-Hershko > > > > wrote: > > > > > > > > > The XDCR plugin indexes the data using an envelope document. Long > > > > > story short, make sure you use the latest XDCR plugin as older ones are > > > > > missing lots of important functions, and use templates and dynamic > > > > > templates with proper field paths for this to work correctly > > > > > > > > > > [http://code972.com/blog/2015/02/80-elasticsearch-one-tip-a-d](http://code972.com/blog/2015/02/80-elasticsearch-one-tip-a-d) > > > > > ay-managing-index-mappings-like-a-pro > > > > > [http://code972.com/blog/2015/02/81-elasticsearch-one-tip-a-d](http://code972.com/blog/2015/02/81-elasticsearch-one-tip-a-d) > > > > > ay-using-dynamic-templates-to-avoid-rigorous-mappings > > > > > > > > > > -- > > > > > > > > > > Itamar Syn-Hershko > > > > > [http://code972.com](http://code972.com) | @synhershko [https://twitter.com/synhershko](https://twitter.com/synhershko) > > > > > Freelance Developer & Consultant > > > > > [Lucene.NET](http://Lucene.NET) committer and PMC member > > > > > > > > > > On Thu, Feb 12, 2015 at 3:59 PM, Nadav Hashimshony [nad...@gmail.com](mailto:nad...@gmail.com) > > > > > wrote: > > > > > > > > > > > Hi, > > > > > > > > > > > > I'm new to the group, hope ill find what i need and share my > > > > > > experience as i go along.. > > > > > > > > > > > > im using ES with the attachment-plugin in order to store and search > > > > > > files. > > > > > > when i set the mapping right and insert the file data in a Base64 > > > > > > manner I'm able to query my data via Kibana. > > > > > > > > > > > > my problem is this. > > > > > > > > > > > > if i create the index + mapping in ES, then insert the data to > > > > > > Couchbase and use XDRC to replicate it to ES, i can't query the Data with > > > > > > Kibana. > > > > > > it looks like the mapping of the index created in ES doesn't index > > > > > > well the data it gets from Couchbase. > > > > > > > > > > > > has anyone encounter such an issue? > > > > > > > > > > > > Thanks You > > > > > > > > > > > > Nadav. > > > > > > > > > > > > -- > > > > > > You received this message because you are subscribed to the Google > > > > > > Groups "elasticsearch" group. > > > > > > To unsubscribe from this group and stop receiving emails from it, > > > > > > send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/) > > > > > > msgid/elasticsearch/8092eaf5-0ef8-4249-8e5d-acff8281a81a%40goo > > > > > > [glegroups.com](http://glegroups.com) > > > > > > [https://groups.google.com/d/msgid/elasticsearch/8092eaf5-0ef8-4249-8e5d-acff8281a81a%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/8092eaf5-0ef8-4249-8e5d-acff8281a81a%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > > > . > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > > > -- > > > > > You received this message because you are subscribed to the Google > > > > > Groups "elasticsearch" group. > > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/) > > > > > msgid/elasticsearch/1d9c8ce5-116f-40cc-a5e3-6ebe47191850% > > > > > [40googlegroups.com](http://40googlegroups.com) > > > > > [https://groups.google.com/d/msgid/elasticsearch/1d9c8ce5-116f-40cc-a5e3-6ebe47191850%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/1d9c8ce5-116f-40cc-a5e3-6ebe47191850%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > > . > > > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > -- > > > You received this message because you are subscribed to the Google Groups > > > "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send an > > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/8628ef07-2e10-423a-9de0-13ebaa37a0e8%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/8628ef07-2e10-423a-9de0-13ebaa37a0e8%40googlegroups.com) > > > [https://groups.google.com/d/msgid/elasticsearch/8628ef07-2e10-423a-9de0-13ebaa37a0e8%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/8628ef07-2e10-423a-9de0-13ebaa37a0e8%40googlegroups.com?utm_medium=email&utm_source=footer) > > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/96f30e99-30dc-4af2-a51b-b12b0dee9e53%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/96f30e99-30dc-4af2-a51b-b12b0dee9e53%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Itamar\_Syn\_Hershko](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/itamar_syn_hershko/32/725_2.png) [@Itamar\_Syn\_Hershko](https://discuss.elastic.co/u/Itamar_Syn_Hershko)\ **Post date:** [February 12, 2015, 3:24pm UTC](https://discuss.elastic.co/t/elasticsearch-attachment-plugin-kibana-couchbase/22133/8 "2015-02-12T15:24:27Z")
Yes, that too 🙂 Also if its a time based data, you will not be able to use kibana's date filtering etc - because it lacks the @timestamp field. Basically, the XDCR elasticsearch plugin was built around the XDCR / Couchbase realm and not around Elasticsearch's. Unfortunately this means many ES features are unavailable / hard to use, e.g. [https://github.com/couchbaselabs/elasticsearch-transport-couchbase/issues/63](https://github.com/couchbaselabs/elasticsearch-transport-couchbase/issues/63) [https://github.com/couchbaselabs/elasticsearch-transport-couchbase/issues/64](https://github.com/couchbaselabs/elasticsearch-transport-couchbase/issues/64) I can help fixing this on the XDCR plugin if you'd like - ping me privately and we can work something out (or I can convince you to avoid using the XDCR replication) -- Itamar Syn-Hershko [http://code972.com](http://code972.com) | @synhershko [https://twitter.com/synhershko](https://twitter.com/synhershko) Freelance Developer & Consultant [Lucene.NET](http://Lucene.NET) committer and PMC member On Thu, Feb 12, 2015 at 5:18 PM, Nadav Hashimshony [nadavh@gmail.com](mailto:nadavh@gmail.com) wrote: > ok, ill try. > > this "envelope document", is it something i need to be concerned about > when I'm querying via Kibana? > > On Thursday, February 12, 2015 at 5:14:39 PM UTC+2, Itamar Syn-Hershko > wrote: > > > Yes. Just make sure the template reflects the actual document structure - > > as I said XDCR wraps your document in an envelope document > > > > -- > > > > Itamar Syn-Hershko > > [http://code972.com](http://code972.com) | @synhershko [https://twitter.com/synhershko](https://twitter.com/synhershko) > > Freelance Developer & Consultant > > [Lucene.NET](http://Lucene.NET) committer and PMC member > > > > On Thu, Feb 12, 2015 at 5:12 PM, Nadav Hashimshony [nad...@gmail.com](mailto:nad...@gmail.com) > > wrote: > > > > > ok, just to be clear. > > > > > > the steps i did was as followed: > > > > > > 1. create the index with the mapping. > > > 2. define the XDCR to replicate my bucket with the index in ES. > > > 3. insert data to couchbase. > > > 4. try to query with kibana > > > > > > What you suggest is to Add another BEFORE step 1: > > > 0. create a template to include my mapping. > > > > > > 1. crate the index in ES > > > and so on... > > > > > > did i get it right? > > > > > > Thanks. > > > Nadav. > > > > > > On Thursday, February 12, 2015 at 5:04:24 PM UTC+2, Itamar Syn-Hershko > > > wrote: > > > > > > > Like I said, you need the mapping to "catch" before the XDCR plugin > > > > begins the replication - so you need to put a template with this mapping > > > > that will override XDCR's > > > > > > > > -- > > > > > > > > Itamar Syn-Hershko > > > > [http://code972.com](http://code972.com) | @synhershko [https://twitter.com/synhershko](https://twitter.com/synhershko) > > > > Freelance Developer & Consultant > > > > [Lucene.NET](http://Lucene.NET) committer and PMC member > > > > > > > > On Thu, Feb 12, 2015 at 4:59 PM, Nadav Hashimshony [nad...@gmail.com](mailto:nad...@gmail.com) > > > > wrote: > > > > > > > > > Thanks you for the response > > > > > > > > > > i am using mapping, i created the following index > > > > > PUT /storage/files/\_mapping > > > > > { > > > > > "files": { > > > > > "properties": { > > > > > "file": { > > > > > "type": "attachment", > > > > > "path": "full", > > > > > "fields": { > > > > > "content\_type": { > > > > > "type": "string", > > > > > "store": true > > > > > } > > > > > } > > > > > } > > > > > } > > > > > } > > > > > } > > > > > > > > > > when i insert data via ES and query it, all is fine. > > > > > the problem is when data is inserted to Couchbase.. > > > > > > > > > > Nadav > > > > > > > > > > On Thursday, February 12, 2015 at 4:03:01 PM UTC+2, Itamar Syn-Hershko > > > > > wrote: > > > > > > > > > > > The XDCR plugin indexes the data using an envelope document. Long > > > > > > story short, make sure you use the latest XDCR plugin as older ones are > > > > > > missing lots of important functions, and use templates and dynamic > > > > > > templates with proper field paths for this to work correctly > > > > > > > > > > > > [http://code972.com/blog/2015/02/80-elasticsearch-one-tip-a-d](http://code972.com/blog/2015/02/80-elasticsearch-one-tip-a-d) > > > > > > ay-managing-index-mappings-like-a-pro > > > > > > [http://code972.com/blog/2015/02/81-elasticsearch-one-tip-a-d](http://code972.com/blog/2015/02/81-elasticsearch-one-tip-a-d) > > > > > > ay-using-dynamic-templates-to-avoid-rigorous-mappings > > > > > > > > > > > > -- > > > > > > > > > > > > Itamar Syn-Hershko > > > > > > [http://code972.com](http://code972.com) | @synhershko [https://twitter.com/synhershko](https://twitter.com/synhershko) > > > > > > Freelance Developer & Consultant > > > > > > [Lucene.NET](http://Lucene.NET) committer and PMC member > > > > > > > > > > > > On Thu, Feb 12, 2015 at 3:59 PM, Nadav Hashimshony [nad...@gmail.com](mailto:nad...@gmail.com) > > > > > > wrote: > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > I'm new to the group, hope ill find what i need and share my > > > > > > > experience as i go along.. > > > > > > > > > > > > > > im using ES with the attachment-plugin in order to store and search > > > > > > > files. > > > > > > > when i set the mapping right and insert the file data in a Base64 > > > > > > > manner I'm able to query my data via Kibana. > > > > > > > > > > > > > > my problem is this. > > > > > > > > > > > > > > if i create the index + mapping in ES, then insert the data to > > > > > > > Couchbase and use XDRC to replicate it to ES, i can't query the Data with > > > > > > > Kibana. > > > > > > > it looks like the mapping of the index created in ES doesn't index > > > > > > > well the data it gets from Couchbase. > > > > > > > > > > > > > > has anyone encounter such an issue? > > > > > > > > > > > > > > Thanks You > > > > > > > > > > > > > > Nadav. > > > > > > > > > > > > > > -- > > > > > > > You received this message because you are subscribed to the Google > > > > > > > Groups "elasticsearch" group. > > > > > > > To unsubscribe from this group and stop receiving emails from it, > > > > > > > send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > > > > To view this discussion on the web visit > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/8092eaf5-0ef](https://groups.google.com/d/msgid/elasticsearch/8092eaf5-0ef) > > > > > > > 8-4249-8e5d-acff8281a81a%[40googlegroups.com](http://40googlegroups.com) > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/8092eaf5-0ef8-4249-8e5d-acff8281a81a%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/8092eaf5-0ef8-4249-8e5d-acff8281a81a%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > > > > . > > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > > > > > -- > > > > > > You received this message because you are subscribed to the Google > > > > > > Groups "elasticsearch" group. > > > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/) > > > > > > msgid/elasticsearch/1d9c8ce5-116f-40cc-a5e3-6ebe47191850%40goo > > > > > > [glegroups.com](http://glegroups.com) > > > > > > [https://groups.google.com/d/msgid/elasticsearch/1d9c8ce5-116f-40cc-a5e3-6ebe47191850%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/1d9c8ce5-116f-40cc-a5e3-6ebe47191850%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > > > . > > > > > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/) > > > > msgid/elasticsearch/8628ef07-2e10-423a-9de0-13ebaa37a0e8% > > > > [40googlegroups.com](http://40googlegroups.com) > > > > [https://groups.google.com/d/msgid/elasticsearch/8628ef07-2e10-423a-9de0-13ebaa37a0e8%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/8628ef07-2e10-423a-9de0-13ebaa37a0e8%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > . > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/96f30e99-30dc-4af2-a51b-b12b0dee9e53%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/96f30e99-30dc-4af2-a51b-b12b0dee9e53%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/96f30e99-30dc-4af2-a51b-b12b0dee9e53%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/96f30e99-30dc-4af2-a51b-b12b0dee9e53%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAHTr4Zt%3Drs%2BnOhoHP2UPM5ZmV%2BWQet\_8FrmiGHFpf\_Ykokdxyw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAHTr4Zt%3Drs%2BnOhoHP2UPM5ZmV%2BWQet_8FrmiGHFpf_Ykokdxyw%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Nadav\_Hashimshony](https://avatars.discourse-cdn.com/v4/letter/n/a88e57/32.png) [@Nadav\_Hashimshony](https://discuss.elastic.co/u/Nadav_Hashimshony)\ **Post date:** [February 15, 2015, 12:24pm UTC](https://discuss.elastic.co/t/elasticsearch-attachment-plugin-kibana-couchbase/22133/9 "2015-02-15T12:24:05Z")
thanks, sent you a pm On Thursday, February 12, 2015 at 5:24:33 PM UTC+2, Itamar Syn-Hershko wrote: > Yes, that too 🙂 > > Also if its a time based data, you will not be able to use kibana's date > filtering etc - because it lacks the @timestamp field. Basically, the XDCR > elasticsearch plugin was built around the XDCR / Couchbase realm and not > around Elasticsearch's. Unfortunately this means many ES features are > unavailable / hard to use, e.g. > [https://github.com/couchbaselabs/elasticsearch-transport-couchbase/issues/63](https://github.com/couchbaselabs/elasticsearch-transport-couchbase/issues/63) > > [https://github.com/couchbaselabs/elasticsearch-transport-couchbase/issues/64](https://github.com/couchbaselabs/elasticsearch-transport-couchbase/issues/64) > > I can help fixing this on the XDCR plugin if you'd like - ping me > privately and we can work something out (or I can convince you to avoid > using the XDCR replication) > > -- > > Itamar Syn-Hershko > [http://code972.com](http://code972.com) | @synhershko [https://twitter.com/synhershko](https://twitter.com/synhershko) > Freelance Developer & Consultant > [Lucene.NET](http://Lucene.NET) committer and PMC member > > On Thu, Feb 12, 2015 at 5:18 PM, Nadav Hashimshony \<[nad...@gmail.com](mailto:nad...@gmail.com) > \\> wrote: > > > ok, ill try. > > > > this "envelope document", is it something i need to be concerned about > > when I'm querying via Kibana? > > > > On Thursday, February 12, 2015 at 5:14:39 PM UTC+2, Itamar Syn-Hershko > > wrote: > > > > > Yes. Just make sure the template reflects the actual document structure > > > > > > - as I said XDCR wraps your document in an envelope document > > > > > > -- > > > > > > Itamar Syn-Hershko > > > [http://code972.com](http://code972.com) | @synhershko [https://twitter.com/synhershko](https://twitter.com/synhershko) > > > Freelance Developer & Consultant > > > [Lucene.NET](http://Lucene.NET) committer and PMC member > > > > > > On Thu, Feb 12, 2015 at 5:12 PM, Nadav Hashimshony [nad...@gmail.com](mailto:nad...@gmail.com) > > > wrote: > > > > > > > ok, just to be clear. > > > > > > > > the steps i did was as followed: > > > > > > > > 1. create the index with the mapping. > > > > 2. define the XDCR to replicate my bucket with the index in ES. > > > > 3. insert data to couchbase. > > > > 4. try to query with kibana > > > > > > > > What you suggest is to Add another BEFORE step 1: > > > > 0. create a template to include my mapping. > > > > > > > > 1. crate the index in ES > > > > and so on... > > > > > > > > did i get it right? > > > > > > > > Thanks. > > > > Nadav. > > > > > > > > On Thursday, February 12, 2015 at 5:04:24 PM UTC+2, Itamar Syn-Hershko > > > > wrote: > > > > > > > > > Like I said, you need the mapping to "catch" before the XDCR plugin > > > > > begins the replication - so you need to put a template with this mapping > > > > > that will override XDCR's > > > > > > > > > > -- > > > > > > > > > > Itamar Syn-Hershko > > > > > [http://code972.com](http://code972.com) | @synhershko [https://twitter.com/synhershko](https://twitter.com/synhershko) > > > > > Freelance Developer & Consultant > > > > > [Lucene.NET](http://Lucene.NET) committer and PMC member > > > > > > > > > > On Thu, Feb 12, 2015 at 4:59 PM, Nadav Hashimshony [nad...@gmail.com](mailto:nad...@gmail.com) > > > > > wrote: > > > > > > > > > > > Thanks you for the response > > > > > > > > > > > > i am using mapping, i created the following index > > > > > > PUT /storage/files/\_mapping > > > > > > { > > > > > > "files": { > > > > > > "properties": { > > > > > > "file": { > > > > > > "type": "attachment", > > > > > > "path": "full", > > > > > > "fields": { > > > > > > "content\_type": { > > > > > > "type": "string", > > > > > > "store": true > > > > > > } > > > > > > } > > > > > > } > > > > > > } > > > > > > } > > > > > > } > > > > > > > > > > > > when i insert data via ES and query it, all is fine. > > > > > > the problem is when data is inserted to Couchbase.. > > > > > > > > > > > > Nadav > > > > > > > > > > > > On Thursday, February 12, 2015 at 4:03:01 PM UTC+2, Itamar > > > > > > Syn-Hershko wrote: > > > > > > > > > > > > > The XDCR plugin indexes the data using an envelope document. Long > > > > > > > story short, make sure you use the latest XDCR plugin as older ones are > > > > > > > missing lots of important functions, and use templates and dynamic > > > > > > > templates with proper field paths for this to work correctly > > > > > > > > > > > > > > [http://code972.com/blog/2015/02/80-elasticsearch-one-tip-a-d](http://code972.com/blog/2015/02/80-elasticsearch-one-tip-a-d) > > > > > > > ay-managing-index-mappings-like-a-pro > > > > > > > [http://code972.com/blog/2015/02/81-elasticsearch-one-tip-a-d](http://code972.com/blog/2015/02/81-elasticsearch-one-tip-a-d) > > > > > > > ay-using-dynamic-templates-to-avoid-rigorous-mappings > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > Itamar Syn-Hershko > > > > > > > [http://code972.com](http://code972.com) | @synhershko [https://twitter.com/synhershko](https://twitter.com/synhershko) > > > > > > > Freelance Developer & Consultant > > > > > > > [Lucene.NET](http://Lucene.NET) committer and PMC member > > > > > > > > > > > > > > On Thu, Feb 12, 2015 at 3:59 PM, Nadav Hashimshony \<[nad...@gmail.com](mailto:nad...@gmail.com) > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > I'm new to the group, hope ill find what i need and share my > > > > > > > > experience as i go along.. > > > > > > > > > > > > > > > > im using ES with the attachment-plugin in order to store and search > > > > > > > > files. > > > > > > > > when i set the mapping right and insert the file data in a Base64 > > > > > > > > manner I'm able to query my data via Kibana. > > > > > > > > > > > > > > > > my problem is this. > > > > > > > > > > > > > > > > if i create the index + mapping in ES, then insert the data to > > > > > > > > Couchbase and use XDRC to replicate it to ES, i can't query the Data with > > > > > > > > Kibana. > > > > > > > > it looks like the mapping of the index created in ES doesn't index > > > > > > > > well the data it gets from Couchbase. > > > > > > > > > > > > > > > > has anyone encounter such an issue? > > > > > > > > > > > > > > > > Thanks You > > > > > > > > > > > > > > > > Nadav. > > > > > > > > > > > > > > > > -- > > > > > > > > You received this message because you are subscribed to the Google > > > > > > > > Groups "elasticsearch" group. > > > > > > > > To unsubscribe from this group and stop receiving emails from it, > > > > > > > > send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > > > > > To view this discussion on the web visit > > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/8092eaf5-0ef](https://groups.google.com/d/msgid/elasticsearch/8092eaf5-0ef) > > > > > > > > 8-4249-8e5d-acff8281a81a%[40googlegroups.com](http://40googlegroups.com) > > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/8092eaf5-0ef8-4249-8e5d-acff8281a81a%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/8092eaf5-0ef8-4249-8e5d-acff8281a81a%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > > > > > . > > > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > > > > > > > -- > > > > > > > You received this message because you are subscribed to the Google > > > > > > > Groups "elasticsearch" group. > > > > > > > To unsubscribe from this group and stop receiving emails from it, > > > > > > > send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/) > > > > > > > msgid/elasticsearch/1d9c8ce5-116f-40cc-a5e3-6ebe47191850%40goo > > > > > > > [glegroups.com](http://glegroups.com) > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/1d9c8ce5-116f-40cc-a5e3-6ebe47191850%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/1d9c8ce5-116f-40cc-a5e3-6ebe47191850%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > > > > . > > > > > > > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > > > -- > > > > > You received this message because you are subscribed to the Google > > > > > Groups "elasticsearch" group. > > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/) > > > > > msgid/elasticsearch/8628ef07-2e10-423a-9de0-13ebaa37a0e8% > > > > > [40googlegroups.com](http://40googlegroups.com) > > > > > [https://groups.google.com/d/msgid/elasticsearch/8628ef07-2e10-423a-9de0-13ebaa37a0e8%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/8628ef07-2e10-423a-9de0-13ebaa37a0e8%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > > . > > > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > -- > > > You received this message because you are subscribed to the Google Groups > > > "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send an > > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/96f30e99-30dc-4af2-a51b-b12b0dee9e53%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/96f30e99-30dc-4af2-a51b-b12b0dee9e53%40googlegroups.com) > > > [https://groups.google.com/d/msgid/elasticsearch/96f30e99-30dc-4af2-a51b-b12b0dee9e53%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/96f30e99-30dc-4af2-a51b-b12b0dee9e53%40googlegroups.com?utm_medium=email&utm_source=footer) > > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/d4a1ccd2-d389-4218-b54c-349976de2a46%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/d4a1ccd2-d389-4218-b54c-349976de2a46%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![tunix](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tunix/32/19441_2.png) [@tunix](https://discuss.elastic.co/u/tunix)\ **Post date:** [June 22, 2017, 7:25pm UTC](https://discuss.elastic.co/t/elasticsearch-attachment-plugin-kibana-couchbase/22133/10 "2017-06-22T19:25:06Z")
> [@Itamar\_Syn\_Hershko](#): > > Also if its a time based data, you will not be able to use kibana's date > > filtering etc - because it lacks the @timestamp field. Basically, the XDCR > > elasticsearch plugin was built around the XDCR / Couchbase realm and not > > around Elasticsearch's. Unfortunately this means many ES features are > > unavailable / hard to use Hi @Itamar_Syn_Hershko, I'm using 5.4.1 for ES & Couchbase and the XDCR transport plugin. I'm able to query my dataset but I cannot use Kibana's time filtering. During index pattern creation, I checked the checkbox for time series data and it was able to show me `doc.createdAt` which is the datetime field (iso8601) in my docs. However no timepicker or histogram in discover tab appears. I couldn't really find a pointer about this issue. Should I have a field with the exact same name: `@timestamp`? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 4, 2022, 4:01am UTC](https://discuss.elastic.co/t/elasticsearch-attachment-plugin-kibana-couchbase/22133/11 "2022-11-04T04:01:25Z")