# Architecture question re. routing and multi DC **URL:** \ **Category:** Elasticsearch\ **Created:** [January 9, 2014, 11:08am UTC](https://discuss.elastic.co/t/architecture-question-re-routing-and-multi-dc/15171 "2014-01-09T11:08:47Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![Arik\_Fraimovich](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/arik_fraimovich/32/1881_2.png) [@Arik\_Fraimovich](https://discuss.elastic.co/u/Arik_Fraimovich)\ **Post date:** [January 9, 2014, 11:08am UTC](https://discuss.elastic.co/t/architecture-question-re-routing-and-multi-dc/15171/1 "2014-01-09T11:08:47Z")
For redundancy purposes, our system is split into two datacenters. One of the DCs is considered central where all the backoffice systems reside and the other is edge. Recently we started using Logstash with ElasticSearch and Kibana. The architecture we had is: - Scribe server on each instance in our cluster forwards logs to a main scribe instance in the DC. - If the DC is the edge, its main scribe instance forwards all logs to the main scribe instance in central. - From the main (central) scribe server we forward message to Logstash, which in turn get written to ES. Because most logs are only stored but never retrieved, to reduce the traffic between DCs, we thought of using custom routing: - Have elastic search node in each DC (currently we have only one). - Tag each log message with the DC it's originated from and route the log messages according to this tag, so each DC's log messages end up in its own ES instance. Will this work? Is this proper use of ElasticSearch's routing? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/0017a4a8-80ca-4fcb-97df-032f9d6858c9%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/0017a4a8-80ca-4fcb-97df-032f9d6858c9%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Ananya\_Sethi](https://avatars.discourse-cdn.com/v4/letter/a/b2d939/32.png) [@Ananya\_Sethi](https://discuss.elastic.co/u/Ananya_Sethi)\ **Post date:** [April 29, 2014, 7:05am UTC](https://discuss.elastic.co/t/architecture-question-re-routing-and-multi-dc/15171/2 "2014-04-29T07:05:01Z")
"From the main (central) scribe server we forward message to Logstash, which in turn get written to ES." Hey. I wanted to know how did you infact forward the message from scribe server to logstash? Did you do it by creating a custom logstash input plugin by yourself? Thanks in advance. Regards, Ananya On Thursday, January 9, 2014 4:38:47 PM UTC+5:30, Arik Fraimovich wrote: > For redundancy purposes, our system is split into two datacenters. One of > the DCs is considered central where all the backoffice systems reside and > the other is edge. Recently we started using Logstash with Elasticsearch > and Kibana. The architecture we had is: > > - Scribe server on each instance in our cluster forwards logs to a > main scribe instance in the DC. > - If the DC is the edge, its main scribe instance forwards all logs to > the main scribe instance in central. > - From the main (central) scribe server we forward message to > Logstash, which in turn get written to ES. > > Because most logs are only stored but never retrieved, to reduce the > traffic between DCs, we thought of using custom routing: > > - Have Elasticsearch node in each DC (currently we have only one). > - Tag each log message with the DC it's originated from and route the > log messages according to this tag, so each DC's log messages end up in its > own ES instance. > > Will this work? Is this proper use of Elasticsearch's routing? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/832ee819-de12-437c-b48a-6688330c2625%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/832ee819-de12-437c-b48a-6688330c2625%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![ananyasethi](https://avatars.discourse-cdn.com/v4/letter/a/f475e1/32.png) [@ananyasethi](https://discuss.elastic.co/u/ananyasethi)\ **Post date:** [April 29, 2014, 7:10am UTC](https://discuss.elastic.co/t/architecture-question-re-routing-and-multi-dc/15171/3 "2014-04-29T07:10:27Z")
From the main (central) scribe server we forward message to Logstash, which in turn get written to ES." Hey. I wanted to know how did you infact forward the message from scribe server to logstash? Did you do it by creating a custom logstash input plugin by yourself? Thanks in advance. Regards, Ananya ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:32am UTC](https://discuss.elastic.co/t/architecture-question-re-routing-and-multi-dc/15171/4 "2017-07-06T01:32:53Z")
Closed on July 6, 2017, 1:32am UTC