# Switching to Big Data Architecture

**URL:** <https://discuss.elastic.co/t/switching-to-big-data-architecture/158478>\
**Category:** Elasticsearch\
**Created:** [November 28, 2018, 6:45am UTC](https://discuss.elastic.co/t/switching-to-big-data-architecture/158478 "2018-11-28T06:45:37Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![josephmanalo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/josephmanalo/32/38163_2.png) [@josephmanalo](https://discuss.elastic.co/u/josephmanalo)\
**Post date:** [November 28, 2018, 6:45am UTC](https://discuss.elastic.co/t/switching-to-big-data-architecture/158478/1 "2018-11-28T06:45:38Z")

</div>

Hi guys. I would like to ask for some advice.

Currently this is what our system architecture looks like.

Current Process:

1. Store data to Elasticsearch
2. Transform data using Python
3. Save the data using Postgres
4. Visualize data using Tableau

Other Info:  
-Elasticsearch servers and Postgre Server are just commodity hardware(laptops with i7,16GB RAM, 1TB HDD)  
-We have a lot of commodity hardwares (i7, 16GB+ RAM, 1TB HDD)  
-We are planning to buy one server (with 32GB RAM, 4TB of storage... this is upgradeable up to 786GB RAM and 95TB of storage).  
-In the long run I don't think we can afford to buy a Server again.

Currently we are having problems especially the data volume is increasing (Approximately 16 Million rows per day)  
And we are thinking of implementing Big Data Architecture(Hadoop) to our system.

Given the details, what's the best thing we can do to maintain this kind of system?

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/1/c/1c0ec256120434496f3495acd9df338f0f2f1a28.png)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 26, 2018, 6:45am UTC](https://discuss.elastic.co/t/switching-to-big-data-architecture/158478/2 "2018-12-26T06:45:41Z")

</div>

Closed on December 26, 2018, 6:45am UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.