Nikhil Das Nomula
I have been reading about data mesh architecture by Zhamak Dehgani and it has been thought provoking in thinking how data is handled in organizations. The typical way that this is handling right now is that we have a central enterprise data team that uses a centralized datawarehouse where analytics are run on top of that
However datamesh architecture challenges that for the right reasons to use data as a product and that data is owned by the team that produces it which is inline with the modern microservices architecture.
For e.g. we have one team that is responsible for checkout that has front end devs, backend devs(or full stack devs) and a data engineer who handles streaming this data to a data warehouse for analytical purposes and responsible for setting the data contracts
The obvious question that comes to mind is what would the enterprise data team do, this is where their expertise comes into picture where they work like what the platforms team does for app development.
Companies can either have a separate data platforms team that enables the principles of datamesh and makes data experience easier for teams or be a part of platforms team
#datamesh #dataengineering