Succeeding with data mesh

Andrea Novara at Agile Lab explains why data guardrails are the key to successful data mesh 

 

Many of today’s organisations, regardless of size or industry sector, still operate a traditional ‘top down’ centralised decision-making structure. While these hierarchical models create a clear chain of command and accountability, they can also lead to inefficiencies and bureaucratic bottlenecks that stifle creativity and innovation while preventing organisations from responding fast to rapidly changing business environments.  

 

To address these challenges, more and more organisations want to distribute decision-making authority to the front line. Having digitalised their operations, they’re looking to leverage data and analytics technologies to empower local teams so they can make faster and more informed decisions and execute these in an agile way. 

 

The key to making all this happen is data mesh: a federated data architecture approach that empowers teams and subject experts to rapidly discover, access, share, and consume the data and insights they need in real-time. However, implementing data mesh is not a simple task and getting it right depends on setting strict ground rules from the offset. 

 

Transforming business performance and agility 

Initiating data mesh involves a significant shift in the way enterprises think about data. Historically, enterprises have operated centralised infrastructures that maintain data ownership across specific domain teams. This centralised approach to data management creates boundaries and information silos that make it difficult to free up data for decentralised decision making. 

 

For example, when a marketing department wants to pull together sales statistics and align these with customer service feedback, it first must approach the corporate IT team and request all this information. This process can involve significant time and effort to complete. Along the way, the marketing team may encounter rejections or justification requests that will prove demotivating and discourage innovation. 

 

By contrast, when central IT teams initiate a more agile self-service approach to data enablement and retrieval, a number of enterprise-wide gains can be realised.  

 

Firstly, IT teams will no longer be burdened with having to handle data requests from multiple teams on a daily basis. By exposing data to all domains and eliminating dependency on centralised systems and IT teams, duplication of effort can be avoided and time-to-market for new products and services can be slashed. Meanwhile, this new era of data autonomy supports rapid and easy collaboration across teams. 

 

The ‘what, how, and why’ of data mesh 

Data mesh facilitates a shift towards the decentralisation of data by placing data ownership and management into the hands of domain owners. This means that functions such as marketing, sales, and finance become responsible for the data they generate as they are subject matter experts who understand it best and care about it the most. Acting as independent entities, these domain owners are accountable for the stewardship of their data, including its accuracy and quality. But that’s not all. 

 

To assure the consistency and control needed to effectively treat data like a product, each department becomes responsible for turning its raw data into clearly defined products that all enterprise-wide users can understand, access, and utilise. To achieve this in a standardised way, tightly defined governance rules are utilised to ensure everyone can access up-to-date information to drive their business initiatives and decision making. 

 

Meanwhile, the IT team itself becomes responsible for providing a secure central platform with easy-to-use tools and plenty of capacity for departments to manage a growing number of data products. This self-service approach effectively negates reliance on IT to access data and ensures that users can quickly analyse and extract insights from the new ‘data as a product’ format. 

 

However, to ensure everything works as intended, careful thought needs to be applied to the creation of governance frameworks that both accommodate domain-specific needs while maintaining data consistency and integrity across the extended data ecosystem. 

 

The importance of data guardrails 

Overarching data governance rules will need to be established to assure quality and coherence across this decentralised environment. Alongside setting out the standardised field types, meta data, and schema flags that will facilitate interoperability, these rules should feature controls for sensitive data to ensure that security and regulatory compliance can be maintained. 

 

Rather than trusting individuals to stick to data guidelines, organisations will need to utilise a computational data governance approach to ensure that all projects follow pre-determined policies governing data quality, architecture, compliance, and security. By ensuring these can’t be bypassed, at a global and local level, organisations can ensure that users are unable to create new data products that don’t meet these pre-set criteria.  

 

To simplify life for users, automated templates speed up how data practitioners set up and initiate projects across technologies and tools and facilitate data access and processing while promoting and enforcing data and access controls. 

 

Transforming how organisations leverage data 

When implemented with computational governance, data mesh can revolutionise how organisations manage and utilise their data to drive innovation and informed decision-making. By enabling data sharing across and between domains without compromising data governance and security, organisations can leverage data mesh to initiate customised business intelligence dashboards. They can also leverage their decentralised and domain-agnostic data architecture to accelerate machine learning and advanced AI-powered analytics programmes.  

 

For organisations that want to manage their data more effectively under the oversight of a centralised governance framework, moving to a data mesh strategy opens the door to scaling up more self-service options for individual business groups – and unlocking more efficiency, innovation, and competitive advantage. 

 

---

 

Andrea Novara is Engineering Lead | Banking & Payments Business Unit Leader at Agile Lab 

 

Main image courtesy of iStockPhoto.com and NicoElNino