Vendor lock-in and the challenge of data gravity

Ravi Mayuram at Couchbase argues that data gravity challenges can derail digital transformation, and describes how to avoid them

 

Digital transformation has reached every corner of the C-suite. Businesses across all sectors now recognise the importance of harnessing data across their operations, especially as organisations look to be both relevant in the new economy as well as bring new experiences that will help retain and attract new customers.

 

There is clear recognition that without a clear data strategy to support future business plans, any digital transformation work  is simply window dressing and not a serious transformative initiative.

 

To achieve proactive digitisation and deliver more sophisticated customer journeys, spot market-trends ahead of competitors, and provide new services, making sense of the growing volume of incoming data is crucial. And over the last decade, organisations have done a good job of collecting this data from a variety of their applications, websites, and internal functions. 

 

As data grows, more applications and services are attracted into its ‘orbit’ – which is what leads us to data gravity. The effects of this come to the fore when enterprises try to move data between different service providers to make use of greater accessibility, extra storage, or new features – and instead are weighed down by this pull of data and are stuck with vendor lock-in.

 

This lock-in dilemma is nothing new, we encounter it daily in all walks of life - printers, Apple, not to mention the original poster child for this - Gillette.

 

But these are consumer products and not to be confused with enterprise software where things are not that ‘disposable’. Fundamentally, the issue prevents customers from making easy, flexible choices, and instead ties them to specific vendors or brands. 

 

In the world of computing and cloud services, vendors can make it difficult, costly, or even impossible to move to another provider’s service; and the primary strategy adopted here is data gravity. 

 

The hypothesis is, the larger the volume of data that is accumulated, the more services that are built on it, the harder and more expensive it is to move.

 

This data gravity then obeys fundamental laws of physics and does not allow services to attain escape velocity. This causes innovation and transformation projects to stall and die on the vine. This can be expensive. The average cost of failed, delayed, or scaled-back projects is over $4 million.

 

The magnitude of this challenge is underlined by Gartner research: “most organisations adopt a multi-cloud strategy out of a desire to avoid vendor lock-in.” 

 

Directing an entire cloud strategy to avoid lock-in is just one technique available to organisations. To navigate data gravity challenges, business leaders need to implement strategies for maintaining true ownership over data and services – just like any other critical business asset. 

 

Flexibility is key

Maintaining flexibility on cloud service providers keeps the power with organisations, providing freedom and negotiating power later down the line to avoid data gravity issues. The best time for businesses to act on this is in the planning stages for new applications. 

 

Rather than building an application to work with a specific cloud service, like Google Cloud or Amazon Web Services, enterprises should strive to maintain flexibility by building the new app as vendor neutral.

 

Alternatively, if partnering with a cloud provider makes sense to utilise the added computing infrastructure and services on offer for faster go to market of their application, then they should build the abstractions within the application to enable the  switch out of equivalent services provided by other cloud providers.

 

This philosophy of development is nothing new, in fact this is what good software architects always do -abstracting away dependencies -  be it operating systems or databases.

 

This step has largely been forgotten among wider cloud migration. Understandably, shiny new services released by cloud service providers have hooked businesses in and consequently locked them in with control over the application architecture and data.

 

But if your business decides to expand into global regions not covered by the specific cloud service provider, for example, then enterprises can be left with an issue of audience reach and consequently loss of revenue.

 

It is vital organisations retain total control over data portability and data migration.

 

Use open standards

Businesses must also take responsibility before working with vendors to ensure they can avoid data gravity challenges. Just as corporate vehicle fleets tend to use common diesel, petrol or electric engines instead of hydrogen, a popular technique for avoiding data gravity is via open standards software – software that is not owned by a particular company or vendor. 

 

Utilising this approach to create software solutions embeds flexibility and portability into the heart of new services, preventing lock-in. The open standards movement has prevented organisations becoming reliant on a single software vendor. This should feed into a wider commitment from enterprises to train their workforce to think and act with a vendor-neutral mindset.

 

It’s all in the detail

Many businesses skim through the fine print of their contracts. Along with any other contractual agreement, the devil is in the detail – and businesses must review them with a fine-toothed comb to make sure vendor lock-in isn’t inadvertently built in.

 

More specifically, businesses should find out whether a given service allows for easy migration; if there are tools or resources available to support integration and migration of applications; and if the service is within budget.

 

They should also be wary of contractual agreements, such as auto-renewals, that could trap them into a service long-term.

 

When possible, negotiating the ownership of data can help alleviate data gravity challenges.

 

Future-proofing your enterprise

There are certain circumstances where vendor lock-in might still make sense. For a new business with limited resources and skills, it might help to use standardised hardware vendors, software vendors, or cloud providers – just as you typically use stabilisers when first starting to ride a bike.

 

But by following simple open standards and abstraction principles early in the cycle, businesses can remain nimble with their long-term strategy. 

 

This is vital given that we are in the midst of a generational transition in computing and specifically software development. This journey spans enterprise applications (fully written by humans), to data applications (written by humans supervising machines using machine learning and data science) and now includes generative applications (written by machines using large language models and supervised by humans).

 

The sophistication of the applications developed in the near future will be much more advanced than what has been developed so far. And the primary enabler of this is data. The quality of any application will entirely depend on the quality and quantity of data that feeds them.

 

Organisations that prioritise this will have an undeniable advantage over their competitors.

 

---

 

Ravi Mayuram is CTO at Couchbase

 

Main image courtesy of iStockPhoto.com