Data streaming goes mainstream

Richard Timperlake at Confluent explores how data streaming is set to become a must-have for businesses, transforming how they operate and stay competitive

 

2024 was an impressive year for technology’s impact on the business mainstream - particularly, as you know, in the case of AI. From ChatGPT’s integration into almost everything to the launch of assistants like Microsoft Copilot, AI has felt like an inescapable phenomenon. The appetite for the technologies and riches that it might unlock has never been higher. 

 

Amid these innovations, one factor stood out: speed. Whether it’s in e-commerce logistics, financial services, or remote work, technology is evolving faster than ever. 

 

That pace will accelerate in the coming year. That’s partially down to data’s ability to travel more quickly, as data streaming – and the platforms that enable it – continues to emerge as a reliable catalyst for digital transformation.

 

The tipping point for data streaming

Data streaming, in simple terms, is the continuous flow of data even as it’s generated. 

 

Traditionally, businesses would collate all their data into huge storage reservoirs and only then begin cutting it into segments to analyse. Data streaming, by contrast, analyses each data point while it’s in motion, which allows data streaming platforms (DSPs) to route the data to the most relevant parts of the business, in real-time. This helps power everything from multiplayer games and real-time fraud detection to social media feeds, stock trading platforms, and GPS tracking. 

 

2025 will be the year that data streaming goes mainstream. After a huge year for elections right across EMEA, a more confident economic outlook will see both government and businesses ramp up investment in data infrastructure, with real-time data streaming at the forefront. 

 

Here in the UK, Keir Starmer’s AI action plan highlights the government’s push to use AI and data to drive smarter public services in everything from healthcare to urban planning. That investment will include an emphasis on improving data use and infrastructure. Only a month into 2025, we’ve already seen the announcement of plans for one of Europe’s largest data centres in Hertfordshire—laying the foundations for the development of AI far into the future.

 

As businesses and governments realise the power of real-time data, it’s clear—data streaming isn’t just a “nice-to-have”; it’s transformative. Investment in infrastructure is an important step in delivering that gold dust; a 2024 report on DSP adoption found that 84% of companies using DSPs could cite an ROI between 2x and 10x their initial investment.

 

Cultural resistance and legacy systems

While speed builds, this transition can’t happen overnight. Traditional models like batch processing still dominate in some organisations – particularly financial services, where the prevalence of legacy technology remains an issue – while scepticism about disrupting workflows can make businesses hesitant to fully embrace DSPs. For many, the question remains: "Why fix what isn’t broken?"

 

The answer to that is that data streaming isn’t just a solution or an upgrade—it’s a fundamental shift of perspective. It democratises data, allowing every department across the business to operate with more context and transparency than ever before, as well as opening doors to new revenue streams. 

 

But overcoming resistance requires more than just technical know-how. It’s about changing the culture of an organisation, getting buy-in from leadership, and providing the right expertise to lead the transition. This is where we’ll see a gradual gap opening between early adopters and latecomers to the technology.

 

Profit and efficiency with real-time data 

The businesses that recognise this will focus on three strategic goals: reducing costs, boosting profitability, and improving customer experience. Real-time data streaming is emerging as the key to achieving all three.

 

For businesses focused on cutting costs, DSPs provide a pathway to streamline operations, eliminate inefficiencies, and unlock previously hidden savings. 

 

For those focused on profitability, real-time customer data—such as website interactions or social media trends—will be crucial in making timely, accurate decisions that directly impact the bottom line.

 

The biggest beneficiary of real-time data, however, is customer experience. By analysing customer behaviours as they happen, businesses can deliver personalised interactions that build loyalty and drive long-term engagement. In 2025, the companies that can leverage real-time data will be the ones that set themselves apart in crowded markets.

 

The power of GenAI and data streaming

If 2024 was the year of Generative AI (GenAI), 2025 will be the year when AI integrates seamlessly with data streaming to transform business operations. Recent reports suggest that 74% of UK executives expect GenAI to continue “trending up” for at least the next two years, compared to the 5% who think we’ve reached the apex. 

 

As GenAI continues to mature, the need for real-time data becomes critical. DSPs ensure AI models are fed with accurate, up-to-date information, as they stream data in real-time. That data can be injected back into the stream, better contextualising future data — a constant cycle of self-refinement.

 

This allows companies to make better, faster decisions. Imagine a customer service chatbot not only answering queries, but also predicting a customer’s next move based on real-time data — thanks to a seamless stream of insights that are contextualised and organised before they even reach you.

 

Decision-making in real-time

Given the pace at which the modern business world moves, leaders can’t afford to wait. Data-driven decision-making is more important than ever, but the older that data is, the less applicable it can be. Real-time data is the only way to guarantee genuine relevance to the decisions you make.

 

Unfortunately, many companies still struggle with siloed systems and slow access to actionable insights. According to a recent study, 90% of executives now need to make decisions in real-time, yet 52% say the data they receive is already outdated by the time it reaches them. 

 

This is where data streaming becomes essential — it bridges the gap, providing timely, actionable insights that help leaders respond swiftly and decisively. Businesses that adopt it will gain a significant competitive advantage, while those that hesitate risk falling behind.

 

---

 

Richard Timperlake is SVP, EMEA at Confluent 

 

Main image courtesy of iStockPhoto.com and sankai