Confluent Reimagines Data Pipelines for the Streaming Era with Stream Designer 

Confluent, Inc.’s new Stream Designer enables developers to build and deploy streaming data pipelines quickly.

“We are in the middle of a major technological shift, where data streaming is making real time the new normal, enabling new business models, better customer experiences, and more efficient operations,” said Jay Kreps, Cofounder and CEO, Confluent. “With Stream Designer we want to democratise this movement towards data streaming and make real time the default for all data flow in an organisation.”

In the streaming era, data streaming is the default mode of data operations for successful modern businesses. The streaming technologies that were once at the edges have become core to critical business functions. This shift is fueled by the growing demand to deliver data instantaneously and scalably across a full range of customer experiences and business operations. Traditional batch processing can no longer keep pace with the growing number of use cases that depend on sub-millisecond updates across an ever-expansive set of data sources.

 Organisations are seeking ways to accelerate their data streaming initiatives as more of their business is operating in real time. Kafka is the de facto standard for data streaming, as it enables over 80% of Fortune 100 companies to reliably handle large volumes and varieties of data in real time. However, building streaming data pipelines on open-source Kafka requires large teams of highly specialised engineering talent and time-consuming development spread across multiple tools. This puts pervasive data streaming out of reach for many organisations and leaves existing legacy pipelines clogged with stale and outdated data.

“A rising number of organisations are realising streaming data is imperative to achieving innovation and maintaining a healthy business,” said Amy Machado, Research Manager, Streaming Data Pipeline, IDC. “Businesses need to add more streaming use cases, but the lack of developer talent and increasing technical debt stand in the way. Visual interfaces, like Stream Designer, are key advancements to overcoming these challenges and make it easier to develop data pipelines for existing teams and the next generation of developers.”

  “Data streaming is quickly becoming the central nervous system of our infrastructure as it powers real-time customer experiences across our 12 countries of operations,” said Enes Hoxha, Enterprise Architect, Raiffeisen Bank International. “Stream Designer’s low-code, visual interface will enable more developers, across our entire organization, to leverage data in motion. With a unified, end-to-end view of our streaming data pipelines, it will improve our developer productivity by making real-time applications, pipeline development, and troubleshooting much easier.”

Stream Designer provides developers a flexible point-and-click canvas to build pipelines in minutes and describe data flows and business logic easily within the Confluent Cloud UI. It takes a developer-centric approach, where users with different skills and needs can seamlessly switch between the UI, a code editor, and command-line interface to declaratively build data flow logic at top speed. It brings developer-oriented practices to pipelines, making it easier for developers new to Kafka to scale data streaming projects faster.

Read the latest edition of PCR’s monthly magazine below:

Like this content? Sign up for the free PCR Daily Digest email service to get the latest tech news straight to your inbox. You can also follow PCR on Twitter and Facebook.