Confluent

Confluent develops two enterprise-grade offerings that both prominently include Apache Kafka: Confluent Cloud, and Confluent Platform. Both offer extended features beyond that of Apache Kafka, including commercial support, with the only substantial difference between the two being that Confluent Cloud is “cloud-native”, meaning it lives in the cloud and is fully managed by Confluent, whereas Confluent Platform is self-managed and can be deployed in-cloud or on-prem. Kafka itself is a distributed streaming platform that remains the most popular open source offering in its class. It offers a number of important capabilities: secure, guaranteed message delivery, stream processing, and data storage are perhaps the most notable.

Confluent takes this core functionality and adds various features to make it suitable for enterprise deployment. These features run the gamut, covering security, development, compatibility, connectivity, governance, and so on, as well as support backed by extensive Kafka expertise. The goal is to create a platform that builds on Kafka to reduce or even eliminate operational requirements, with a cloud-native solution that’s available everywhere.

Customer Quotes

Confluent customers report saving six to nine months of development time by using Confluent connectors rather than building and configuring their own.

Apache Kafka enables its users to process data in-stream. It can combine stream processing with historic data, and it provides a Kappa architecture capable of processing all data in a single environment. Several performance-oriented features are provided, including “log compaction” (a Kafka exclusive), partitions to support parallel processing, and both offset and timestamp indexes.

By default, Kafka supports applications written in Java and Scala. Confluent adds development support for other languages, including C, C++, Go, Python, and .NET. Confluent Connect provides a wide range of connectors – 120 are currently available, a substantial number of which are fully managed – and Confluent Control Center offers end-to-end monitoring and alerting. A range of operational capabilities are also provided, including automated load balancing and replication capabilities that support multi-data centre Kafka implementations.

Confluent Stream Governance, a recent addition, is a data governance suite designed specifically for governing streaming data. It includes Confluent’s Schema Registry, a longstanding capability that acts as a central repository for the format of Kafka data, as well as a streaming data catalogue. Data discovery and data lineage capabilities are also provided. This is both a notable addition and a significant differentiator, and it will likely prove particularly relevant for managing your streaming data as streaming technologies become increasingly pervasive and widespread. In previous years, we have praised Confluent Control Center for adding manageability to Kafka: Confluent Stream Governance builds on this to the nth degree, raising the platform to another level in this regard.

Kafka is extremely popular. It is deployed within approximately 70% of the Fortune 500, and 80% of the Fortune 100. It is also notable that, at least for the former figure, this percentage has doubled between this report and its immediate predecessor. This speaks to both the growth of streaming as a technology and the persistent popularity that Kafka enjoys. This level of popularity is an advantage in itself: Kafka integrates with everything, because everything wants to integrate with Kafka.

Moreover, Kafka is very capable of forming the bedrock of your streaming solution. However, it is not a solution in and of itself: you will either need to build on top of it yourself, or adopt a streaming platform based on it, such as Confluent. There are very good reasons to do the latter, not the former: it is far easier; likely cheaper (at least in the medium-long term); and at least in Confluent’s case, it offers a high quality, multi-cloud, hybrid, fully-managed solution that is well-equipped for enterprise deployment. We also find Confluent Stream Governance to be particularly impressive and forward-thinking.

The Bottom Line

Kafka is almost a de facto standard for distributed streaming environments, and Confluent expands on it in almost every conceivable way. If you are at all serious about streaming technologies, you should at least be considering it.