Confluent says it will release a series of updates to its data streaming platform every quarter.
In this quarter, the updates consist of a number of new features built from the Apache Kafka open source distributed event streaming platform. They include schema linking, new controls to shrink capacity for clusters on-demand and new fully managed Kafka connectors.
The new capabilities “can make a huge difference in creating a data mesh versus a data mess,” Schema Linking gives organizations the freedom to develop without the risk of damaging production, Rosanova told The New Stack.
“Dev and prod generally don’t talk to one another — because production environments are so sensitive, you don’t want to give everyone access,” Rosanova said. With Schema Linking, built on top of Cluster Linking, schemas can be shared that sync in real-time across teams, organizations and environments, such with hybrid and multicloud environments. “This is far more scalable and efficient compared to workarounds I’ve seen where people are literally sharing schemas through spreadsheets,” Rosanova said.
Much verbiage is devoted to scaling, but how to dynamically adjust network resources for resource savings when needed to avoid redundancy is often not addressed. As Rosanova noted, organizations maintain high availability by beefing up their capacity to handle spikes in traffic and avoid downtime.
“We added a simple, self-service way to scale back capacity so customers no longer have to worry about wasting resources on capacity they don’t use. These clusters also automatically rebalance your data every time you scale up or down,” Rosanova said. “This solves the really hard challenge of rebalancing workloads while they are running. It’s like changing the tires on a moving car. Now you can optimize data placement without disrupting the real-time flow of information.”
Confluent’s new release now features over 50 managed connectors for Confluent Cloud. The idea behind Confluent’s Apache Kafka connectors is to facilitate network connections for data streaming with data sources and sinks that organizations select.
In the last six months, Confluent more than doubled the number of managed connectors it offers, Rosanova said. “Once one system is connected, two more need to be added, and so on,” he said. “We are bringing real-time data to traditional, non-real-time places to quickly modernize companies’ applications. This is a significant need that continues to grow.”
Kafka has emerged as a leading data streaming platform and Confluent continues to evolve with it, Rosanova said.
“We are improving what businesses can accomplish with Kafka through these new capabilities. Real-time data streaming continues to play an important role in the services and experiences that set organizations apart,” Rosanova said. “We want to make real-time data streaming within reach for any organization and are continuing to build a platform that is cloud native, complete, and available everywhere.”
Confluent’s connector list now includes:
Data warehouse connectors: Snowflake, Google BigQuery, Azure Synapse Analytics, Amazon Redshift.
Database connectors: MongoDB Atlas, PostgreSQL, MySQL, Microsoft SQL Server, Azure. Cosmos DB, Amazon DynamoDB, Oracle Database, Redis, Google BigTable.
Data lake connectors: Amazon S3, Google Cloud Storage, Azure Blob Storage, Azure Data. Lake Storage Gen 2, Databricks Delta Lake.
Additionally, Confluent has improved access to popular tools for network monitoring. The platform now offers integrations with Datadog and Prometheus. “With a few clicks, operators have deeper, end-to-end visibility into Confluent Cloud within the monitoring tools they already use,”blog post.
The post Confluent’s Q1 Updates: ‘Data Mesh vs. Data Mess’ appeared first on The New Stack.
The challenge is clear: How to offer real- or near real-time access to data that is continually refreshed across a number of different distributed environments. With different types of data streaming from various sources such as multicloud and on-premises environments, the data, often in shared digital layers such as so-called digital information hubs (DIHs), must be updated asynchronously. This is necessary in order to maintain a consistent user experience.
To that end, data streaming platform provider Apache Kafka, hundreds of different applications and data systems can use it to migrate to the cloud or share data between their data center and the public cloud, Confluent says.
Traditionally, syncing data between multiple clouds or between on-premises and the cloud was “like a bad game of telephone,”
Edge computing is getting more attention of late — because there are advantages to having computing power and data storage near the location where it’s needed. As Edge computing needs grow, users are likely to take a hard look at whether public cloud giants like AWS, Google are their best choice, or whether their local ISP is best suited for the job.
ISPs — including cable, DSL and mobile providers — claim to offer benefits when delivering SaaS and other services compared to public cloud providers: low latency, high-bandwidth connections, fewer security vulnerabilities, regional regulation compliance, and greater data sovereignty. While they must also demonstrate that they can deliver services robust enough to meet DevOps needs, ISPs can offer tremendous benefits and fill gaps in current cloud computing offerings.
“A key concern cloud customers have when leveraging their microservices architecture for the applications they offer or rely on is how to achieve and maintain ultra-low latency,” said
The goal is straightforward, but getting there has proven to be a challenge: how to offer real- or near real-time access to data that is continually refreshed on an as-needed basis across a number of different distributed environments.
Consequently, as different systems of data and their locations can proliferate across different network environments — including multiclouds and on-premises and, in many cases, geographic zones — organizations can struggle to maintain low-latency connections to the data their applications require. The challenges are especially manifest when users require and increasingly demand that their experiences, which are often transactional-based, are met in near- or real-time that require data-intensive backend support.
Many organizations continue to struggle with the challenges of maintaining and relying on data streaming and other ways, such as through so-called “speed layers” with cached memory, to maintain low-latency connections between multicloud and on-premises environments.
In this article, we describe the different components necessary to maintain asynchronously updated data sources consisting of different systems of record for which real-time access is essential for the end-user experience.
For the CIO, the challenges consist of the ability for applications to have low-latency access to data, often dispersed across a number of often highly distributed Continue reading
Network connections can be likened to attending an amusement park, where Dynamic Host Configuration Protocol (DHCP), serves as the ticket to enter the park and the domain name system (DNS) is the map around the park. Network management and security provider Infoblox made a name for itself by collapsing those two core pieces into a single platform for enterprises to be able to control where IP addresses are assigned and how they manage network creation and movement.
“They control their own DNS so that they can have better control over their traffic,” explained Infoblox: How DDI Can Help Solve Network Security and Management Ills
Also available on Google Podcasts, PlayerFM, Spotify, TuneIn
Infoblox’s name for this unified service is DDI, which is
DevOps platform provider JFrog is touting its new Private Distribution Network (PDN) as an “industry first,” because it allows DevOps teams to accelerate application updates through a lightweight solution that combines two network acceleration and optimization technologies —
Video games continue to Bharat Bhat (Okta marketing lead for developer relations) cover why and how video game platforms and connections should be more secure, with guest Okta senior developer advocate Video Game Security Should Be Simple for Developers
Also available on Google Podcasts, PlayerFM, Spotify, TuneIn
The gaming industry has often served as a showcase for some of the industry’s greatest programming talents. As a case in point,
Service mesh integration software provider Solo.io has released into general availability (GA) version 1.8 of its Gloo Edge Kubernetes-native ingress controller and API gateway. Version 1.8 offers integration for legacy SOAP (Simple Object Access Protocol) web services and other features, as Solo seeks to improve API-centric support for scaling needs across cloud native environments.
Based on the Gloo Edge now helps DevOps teams integrate decades-old SOAP through a single API.
Gloo Edge 1.8’s support for SOAP is “the biggest breakout feature” of the release, blog post, Gaun described how SOAP, an XML messaging protocol from the turn of the century, “remains prevalent today for enterprise web services across a number of industries, including financial services and healthcare.”
Yet, “Unfortunately, SOAP (and associated legacy middleware applications) hold back large-scale modernization efforts because there hasn’t been a viable migration approach in the market,” Gaun wrote. “Organizations haven’t been able to tackle incremental deprecation of SOAP web services over time without great difficulty.”
Gloo Edge Enterprise 1.8, with the addition of
Enterprise virtualization software giant VMware says it is “redefining” security as it seeks to help customers meet the challenges associated with a skyrocketing number of threats, more numerous attack vectors, and having fewer human resources at their disposal to help keep attacks at bay.
“So what we’re asking all of these IT security teams to do is essentially to do more — and there’s a lot more complexity,” 2020 Threat Landscape report results, 81% of the survey respondents reported a breach during the past 12 months — with four out of the five breaches (82%) deemed material. At the Continue reading
The open source Istio has emerged as the “dominant” service mesh to manage microservices and Kubernetes environments, solo.io executives say.
Gloo Edge 2.0, to be released in beta in the middle of the year is the “first and the only” Istio-native API gateway with all of Istio’s native functionality, Posta said. The ingress controller will integrate #SoloCon2021 https://t.co/VKAxWqk5KJ is fully committed to Istio. We see it as the dominant service mesh—it’s the one that’s most deployed to production and the most mature. #Gloo @soloio_inc #sponsored March 24, 2021
Solo.io’s proclamation also coincides with a number of new improvements for solo.io’s Gloo Edge platforms announced the new capabilities feature, among other things, an even tighter integration between #SoloCon2021 Continue reading
Version 2.0 of Solo.io’s Gloo Edge will integrate the Gloo Edge, an ingress controller, and the open source Istio service mesh will form a single control plane, Solo.io said this week during its Torsten Volk, an analyst for Enterprise Management Associates (EMA), said.
“Most organizations have regarded Istio as something to ‘attack once it’s become more approachable and easier to manage,’” Volk said. “These Solo.io announcements might ring in this new age of “service mesh for everyone.”
In a Continue reading
Version 2.2 of offers service discovery and native support for the HashiCorp’s Daniel Corbett, head of product, HAProxy Technologies, in a blog post.
Through a RESTful HTTP API, HAProxy connects directly to a defined Consul server and ingests the list of services and nodes from a Consul catalog, Corbett later told The New Stack.
The API will set off a process that can “define an HAProxy backend and pool of servers to match this catalog and automatically scale up or down nodes/servers on-demand based on changes within the Consul catalog,” Corbett said.
Corbett noted in the has also released version 2.3 of HAProxy itself, adding features such as forwarding, prioritizing, and translating of messages sent over the Syslog Protocol on both UDP and TCP, an OpenTracing SPOA, Stats Contexts, SSL/TLS enhancements, an improved cache, and changes in the connection layer that lay the foundation for support for HTTP/3/QUIC.
For more information on the HAProxy’s Data Plane API,
Third-party DNS providers have seen tremendous consolidation during the past few years, resulting in dependence on a smaller pool of providers that maintain the world’s largest website lookups. Reliance on only one of a few single DNS providers also represents a heightened risk in the event of a Carnegie Mellon University, 89.2% of the CDN MaxCDN, the researchers noted. A
“There’s a lot to say about each of these service meshes and how they work: their architecture, why they’re made, what they’re focused on, what they do when they came about and why some of them aren’t here anymore and why we’re still seeing new ones,” Layer5, explained during his talk with “Service Mesh Specifications and Why They Matter in Your Deployment.”
Service mesh is increasingly seen as a requirement to manage microservices in Kubernetes environments, offering a central control plane to manage microservices access, testing, metrics and other functionalities. One-third of the respondents in The New Stack survey of our readers said their organizations already use service mesh. Among the numerous service mesh options available; Envoy, Linkerd and
The release of HashiCorp’s push to widen the scope of its managed services offerings on the cloud.
The GA release of HCS on Armon Dadgar, co-founder and CTO of HashiCorp, said the Azure HCS release is part of HashiCorp’s shift to a more managed-services business model.
“We are transitioning from being a desktop software vendor to becoming more of a cloud software vendor,” said Dadgar.
Dadgar said HashiCorp opted for Azure since there is a lot of overlap between the kinds of customer organizations HashiCorp and Microsoft tend to focus on.
The launch Continue reading
Cloudflare’s Jonathan Sullivan, NS1 chief technology officer and co-founder, told The New Stack.
While Cloudflare — an NS1 competitor — did have DNS redundancy built into its infrastructure, the resulting traffic drop in its network infrastructure was about 50% throughout its network and resulted in a 27-minutes outage of Cloudflare Internet properties and services, Cloudflare Chief Technology Officer blog post.
A router overload in the state of Georgia resulted in the Cloudflare outage. One way Cloudflare learned to prevent such an event from recurring was to set a limit on the Georgia router’s traffic for BGP sessions. This will result in the shutdown of Continue reading