While conventional data warehouses and data lakes have become common practice for analytics workloads, they don’t solve the broader enterprise problems of sharing real-time operational data among departments or across companies. This three-part series explores the challenges and solutions that arise when integrating business data across different applications, clouds and organizations in a modern IT stack. Part 1 highlights the challenges of real-time data sharing, discusses operational vs. analytical data, and legacy solutions and their limitations. Part 2 defines the real-time data mesh and discusses the key tenets for incorporating them into modern IT stacks. Part 3 focuses on what’s needed to effectively evaluate real-time data-sharing solutions. Canyon Spanning — The Foundational IT Challenge Tim Wagner Tim is the inventor of AWS Lambda and a former general manager of AWS Lambda and Amazon API Gateway services. He has also served as vice president of engineering at Coinbase, where he managed design, security and product management teams. Tim co-founded Vendia to help organizations of all sizes share data more effectively across clouds and companies, and he serves as its CEO. One of the most enduring and foundational challenges for IT professionals regardless of their organization’s size or industry is getting data Continue reading
Modern, cloud-based, distributed networks may lack a defined perimeter to protect, but they still need network security. And nearly all organizations know that:
The KubeCon+CloudNativeCon EU, the open source project revealed that is has been working on an extension, explained Envoy creator
In the first article in this series, we discussed what zero trust security is and why it matters. In this article, we will take a deep dive into zero trust network access, how it works, and its benefits to the modern organization. What Is Zero Trust Network Access? Zero Trust Network Access, or ZTNA, is a security solution that many IT departments and IT organizations use to ensure secure remote access to a range of data, applications, networks, and services within an organization. ZTNA is based on defined access control policies that clearly communicate who has access to what and for how long that access is granted. ZTNA is a gap-filler when it comes to secure remote access tools, methods, and technologies. VPNs or virtual private networks are different from zero trust network access because VPNs give access to an entire network rather than specific applications or data. As the remote workforce continues to take shape and many companies lean on it as a source of employee satisfaction and employer productivity. Zero trust network access is pertinent to keeping the right people in and the wrong people out of your organization’s systems. How Does Zero Trust Network Access Work Continue reading
The web development platform Matt Biilmann. In an interview with The New Stack, he described how looking to the future inspired the vision for the company’s latest product. “As we’re building out our edge network and as we start seeing this category mature, with people building more complex projects in a variety of industries, I believed we’d see a new layer emerge where developers could easily write code that would run on the edge. In the beginning, we weren’t sure what that layer would look like or what it would do. We spent a lot of time investigating WebAssembly as the runtime mechanism but ultimately decided against it. In 2020, we moved our efforts from WASM to our own JavaScript-based edge runtime.” “The standard JS runtimes like Node.js aren’t really built to be run in a totally multitenant environment or unique process isolation, so we had to start building our own.” Matt Biilmann, Netlify CEO A year ago, Netlify’s first version of Edge Functions (named Edge Handlers at the Continue reading
E.G. Galano, who co-founded the company in 2016 and is now a key member of the Consensys product team. We also spoke about the growth of NFTs and blockchain apps over the past several years, and their potential moving forward. What Exactly Is Infura? To compare Infura to a Web 2.0 service, Galano said that it’s similar to a web host — except that while a web host actually holds user data, Infura is more like a gateway to that data. “There’s a public data set that you need to connect to, and pull from, and read from, and interact with when Continue reading
Dhruba Borthakur Dhruba is CTO and co-founder of Rockset and is responsible for the company's technical direction. He was an engineer on the database team at Facebook, where he was the founding engineer of the RocksDB data store. Earlier at Yahoo, he was one of the founding engineers of the Hadoop Distributed File System. He was also a contributor to the open source Apache HBase project. Note: This post is the third in the series “can spike 10x during Black Friday. There are many other occasions where data traffic balloons suddenly. Halloween causes consumer social media apps to be inundated with photos. Major news events can set the markets afire with electronic trades. A meme can suddenly go viral among teenagers. In the old days of batch analytics, bursts of data traffic were easier to manage. Executives didn’t expect reports more than once a week nor dashboards to have up-to-the-minute data. Though some data Continue reading
At first glance, it seems clear that the cloud era has fundamentally changed the way we think about networking. We’re now operating outside defined perimeters, and networks can span multiple data centers or clouds. But has networking really changed all that much from the days when everything lived in on-premises data centers? Peter McCarron Peter is a senior product marketing manager for Consul at HashiCorp and based in San Francisco. If he's not studying the best way to discover and manage microservices or talking about cloud-based networking, you'll likely find him discovering real clouds in the great outdoors. After all, it’s still all about establishing consistent connectivity and enforcing security policies. So why does everything seem so different and complicated when it comes to the cloud? To better understand the evolution to modern networking, it’s important to step back and identify the core workflows that have defined those changes, including: Discovering services Securing networks Automating networking tasks Controlling access In this article, we will walk through each of these workflows and talk about how they are combined to achieve a modern service-based networking solution. Since I work at HashiCorp, I’m going to use
It all starts with the dreaded alert. Something went awry, and it needs to be fixed ASAP. Whether it’s the middle of the night and you’re the on-call responder, or it’s the middle of the afternoon, and your whole team is working together to ship a bundle of diffs, having an incident happen is extremely disruptive to your business — and often very expensive, making every minute count. So how can observability (o11y for short) help teams save precious time and resolve incidents faster? First, let’s explore the changing landscape from monitoring to observability. Debugging Using Traditional Monitoring Tools Savannah Morgan Savannah is senior technical customer success manager at Honeycomb. She is passionate about helping users find creative solutions for complex problems. When she is off the clock, Savannah can be found at the park with her family, binge-watching Netflix or spoiling her big pup, Bruce. The key to resolving an incident quickly is to rapidly understand why things went wrong, where in your code it’s happening, and most of all, who it affects and how to fix it. Most of us learned to debug using static dashboards powered by metrics-based monitoring tools like Prometheus or Datadog, plus a whole Continue reading
Zero Trust is a framework for security in which all users of an application, software, system, or network, inside or outside of an organization, must be authenticated, verified, and frequently validated before being granted access to specific data or tools within the company’s network. In the zero trust framework, networks can be in the cloud, hybrid, or on-premise with employees in any location. The assumption is that no users or devices are to be trusted with access without meeting the necessary validation requirements. In today’s modern digital transformation forward environment, the zero-trust security framework helps to ensure infrastructure and data are kept safe, and more modern business challenges are handled appropriately. For example, as the pandemic has evolved, securing remote workers and their access will be of greater importance for organizations that want to scale their workforce. Ransomware threats and attacks are increasing, and zero trust implementation can detect these threats, from novel ones to custom-crafted malware, far before they cause harm. What Foundation Makes up Zero Trust? Zero Trust security is built on the architecture established by the National Institute of Standards & Technology (NIST). The
Nicolas Vermandé Nicolas is the principal developer advocate at Ondat. He is an experienced hands-on technologist, evangelist and product owner who has been working in the fields of cloud native technologies, open source software, virtualization and data center networking for the past 17 years. Passionate about enabling users and building cool tech solving real-life problems, you'll often see him speaking at global tech conferences and online events. Outages and degraded performance are inevitable. Operators make mistakes; new protocols introduce errors, natural disasters damage equipment and more. That’s why rather than trust Amazon’s ability to design a hurricane-proof data center, most platform managers opt to spread their application’s infrastructure across multiple availability zones (AZs). AZ outages aren’t terribly common, but
Michael Kalantar Michael is a senior software engineer who has contributed to the design and development of a number of scalable distributed and cloud-based enterprise systems. He is a co-founder of the Iter8 project. In this article, we show how to use GitHub Actions to load-test, benchmark and validate HTTP and gRPC services with service-level objectives (SLOs). When developing a new version of an HTTP or gRPC service, it is desirable to benchmark its performance and to validate that it satisfies desired service-level objectives (SLOs) before upgrading the current version. We describe a no-code approach based on GitHub Actions that can be used to automate such testing at any point in a continuous integration/continuous delivery (CI/CD) pipeline. For example, at build time it can be used to validate the new version as soon as possible. Alternatively, at deployment time it can be used to validate SLOs in the production environment. HTTP Load Testing with the Iter8 GitHub Action The Iter8 GitHub Action, iter8-tools/iter8-action@v1, enables automated Iter8 experiments in a GitHub workflow. To use the action, specify an experiment chart and its configuration via a Helm valuesFile. No programming is necessary — all configuration is declarative. Typical use is to Continue reading
Grace Andrews An enthusiastic technologist with a cross-cultural focus and experience managing, facilitating and executing entrepreneurial training and processes, Grace has a keen eye for public relations, marketing, consulting and networking. Did you know that the fiber cables that helped bring you this web page may be buried inside a pipeline originally built to carry oil and gas? Or that Cold War military researchers were instrumental in birthing the concepts that gave rise to those cables in the first place? How about the fact that people once tried to build their own cellular phone networks using analog modems? Few of the people who use the internet daily, from those creating GitHub repos to those simply scrolling through Twitter, are aware of the fascinating backstory of the physical infrastructure that makes it all work. The idea behind Apple, RSS. For more creative content by and about the humans that build and scale the internet, follow Twitter, Instagram. Finally, be sure to check out the
Charles Mahler Charles is a technical marketing writer at InfluxData. Charles’ background includes working in digital marketing and full-stack software development. As the technology landscape continues to change at a rapid pace, enterprise companies are in a rush to catch up and modernize their legacy IT and network infrastructure to capture the benefits of newly developed tools and best practices. By adopting modern DevOps techniques, they can reduce their operational costs, increase the reliability of their services and improve the overall speed and agility at which their IT teams are able to move. Background
Srinivasan Parthasarathy Sri is an applied machine learning researcher with a track record of creating scalable AI/ML/advanced optimization-based enterprise solutions for hybrid cloud, cybersecurity and data-exploration problem domains. A co-founder of Iter8, he has presented at Kubecon 2020 and 2021, and at community meetups like Knative and KFServing. gRPC is an open source remote procedure call (RPC) system that is becoming increasingly popular for connecting microservices and connecting mobile/web clients to backend services. Benchmarking and performance validation is an essential building block in the continuous integration and delivery (CI/CD) of robust gRPC services. In this hands-on article, we show how Iter8 unifies performance validation for HTTP and gRPC services. What Is Iter8?
It was only a matter of time. While multifactor authentication (MFA) makes logging into systems safer, it doesn’t make it “safe.” As well-known hacker KnownBe4, showed in 2018 it’s easy to Proofpoint has found transparent reverse proxy. Typically transparent reverse proxies, such as the open source man-in-the-middle (MitM) attacks to steal credentials and session cookies. Why go to this trouble? Because, as an MFA company 78% of users now use MFA, compared to just 28% in 2017. That’s good news, but it’s also given cybercrooks the incentive they needed to target MFA. A Range of Kits To make it easy for wannabe hackers. Proofpoint found today’s phishing kits range from “simple open-source kits with human-readable code and no-frills functionality Continue reading
In traditional systems, environment variables play an important role, but not always a crucial one. Some applications make more use of environment variables than others. Some prefer configuration files over environment variables. However, when it comes to Kubernetes, environment variables are more important than you might think. It’s partially due to the way containers work in general and partially due to the specifics of Kubernetes. In this post, you’ll learn all about environment variables in Kubernetes. Traditionally, environment variables are dynamic key-value variables that are accessible to any process running on the system. The Basics Let’s start with the basics. What are environment variables and why do they exist? Traditionally, environment variables are dynamic key-value variables that are accessible to any process running on the system. The operating system itself will set many environment variables that help running processes understand the specifics of the system. Thanks to this, software developers can include logic in their software that makes the programs adjustable to a specific operating system. Environment variables also hold a lot of important information about the user, things like username, preferred language, user home directory path and many other useful bits of information. User-Defined Environment Variables Dawid Ziolkowski Dawid Continue reading
Romaric Philogène Romaric is CEO and co-founder of Qovery with more than 10 years of experience in site reliability engineering and software development. For two years at Qovery, we built a Qovery Engine. Environment On Qovery, every application and database belong to an environment. It is a logical entity that links all resources together. When turning on the Preview Environment feature, Qovery will duplicate an Continue reading
If you want to run code to provide observability, security or network functionality, running it in the kernel of your operating system gives you a lot of power because that kernel can see and control everything on the system. That’s powerful, but potentially intrusive or dangerous if you get it wrong, whether that’s introducing a vulnerability or just slowing the system down. If you’re looking for a way to take advantage of that kind of privileged context without the potential danger, eBPF is emerging as an alternative — and now it’s coming to Windows. Not Just Networking Originally eBPF stood for “extended Berkeley Packet Filter”, updating the open source networking tool that puts a packet filter in the Linux kernel for higher performance packet tracing (now often called cBPF for classic BPF). But it’s now a generic mechanism for running many kinds of code safely in a privileged context by using a sandbox, with application monitoring, profiling and security workloads as well as networking, so it’s not really an acronym anymore. That privileged context doesn’t even have to be an OS kernel, although it still tends to be, with eBPF being a more stable and secure alternative to kernel modules Continue reading
Sheraline Barthelmy Sheraline is the head of product, marketing and customer success at Cox Edge, an edge cloud startup from Cox Communications. At Cox Edge, she's focused on developing the tools and systems that customers and developers rely on to build the next generation of edge applications. These days, brick-and-mortar or television-based bank robberies and heists seem old-fashioned no matter how well planned or executed. What the new “money” criminals are after is personal data. And the “banks” being attacked are the growing number of web applications. Studies show that web application attacks have become the single most significant cause of data breaches. According to NTT’s 2020 Global Threat Intelligence Report (GTIR), more than half (55%) of all attacks in 2019 were a mix of web application and application-specific attacks, up from 32% the year before. As organizations move away from VPNs, virtual machines and centralized management systems to distributing and even running applications at the edge, conventional perimeter-based security like network firewalls isn’t enough. The best defense is a firewall that can mitigate application-layer attacks. Web Application Firewall (WAF) A WAF helps protect web applications from application-layer attacks like cross-site scripting, SQL injection attacks, remote file inclusion and cookie Continue reading