On today’s episode of “The Interview” with The Next Platform we talk about the growing problem of networks within networks (within networks) and what that means for future algorithms and systems that will support smart cities, smart grids, and other highly complex and interdependent optimization problems.
Our guest on this audio interview episode (player below) is Hadi Amini, a researcher at Carnegie Mellon who has focused on the interdependency of many factors for power grids and smart cities in a recent book series on these and related interdependent network topics. Here, as in the podcast, the focus is on the …
Networks Within Networks: Optimization at Massive Scale was written by Nicole Hemsoth at The Next Platform.
The idea of bringing liquids in the datacenter to cool off hot-running systems and components has often unnerved many in the IT field. Organizations are doing it as they look for more efficient and cost-effective ways to run their infrastructures, particularly as the workloads become larger and more complex, more compute resources are needed, parts like processors become more powerful and density increases.
But the concept of running water and other liquids through a system, and the threat of the liquids leaking into the various components and into the datacenter, has created uneasiness with the idea.
Still, the growing demands …
Sandia, NREL Look to Aquarius to Cool HPC Systems was written by Jeffrey Burt at The Next Platform.
Changes to workloads in HPC mean alterations are needed up and down the stack—and that certainly includes storage. Traditionally these workloads were dominated by large file handling needs, but as newer applications (OpenFOAM is a good example) bring small file and mixed workload requirements to the HPC environment, it means storage approaches need to shift to meet the need.
With these changing workload demands in mind, recall that in the first part of our series on future directions for storage for enterprise HPC shops we focused on the ways open source parallel file systems like Lustre fall short for users …
Changing HPC Workloads Mean Tighter Storage Stacks for Panasas was written by Nicole Hemsoth at The Next Platform.
To keep their niche in computing, field programmable gate arrays not only need to stay on the cutting edge of chip manufacturing processes. They also have to include the most advanced networking to balance out that compute, rivalling that which the makers of switch ASICs provide in their chips.
By comparison, CPUs have it easy. They don’t have the serializer/deserializer (SerDes) circuits that switch chips have as the foundation of their switch fabric. Rather, they might have a couple of integrated Ethernet network interface controllers embedded on the die, maybe running at 1 Gb/sec or 10 Gb/sec, and they offload …
FPGA Interconnect Boosted In Concert With Compute was written by Timothy Prickett Morgan at The Next Platform.
Flash memory has become absolutely normal in the datacenter, but that does not mean it is ubiquitous and it most certainly does not mean that all flash arrays, whether homegrown and embedded in servers or purchased as appliances, are created equal. They are not, and you can tell not only from the feeds and speeds, but from the dollars and sense.
It has been nine years since Pure Storage, one of the original flash array upstarts, was founded and seven years since the company dropped out of stealth with its first generation of FlashArray products. In that relatively short time, …
Why Cisco Should – And Should Not – Acquire Pure Storage was written by Timothy Prickett Morgan at The Next Platform.
Of the three pillars of the datacenter – compute, storage, and networking – the one that consistently still has some margins and yet does not dominate the overall system budget is networking. While these elements affect each other, they are still largely standalone realms, with their own specialized devices and suppliers. And so it is important to know the trends in the technologies.
Until fairly recently, the box counters like IDC and Gartner have been pretty secretive about the data they have about the networking business. But IDC has been gradually giving a little more flavor than just saying Cisco …
Drilling Down Into Ethernet Switch Trends was written by Timothy Prickett Morgan at The Next Platform.
Today’s episode of “The Interview” with The Next Platform is focused on the evolution of stream processing—from the early days to more recent times with vast volumes of social, financial, and other data challenging data analysts and systems designers alike.
Our guest is Nathan Trueblood, a veteran of several companies like Mirantis, Western Digital, EMC, and current VP of product management at DataTorrent—a company comprised of many ex-Yahoo employees who worked with the Hadoop platform and have pushed the evolution of that framework to include more real-time requirements with Apache Apex.
Trueblood’s career has roots in high performance computing …
Pushing Greater Stream Processing Platform Evolution was written by Nicole Hemsoth at The Next Platform.
High performance computing experts came together recently at Stanford for their annual HPC Advisory Council Meeting to share strategies after what has been an interesting year in supercomputing thus far.
As always, there was a vast amount of material covering everything from interconnects to containerized compute. In the midst of this, The Next Platform noted an obvious and critical thread over the two days–how to best map infrastructure to software in order to reduce “computational back pressure” associated with new “data heavy” AI workloads.
In the “real world” back pressure results from a bottleneck as opposed to desired …
Spinning the Bottleneck for Data, AI, Analytics and Cloud was written by James Cuff at The Next Platform.
On today’s episode of “The Interview” with The Next Platform we talk about the past, present, and future of tape storage with industry veteran Matt Starr.
Starr is CTO at tape giant, Spectra Logic and has been with the company for almost twenty-five years. He was the lead engineer and architect forthe design and production of Spectra’s enterprise tape library family, which is still a core product.
We talk about some of the key evolutions in tape capacity and access speeds over the course of his career before moving into where the new use cases at massive scale are. In …
Expanding Use Cases Mean Tape Storage is Here to Stay was written by Nicole Hemsoth at The Next Platform.
Hewlett Packard Enterprise (HPE) and NVIDIA have partnered to accelerate innovation, combining the extreme compute capabilities of high performance computing (HPC) with the groundbreaking processing power of NVIDIA GPUs.
In this fast-paced digital climate, traditional CPU technology is no longer sufficient to support growing data centers. Many enterprises are struggling to keep pace with escalating compute and graphics requirements, particularly as computational models become larger and more complex. NVIDIA GPU accelerators for HPC seamlessly integrate with HPE servers to achieve greater speed, optimal power efficiency, and dramatically higher application performance than CPUs. High-end data centers rely on these high performance …
Leverage Extreme Performance with GPU Acceleration was written by Nicole Hemsoth at The Next Platform.
The hyperscalers of the world have to deal with dataset sizes – both streaming and at rest – and real-time processing requirements that put them into an entirely different class of computing.
They are constantly inventing and reinventing what they do in compute, storage, and networking not just because they enjoy the intellectual challenge, but because they have swelling customer bases that hammer on their systems so hard they can break them.
This is one of the reasons why an upstart called Streamlio has created a new event-driven platform that is based the work of software engineers at Twitter, Yahoo, …
Weaving A Streaming Stack Like Twitter And Yahoo was written by Timothy Prickett Morgan at The Next Platform.
Gone are the days of early warehouse scale computing pioneers that were based in the U.S.. Over the last several years, China’s web giants are extending their reach through robust shared AI and cloud efforts—and those are pushing ever further into territory once thought separate.
Alibaba is much like compatriots Baidu and Tencent in its desire to expand well beyond the borders of China and compete with global players like Amazon Web Services, Google, Microsoft and Facebook in such fast-growing areas like the cloud, supercomputing and artificial intelligence (AI).
The tech giant has significant resources at its disposal, pulling in …
China’s Global Cloud and AI Ambitions Keep Extending was written by Jeffrey Burt at The Next Platform.
On today’s episode of “The Interview” with The Next Platform we discuss the art and science of tuning high performance systems for maximum performance—something that has traditionally come at high time cost for performance engineering experts.
While the role of performance engineer will not disappear anytime soon, machine learning is making tuning systems—everything from CPUs to application specific parameters—less of a burden. Despite the highly custom nature of systems and applications, reinforcement learning is allowing new leaps in time-saving tuning as software learns what works best for user applications and architectures, freeing up performance engineers to focus on the finer …
Machine Learning for Auto-Tuning HPC Systems was written by Nicole Hemsoth at The Next Platform.
The first step in rolling out a massive supercomputer installed at a government sponsored HPC laboratory is to figure out when you want to get it installed and doing useful work. The second is consider the different technologies that will be available to reach performance and power envelope goals. And the third is to give it a cool name.
Last but not least is to put a stake in the ground by telling the world about the name of the supercomputer and its rough timing, thereby confirming. These being publicly funded machines, this is only right.
As of today, it’s …
The Roadmap Ahead For Exascale HPC In The US was written by Timothy Prickett Morgan at The Next Platform.
If you are running applications in the HPC or AI realms, you might be in for some sticker shock when you shop for GPU accelerators – thanks in part to the growing demand of Nvidia’s Tesla cards in those markets but also because cryptocurrency miners who can’t afford to etch their own ASICs are creating a huge demand for the company’s top-end GPUs.
Nvidia does not provide list prices or suggested street prices for its Tesla line of GPU accelerator cards, so it is somewhat more problematic to try to get a handle on the bang for the buck over …
Tesla GPU Accelerator Bang For The Buck, Kepler To Volta was written by Timothy Prickett Morgan at The Next Platform.
Details about the technologies being used in Canada’s newest and most powerful research supercomputer have been coming out in a piecemeal fashion over the past several months, but now the complete story.
At the SC17 show in November, it was revealed that the HPC system will use Mellanox’s Dragonfly+ network topology and a NVM Express burst buffer fabric from Excelero as key part of a cluster that will offer a peak performance of more than 4.6 petaflops.
Now Lenovo, which last fall won the contract for the Niagara system over 11 other vendors, is unveiling this week that it is …
Accelerating HPC Investments In Canada was written by Jeffrey Burt at The Next Platform.
File system changes in high performance computing, take time. Good file systems are long-lived. It took several years for some parallel file systems to win out over others and it will be many more decades before the file system as we know it is replaced by something entirely different.
In the meantime, however, there are important points to consider for real-world production HPC deployments that go beyond mere performance comparisons, especially as these workloads grow more complex, become more common, and put new pressures on storage and I/O systems.
The right combination of performance, stability, reliability, and ease of management …
Enterprise HPC Tightens Storage, I/O Strategy Around Support was written by Nicole Hemsoth at The Next Platform.
There was concern in some scientific quarters last year that President Trump’s election could mean budget cuts to the Department of Energy (DoE) that could cascade down to the country’s exascale program at a time when China was ramping up investments in its own initiatives.
The worry was that any cuts that could slow down the work of the Exascale Computing Project would hand the advantage to China in this critical race that will have far-reaching implications in a wide range of scientific and commercial fields like oil and gas exploration, financial services, high-end healthcare, national security and the military. …
U.S. Exascale Efforts Benefit in FY 2019 Budget was written by Jeffrey Burt at The Next Platform.
On today’s episode of “The Interview” with The Next Platform we talk with Doug Miles who runs the PGI compilers and tools team at Nvidia about the past, present, and future of OpenACC with an emphasis on what lies ahead in the next release.
Over the last few years we have described momentum with OpenACC in a number of articles covering everything from what it means in the wake of new memory options to how it pushes on OpenMP to develop alongside. Today, however, we take a step back with an HPC code veteran for the bigger picture and …
OpenACC Developments: Past, Present, and Future was written by Nicole Hemsoth at The Next Platform.
Computing used to be far away.
It was accessed via remote command terminals, through time sliced services. It was a pretty miserable experience. During the personal computing revolution, computing once again became local. It would fit under your desk, or in a small dedicated “computer rooms”. You could touch it. It was once more, a happy and contented time for computer users. The computer was personal again. There was a clue in the name.
However, as complexity grew, and as networks improved, computing was effectively taken away again and placed in cold dark rooms once more far, far away for …
Hardware as a Service: The New Missing Middle? was written by James Cuff at The Next Platform.