The hyperscalers of the world have to deal with dataset sizes – both streaming and at rest – and real-time processing requirements that put them into an entirely different class of computing.
They are constantly inventing and reinventing what they do in compute, storage, and networking not just because they enjoy the intellectual challenge, but because they have swelling customer bases that hammer on their systems so hard they can break them.
This is one of the reasons why an upstart called Streamlio has created a new event-driven platform that is based the work of software engineers at Twitter, Yahoo, …
Weaving A Streaming Stack Like Twitter And Yahoo was written by Timothy Prickett Morgan at The Next Platform.
Gone are the days of early warehouse scale computing pioneers that were based in the U.S.. Over the last several years, China’s web giants are extending their reach through robust shared AI and cloud efforts—and those are pushing ever further into territory once thought separate.
Alibaba is much like compatriots Baidu and Tencent in its desire to expand well beyond the borders of China and compete with global players like Amazon Web Services, Google, Microsoft and Facebook in such fast-growing areas like the cloud, supercomputing and artificial intelligence (AI).
The tech giant has significant resources at its disposal, pulling in …
China’s Global Cloud and AI Ambitions Keep Extending was written by Jeffrey Burt at The Next Platform.
On today’s episode of “The Interview” with The Next Platform we discuss the art and science of tuning high performance systems for maximum performance—something that has traditionally come at high time cost for performance engineering experts.
While the role of performance engineer will not disappear anytime soon, machine learning is making tuning systems—everything from CPUs to application specific parameters—less of a burden. Despite the highly custom nature of systems and applications, reinforcement learning is allowing new leaps in time-saving tuning as software learns what works best for user applications and architectures, freeing up performance engineers to focus on the finer …
Machine Learning for Auto-Tuning HPC Systems was written by Nicole Hemsoth at The Next Platform.
The first step in rolling out a massive supercomputer installed at a government sponsored HPC laboratory is to figure out when you want to get it installed and doing useful work. The second is consider the different technologies that will be available to reach performance and power envelope goals. And the third is to give it a cool name.
Last but not least is to put a stake in the ground by telling the world about the name of the supercomputer and its rough timing, thereby confirming. These being publicly funded machines, this is only right.
As of today, it’s …
The Roadmap Ahead For Exascale HPC In The US was written by Timothy Prickett Morgan at The Next Platform.
If you are running applications in the HPC or AI realms, you might be in for some sticker shock when you shop for GPU accelerators – thanks in part to the growing demand of Nvidia’s Tesla cards in those markets but also because cryptocurrency miners who can’t afford to etch their own ASICs are creating a huge demand for the company’s top-end GPUs.
Nvidia does not provide list prices or suggested street prices for its Tesla line of GPU accelerator cards, so it is somewhat more problematic to try to get a handle on the bang for the buck over …
Tesla GPU Accelerator Bang For The Buck, Kepler To Volta was written by Timothy Prickett Morgan at The Next Platform.
Details about the technologies being used in Canada’s newest and most powerful research supercomputer have been coming out in a piecemeal fashion over the past several months, but now the complete story.
At the SC17 show in November, it was revealed that the HPC system will use Mellanox’s Dragonfly+ network topology and a NVM Express burst buffer fabric from Excelero as key part of a cluster that will offer a peak performance of more than 4.6 petaflops.
Now Lenovo, which last fall won the contract for the Niagara system over 11 other vendors, is unveiling this week that it is …
Accelerating HPC Investments In Canada was written by Jeffrey Burt at The Next Platform.
File system changes in high performance computing, take time. Good file systems are long-lived. It took several years for some parallel file systems to win out over others and it will be many more decades before the file system as we know it is replaced by something entirely different.
In the meantime, however, there are important points to consider for real-world production HPC deployments that go beyond mere performance comparisons, especially as these workloads grow more complex, become more common, and put new pressures on storage and I/O systems.
The right combination of performance, stability, reliability, and ease of management …
Enterprise HPC Tightens Storage, I/O Strategy Around Support was written by Nicole Hemsoth at The Next Platform.
There was concern in some scientific quarters last year that President Trump’s election could mean budget cuts to the Department of Energy (DoE) that could cascade down to the country’s exascale program at a time when China was ramping up investments in its own initiatives.
The worry was that any cuts that could slow down the work of the Exascale Computing Project would hand the advantage to China in this critical race that will have far-reaching implications in a wide range of scientific and commercial fields like oil and gas exploration, financial services, high-end healthcare, national security and the military. …
U.S. Exascale Efforts Benefit in FY 2019 Budget was written by Jeffrey Burt at The Next Platform.
On today’s episode of “The Interview” with The Next Platform we talk with Doug Miles who runs the PGI compilers and tools team at Nvidia about the past, present, and future of OpenACC with an emphasis on what lies ahead in the next release.
Over the last few years we have described momentum with OpenACC in a number of articles covering everything from what it means in the wake of new memory options to how it pushes on OpenMP to develop alongside. Today, however, we take a step back with an HPC code veteran for the bigger picture and …
OpenACC Developments: Past, Present, and Future was written by Nicole Hemsoth at The Next Platform.
Computing used to be far away.
It was accessed via remote command terminals, through time sliced services. It was a pretty miserable experience. During the personal computing revolution, computing once again became local. It would fit under your desk, or in a small dedicated “computer rooms”. You could touch it. It was once more, a happy and contented time for computer users. The computer was personal again. There was a clue in the name.
However, as complexity grew, and as networks improved, computing was effectively taken away again and placed in cold dark rooms once more far, far away for …
Hardware as a Service: The New Missing Middle? was written by James Cuff at The Next Platform.
The general consensus, for as long as anyone can remember, is that there is an insatiable appetite for compute in the datacenter. Were it not for the constraints of budgets, for both the acquisition of iron, the facilities to house it, the electricity to power and cool it, and the people to manage it, it is safe to say that the installed base of servers worldwide would be much larger than it is.
But the world doesn’t work that way, and there are constraints. But thanks to ever more complex applications, ever richer media, and a burning desire to save …
The Server Market Booms, And It Could Last For A While was written by Timothy Prickett Morgan at The Next Platform.
The high performance computing market might get some windfall from the processing requirements of IoT and edge devices, but the real driver for spending will come well before the device ever hits the market.
The rise in IoT and edge device demand means an exponential increase in the number of devices that need to be modeled, simulated and tested and this means greater investment in HPC hardware as well as the engineering software tools that have generally served the HPC set.
In other words, it is not the new, enlarged, complex dataset from IoT and edge that presents the next …
IoT Will Force HPC Spending–But Not For the Reasons You Think was written by Nicole Hemsoth at The Next Platform.
Since its inception, the OpenStack cloud controller co-created by NASA and Rackspace Hosting, with these respective organizations supplying the core Nova compute and Swift object storage foundations, has been focused on the datacenter. But as the “Queens” release of OpenStack is being made available, the open source community that controls that cloud controller is being pulled out of the datacenter and out to the edge, where a minimalist variant of the software is expected to have a major presence in managing edge computing devices.
The Queens release of OpenStack is the 17th drop of software since NASA and Rackspace first …
OpenStack Stretches To The Edge, Embraces Accelerators was written by Timothy Prickett Morgan at The Next Platform.
The rapid proliferation of connected devices and the huge amounts of data they are generating is forcing tech vendors and enterprises alike to cast their eyes to the network edge, which has become a focus of the distributed computing movement as more compute, storage, network, analytics and other resources are moving closer to where these devices live.
Compute is being embedded in everything, and this makes sense. The costs in time, due to latency issues, and money, due to budgetary issues, from transferring all that data from those distributed devices back to a private or public datacenter are too high …
VMware Crafts Compute And Storage Stacks For The Edge was written by Jeffrey Burt at The Next Platform.
On today’s episode of “The Interview” with The Next Platform, we focus on how geographic information systems (GIS) is, as a field, being revolutionized by deep learning.
This stands to reason given the large volumes of satellite image data and robust deep learning frameworks that excel at image classification and analysis–a volume issue that has been compounded by more satellites with ever-higher resolution output.
Unlike other areas of large-scale scientific data analysis that have traditionally relied on massive supercomputers, our audio interview (player below) reveals that a great deal of GIS analysis can be done on smaller systems. However, …
Geographic Information Systems (GIS) Field Upended by Neural Networks was written by Nicole Hemsoth at The Next Platform.
There is no question right now that if you have a big computing job in either high performance computing – the colloquial name for traditional massively parallel simulation and modeling applications – or in machine learning – the set of statistical analysis routines with feedback loops that can do identification and transformation tasks that used to be solely the realm of humans – then an Nvidia GPU accelerator is the engine of choice to run that work at the best efficiency.
It is usually difficult to make such clean proclamations in the IT industry, with so many different kinds of …
The Engine Of HPC And Machine Learning was written by Timothy Prickett Morgan at The Next Platform.
On today’s episode of “The Interview” with The Next Platform we talk about the use of petascale supercomputers for training deep learning algorithms. More specifically, how this happening in Astronomy to enable real-time analysis of LIGO detector data.
We are joined by Daniel George, a researcher in the Gravity Group at the National Center for Supercomputing Applications, or NCSA. His team garnered a great deal of attention at the annual supercomputing conference in November with work blending traditional HPC simulation data and deep learning.
George and his team have shown that deep learning with convolutional neural networks can provide many …
Deep Learning on HPC Systems for Astronomy was written by Nicole Hemsoth at The Next Platform.
There is a new sheriff in town at Hewlett Packard Enterprise – that would be chief executive officer Antonio Neri – and that means a new way of looking at the books and therefore steering the business. No, we didn’t mean that the other way around.
In opening up its books for the first quarter of its fiscal 2018 year, which ended in January, we can see some important things about HPE’s business, and at the same time, we have lost some visibility about core parts of its business.
First of all, the books have been reclassified significantly in the …
The New HPE Sheriff Lays Down The Hybrid IT Law was written by Timothy Prickett Morgan at The Next Platform.
Among all of the hardware and software that is in a datacenter, the database – whether it is SQL, NoSQL, NewSQL or some other contraption in which to pour data and ask questions about it – is probably the stickiest. Companies can and do change server, storage, or networking hardware, and they change operating systems and even applications, but they are loath to mess with repository of the information that is used to run the company.
This is understandably so, given the risk of inadvertently altering or losing that vital data. Ironically, this is one reason why databases proliferate at …
Bringing GPUs To Bear On Big Standard Relational Databases was written by Timothy Prickett Morgan at The Next Platform.
On today’s episode of “The Interview” with The Next Platform, we take a look at the evolution of the NAMD molecular dynamics and how the introduction of GPU computing upended performance expectations and set the stage for new metrics now that the Volta GPU architecture will be available on large supercomputers like the Summit machine coming to Oak Ridge National Lab.
Our guest is well known for being part of the team that won a Gordon Bell Prize in 2002 for work on scaling NAMD. Dr. Jim Phillips is a Senior Research Programmer in the NCSA Blue Waters Project …
The Evolution of NAMD: A Scalability Story from Single-Core to GPU Boosted was written by Nicole Hemsoth at The Next Platform.