When considering system and software needs at massive scale, one application area that tends to shed light on what lies ahead is weather prediction and modeling.
Over the last year, we have had a number of pieces about what centers that deliver forecasts (and carry out research to improve those predictions) need to do to stay ahead, and while conversations about hardware and software are important, what is emerging is that weather, like many other areas of computing at scale, actually needs a platform versus innovation at one or two levels of the stack.
With that idea of a platform …
The Weather Company Seeks Next Data-Driven Platform was written by Nicole Hemsoth at The Next Platform.
One of the things that high-end network adapter and switch maker Mellanox Technologies got through its $811 million acquisition of network processing chip maker EZchip last September was a team that was well versed in massively parallel processor chip design, and one that could make Mellanox a potential player in the server chip space.
But not necessarily in the way you might be thinking about it.
The reason this is the case is that in July 2014, EZchip, wanting to expand out beyond its networking chip business as Applied Micro, Cavium, and Broadcom have all done with ARM-based server chips, …
Putting More Brains In The Network Frees Up Compute was written by Timothy Prickett Morgan at The Next Platform.
There are plenty of people in industry, academia, and government that believe there is a direct correlation between the investment in supercomputing technologies and the healthy and vibrancy of the regional or national economy. So getting a big bump up in performance, as South Africa’s Center for High Performance Computing has just done this week, is a big deal.
Up until now, CHPC has had fairly modest sized systems, but thanks to Moore’s Law advancements that have radically brought down the cost of compute and a more aggressive plan to invest in HPC within South Africa, CHPC is breaking into …
South African Lengau System Leaps Towards Petaflops was written by Timothy Prickett Morgan at The Next Platform.
The datacenter is going through tremendous change, and many long-held assumptions are now being called into question. Even the basic process of separating data onto a separate storage area network, growing it, and pulling it across the network and processing it, is no longer necessarily the best way to handle data. The separation between production and analytics, which has evolved into an art form, is also breaking down because it takes a day or longer to get operational data into analytic systems.
As a backdrop to all of these technology changes, organizations say they need more agility. The ability to …
What’s Fueling the Move to a Converged Data Platform? was written by Nicole Hemsoth at The Next Platform.
Chip startups come and go. Generally, we cover them because of novel architectures or potential for specific applications. But in some cases, like today, it is for those reasons and because of the people behind an effort to bring a new architecture into a crowded, and ultimately limited, landscape.
With $100 million in “patience money” from a few individual investors who believe in the future of sparse matrix-based computing on low-power and reprogrammable devices, Austin-based Knupath, has spent a decade in stealth mode designing and fabricating a custom digital signal processor (DSP) chip to target deep learning training, machine …
Former NASA Exec Brings Stealth Machine Learning Chip to Light was written by Nicole Hemsoth at The Next Platform.
The server cycle has some long waves that are not always in phase with each other, and that is generally a good thing. But every now and then, the waves synchronize, and it is either really exciting as the market rises or something of a bummer as it falls. In the first quarter of this year, there was a bit of a dip but still an order of magnitude less dramatic than the collapse in shipments and sales during the Great Recession.
It is important to keep perspective, and it is actually quite remarkable that the server market is as …
Hyperscalers, Enterprises Pull Back On Server Spending was written by Timothy Prickett Morgan at The Next Platform.
Intel is coming to the finish line with its 14 nanometer chip making process with the launch of the “Broadwell” generation of Xeon E7 server processors in China today.
Why China? Because for reasons that are not immediately obvious but are completely beneficial to Intel, the Chinese market has for the past several years been adopting four-socket servers in large scale datacenters at a rate that is considerably higher than their peers in the rest of the world. These new Xeon E7 v4 processors will be a big hit for big iron there and anywhere else where having a big …
Big Iron Xeons Get A Broadwell Compute And Memory Boost was written by Timothy Prickett Morgan at The Next Platform.
Networking chip maker Cavium is one of the ARM server chip upstarts that is taking on Intel’s hegemony in the datacenter, and is probably getting the most traction among its ARM peers in the past year with its ThunderX multicore processors.
The first-generation ThunderX chips are seeing the most interest from hyperscalers and HPC centers, plus a few cloud builders, telcos, and large enterprises, that want to explore the possibilities of a different server architecture, and they will be even more intrigued by the second-generation ThunderX2 processors, which Cavium unveiled earlier this week at the Computex trade show in Taipei, …
Next-Generation ThunderX2 ARM Targets Skylake Xeons was written by Timothy Prickett Morgan at The Next Platform.
During a trip to Dell in Austin, Texas this week, little did The Next Platform know that the hardware giant and nearby Texas Advanced Computing Center (TACC) had major news to share on the supercomputing front.
It appears the top ten-ranked Stampede supercomputer is set for an extensive, rolling upgrade—one that will keep TACC’s spot in the top tier of supercomputing sites and which will feature the latest Knights Landing processors and over time, a homogeneous Omni-Path fabric. The net effect of the upgrade will be a whopping 18 petaflops of peak performance by 2018.
The new system will begin …
Knights Landing Upgrade Will Push TACC Supercomputer to 18PF was written by Nicole Hemsoth at The Next Platform.
While online auctioneer eBay does not run the largest search engine in the world, search is a very key component of its service, which has over 162 million active buyers and 900 million product listings. Looking to improve upon its current hardware infrastructure underpinning its search engine, eBay has tapped Dell to create a new water-cooled, hyperscale-style rack system that will let it overclock its servers and boost their performance on compute-intensive search algorithms.
The hyperscalers are all a bit cagey about the search engine infrastructure that they use because it is such a critical component of what they do, …
eBay Taps Dell Triton Systems To Overclock Search Engines was written by Timothy Prickett Morgan at The Next Platform.
Back in 2009, when yours truly was assigned the primary beat of covering supercomputing on remote hardware (then dubbed the mysterious “cloud”), the possibility that cloud-based high performance computing was little more than a pipe dream.
At that time, most scientific and technical computing communities had already developed extensive grids to extend their research beyond physical borders, and the idea of introducing new levels of latency, software, and management interfaces did not appear to be anything most HPC centers were looking forward to—even with the promise of cost-savings (as easy “bursting” was still some time off).
Just as Amazon Web …
AWS Brings Supercomputing Set Further into Fold was written by Nicole Hemsoth at The Next Platform.
Computing for neuroscience, which has aided in our understanding of the structure and function of the brain, has been around for decades already. More recently, however, there has been neuroscience for computing, or the use of computational principles of the brain for generic data processing. For each of these neuroscience-driven areas there is a key limitation—scalability.
This is not just scalability in terms of software or hardware systems, but on the application side, limits in terms of efficiently deploying computational tools at sufficient size and time scales to yield far greater insight. While adding more compute to the problem …
Strong Scaling Key to Redrawing Neuroscience Borders was written by Nicole Hemsoth at The Next Platform.
The datacenter is a battleground with many fronts these days, with intense competition between compute, memory, storage, and networking components. In terms of revenues, profits, and prestige, the compute territory is the most valuable that chip makers and their system partners are fighting for, and the ARM and OpenPower collectives are doing their best to take some ground from a very powerful Intel.
As such, chip makers end up comparing themselves to Intel Xeon or Atom processors, and Intel sometimes makes comparisons back. At the high end, Intel is battling the Power8 processor championed by IBM and to a lesser …
Intel Lines Up ThunderX ARM Against Xeons was written by Timothy Prickett Morgan at The Next Platform.
Having made the improbable jump from the game console to the supercomputer, GPUs are now invading the datacenter. This movement is led by Google, Facebook, Amazon, Microsoft, Tesla, Baidu and others who have quietly but rapidly shifted their hardware philosophy over the past twelve months. Each of these companies have significantly upgraded their investment in GPU hardware and in doing so have put legacy CPU infrastructure on notice.
The driver of this change has been deep learning and machine intelligence, but the movement continues to downstream into more and more enterprise-grade applications – led in part by the explosion …
The Age of the GPU is Upon Us was written by Nicole Hemsoth at The Next Platform.
Any aspirations that the Hewlett-Packard that we knew for nearly a decade and a half to build a conglomerate that resembled IBM in its own former enterprise breadth and depth of software, services, and systems is now over with the company spinning out its Enterprise Services business and focusing very tightly on its core hardware and related software businesses.
In conjunction with the posting of its financial results for the first quarter of its fiscal 206, the trimmed down Hewlett Packard Enterprise, which has not included the PC and printer businesses since last year, announced that it was going to …
HPE Hunkers Down On Datacenter Hardware was written by Timothy Prickett Morgan at The Next Platform.
Don’t just call it “the cloud.” Even if you think you know what cloud means, the word is fraught with too many different interpretations for too many people. Nevertheless, the effect of cloud computing, the web, and their assorted massive datacenters has had a profound impact on enterprise computing, creating new application segments and consolidating IT resources into a smaller number of mega-players with tremendous buying power and influence.
Welcome to the hyperscale market.
At the top end of the market, ten companies – behemoths like Google, Amazon, eBay, and Alibaba – each spend over $1 billion per year on …
The Hyperscale Effect: Tracking the Newest High-Growth IT Segment was written by Nicole Hemsoth at The Next Platform.
Having access to fairly reliable 10-day forecasts is a luxury, but it comes with high computational costs for centers in the business of providing predictability. This ability to accurately predict weather patterns, dangerous and seasonal alike, has tremendous economic value and accordingly, significant investment goes into powering ever-more extended and on-target forecast.
What is interesting on the computational front is that the future of weather prediction accuracy, timeliness, efficiency, and scalability seems to be riding a curve not so dissimilar to that of Moore’s Law. Big leaps, followed by steady progress up the trend line, and a moderately predictable sense …
Large-Scale Weather Prediction at the Edge of Moore’s Law was written by Nicole Hemsoth at The Next Platform.
While legacy monolithic applications will linger in virtual machines for an incredibly long time in the datacenter, new scale-out applications run best on new architectures. And that means the underlying hardware will look a lot more like what the hyperscalers have built than traditional siloed enterprise systems.
But most enterprises can’t design their own systems and interconnects, as Google, Facebook, and others have done, and as such, they will rely on others to forge their machines. A group of hot-shot system engineers that were instrumental in creating systems at Sun Microsystems and Cisco Systems in the past two decades have …
Driving Compute And Storage Scale Independently was written by Timothy Prickett Morgan at The Next Platform.
For those in enterprise circles who still conjure black and white images of hulking supercomputers when they hear the name “Cray,” it is worth noting that the long-standing company has done a rather successful job of shifting a critical side of its business to graph analytics and large-scale data processing.
In addition to the data-driven capabilities cooked into its XC line of supercomputers, and now with their DataWarp burst buffer adding to the I/O bottom line on supercomputers including Cori, among others, Cray has managed to take supercomputing to the enterprise big data set by blending high performance hardware with …
Cray Sharpens Approach to Large-Scale Graph Analytics was written by Nicole Hemsoth at The Next Platform.
No one expects that setting up management tools for complex distributed computing frameworks to be an easy thing, but there is always room for improvement–and always a chance to take out unnecessary steps and improve the automated deployment of such tools.
The hassle of setting up such frameworks, such as Hadoop for data analytics, OpenStack for virtualized infrastructure, or Kubernetes or Mesos for software container management is an inhibitor to the adoption of those new technologies. Working with raw open source software and weaving it together into a successful management control plane is not something all enterprises have the skills …
Samsung Experts Put Kubernetes Through The Paces was written by Timothy Prickett Morgan at The Next Platform.