From observing binary black holes to synthesizing all of the work in the largest publication journal, HPC processes the world’s data and attempts to answer science’s largest problems. Despite this field’s rapid growth and acute use in problem solving, the gap in representation for those in the community remains apparent. This is why WHPC’s international mission to enhance equity, inclusion, and access for women and minorities to the field of HPC is so critical.

Currently, Georgia Tech awards more engineering degrees to women than any other institution in the country. As HPC represents the intersection of engineering and computing, carrying forward the WHPC mission to promote, build, and leverage a diverse and inclusive HPC community is a natural next step for Georgia Tech.

This next step is also complimented by several high-profile initiatives launched over the past few years including the unveiling of a $5.3 million HPC system known as the Hive supercomputer. 

“Our geography and history provide a unique context for engaging in diversity, equity, and inclusion efforts given the racial and ethnic diversity of our broader community around Georgia Tech,” said Lorna Rivera, research scientist and founding WHPC member.

The Georgia Tech chapter’s primary goal is to provide a platform for all HPC researchers – regardless of gender, area of study, and level of expertise. The chapter’s main drivers for this goal include:

1. Helping students toward HPC professional career paths
2. Providing classroom HPC resources for teachers 
3. Offering HPC consulting to teachers
4. Involving industry partners on real-world projects

The chapter aims to achieve these goals by engaging researchers from across academic institutions including Georgia Tech, Atlanta’s Historically Black Colleges and Universities, minority serving institutions, and others interested in learning more, or providing support for HPC research. 

“We believe that by increasing exposure as well as access to resources for teaching through student-run seminars, workshops, online classes offered by GT’s Partnership for an Advanced Computing Environment (PACE), and mentorship efforts, we can build a more diverse, equitable, and inclusive supercomputing community,” said Neil Bright, PACE Associate Director of Research Cyberinfrastructure.

The WHPC will host a public event to celebrate its launch on March 26 from 2-3 p.m. EST.

The event will feature some of HPC’s heavy hitters from across the country. This list includes high-profile speakers experienced in navigating the world of HPC, passionate about building equity in this arena, and enthusiastic about sharing their professional journey. 

Register for the event here!

The Student Cluster Competition (SCC) is an immersive 72-hour HPC challenge held as part of the Supercomputing Conference Series (SC). SCC presents undergraduate students with real-world scientific workloads, a series of benchmarks, and even a mystery challenge to overcome.

[Related News: High-Performance Computing for All, Everywhere]

This achievement marks just the second time that a Georgia Tech team was accepted to the competition. Despite a Georgia Tech group not competing since 2017, this year’s challengers were prepped and ready to become the only team outside of China to place in the top three of 19 competing organizations.

“The success of Georgia Tech's SC20 Student Cluster Competition team was a thrill to witness. We feel strongly about providing students with opportunities to experience HPC. Our strategic partners from industry and national labs tell us demand for HPC skills continues to rise and initiatives like the SCC help cultivate interest in HPC careers,” said Team Phoenix coach and School of Computational Science and Engineering (CSE) Research Technologist Will Powell.

Team Phoenix’s student competitors include computer science undergraduates Sudhanshu Agarwal, Albert Chen, Aman Jain, Ali Kazmi, Nicole Prindle, and Marissa Sorkin. This team of six was selected from a Spring 2020 Vertically Integrated Project (VIP) course focused on using HPC systems and applications. 

VIP instructors Rich Vuduc, Aaron Jezghani, Will Powell, and Jeff Young helped lead the group along with CSE graduate student Vijay Thakkar, who served as the team mentor.

Supported by the VIP program and the leadership of the five coaches, the students engaged in the long-term, large-scale project that took nearly a year of planning to achieve before the competition had even started.

Part of this initial planning period was used to identify and recruit sponsorship for the team’s machine configuration. Vendor partner Penguin Computing answered the call by providing hardware support and sponsorship of Team Phoenix, while Intel and NVIDIA both shared expertise related to application optimization and HPC tools.

Due to Covid-19, the competition was moved to the Microsoft Azure cloud to accommodate remote participation, becoming the first-ever Virtual Student Cluster Competition of the Supercomputing Conference series. Undeterred by the change in format, the students exhibited mastery of both HPC and cloud skills while building virtual clusters, learning scientific applications, and applying optimization techniques for chosen cloud configurations. 

As part of the competition’s new parameters, each team received an initial $3,200 debit balance for the use of cloud resources. Benchmark results were then compared to calculate standings followed by a $500 boost 12 hours before the competition’s end which presented new opportunities for teams looking to maximize their respective scores. In total, all 19 teams used a collective $61,300 of the $70,300 budget, demonstrating not only effective HPC skills but also effective managerial and budgeting skills as well.

“Team Phoenix was supportive of one-another, competitive, clever, resourceful, and focused. It was a marvelous adventure. The end result is something all Yellow Jackets can be proud of and the team itself will never forget. We now have a reputation to defend and look forward to embarking on the adventure again in 2021,” said Powell.

Watch the team’s daily interviews at the link below.

https://youtube.com/playlist?list=PLl2dezBNo_BksHs_emoprAjYc-ZEbr-V_

However, a breakthrough created by researchers from Georgia Tech and Oak Ridge National Laboratory (ORNL) presents a way to link the millions upon millions of data points found throughout volumes of information. 

The paper outlining this breakthrough method has been nominated for the 2020 Gordon Bell Prize, which awards outstanding achievements in high-performance (HPC) computing with an emphasis on applying HPC to applications across science, engineering, and large-scale data analytics.

Co-authored by School of Computational Science and Engineering (CSE) Master’s degree student Vijay Thakkar and Professor Rich Vuduc and ORNL researchers Piyush Sao, Hao Lu, Drahomira Herrmannova, Robert Patton, Thomas Patok, Ramakrishnan Kannan this work presents a novel data mining approach to analyze large corpora of scholarly publications with HPC to discover how concepts relate to one another.

“In order to keep pushing the exponential progress of science, we need some level of analysis of different research papers so we can synthesize that into digestible information,” said Thakkar, who currently works in the HPC Garage research lab.

Thakkar joined the HPC Garage, led by Vuduc, one year ago when pursuing research in sparse linear algebra – Vuduc’s area of expertise. However, a few weeks into the independent study, Vuduc approached Thakaar about an opportunity to work with the ORNL team as a CUDA developer. Since then, Thakkar’s research emphasis has focused on facilitating this robust project.

“To me, this project boils down to a very important issue which is that as technological progress becomes faster and faster, for each individual scientist, there is too much information out there to distill into something that is comprehensible. Making connections between different research groups gets harder and harder to do, stalling the progress science can make,” he said.

In this initial project, the team is focusing on processing the articles across PubMed as a particular case study. PubMed is a database of biomedical literature maintained by the National Library of Medicine and currently hosts approximately 18 million papers. 

By mining key terms in the papers’ abstracts, the team is able to turn each term into a point, or vertex, in a large-scale graph. 

“A term might be the name of a drug or a disease or a symptom, for instance. When two terms appear in the same paper, that means we know there is a direct relationship between them, which becomes an edge in the graph,” said Vuduc.

But just because two terms are not directly linked does not mean they are unrelated; it only means they are not known yet. To find these as-yet undiscovered connections, the idea is to look for short paths that bridge two terms. 

Vuduc said, “A useful analogy might be the following: Suppose you are driving from point A to point B. Maybe you know one path to get there. But what if there is a better, shorter way? That’s what this method does. Points are these biological or medical terms or concepts, and road segments are the papers that correspond to known routes. It’s the unknown routes that are interesting.”

These paths are analyzed using a shortest path algorithm referred to as DSNAPSHOT. While shortest path algorithms are not a new concept and are fairly common, the scale at which this particular shortest path algorithm is applied is unheard of.

“If you think about it, six million papers may seem like a small number, but the size of the problem is not really six million. It’s six million times six million because in the worst-case scenario, each paper can have at least one connection to every paper in the database and that is where the complexity of the problem comes from,” said Thakkar.

However, according to Thakkar, the problem is more complex and more difficult to solve than this example – even with today’s supercomputers.

“It’s a much harder problem because you’re visiting each vertex once making this a V³ problem. At the scales we are talking about, this is 18 million times 18 million times 18 million and those orders of magnitude add up really quickly, which is why we need something like the Summit Supercomputer to crunch these numbers,” Thakkar said.

For the team’s first finalist run of 4 million papers it took 30 minutes to compute using over twenty-four thousand GPUs of the Summit machine. What’s more, the time this type of problem will require to solve will continue to grow in a cubic manner with each paper added to PubMed database. 

The announcement of the Gordon Bell Prize winner will be made at the 2020 Supercomputing Conference award ceremony held, Nov. 19 at 2 p.m. EST.

The Supercomputing Conference (SC), held from Nov. 9 to 19, is the premier event for this rapidly growing field. Mathematicians, engineers, and developers come together to present the most groundbreaking HPC research, and each year, the Georgia Tech name is one of the leading names in those presentations.

Although it is being hosted virtually, this year is no different for SC20 presentations. With several papers, awards, and workshops, Georgia Tech, represented by the School of Computational Science and Engineering (CSE), Partnership for Advanced Computing Environment (PACE), School of Computer Science (SCS), and the Georgia Tech Research Institute, is leading charge in the HPC discussion.

This year, Georgia Tech’s SC20 proceedings includes two notable awards. 

SCS Chair Vivek Sarkar is being recognized with the ACM-IEEE CS Ken Kennedy Award, Nov. 16. Sarkar’s recognition honors his leadership in several areas including foundational technical contributions to programmability and productivity.  

[Related News: ACM/IEEE Recognizes Chair's Research, Service to Computer Science Community]

CSE Ph.D. student Vijay Thakkar and Professor Rich Vuduc are being recognized with a Gordon Bell Prize nomination for their paper, Scalable Knowledge Graph Analytics at 136 PetaFLOPS. Their work, developed with a team from Oak Ridge National Laboratory, is motivated by data mining large-scale bodies of scholarly publications to discover concepts that relate to one another. The announcement of the Gordon Bell Prize winner for 2020 will be held, Nov. 19 from 2 to 2:30 p.m. EST.

An invited talk by SCS and School of Electrical and Computer Engineering Professor Tom Conte is another notable agenda item. Conte’s talk, HPC After Moore’s Law, discusses the post-Moore computing era as seen from the IEEE Rebooting Computing Initiative perspective and presents promising technologies to keep a close eye on.

Excitingly, Georgia Tech students are also participating in a virtual student cluster competition (VSCC) this year. While this event hosts the non-stop 72-hour track of a traditional hackathon, the VSCC separates itself from the rest in that the students prepare up to 6 months in advance for the competition. 

The Georgia Tech team, Team Phoenix, with sponsorship from vendor partner Penguin Computing, will compete with students from around the globe to complete a set of benchmarks and real-world scientific workloads from Nov. 8 to 11. Each team is tasked to design and build virtual clusters in the Microsoft Azure cloud, learn scientific applications, and apply optimization techniques for their chosen cloud configurations. Daily video interviews with the team can be watched here.

While the conference’s papers, presentations, and competitions are underway, this extraordinary year has required extraordinary efforts to transcend the physical conference barriers. With its first ever fully virtual format, SC20’s platform is bringing the HPC discussion into the hands of more than ever before.

To capitalize on this opportunity of enhancing equity in HPC, Georgia Tech is hosting special virtual programming to run parallel to the SC20 agenda. This programming includes a virtual party and poster show that is open to the public – while space lasts – and a 360-degree data center tour. 

The virtual data center tour will be featured on the PACE website and allows users to navigate through Georgia Tech’s premier data center housed at the Coda Building. While walking through the data center, viewers will see the Institute’s two premier HPC resources, the Hive supercomputer and the newly installed Phoenix cluster.

Vuduc is the newly appointed co-director of the Center for Research into Novel Computing Hierarchies(CRNCH), a unit built on the mission of reexamining and building computing technologies for the post-Moore computing era. He is succeeding CRNCH Co-Director Vivek Sarkar and will direct the center alongside School of Computer Science and School of Electrical and Computer Engineering Professor Tom Conte

"CRNCH was conceived as a center for post-Moore computing that spans the entire computing stack, including hardware, software, and algorithms. Rich's expertise in high-performance computing (HPC) algorithms and performance engineering will further contribute to CRNCH's success in this broad vision,” said Sarkar.

Post-Moore Computing’s Challenges

“In HPC especially, we are in a crazy time right now. Basically, for the last 50 years, we got a free ride. Computers got faster every year, and this transformed entire industries,” said Vuduc.

The improvements Vuduc is referring to, came about from the ability to make transistors smaller each year, allowing for faster computing year-over-year. The problem computing is facing now is that the scientific community does not know how to make transistors any smaller, and there is no defined way to move forward.

According to Vuduc, “It’s an incredibly fertile time to be exploring all kinds of wacky, very radical new ways of thinking about computing systems and how we might build them. And CRNCH has been investigating these questions since it was launched in 2016.”

Bringing a Fresh Perspective to the Center

From the way algorithms are designed to the way machines are programmed, software is still being built around the 1970s concept of a general-purpose computer.

“A lot of people are working to figure out what the physical and logical form of the computer is going to look like in the future. But a related question is, ‘What is the thing that is going to run on top?’” said Vuduc.

“In my joining CRNCH, I hope to think a lot more about those kinds of issues. That’s why I think this new role is exciting.”

Building a CRNCH Community

In his new role, Vuduc wants to launch projects with the goal of developing algorithms and applications to run on these hypothetical future systems. However, his long-term goals are considerably more community focused. 

Currently, he is hoping to excite a critical mass of people at Georgia Tech to begin thinking about the types of problems associated with post-Moore computing.

“From my point of view, the most exciting thing would be if we can bring new people on board, whether it be new faculty or students, to really think about these upper levels in the ‘computing stack.’ What are the kinds of applications and algorithms that we might run on these future machines?” he said.

The winning dissertation, Parallel and Scalable Combinatorial String Algorithms on Distributed Memory Systems, offers a new approach to solve large-scale string and graph problems used throughout computational biology applications. The computational methods introduced in Dr. Flick’s work achieve efficient and scalable execution on large-scale distributed compute clusters, achieving solutions to increasingly larger problems.

Inspired by the advent of high-throughput DNA sequencing which enables generations of billions of reads per minute, and the growing need to find a computational approach that can keep pace, this research expands on prior theoretical approaches. The resulting algorithms and data structures implemented by Flick advance the state-of-the-art by providing improved theoretical complexity and better practical performance, while minimizing overall and per-node communication volume within a computer’s distributed memory architecture.

Ultimately, these findings offer a more efficient method to represent, construct, and query data structures for large-scale and memory intensive applications in text processing, information retrieval, and computational biology.

Flick joined CSE for his Ph.D. in 2014 under the guidance of CSE Professor and Interim Chair Srinivas Aluru.

According to Aluru, “Patrick's Ph.D. work addresses some notoriously difficult problems in parallel string algorithms, and his dissertation gets it just right by providing both theoretical optimality and practical efficiency. His work, all published in top forums in the field, has lasting value. It is gratifying to see him win this year's ACM SIGHPC Dissertation Award.”

Flick defended his thesis in March 2019 and officially graduated the following May. He is now a software engineer at Google. 

Flick’s previous successes include authoring the first paper used for the Student Cluster Reproducibility Challenge at Supercomputing 2016 and winning the Best Student Paper Award at Supercomputing 2015.

SIGHPC is the ACM’s special interest group that focuses on providing a platform for high-performance computing (HPC) research and efforts internationally. The ACM SIGHPC Dissertation Award pulls from this professional society in an effort to highlight innovative and prolific research in the supercomputing and parallel processing fields.  

The 2020 ACM SIGHPC Dissertation Award includes a $2,000 honorarium, travel support to the Supercomputing Conference, and an award plaque.

Rivera is a research scientist at Georgia Tech’s Center for Education Integrating Science, Mathematics, and Computing (CEISMC) and for the Women in High Performance Computing (WHPC). 

Her research for both of these programs helps aggregate and analyze data in meaningful ways to determine custom-tailored approaches to advance initiatives focused on minority or underserved groups. 

After providing a booth talk detailing initiatives in CEISMC and WHPC during SC19, the School of Computational Science and Engineering had the opportunity to discuss with Rivera updates on her latest efforts.

[Related News: Diversifying the HPC Community: Q&A with Georgia Tech’s Lorna Rivera]

For those unfamiliar, can you give a CEISMC from 30,000-foot view?

CEISMC connects Georgia Tech to surrounding K12 schools, focusing on underserved and minority areas, and works to move the needle in the state of Georgia to improve representation in the scientific areas of STEM and computing. It is led by Dr. Lizanne DeStefano who has a reputation for not only pushing those programs but also in evaluating their effectiveness.

A lot of what CEISMC does is focused on evaluation of these programs. Can you tell us a what this evaluation looks like and what you are looking for?

Evaluation can mean a lot of things and it differs from research, although you can do both research and evaluation. Evaluation differs in that you want to have a deep understanding of the context in which you are working. In research, you often want to wash out context because you want to be able to replicate things, and you can do that with evaluation too, they’re not completely separate boxes. But sometimes in evaluation, you study things not necessarily with the intention of replicating but because you want to understand the unique factors of that particular situation.

What kind of communities are you looking to serve or help enhance in your initiatives?

I personally try to always pay attention to those who are underserved in a particular community. I use an evaluation approach that accounts for what we look at in thirds. So, one third is focused on the quality of science being done, another third is focused on the way that science is being taught, and the remaining third is focused on the disadvantaged in that group that is being served. Then, in the middle, if you think about it like a Venn Diagram, those three come together and that is where you find the program and evaluation. 

For more information about the approach, which was pioneered by CEISMC Executive Director Lizanne DeStefano, click here.

Does this approach transcend or translate to your efforts with the WHPC program?

Absolutely! I am the director of research for Women in HPC and it’s an organization that started at the University of Edinburgh’s EPCC  and it is now an international organization. My role with WHPC is to understand the unique experiences of women in the international HPC community. You may notice that women of color are extremely underrepresented in this community, so, I am particularly passionate about moving the needle in this space. It is very important to me and, the way that we can accomplish this is, yet again, by evaluation. The WHPC programs are all new and are being rolled out now, so I am in the initial phases of evaluating. We have the mentoring program, which I got to talk about today in the Georgia Tech booth, we have a fellowship program that we just started, and we have our first three fellows with us this year. Then we are looking to broaden and see what else we can do in this space while growing sustainably because we do not want to grow too fast.

This sounds like WHPC has a lot going on – not to mention your first inaugural event is coming up! Can you tell us a little about it and what to expect? 

We have our first Women in HPC Summit at the end of April and early May that is going to be held together with Simon Fraser University and will be held in Vancouver, Canada. We are very excited! There will be a series of technical talks, workshops, and tutorials. Sometimes, we hear from our members that they do not want to focus on the technical content and rather they want to focus on the experience of women, or vice versa and that they want to focus on the technical. We are hoping, now that we have our own event and are not limited to a one-day workshop, we can meet both of those needs.

Is there anything else you want to stress to people regarding the WHPC?

If you are interested in this topic, regardless of your gender, WHPC is for everybody. We need everybody involved and as many advocates as possible. Please don’t let the name turn anyone off, we are open to all and we want to make this a better space for everyone!

Known as Hive, this newly operational supercomputer supports research for over 33 faculty, 54 research scientists and postdocs, 195 graduate students, and 56 undergraduate students from the colleges of Computing, Engineering, and Sciences.

Hive was acquired by the Institute for Data Engineering and Science (IDEaS) through a $3.7 million National Science Foundation (NSF) Major Research and Instrumentation Program grant and a $1.6 million contribution by Georgia Tech in 2018.

[Related News: Georgia Tech Award Equips Coda’s Data Center with New Supercomputer]

IDEaS Director and School of Computational Science and Engineering (CSE) Professor and Interim Chair Srinivas Aluru is the primary investigator on the grant. According to Aluru, “Hive allows us to solve large scale of data-intensive problems and will be an asset to Georgia Tech for the next five years or so.”

The supercomputer has over 100 trillion bytes of memory, 11,500 compute cores, and 2.5 quadrillion bytes of storage. To put it simply, this cluster, which delivers 0.7 petaflops of performance based on the LINPACK Benchmark, is fast.

[VIDEO::https://youtu.be/ZHV8fxzhlDw::aVideoStyle]

Local and National Collaborations

Hive represents several interdisciplinary, cross-institute collaborative efforts. One of these is the attachment of this supercomputer to the Extreme Science and Engineering Discovery Environment (XSEDE) program, which funds and interconnects supercomputers nationwide, enabling national collaborative use of this resource.

“We have reserved about 20 percent capacity of the machine to support research activities of regional partners, minority-serving institutions, and others through the XSEDE Program,” Aluru said.

“One particular segment that we are focusing on is supporting historically black colleges and universities (HBCUs) and minority institutions in or around Atlanta. We are also providing active training programs to bring faculty and students from these institutions on board to use high-performance computing,” he said.

Currently, Morehouse College, Spelman College, and Clark Atlanta University all have research on the machine with the support of this program.

Georgia Tech’s HPC Future

Hive made its official debut with a ribbon-cutting ceremony on Oct. 21 at the Coda datacenter. Georgia Tech Executive Vice President of Research Chouki Abdallah delivered opening remarks before attendees were invited for a private tour of the new facility and an up-close view of the Hive computer – which featured newly-added blue LED lights across its many stacks.

“The institute is trying to come up with a long-term plan for high-performance computing, and this is a manifestation of our performance so far,” said Abdallah during the ceremony.

Part of this plan includes moving all of Georgia Tech’s computing services and the Office of Information Technology to the new data center by the end of the school year.

Six Years in the Making

Hive’s existence represents roughly six years of planning and preparation across Georgia Tech units and several external organizations. This includes the construction of an entirely new 80,000 sq. ft. data center in the Coda building which is managed by DataBank.

Integral to this datacenter planning and readying for the Hive supercomputer was Georgia Tech’s Partnership for an Advanced Computing Environment (PACE) team.

According to PACE Senior HPC Architect Paul Manno, “We have partnered with DataBank and we are working with them to create what is essentially a great research resource, but also a great operations datacenter, for Georgia Tech and others. Before, we were limited by space, power, and cooling in the old datacenter on campus.”

He said, “Now, in addition to having much more space, we are starting out with roughly two megawatts of power for Georgia Tech and we can expand that up to 8-10 megawatts, which gives us the ability to expand well beyond what people had originally envisioned and provides Georgia Tech a means to grow into the future.”

Georgia Tech researchers are stepping in to answer this growing need by leading HPC research from a variety of approaches and are preparing to present their latest findings in Denver, Colorado next week at the 2019 International Conference for High Performance Computing, Networking, Storage, and Analysis.

Also known as Supercomputing or SC19, the annual conference features leading research in the fields of HPC and exascale computing with an emphasis on real-world application.

This year, Georgia Tech’s presence boasts four papers, four workshops, two posters, and one Birds of a Feather discussion.

Two of the four papers, CARE: Compiler-Assisted Recovery from Soft Failures and GPU Acceleration of Extreme Scale Pseudo-Spectral Simulations of Turbulence Using Asynchronism are best student paper award finalists. The announcement naming the winner of the award will be made at the SC19 award ceremony on Nov. 21.

Georgia Tech’s presence also comprises a number of activities outside of the conference agenda, including the Georgia Tech booth (#1809) on the vendor floor, which will give SC19 attendees an opportunity to engage directly with researchers, students, and staff alike.

This year, booth #1809 features four live demos provided by researchers and students, including a presentation about the Women in High Performance Computing (WHPC) program by WHPC Director and Center for Education Integrating Science, Mathematics, and Computing Research Scientist II Lorna Rivera.

To see a full listing of Georgia Tech’s presence in the Supercomputing proceedings and a demo schedule click here.

A full listing of papers with Georgia Tech Affiliations this year are below:

• CARE: Compiler-Assisted Recovery from Soft Failures

Chao Chen, Greg Eisenhauer, Santosh Pande, Qiang Guan

• Distributed Enhanced Suffix Arrays: Efficient Algorithms for Construction and Querying

Patrick Flick, Srinivas Aluru

• GPU Acceleration of Extreme Scale Pseudo-Spectral Simulations of Turbulence Using Asynchronism

Kiran Ravikumar, David Appelhans, P.K. Yeung

• Practical and Efficient Incremental Adaptive Routing for HyperX Networks

Nic McDonald, Mikhail Isaev, Adriana Flores, Al Davis, John Kim

• AutoFFT: A Template-Based FFT Codes Auto-Generation Framework for ARM and X86 CPUs

*Zhihao Li, Haipeng Jia, Yunquan Zhang, Tun Chen, Liang Yuan, Luning, Cao, Xiao Wang
*Note: This paper is by a Georgia Tech visiting Ph.D. student but is not a Georgia Tech published item.
 

The Center for High Performance Computing (CHiPC) leads this year’s Georgia Tech efforts under CHiPC Director and School of Computational Science and Engineering Professor Rich Vuduc.

"I’m really proud of Georgia Tech’s presence at SC19 this year, especially the two best paper finalists. It’s a good warm-up for SC20, where we will be the home team,” he said.

Vuduc is referencing the fact that Supercomputing is making its way for the first time to Atlanta next year. This means Georgia Tech’s HPC community is buzzing with anticipation to showcase its newest HPC resource, the Hive.

The National Science Foundation (NSF) announced the award, which will support the acquisition and deployment of the new supercomputer to TACC at The University of Texas at Austin, today. Anticipated to begin operations in 2019, Frontera will be the fastest at any U.S. university, among the most powerful in the world, and will allow the nation’s academic researchers to make important discoveries in all fields of science, from astrophysics to zoology.

The team from Georgia Tech is led by Srinivas Aluru, co-executive director of the Institute for Data Engineering and Science, and also includes Rich Vuduc, Edmond Chow, and David Bader. All are professors in the School of Computational Science and Engineering, where Bader serves as chair, and in the College of Computing.

“Georgia Tech is excited to be part of the NSF leadership-class computing facility project, which will produce the design and operation of the leading academic supercomputer of our times, and guide its path to reaching exascale capabilities,” Aluru said.

His team will be working on the evaluation of possible architectures and technical design of the Phase Two system, and develop exascale-ready code for application areas including computational biology and computational chemistry. Georgia Tech faculty will also serve on the technical advisory committee for the project.

“Many of the frontiers of research today can only be advanced by computing, and Frontera will be an important tool to solve grand challenges that will improve our nation’s health, well-being, competitiveness, and security,” said Dan Stanzione, TACC executive director.

If completed today, Frontera would be the fifth most powerful system in the world, the third fastest in the U.S., and the largest at any university. For comparison, Frontera will be roughly twice as powerful as Stampede2 (currently the fastest university supercomputer), and 70 times larger than Ranger, which operated until 2013.

To match what Frontera will compute in just one second, a person would have to perform one calculation every second for roughly one billion years. 

Through its involvement, Georgia Tech researchers will influence the design, operation, and science conducted on the leading NSF supercomputer. Tech researchers will also receive substantial access during the early operations phase.

Anticipated early projects on Frontera include analyses of particle collisions from the Large Hadron Collider, global climate modeling, improved hurricane forecasting, and multi-messenger astronomy.

The primary computing system will be provided by Dell EMC and powered by Intel processors. Data Direct Networks will contribute the primary storage system and Mellanox will provide the high-performance interconnect for the machine. GRC (Green Revolution Cooling), NVIDIA, and the cloud providers Amazon, Google, and Microsoft will also have roles in the project.

Faculty at the Institute for Computational Engineering and Sciences at UT Austin will lead the world-class science applications and technology team, with partners from the California Institute of Technology, Cornell University, Princeton University, Stanford University, the University of Chicago, the University of Utah, and the University of California, Davis.

Experienced technologists and operations partners from the sites above, as well as The Ohio State University, the Georgia Institute of Technology, and Texas A&M University will ensure the system runs effectively in all areas, including security, user engagement, and workforce development.

Frontera will enter production in the summer of 2019 and will operate for five years. In addition to serving as a resource for the nation’s scientists and engineers, the award will support efforts to test and demonstrate the feasibility of an even larger future leadership-class system – ten times faster than Frontera – to potentially be deployed as Phase 2 of the project.

“Keeping the U.S. at the forefront of advanced computing capabilities and providing researchers across the country access to those resources are key elements in maintaining our status as a global leader in research and education,” said NSF Director France Córdova.

“This award is an investment in the entire U.S. research ecosystem that will enable leap-ahead discoveries.”

The new $1.26 million National Science Foundation (NSF) project seeks to develop compiler tools and runtime systems to create a framework, named Parallel Algorithms by Blocks (PAbB), built specifically to facilitate simpler programming of scalable parallel systems in high-performance computing (HPC) and exascale machines.

“Current supercomputers and exascale machines are getting harder to program because of how technology is evolving,” said School of Computational Science and Engineering (CSE) Professor Ümit Çatalyürek.

Çatalyürek is Georgia Tech’s principal investigator (PI) for the project and joins the project’s lead PI, University of Utah Professor Ponnuswamy Sadayappan, and co-PIs, Ananth Kalyanaraman, Aravind Sukumaran Rajam, and Sriram Krishnamoorthy of Washington State University. The team of researchers plan to combine user insights, new compiler optimizations, and advanced runtime support to create the PAbB framework which will ultimately create building blocks of parallel code for heterogeneous environments to use across a number of applications from computational science and data science.

From caches to networks, architectures are written so that a system inherently wants to transfer multiple items at once. And, according to Çatalyürek, when looking at algorithms and problems, and thinking of them in terms of blocks – or packages of data – they are able to take advantage of hardware transfer and have better scheduling of communication and computation in these heterogeneous systems.

“We cannot make the single core in a computer much faster anymore, which is why sequential programs are not gettingfaster, and why we have to do everything in parallel computing,” he said.

“If you look at today’s supercomputers you will see that all of the architectures are becoming more and more heterogenous. So, writing parallel code by itself without heterogeneity is difficult, but, when combined, it becomes a barrier for many engineers.”

Heterogenous systems are made up of hardware and software components that necessitate the use of different languages, run on different operations, and usually incorporate specialized processing capabilities to handle particular tasks. 

Researchers, particularly those in the HPC and exascale spaces, need a way to ensure they write their data and computation to work in these heterogenous environments, on multiple nodes, and are still able to communicate effectively with the rest of a program while producing the results fast. PAbB aims to become the first framework that achieves both high productivity and high performance in such environments by creating a framework that utilizes block programming. 

Jordi Wolfson-Pou, a Ph.D. student in the School of Computational Science and Engineering (CSE), is an HPC researcher who has spent the past few months traveling the globe, presenting new insights on an old solution that aims to tackle synchronization bottlenecks in supercomputers.

These new insights highlight the efficacy of using asynchronous multigrid iterative methods for solving large linear systems on exascale computers. The researchers who discovered these findings believe this approach can quicken the computing processes used in a variety of fields, such as physics and engineering. 

“Solving equations in physics and engineering often requires highly accurate solutions, which means very large problems need to be solved.  This is where supercomputers come in. The next generation of supercomputers will be capable of doing calculations at the exascale and will certainly be fast, but synchronization will limit their speed,” said Wolfson-Pou.

Iterative methods are old computing techniques that start with an initial guess and generate a sequence of improved approximations to the true solution. These methods have achieved remarkable results and showed measurable progress for completing calculations simultaneously. However, the nature of these methods involves one or more synchronization points within each iteration.

“Supercomputers are composed of many parallel processes doing calculations concurrently. If many processes have to synchronize, some may be idle while waiting for others to finish,” said Wolfson-Pou.  “For example, this could be due to some processes having to do more calculations than others, or the underlying hardware that a process uses is slower than the hardware another uses. In asynchronous methods, the faster processes simply move on to the next step using the most available information.”

From Brazil to China, Wolfson-Pou presented the new observations he and CSE Professor Edmond Chow discovered while examining multigrid methods in an effort to understand how they can be executed asynchronously.

Their findings are detailed in the paper, Asynchronous Multigrid Methods, which was presented at the following:

1. International Parallel & Distributed Computing Symposium (IPDPS), May 20-24, Rio de Janeiro, Brazil
2. International Conference on Industrial and Applied Mathematics (ICIAM), July 15-19, Valencia, Spain
3. International Multigrid Conference (IMG), August 11-16, Kunming, China
4. AMG Summit, September 30 – October 3, Santa Fe, New Mexico

The paper’s experimental results show that asynchronous multigrid can be faster than classical multigrid in terms of reducing the time it takes to converge to the solution.

These applications will ultimately speed processing rates and cut computational costs for embedded computing systems used in transportation, healthcare, manufacturing, retail, and a host of other industries.

But, there are specific requirements and unique challenges – known and unknown – to deploying HPC applications outside of a data center. To address these challenges and advance this growing research area, School of Computational Science and Engineering (CSE) HPC researchers are bringing their expertise to this week’s 2019 IEEE High Performance Extreme Computing Conference (HPEC).

HPEC is one of the leading conferences of its kind. It brings HPC and embedded systems researchers together to identify obstacles and develop effective solutions for delivering HPC capabilities to edge computing applications, augment big data with GPUs, and more.

Researchers from CSE are set to present several papers at this year’s HPEC, which runs from Sept. 24 to 26 in Waltham, Massachusetts. Of these, two papers were submitted to the HPEC 2019 GraphChallenge and are set to receive Innovation Awards. 

“The GraphChallenge creates an annual benchmark that drives community development of new solutions for analyzing graphs and sparse data from social media, sensor feeds, and scientific data to discover relationships between events,” said CSE Ph.D. student Abdurrahman Yasar, an investigator on two of the four Innovation Award winning papers, and a 2018 GraphChallenge champion.

One of the two CSE winning papers, Linear Algebra-Based Triangle Counting via Fine-Grained Tasking on Heterogeneous Environments, describes an update to the linear-algebraic formulation of the classic triangle-counting problem. 

“Triangle counting is a representative graph analysis algorithm with several applications and is also one of the three benchmarks used in the IEEE HPEC GraphChallenge,” said Yasar.

“In this work we propose a novel multi-core multi-GPU triangle counting algorithm. Our new approach does not require architecture specific changes on code which is crucial for portability on heterogenous environments and the way we distribute the tasks between GPUs and CPUs is highly appreciated.”

While triangle counting remains an important benchmark in the GraphChallenge, several other research areas are also prevalent in this year’s CSE HPEC proceedings.  

See the list below to view all Georgia Tech papers being presented at this year’s HPEC conference:

• Concurrent Katz Centrality for Streaming Graphs - Chunxing Yin, Jason Riedy
• Skip the Intersection: Quickly Counting Common Neighbors on Shared-Memory Systems – Xiaojing An, Kasimir Gabert, James Fox, Oded Green, David A. Bader
• Improving Scheduling for Irregular Applications with Logarithmic Radix Binning – James Fox, Alok Tripathy, Oded Green

Graph Challenge Champions

• Linear Algebra-Based Triangle Counting via Fine-Grained Tasking on Heterogeneous Environments - Abdurrahman Yasar, Sivasankaran Rajamanickam, Jonathan Berry, Michael Wolf, Jeff Young, Ümit V. Çatalyürek
• Scalable Triangle Counting on Distributed-Memory Systems – Seher Acer, Abdurrahman Yasar, Sivasankaran Rajamanickam, Michael Wolf, Ümit V. Çatalyürek

[Related News: School of CSE Speeds up Graph Applications and Presents GraphChallenge Winning Paper at HPEC 2018]

CSE Professor and Associate Chair Ümit Çatalyürek served as the program chair of HiPC2017, which, coincidentally, was where he presented his first peer-reviewed paper in 1995. Now, 22 years later, Çatalyürek oversaw all aspects of the conference, from the technical program – with a committee of more than 100 individuals – that featured three keynote talks and 41 papers, selected from over 180 submissions. CSE Chair David Bader and CSE Professor and IDEaS co-Director Srinivas Aluru are members of the HiPC Steering Committee and attended the conference.

“The quality of the papers was very good this year and I was very happy. It can be challenging to attract such good submissions when you have HPC conferences in locations that are not historically known for being pillars for that field of research,” said Çatalyürek.

Among these papers, was CSE Professor Srinivas Aluru’s, Parallel Exact Dynamic Bayesian Network Structure Learning with Application to Gene Networks, which earned a best paper finalist award.

CSE recent graduate, Devaret Makkar, presented the paper Exact and Parallel Triangle Counting in Dynamic Graphs, which discusses a new dynamic graph algorithm for counting triangles, on behalf of co-authors CSE Research Scientist Oded Green and CSE Chair David Bader.

Several days before HiPC2017, Bader attended ICMLDS2017, the first of an international conference series hosted by Bennett University, where he participated as as a keynote speaker.

Bennett University was established – just 18 months ago prior to the conference – as a startup academic venture in the suburbs of Delhi. It was founded by The Times Group, a conglomerate that owns several media brands throughout India.

One of these brands, The Economic Times, interviewed Bader about his keynote at ICMLDS2017, publishing it in the paper the next morning. The Times of India, the sister platform to The Economic Times, also interviewed Bader about 5G abilities and published the Q&A online.

The Times Group established Bennett University as an independent institute with the goals of revitalizing the research industry, addressing research needs across the region, and ultimately encouraging economic development of the country. The university, still in its infancy, hosted the conference as a means to introduce those in academia, industry, and international audiences, to the new concept.

“The Times of India realized they needed a competitive advantage to encourage economic development of the country. In order to accomplish this with the creation of Bennett University, they gave the new school latitude to higher faculty and started collaborative research labs that are often joint with industry. In computing, they’ve started labs centered around companies such as Dell, NVIDIA, and others,” said Bader.

“One area they are investing in is using HPC for data science, and they have formed unique departments at the interface of traditional disciplines similar to our school [CSE] at Georgia Tech.”

The Student Cluster Competition is a high-performance computing (HPC) multi-disciplinary experience integrated within the HPC community’s biggest gathering, SC'17. The event is a real-time, non-stop, 48-hour challenge that demonstrates the breadth of skills, technologies and science required to build, maintain, and utilize a computer.

Teams of six comprising undergraduate and high school students are selected via a team proposal. The groups then assemble a small cluster on the exhibit floor as they race to complete a real-world workload across a series of applications while rotating personnel to accomplish the varying tasks associated with the competition agenda.

This year’s conference takes place November 12-17 in Denver, Colorado. Georgia Tech’s team, Team Swarm, is ready to compete against the 16 others selected from across the globe, each ranging in age, experience, and nationality. The team is advised by CSE Research Scientist Oded Green alongside co-advisors, Chirag Jain, fourth-year CSE Ph.D. student, and CSE Research Technologist Will Powell.

Before 2015, the competition was mostly about throughput and judging was based primarily on how many datasets could be computed in the shortest amount of time. Since then, more elements that replicate what scientists and staff at research centers contend with every day have been included into the completion scorecard and agenda.

The two-day competition is now scored on a number of categories including the ability to describe methods used during an interview and in written format, group engagement in the conference, a poster presentation, benchmarking, reproducibility and a number of apps, as well as a surprise competition. Teams can expect more datasets than would be possible to compute within their power budget, timeframe and cloud resources. This is meant to encourage teams to make decisions based on their architecture and how to best use their resources in live time.

Accoridng to Green, students have been preparing for the competition since last January through a mix of weekly meetings, registering for advanced classes, and gaining valuable insight through hands on experience. While these students are still in their undergrad, the training and education they have received in preparation for this competition expands well beyond HPC and produces an educational foundation they will continue using for years to come.

“This may be Georgia Tech’s first year participating, but our first-rate dedication and preparation certainly makes me confident that our students will succeed far beyond the competition itself,” said Green.

This year, Team Swarm includes Petros Eskinder, a fourth year computer science major; Dezhi “Andy” Fang, a second year science major; Nick Fahrenkrog, a fourth year electrical engineering student; Manas George, a third year computer science student; David Meyer, a third year computer science major; Jessica Rosenfield, a third year computer science major; and Alok Tripathy, a second year computer science major.

“Each year, Georgia Tech’s presence at the Supercomputing Conference is felt: from papers in the main conference and workshops, to faculty participation in panels and keynotes, to students presenting at poster sessions and exhibition booths,” said Jain.

“However, up until this year, we have been absent from the Cluster Competition. Now, with our team competing, we believe Georgia Tech has officially swarmed SC17 from all fronts. It also helps that our mascot is a yellow jacket.”

“Georgia Tech is using high-performance computing (HPC) to help solve some of the world’s toughest technical challenges in energy, sustainability, human health, materials science, and national security,” said Center for High Performance Computing Director and School Computational Science and Engineering (CSE) Professor Rich Vuduc.

“We believe research this powerful deserves the superhero treatment. So, we have created the first-ever Superheroes of Supercomputing card deck to showcase Georgia Tech's HPC findings.”

Each card represents one of four technical papers being presented at SC’18, which is being held in Dallas, Texas, Nov. 11 to 16. A card touting the Rogue’s Gallery, Georgia Tech’s growing collection of next generation computing hardware – has also been created.

Along with a unique superhero for each subject, the two-sided cards include details about the project and list the primary researchers along with their Georgia Tech affiliations. All artwork and design work was done by Georgia Tech communications officer Kristen Perez.

Also included on each card are details about how SC’18 conference attendees can enter a contest for the chance to win an Amazon Echo Show courtesy of Amazon through the cards. To enter, attendees can take a photo of any character from the deck out in the world, at home, or at the conference itself and post to Twitter or Instagram with the hashtag: #gtsuperherosc18. Downloadable PDF versions are available online for printing purposes. Winners will be announced Nov. 28. 

The paper and center contributions, each with a representative superhero card, are below. A complete list of Georgia Tech activities at SC’18 can be viewed here.

• Rogues Gallery
• HiCOO: Hierarchical Storage of Sparse Tensors
• Accelerating Quantum Chemistry with Vectorized and Batched Integrals
• Detecting MPI Usage Anomalies via Partial Program Symbolic Execution
• Optimizing High Performance Distributed Memory Parallel Hash Tables for DNA k-mer Counting

As it has in the past, Georgia Tech is boasting a large presence at SC17. This year, the Institute has over three dozen attendees, including faculty, staff, and students. Along with staffing a booth (335), Georgia Tech is participating in a broad array of events showcasing recent technical applications and advances in HPC.

“The power to apply the latest computation-based innovation is vital in today’s marketplace. HPC and data sciences, from analytics to machine learning, at Georgia Tech goes well-beyond merely keeping pace with market trends,” said David Bader, chair of the School of Computational Science and Engineering (CSE).

“As we continue to attract the best and the brightest student, faculty, and research minds in HPC, we are pushing forward on what is possible and delivering results that can lead to groundbreaking advances in cancer research, cybersecurity and other areas of vital importance to our nation and the world.”

The six-day SC17 affair includes invited talks, panels, research papers, tutorials, workshops, posters, and Birds of a Feather (BoF) sessions. Participants include leading HPC networking, storage, and analysis experts from industry, academia, and government.

One of the workshops will feature keynote remarks from School of Computer Science Professor Vivek Sarkar. During a session on memory centric programming for HPC. Sarkar will present, Compiler and Runtime Challenges for Memory Centric Programming, which begins at 9:10 a.m. on Nov. 12.

Another expected highlight at SC17 is a panel discussion featuring Debra Lam – Managing Director of Smart Cities and Inclusive Innovation for Georgia Tech – is participating in HPC Connects Plenary: The Century of the City, Nov. 13 from 5:30-6:30 p.m. Along with two other HPC experts, Lam will discuss emerging needs and opportunities suggesting an increasing role for HPC in cities, with perspectives from city government, planning and design, and embedded urban HPC systems.

One of the most anticipated events at SuperComputing each year is the annual announcement of the Graph500 list. Developed by Bader and a small cadre of well-known supercomputing experts, the Graph500 is recognized as a leading indicator of global HPC development and investment, and often reveals trends regarding which technologies are popular in the machines. This year, the 15^th Graph500 List will be announced during a BoF session, which is set for Nov. 15 at 12:15 p.m.

The conference agenda also includes awards, an evening gala, exhibit hall, and the Students@SC mission.

The mission encompasses development programs, opportunities to learn from mentors, and engagement with Sc’s technical sessions for the HPC student community at both graduate and undergraduate levels.

The mission also organizes the Student Cluster Competition (SCC), Beginning early on Nov. 13, the SCC is a non-stop, 48-hour HPC multi-disciplinary challenge, in which student teams demonstrate their abilities to build, maintain, and utilize a computer.

This is the first year a Georgia Tech team will be participating in the SCC. Known as Team Swarm, the team of six students is advised by Research Scientist Oded Green, and co-advised by Ph.D. student Chirag Jain and CSE Research Technologist Will Powell.

Additional Georgia Tech participation includes:

Nov. 12 

• Workshop – Women in HPC: Diversifying the HPC Community – Lorna Rivera
• Workshop – Keynote: Compiler and Runtime Challenges for Memory Centric Programming – Vivek Sarkar
• Workshop – Panel Discussion: Diversifying the HPC Workforce – Lorna Rivera
• Workshop – Parallel Streaming for In Transit Analysis with Heterogeneous Data Layout – Greg Eisenhauer
• Workshop – Why Subtle Bias is Often Worse than Blatant Discrimination – Lorna Rivera

Nov. 13

• Workshop – Energy Efficient Supercomputing (E2SC) – Sudhakar Yalamanchili
• HPC Connects Plenary, Keynote & Plenary Talk – HPC Connects Plenary: The Century of the City – Debra Lam
• Workshop – Seventh SC Workshop on Irregular Applications: Architectures and Algorithms – Richard Vuduc
• Workshop – Parallel Tensor Infrastructure (ParTI!) – Jiajia Li
• Student Cluster Competition (Nov. 13-16) - Team Swarm - Oded Green and Will Powell

Nov. 14

• Poster Reception –  A:25 Investigating Performance of Serialization Methods for Networked Data Transfer in HPC Applications – Max Yang
• Birds of a Feather – Memory-Centric Architectures for the Cloud and HPC – Sudhakar Yalamanchili
• ACM Student Research Competition, Poster, Reception – P10: HiCOO: A Hierarchical Sparse Tensor Format for Tensor Decompositions – Jiajia Li, Jimeng Sun, Richard Vuduc
• Birds of a Feather – Message Passing Can’t Solve Your Graph Problems – David Bader
• Birds of a Feather – A Proposal for a Next-Generation BLAS – Jason Riedy

Nov. 15

•  15^th Graph500 List – David Bader
• Birds of a Feather – From Outreach to Education to Researcher – Innovative Ways of Expanding the HPC Community – Lorna Rivera
• Birds of a Feather – OpenSHMEM in the Era of Exascale – Vivek Sarkar

Nov. 16

• Paper Presentation – Distributed Southwell: An Iterative Method with Low Communication Costs – Jordi Wolfson-Pou and Edmond Chow
• Poster, Scientific Visualization & Data Analytics Showcase – First Light in the Renaissance Simulation Visualization: Formation of the Very First Galaxies in the Universe – John H. Wise
• Birds of a Feather – Recruitment: How to Build Diverse Teams – Lorna Rivera 

Additionally, Georgia Tech, along with with Texas A&M, and U.C. Davis, will all contribute to the progress of the NVIDIA data science curriculum partnership this year, expanding the open source RAPIDS graph analytics algorithms.

Through these partnerships, Georgia Tech will work closely with NVIDIA researchers on GPU technologies and their application to data analytics. The goal of this collaboration is to build the world’s most advanced accelerated end-to-end pipeline for data analytics, and to support groundbreaking work of the world’s leading AI researchers and labs.

School of Computational Science and Engineering (CSE) Chair David Bader led the NVAIL program proposal effort as part of continuing work at Georgia Tech on GPU technology and scalable graph algorithms.

“Data science is the fastest growing field of computer science today. With this growth in mind, we are excited to further the collaborative relationship of Georgia Tech and NVIDIA,” said Bader. “We will focus on the design and implementation of scalable graph algorithms and primitives for integrating into cuGRAPH, leveraging their Hornet framework that will foster new abilities in the broader community of data scientists.”

As part of the awards, NVIDIA is providing:

• $100,000 cash award for one year toward research into scalable graph algorithms
• (1) NVIDIA DGX Station
• Collaboration on Deep Learning Institute Teaching Kits and educator outreach
• Opportunities for faculty and students of the Georgia Tech NVIDIA AI Lab to work at NVIDIA as visiting professors and interns

HiCOO was awarded the prestigious title of best student paper at this year’s International Conference for High Performance Computing, Networking, Storage, and Analysis, commonly referred to as Supercomputing (SC).

Winning the best student paper award at this well-established and increasingly competitive conference program, which boasted an impressive 288 technical paper submissions with 68 ultimately being accepted this year, is no small feat.

However, for Li, who holds a second Ph.D. degree in computer architecture, the journey to effectively alter a process is half the battle and half the fun in research.

“For me, it isn’t about the outcome, but rather what I can explore in the process along the way,” she said. “Tensors algorithms are a way to break down data by organizing or viewing it in a certain manner to find what connects the different factors. Sparse tensors are just a specific kind of tensor.”

For sparse tensor computations, there is tension among storage, speed, and flexibility. With traditional methods, you can usually get just two of these three traits. For example, a computation can be fast and flexible at the price of more storage. Or, it can be compact and flexible, but also slow.

“We wanted to create a storage format for tensors that was all three: compact, fast, and flexible,” said Li.

Authors of HiCOO, which includes Li and CSE Associate Professors Rich Vuduc and Jimeng Sun, were inspired by the work of Lawrence Berkeley National Lab Staff Scientist Aydin Buluç which focuses on sparse matrices, the lower dimensional analogue of a tensor. 

“We saw an opportunity to extend those ideas to sparse tensors,” explained Li. “We believe HiCOO is smaller, faster, and simpler to update [when the data is changing] than state-of-the-art alternatives. So, we think it will be easier to use in tensor libraries, tools, and data mining applications, which today include e-commerce, healthcare, security, and deep learning, to name a few.”

Vuduc said, “An example of this use comes from Jimeng Sun’s work where his team is trying to find structure in electronic health records (EHRs). There is always a way to organize the data along certain dimensions, for example, a patient could be considered one dimension, the diagnosis they receive would be a second, and their treatment a third.”

As one could imagine, the way in which these dimensions could be combined could yield endless outcomes. Which is why the ability to store data of this size in a way that respects order, quickly computes, and uses as minimal amount of storage as possible is a critically needed function.  

“Basically, the formats that existed already could give you two of those areas: All the competition could either have a small data structure which took less storage or if you wanted to access the data in a different order it could but would be slower,” said Li. “HiCOO breaks the mold of its predecessors by proposing a new storage format for sparse tensors [called Hierarchical COOrdinate], that accomplishes all three.”

Li currently works as a computer scientist at the High Performance Computing Group of Pacific Northwest National Laboratory (PNNL) in Richland, Washington. 

Read the PNNL press release of HiCOO here.

“Georgia Tech researchers are presenting 23 separate events this week, including four technical paper presentations, several workshops, panels, and even a doctoral showcase with CSE [School of Computational Science and Engineering] Ph.D. student Patrick Flick,” said Center for High Performance Computing Director and CSE Professor Rich Vuduc.  

One of the papers being presented, HiCOO: Hierarchical Storage of Sparse Tensors, by CSE recent graduate Jiajia Li and CSE Associate Professors Vuduc and Jimeng Sun was nominated as one of the five best student papers of 288 submissions in this year’s program.

Attendees can also stop by the Georgia Tech booth (#1217) to talk with faculty and students, take a moment to watch the live feed of fish from the Georgia Aquarium, and view additional Georgia Tech HPC research with the research slide deck. Swag items are also available, including Superheroes of Supercomputing Cards and Georgia Tech-branded bags.  

The complete Georgia Tech SC’18 proceedings, projects, and research slides can be viewed here.

See below for a list of Georgia Tech events at SC’18 (broken down by event type):

Workshops: 

• Hot Topics Discussion II: Thriving at Work
  • Lorna Rivera, Lucy Nowell, Carissa Holohan
• A Preliminary Study of Compiler Transformations for Graph Applications on the Emu System
  • AUTHOR/PRESENTERS: Prasanth Chatarasi, Vivek Sarkar
• A One Year Retrospective on a MOOC in Parallel, Concurrent, and Distributed Programming in Java
  • Vivek Sarkar, Max Grossman, Zoran Budimlic, Shams Imam
• Shortest Path and Neighborhood Subgraph Extraction on a Spiking Memristive Neuromorphic Implementation
  • Catherine Schuman, Kathleen Hamilton, Tiffany Mintz, Md Musabbir Adnan, Bon Woong Ku, Sung-Kyu Lim, Garrett S. Rose
• Automated Parallel Data Processing Engine with Application to Large-Scale Feature Extraction
  • Xin Xing, Bin Dong, Jonathan Ajo-Franklin, Kesheng Wu
• A Fast and Simple Approach to Merge and Merge Sorting Using Wide Vector Instructions
  • Alex Watkins, Oded Green
• A Unified Runtime for PGAS and Event-Driven Programming
  • Sri Raj Paul, Kun Chen, Akihiro Hayashi, Max Grossman, Vivek Sarkar
• A Study of OpenMP Device Offloading in LLVM: Correctness and Consistency
  • Lechen Yu

Birds of a Feather:

• 17th Graph500 List
  • Richard Murphy, David Bader, Peter Kogge, Andrew Lumsdaine, Anton Korzh
• Strategies for Inclusive and Scalable HPC Outreach and Education
  • Julie Mullen, Weronika Filinger, Tom Maiden, Nicholas Brown, Lorna Rivera, John Urbanic, Karina Nunez, Bryan Johnston, Karina Pesatova, Martin Quinson
• Advanced Architecture Testbeds: A Catalyst for Co-design Collaborations
  • Kevin Barker, Jeffrey Young, Alice Koniges, Jeffrey Vetter, James Laros
• HPC Graph Toolkits and GraphBLAS Forum
  • José Moreira (IBM), Antonino Tumeo (PNNL), Aydin Buluç (LBNL), Mahantesh Halappanavar (PNNL), John Feo (PNNL), David Bader
• An Update on the Next-Generation BLAS Proposal
  • Piotr Luszczek, Jack Dongarra, Jason Riedy, Greg Henry, James Demmel, Mark Gates, Xiaoye S. Li, Ping Tak P. Tang
• Workloads and Benchmarks for System Acquisition
  • Neil Bright, Laura Brown, Henry Neeman, Alex Younts

Papers

• Optimizing High Performance Distributed Memory Parallel Hash Tables for DNA k-mer Counting
  • Tony C. Pan, Sanchit Misra, Srinivas Aluru
• HiCOO: Hierarchical Storage of Sparse Tensors
  • Jiajia Li, Jimeng Sun, Rich Vuduc
• Accelerating Quantum Chemistry with Vectorized and Batched Integrals
  • Hua Huang, Edmond Chow
• Detecting MPI Usage Anomalies via Partial Program Symbolic Execution
  • Fangke Ye, Jisheng Zhao, Vivek Sarkar
• Poster: Modeling Single-Source Shortest Path Algorithm Dynamics to Control Performance and Power Tradeoffs
  • Sara Karamati, Jeffrey Young, Rich Vuduc

Panels

• Runtime for Exascale and Beyond: Convergence or Divergence?
  • Marc Snir, Vivek Sarkar, Pavan Balaji, Laxmikant Kale, Sean Treichler, Hartmut Kaiser, Raymond Namyst
• Convergence between HPC and Big Data: The Day After Tomorrow
  • Bilel Hadri, Sadaf Alam, Katie Antypas, David Bader, Dan Reed, Rio Yokota
• Students@SC: Making the Best of Your HPC Education
  • Rebecca Hartman, Elisa Heymann, Jesmin Jahan Tithi, Rich Vuduc, Kenneth Weiss, Xinghui Zhao

Doctoral Showcase

• Parallel and Scalable Combinatorial String and Graph Algorithms on Distributed Memory Systems
  • Patrick Flick, Srinivas Aluru

According to the DOE CSGF alumni listing, Hayashi is the first Georgia Tech recipient of this award since 2011 and the first-ever recipient of this award from the College of Computing. 

Since its establishment in 1991, the DOE CSGF has provided benefits and opportunities to students pursuing doctoral degrees in fields that use high-performance computing (HPC) to solve complex science and engineering problems. 

“Many fields at this time are suffering from a number of big data problems. HPC is not exempt from this issue,” said Hayashi. “Through my research, I am aiming to develop tools that alleviate issues for applications that produce data that is too large or too complex to be stored and analyzed with HPC.”

Hayashi’s focus in HPC examines applications in data analysis and emphasizes scalable algorithms and software for the mining, analysis, and compression of data that may be modeled by tensors and hypergraphs.  

“Specifically, my focus is on non-negative matrix factorization (NMF) and its variants, tensor factorizations and joint factorizations,” he said. “My goal is to demonstrate the usefulness of these methods combined with developing efficient implementations to allow researchers in various domains to utilize them in their work.”

CSE Professors Haesun Park and Rich Vuduc advise Hayashi.

“This is a very exciting and well-deserved opportunity,” said Vuduc. “What makes Koby’s proposed research, which is on graph and hypergraph clustering using NMF and tensor methods, especially impactful is his emphasis on interpretability, which is basically giving end-users analysis results that are easier to understand."

The DOE CSGF will provide Hayashi with a yearly stipend of $37,000, payment of full tuition and required fees during the appointment period, an academic allowance, up to four years of total support, and a 12-week practicum experience at one of the 21 DOE national laboratories or sites.

On modern multicore central processing units, the reordering algorithm Lexi-Order obtains up to 4.14 times speedup on sequential HiCOO-MTTKRP – a sparse tensor format with one of the most computationally expensive kernels in sparse tensor computations – and 11.88 times speedup on its parallel counterpart. 

The full findings from the paper introducing these methods will be presented this Thursday, June 27, at the International Conference on Supercomputing (ICS).

For those new to tensors, a tensor is an algebraic object related to space that provides a natural and concise mathematical framework for formulating and solving problems in computational science. These objects are represented in the form of cubes, with the X and Y axis depths ranging according to the size of data entries they can hold.

Tensors are fed information in the form of numeric data from a variety of applications but, oftentimes, these entries mainly consist of zeroes. In these instances, the tensors are considered sparse and do not need to be stored or explicitly computed on. 

The primary investigator of this research, Jiajia Li, is a staff scientist at the High Performance Computing (HPC) group of Pacific Northwest National Laboratory (PNNL) and recent doctoral graduate of CSE. Li has led research on sparse tensors in the past, including Supercomputing 2018 Best Student Paper Award winning work, HiCOO, which this new work builds upon.

[See Related: HiCOO Takes Home Best Student Paper Title of Supercomputing 2018 by Creating a New Storage Format]

“A sparse tensor is often a natural way to represent a multifactor or multi-relational dataset, and has found numerous applications in data analysis and mining for healthcare, natural language processing, machine learning, and social network analytics, among many others,” she said.

“This paper formalizes the problem of reordering a sparse tensor to improve the spatial and temporal locality of operations within it, and proposes two reordering algorithms for this problem.” 

According to Li, there are a variety of data structures and techniques for storing and operating efficiently with sparse tensors. One technique is to reorder the tensor, which means relabeling the indices – and, therefore, reorganizing the nonzero structure of the tensor.

One form of reordering explored in prior work is to sort the input tensor which can improve locality, but simultaneously aggravates load balance.

She said, “We observe that reordering increases imbalance by as much as 6.7 times in practice. In this paper, we propose improvements to the data structure and corresponding algorithms that address this problem. A new partition-based superblock scheduling is also proposed for the HiCOO format to improve load balance.”

The team of investigators for this paper include Li, PNNL Team Lead Scientist Kevin Barker, French National Center for Scientific Research Scientist Bora Ucar, and CSE Professors Umit Catalyurek, Jimeng Sun, and Rich Vuduc.

The code is released as part of Parallel Tensor Infrastructure (ParTI!) and can be found here.

The NVIDIA DGX-1 supercomputer is a specialty graphics processing unit (GPU)-based platform designed to facilitate faster and more efficient big data sequencing, machine learning (ML), and deep learning processes.

“There aren’t many computers or servers significant enough to be recognizable by model name. But, a DGX-1 is well-known throughout the computing community, particularly by the artificial intelligence and machine learning crowd,” said School of Computational Science and Engineering (CSE) Research Technologist Will Powell.

Although the DGX-1 is a well-known system, these high-profile computers are not normally accessible to the broader computational science community because of their cost, which is why this opportunity is unique for Georgia Tech students.

CSE won funding for the DGX-1 as part of a bid approved in this fiscal year’s campus Technology Fee Funding. This institute-level program is orchestrated by a committee comprised of faculty, staff, and students that review proposals from across campus to purchase various technologies for instruction and learning. 

“CSE strives to cultivate a diverse computing environment with ground-breaking technologies, such as the DGX-1. Providing students with the opportunity to use and study best-in-class, forward-thinking architectures allows us to further pioneer new computational methods,” said Powell.

The DGX-1 will become available for the Fall 2019 semester in the data and visual analytics classes (CX4242 and CSE6242) taught by Lecturer Mahdi Roozbahani and Associate Professor Polo Chau. Students in these classes will use this platform to accelerate computational analysis performed on large-scale real-world datasets.

“This is a great opportunity for students to have access to a high performance platform designed for artificial intelligence, ML, deep learning, and analytics,” said Joint CSE and NVIDIA Senior Graph Software Engineer Oded Green, whose research focuses on improving the performance and scalability of large-scale graph analytics using platforms such as the DGX-1.

The DGX-1 is powered by 8 NVIDIA TESLA V100 GPUs, has over 40,000 CUDA Cores, 5,000 Tensor cores, and 1,000 TFLOPS built specially for deep learning.

And while the DGX-1 arriving at Georgia Tech for student-use is exciting enough, there is cause for more celebration as a DGX Station also arrived this year as part of the new NVIDIA Artificial Intelligence (AI) Lab (NVAIL) grant awarded to CSE.

The NVAIL grant focuses on developing multi-GPU graph analytics and the DGX station is constructed specifically for data science and artificial intelligence development. The station will be used for graduate students and researchers associated with the NVAIL.

[Related News: Two New NVIDIA Collaborations Awarded to Georgia Tech with School of CSE’s Leadership]

“The program is looking at how do we come to a new paradigm of computing where running time isn’t necessarily the constraint, but how much power and battery that we have available is really the new constraint,” says David Bader, executive director of high-performance computing at the School of Computational Science and Engineering.

If the project is successful, it could result in computers far smaller and orders of magnitude more efficient than today’s machines. It could also mean that the computer mounted tomorrow on an unmanned aircraft or ground vehicle, or even worn by a soldier would use less energy than a larger device, while still being as powerful.

Georgia Tech’s part in the DARPA-led PERFECT effort is called GRATEFUL, which stands for Graph Analysis Tackling power-Efficiency, Uncertainty and Locality. Headed by Bader and co-investigator Jason Riedy, GRATEFUL focuses on algorithms that would process vast stores of data and turn it into a graphical representation in the most energy-efficient way possible.

The ultimate goal is to get an algorithmic framework that delivers supercomputer capabilities on a small, power-restricted platform.

One approach to reducing power consumption is to reduce the level of data collection. For example, when looking for a needle in a haystack, you don’t necessarily need to inspect every piece of hay. “What we’re looking at is collecting the minimal data necessary to make accurate decisions,” Bader says.

For now, the Tech team is applying GRATEFUL to social network analysis. But that same technology could also be used for any number of security applications, such as identifying hackers trying to break into a network. And, eventually, the technology developed under GRATEFUL could find its way onto smaller, more efficient computers in unmanned aerial vehicles or worn by soldiers.

The team is currently one year into a potentially five-year effort. Bader says most of the work is still in the elementary stages, but the team is developing proofs of concept software. “Our goal is to create architecture-independent software that can run across multiple hardware platforms and still perform extremely well,” he says.

The 2014 list is a compilation of the 16 best and brightest minds from academia, science, and technology whose contributions in high performance computing (HPC) have the potential to profoundly impact the world this year and beyond. Finalists are selected following an extensive review process by the HPCwire editorial and executive staff along with guidance from industry analysts and luminaries across the HPC community.

Although HPCwire has recognized Bader as a “person to watch” before, he continues to garner attention and recognition for his work in big data at the intersection where high performance computing meets real-world applications.

Among his many achievements, Bader led an effort that resulted in a nearly $2 million dollar grant for researchers at Georgia Tech and the University of Southern California to bring supercomputing capabilities into the grasp of tablets, smart phones, and other devices.

Read more about Bader and his thoughts on the trends in HPC for 2014.

Georgia Tech is close to choosing the project manager for its nearly 700,000-square-foot High Performance Computing Center. The decision could come in weeks, said John Majeroni, Georgia Tech’s executive director of real estate development. The proposed project is an expansion of Technology Square, an influential development that has helped anchor the southern end of Midtown at Spring and Fifth streets.

One of the most likely sites for the High Performance Computing Center is the historic Crum & Forster building. All that remains of that building is, basically, the facade. Preservationists wanted to keep the entire building. Georgia Tech said two-thirds of it needed to be demolished to make room for the computing center. Earlier this year, a consent order issued by a Fulton County Superior Court judge paved the way for plans to move ahead.

Other sites could be chosen, instead, Majeroni said. “We always consider other locations before a project is finalized,” he said.

While a project manager may be named in a matter of weeks, it could take at least a year to find out who wins the job as developer. The earliest Tech might issue requests for qualifications is next fall, Majeroni said. The developer would design, finance and own the project.

Tech would help anchor the building. The school would enter a long-term lease for about 50 percent of the space, and that could include Georgia Tech business functions. The space could also house data center operations, among other uses. Less is known about what kinds of tenants a developer might target for the project, though companies that depend on high-performance data and network applications and others that involve genetic modeling for biotech and medical firms are options. Three years ago, Princeton University launched a similar project. 

Technology Square is a more than 1 million-square-foot development Georgia Tech started 10 years ago that expanded the campus across the Downtown Connector. Technology Square spans eight blocks, and it has been a catalyst for new real estate development. The latest example: a 22-story tower to be developed by Gateway Development and South City Partners LLC. The residential tower is meant to add a missing piece to the overall project.

“Keeneland provides an important capability for the NSF computational science community,” says Jeffrey Vetter, Principal Investigator and Project Director, with a joint appointment to Georgia Tech's College of Computing and Oak Ridge National Laboratory. “Many users are running production science applications on GPUs with performance that would not be possible on other systems.”

Scientists will be able to use the resource to create breakthroughs in many fields of science. For the past 20 months, the Keeneland Initial Delivery System (KIDS) has been used for research in both computer science and computational science, and has included applications in astronomical sciences, atmospheric sciences, behavioral and neural sciences, biological and critical systems, materials research and mechanical and structural systems, along with many other application areas. Much of the research will continue on KFS.

Keeneland’s early users note how the system’s capabilities have significantly advanced their research application areas.

“The Infiniband communication is now fast enough so that I can run my program on more GPUs to achieve better performance,” says Jens Glaser, a post-doctoral associate in chemical engineering and materials science at the University of Minnesota. Glaser believes his research results demonstrate that the KFS' hardware is a significant step forward in supercomputing.

Astrophysics researcher Jamie Lombardi, an associate professor in the Department of Physics at Allegheny College, says Keeneland is easily the fastest system he has used. Lombardi uses his hydrodynamics code Starsmasher to simulate the collision and merger of two stars. The dynamics of the gas are parallelized on the CPU cores, while the gravity calculations are parallelized on the GPUs. 

“Running on one node of KFS is nearly a factor of three faster than running on one node of my local cluster,” says Lombardi. “The availability of such a large number of nodes on KFS makes it possible for me to run higher resolution simulations than I have ever run before.”

The Keeneland Full Scale System is a 615 TFLOPS HP Proliant SL250-based supercomputer with 264 nodes, where each node contains two Intel Sandy Bridge processors, three NVIDIA M2090 GPU accelerators, 32 GB of host memory, and a Mellanox InfiniBand FDR interconnection network. KFS has delivered sustained performance of over a quarter of a PetaFLOP (one quadrillion calculations per second) in initial testing. The system is space efficient in that it occupies about 400 square feet, including the space for in-row cooling and service areas.

During the KFS installation and acceptance testing, the initial delivery system, KIDS, was used to start production capacity for XSEDE users seeking to run their applications on the system and who had received allocations for Keeneland through a peer review process. KIDS was upgraded with newer GPUs and used for software and application development and for pre-production testing of codes that utilize the GPU accelerators in the Keeneland systems. Even before KFS began production, allocation requests for time greater than the total available for its lifecycle had been received from XSEDE application users.

“Our Keeneland Initial Delivery system has hosted over 130 projects and 200 users over the past two years,” says Vetter. “Requests for access to Keeneland have far outstripped the planned resource delivery, sometimes by as much as twice the availability.”

The Keeneland Project is a five-year Track 2D cooperative agreement, which was awarded by NSF under Contract OCI-0910735 in 2009 for the deployment of an innovative high performance computing system to the open science community. The Georgia Institute of Technology, University of Tennessee-Knoxville, the National Institute for Computational Sciences, and Oak Ridge National Laboratory manage the facility, perform education and outreach activities for advanced architectures, develop and deploy software tools for this class of architecture to ensure productivity, and team with early adopters to map their applications to Keeneland architectures.

To learn more about Keeneland or XSEDE, visit http://keeneland.gatech.edu or https://www.xsede.org/, respectively.

###

Contacts

Joshua Preston

Communications Officer

College of Computing at Georgia Tech

jpreston@cc.gatech.edu

678-231-0787

Biros and his 11 teammates, which included colleagues not only from Georgia Tech but also from Oak Ridge National Laboratory (ORNL) and New York University (NYU), created a blood-flow simulation of 260 million deformable red blood cells flowing in plasma. The team ran its application on ORNL’s Jaguar supercomputer. Using 196,000 of Jaguar’s 224,000 processor cores, the application pushed the machine to 700 trillion calculations per second, or 700 teraflops. The simulation amounted to successful resolution of 90 billion unknown dimensions in space, and topped the previous largest blood-flow simulation (of 14,000 cells) by four orders of magnitude.

“We put this team together to tackle the mathematical and computational challenges associated with blood-flow simulation, and while our research is an important step, it is only a first step, which we hope to expand upon in the coming years,” said Biros, who has a joint appointment in the Wallace H. Coulter Department of Biomedical Engineering. “Moving forward, I hope this award spotlights Georgia Tech’s research and contributions to high-performance computing. We are honored to have received the prize and are truly grateful for the recognition of our work by our colleagues.”

The simulation is described in a paper, "Petascale Direct Numerical Simulation of Blood Flow on 200K Cores and Heterogeneous Architectures,” presented at SC10. Georgia Tech team members included assistant professor Richard Vuduc; research technologist Logan Moon; adjunct scientist Ilya Lashuk; graduate students Abtin Rahimian and Aparna Chandramowlishwaran (both Ph.D. students in CSE), and Aashay Shringarpure (M.S. student in computer science); and former undergraduate intern Dhairya Malhotra. ORNL team members included Future Technologies group leader Jeffrey Vetter (who has a joint appointment in Georgia Tech's School of CSE) and postdoctoral researcher Rahul Sampath. Professor Dennis Zorin and postdoctoral researcher Shravan Veerapaneni were the NYU team members.

“We are very proud of George and all his teammates from such esteemed partner institutions as Oak Ridge National Lab and NYU,” said Zvi Galil, John P. Imlay Jr. Dean of Computing at Georgia Tech. “Supercomputing is the tool that will enable science to address some of the truly grand challenges of our time, from curing our most pernicious diseases to successfully predicting severe weather. At Georgia Tech we are committed to attracting the best high-performance computing researchers in the country—and giving them the tools they need to do their work.”

Part of what made the team’s accomplishment so impressive is that their application simulated not artificially spherical blood cells that retain their shape, but realistic cells that deform as they move through plasma. In announcing the award, Horst Simon, associate laboratory director for computing sciences at Lawrence Berkeley National Laboratory (and a former Bell Prize winner), called the simulation a “very challenging multiscale, multiphysics problem” and its successful execution “a very significant accomplishment.”

“We would like to thank the National Science Foundation and the Department of Energy for their support and for providing the computational resources that made these calculations possible,” Biros said. “Our long-term goal is to investigate the design of diagnostic microfluidic devices and develop a quantitative understanding of blood clotting mechanisms. The main challenge in modeling blood flow is resolving the hydrodynamic interactions between erythrocytes with the surrounding plasma.”

The Bell Prize, named for supercomputing pioneer Gordon Bell, has been awarded every year since 1987 in recognition of the world’s fastest supercomputing application. It carries a $10,000 cash prize for the winner.

###

About the Georgia Tech College of Computing

The Georgia Tech College of Computing is a national leader in the creation of real-world computing breakthroughs that drive social and scientific progress. With its graduate program ranked 10th nationally by U.S. News and World Report, the College’s unconventional approach to education is defining the new face of computing by expanding the horizons of traditional computer science students through interdisciplinary collaboration and a focus on human centered solutions. For more information about the Georgia Tech College of Computing, its academic divisions and research centers, please visit http://www.cc.gatech.edu.

Contacts

Michael Terrazas
Assistant Director of Communications
College of Computing at Georgia Tech
mterraza@cc.gatech.edu
404-245-0707