HTC News Shorts: January 2025

This feature centers on news updates from the Center For High Throughput Computing; ranging from the growing HTC international community to fostering new relationships with other American institutions to centering on the ethics of throughput computing use and distribution.

---

Graduate Students at Doshisha University, Japan, Learn about HTCondor

Ria Dhingra

Since attending the OSG Consortium’s OSG School, a week-long intensive open to researchers and facilitators for implementation of high throughput computing (HTC) methodologies, Professor Takahiro Koita has been introducing his students to the use of HTCondor and activities of the OSG Consortium. Inviting his students to learn the opportunities afforded by HTC, Takahiro is developing the next generation of HTC users and building their relationship with the Center for High Throughput Computing (CHTC).

A professor in the Graduate School of Engineering at Doshisha University, one of Japan’s oldest private institutions, Professor Koita regularly teaches a graduate seminar on Advanced Distributed Systems. In partnership with OSG Consortium, Koita’s class has access to HTC systems, allowing them to deepen their learning of CHTC developed and distributed software: HTCondor. This year, in a class of 58 students, he instructed the group about both HTCondor and the OSG Consortium. Furthermore, he encouraged his class to virtually attend and participate in HCT24, the annual Throughput Computing event held in Madison, WI by CHTC. Bringing together researchers, campuses, science collaborations, facilitators, administrators, government representatives, and professionals interested in high throughput computing, the HTC24 event was a way to engage with the throughput computing community and focused on themes ranging from cyberinfrastructure to AI and machine learning.

Professor Koita’s class made up a portion of the 250 remote participants who attended HTC24 and, together, represented one of the 122 institutions who attended and shared their insights of all things high throughput computing. Despite the international time difference, Koita’s graduate students attended lectures and events, logging in at 3:00 am Japan Standard Time (JST) to attend lectures such as “Providing Value to Campuses: OSG Campus Services” by Tim Cartwright and “Optimizing Cost and Performance: Best practices for Efficient HTCondor Workload Deployment in AWS” by Sudheendra Bhat. Their attendance of Professor Koita’s students is a testament to the international collaboration of the growing HTC community.

…

You can learn more about HTC24 here in our news recap story and learn more about the upcoming HTC25 conference to be held June 2-6, 2025, here.

---

Expanding the Implementation of HTCondor to the National Autonomous University of Mexico

Kallen Wank

The National Autonomous University of Mexico (UNAM) developed Grid UNAM as a project to optimize and consolidate computing across its academic entities. The result was the creation of a distributed computing infrastructure utilizing HTCondor as one of its key technologies. The Grid UNAM infrastructure allows sharing the computing and storage power through the integration of the HTCondor-CE for the orchestration and sharing of computing resources. UNAM’S computing infrastructure is designed to support cutting-edge research in fields such as astrophysics, data science, and artificial intelligence and leverages the powerful job scheduling and resource management capabilities of HTCondor.

HTCondor, an open-source software widely used in computing, serves as the backbone for managing tasks across computing environments in Grid UNAM. Its ability to efficiently distribute workloads allows researchers to run complex simulations and analyses without worrying about system compatibility.

Grid UNAM represents a step forward for scientific computing at UNAM, that offers researchers a reliable, scalable platform for high-performance tasks, and also provides flexibility to collaborate across disciplines.

…

Read about this project of the Computing and Information and Communication Technologies, National Autonomous University of Mexico article here.

---

A Discussion of Inclusivity in Computing: Fostering New Relationships at SC24

Ria Dhingra

Discussing democratizing data access and and usage in the world of high throughput computing (HTC) and high processing computing *(HPC), Professor Michela Taufer, (University of Tennessee, Knoxville) organized at **SC24 a panel entitled “Creating Inclusive Scientific Communities in the Information Age: Can Equity Democratize Data-Intensive Research, from Experimental Facilities and HPC to Scientific Discovery?” The panel discussion focused on efforts to advance equity and empower minority serving institutions with computational facilities and data access.

Miron Livny, founding Director of CHTC, a group that develops advanced HTC software suite HTCondor, was invited to speak in this panel discussion alongside scholars Joel D. Brock (Cornell University), Amy Roberts (University of Colorado Denver), Valerio Pascucci (University of Utah), and Amber Boehnlein (Thomas Jefferson National Accelerator Facility). Serving as an oppositional voice to HPC approaches in an HPC conference environment, Livny shared computing alternatives and the benefits of HTC and its advantages in reaching individual researchers from smaller universities with less computing access.

Also in support of equity and access, Livny discussed the Pelican Platform Project, a data transfer service, Pelican facilitates the ability for researchers to access and transfer data from where it is stored to where it needs to run. Inspired by the Obama administration’s proposition to make all government funded research data publicly accessible, Pelican works to efficiently transfer this open data to researchers who require it.

In discussing the work of CHTC such as Pelican and HTCondor, Livny engaged in group discussion that showcased projects such as the National Science Data Fabric and the High-Performance Data Facility Hub. These initiatives pave the way for an inclusive future in computing research, and the panel served as a place for opposing viewpoints to consider common goals and individual shortcomings.

Additionally, Livny’s experience of participating in SC Atlanta resulted in exciting collaboration for the CHTC communications team. During the Q&A position of the panel, National Data Science Fellow, Michael Sutherhlin (University of Tennessee) proposed the idea of considering narratives, real user stories, as a metric for evaluating how equitable a computing project/resource is.

In response to Sutherlin’s inquisition and a discussion following the panel, Livny put Sutherlin in touch with the CHTC communications team to have a conversation that considered aspects of public good regarding the impact of the work done here at CHTC.

As the goal of HTC as a methodology is to cluster computing resources, collaborations across institutions mimics this form of computing; with the SC Atlanta experience broadening a collective network that continues to further research computing.

…

*Established in 1988, the International Conference for High Performance Computing, Networking, Storage, and Analysis is an environment to discuss innovation in the field of High Processing Computing (HPC).

**HPC is a form of computing that consists of running complex, large-scale, tasks on a computing resource that relies on multiple processors (i.e. a supercomputer). By contrast, High Throughput Computing (HTC) is a form of computing that parallelly executes a large number of simple computational tasks, splitting a large job into simultaneously running smaller ones. The approaches differ in how they handle aspects such as workload, cost, efficiency, processing power, resource management, fault tolerance, accessibility, and scalability.