photo by Michael Miller

Terascala Launches Product to Make HPC Workflows More Efficient

BOSTON, Mass., Oct. 8 -- Terascala, the fast data company, today announced the availability of the Terascala Intelligent Storage Bridge (ISB), a high-performance workflow manager designed to transfer large data sets quickly and automatically between fast scratch storage and enterprise storage. The ISB is the first workflow manager that is purpose-built for application users working in high-performance computing (HPC) environments that use the Lustre parallel file system and need direct access to project data at any point in the application workflow.

The rate at which technical data is collected grows larger every day. Incorporating large volumes of data into research workflows requires many iterations and causes extended application run times and longer time to insight. Users routinely work with gigabyte and terabyte-size active data sets that need to be shared among colleagues now and preserved by the organization later. However, data movement can be tedious, time consuming and error-prone—even when handled by an HPC administrator. It can take days to manually mount source and destination directories, copy files, and move large data sets when using a general-purpose server or workstation. Even script-based automation requires continuous maintenance and monitoring.

The Terascala ISB solves these challenges by automatically moving large data sets on and off scratch storage using the maximum available network, compute and storage resources. The data movement is parallelized, saving time as well as costs because organizations no longer need to over-provision the infrastructure in order to compensate for slow data transfer rates. In addition, the ISB is the first workflow manager that puts primary control of the data in the hands of the user rather than the HPC administrator, removing the need for data management expertise through wizard-based tools and user-defined automation policies.

“Our users are most efficient when they have the data they need, when they need it,” said Kevin Campbell, Director of HPC at TGen. “The Intelligent Storage Bridge does just that—it gives our user community a tool to automate data movement in and out of our fast scratch environment for long-term storage and protection. It’s a better use of our resources because it lets us keep scratch for what it’s meant for—processing high-performance workflows.”

“Because it is difficult to tie departmental HPC environments to IT, we often find that research data ends up stranded within the workgroup,” said Terri McClure, Senior Analyst at Enterprise Strategy Group. “That leaves it untracked, unprotected, and more importantly the company can’t leverage it for other projects. Terascala’s ISB makes that connection and can help users centralize data within, across and between departments to make it easily available for new or shared projects, ultimately speeding the time to results.”

The Terascala ISB helps organizations: 

  • Manage data transfer tasks more easily: The ISB user interface is fully integrated into the Terascala Operating System (TeraOS) portal. It gives users the ability to create and monitor the progress of their data transfer tasks, and gives administrators the information they need to ensure that resources are optimally allocated between data transfer and compute jobs.
  • Save time, speed processes: Intuitive wizard-based menus allow users and administrators to create and manage jobs and view job history, accelerating and automating the data management process. Polices can be defined by time, event, or file age.
  • Ensure high availability and redundancy: In the event of an ISB node failure, other nodes will automatically assume the workload, minimizing the impact a hardware failure can have on the workflow. Administrators are automatically notified about all hardware failures. 

Once confined to large academic and government laboratories, HPC is becoming more prevalent in small and medium-size companies as compute-intensive applications for genomics sequencing, seismic processing, and product design are more readily available. For HPC to be effective in the enterprise, organizations must optimize their entire HPC workflow by adopting a holistic approach that seamlessly integrates high-performance solutions, such as those provided by Terascala.

“The Terascala Intelligent Storage Bridge fulfills our vision of optimizing HPC workflows for faster time to results,” said Steve Butler, CEO of Terascala. “The ISB eliminates the chasm that exists between HPC and enterprise storage. Now, those two storage environments are connected, which is vital for project data sharing, long-term data protection and regulatory compliance.”

The ISB is integrated with Terascala-powered high-performance storage appliances and is managed via a single user interface. Sold by partners and through Terascala directly, the entry-level ISB is packaged as a three-node, highly available cluster. Additional nodes can be added to achieve higher performance.

About Terascala 

Terascala is the fast data company. Terascala storage appliances dramatically accelerate the time to insight for organizations that rely on simulation, analysis, and modeling tools to bring new products and innovation to market. Exclusively available through strategic partners Dell and NetApp, Terascala storage appliances provide on-demand throughput at multiple gigabytes per second while leveraging industry-leading storage platforms for long-term data protection. Learn more at


Source: Terascala