Using Trusted Execution Environments on High Performance Computing Platforms

Presentation Download Presentation Video

Many scientific domains, like genomic research or network diagnostics, require high-performance and reliable analysis of large, varied datasets. The analyzed data may be privacy sensitive as they might contain information regarding someone’s susceptibility to specific diseases (genome data) or communication patterns (network traffic).

Today, this analysis is performed on High Performance Computing (HPC) machines where multiple users may share computing resources. HPC is a unique environment for performing secure computation. Most HPC applications (including data analytics) require a complex version-dependent software stack, researchers are only granted access on HPC machines for limited time slots, and applications often use a large number of intercommunicating processes.

As an initial exploration of using trusted execution environments (TEEs) for HPC workloads, we analyze the performance of a suite of HPC applications on two commercially available TEEs, Intel’s SGX and AMD’s SEV, as shown below.

Performance of NPB, GAPBS, and LGBM on SGX and SEV

The figure above shows the geometric mean slowdown for three different sets of benchmarks on both platforms. We investigate traditional HPC workloads with the NAS parallel benchmarks suite (NPB), graph workloads with the GAPBS workloads running two different inputs, and a decision tree workload as a proxy for some types of machine learning training applications.

This figure shows that SEV has less slowdown compared to the native unsecure baseline than SGX. The main reason for this difference is that SGX has a limited secure memory (about 100 MB) and HPC workloads have large working set sizes. This forces the SGX runtime to move data between secure and unsecure memory which significantly impacts performance.

Digging deeper into the SEV performance, we believe most of the slowdown comes from the requirement that applications run in a virtualized environment. Virtualization causes a number of overheads for applications, and the large working set sizes of HPC applications exacerbate this.

In future work, we will explore the requirements for future TEEs to support HPC applications with lower overhead.

If you’re interested in more details or would like to work on this project, we are currently recruiting students to help us! Feel free to reach out to Jason (jlowepower@ucdavis.edu).

This is a collaboration between the DArchR group (Ayaz, Venkatesh, and Jason) at UC Davis and Lawrence Berkeley National Laboratory (Anna and Sean).

Ayaz Akram, Anna Giannakou, Venkatesh Akella, Jason Lowe-Power, and Sean Peisert. Using Trusted Execution Environments on High Performance Computing Platforms. Open-source Enclaves Workshop (OSEW 2019). 2019.

@inproceedings{hpc-enclaves:Akram:2019,
    booktitle = {Open-source Enclaves Workshop (OSEW 2019)},
    title = {Using Trusted Execution Environments on High Performance Computing Platforms},
    year = {2019},
    month = {july},
    day = {25},
    author = {Ayaz Akram and Anna Giannakou, Venkatesh Akella and Jason Lowe-Power and Sean Peisert}
}