Data Warehouse Appliances for Mesh Analysis

2010-01-07 Thu
io mesh pub

We published an unclassified unlimited release (UUR) paper and technical report comparing different storage-focused analytic platforms.

Abstract

As scientific computing users migrate to petaflop platforms that promise to generate multi-terabyte datasets, there is a growing need in the community to be able to embed sophisticated data analysis algorithms in the storage systems for the computing platforms. Data Warehouse Appliances (DWAs) are an attractive option for this work, due to their ability to process massive datasets efficiently. While DWAs have been proven effective in data mining and informatics applications, there are relatively few examples of how DWAs can be integrated into the scientific computing workflow. In this paper we present our experiences in adapting two mesh analysis algorithms to function on two different DWAs: a SQL-based Netezza database appliance and a Map/Reduce-based Hadoop cluster. The main contribution of this work is insight into the differences between the two platforms' programming environments. In addition, we present performance measurements for entry-level DWAs to help provide a first-order comparison of the hardware.

Publications

Long Road to Reconfigurable Supercomputing

2009-09-01 Tue
fpga pub

We published an unclassified unlimited release (UUR) paper.

Abstract

The field of high performance computing (HPC) currently abounds with excitement about the potential of a broad class of things called accelerators. And, yet, few accelerator based systems are being deployed in general purpose HPC environments. Why is that? This article explores the challenges that accelerators face in the HPC world, with a specific focus on FPGA based systems. We begin with an overview of the characteristics and challenges of typical HPC systems and applications and discuss why FPGAs have the potential to have a significant impact. The bulk of the article is focused on twelve specific areas where FPGA researchers can make contributions to hasten the adoption of FPGAs in HPC environments.

Publications

Service Node Proxies

2009-05-06 Wed
hpc io pub

We published an unclassified unlimited release (UUR) paper.

Abstract

Partitioning massively parallel supercomputers into service nodes running a full-fledged OS and compute nodes running a lightweight kernel has many well-known advantages but renders it difficult to access externally located resources such as high-performance databases that may only communicate via TCP. We describe an implementation of a proxy service that allows service nodes to act as a relay for SQL requests issued by processes running on the compute nodes. This implementation allows us to move toward using HPC systems for scalable informatics on large data sets that simply cannot be processed on smaller machines.

Publications

Presentations

Feature Characterization Library (FCLib)

2008-11-08 Sat
mesh code pub

We published an unclassified unlimited release (UUR) technical report and received permission to release the software.

Publications

Code

Threading Opportunities in Flash-Memory

2008-09-24 Wed
io pub

We presented this unclassified unlimited release (UUR) poster/paper.

Publications

Presentations