2018-11-30 Fri
faodel code
We received DOE approval to relase version 1.1811.1 ("DIO") of FAODEL on Github.
Release Improvements
- Kelpie has rank-folding table (RFT) pool
- Kelpie has new experimental IOMs (IomHDF5, IomLevelDB)
- New MPISyncStart service for auto-filling node ids in configurations
- DirMan is now its own service (previously a child of opbox)
- Lunasa has templated containers for storing a bundle of items in an LDO. The Generic(Sequential|Random)DataBundle classes are optimized for cases where data is accessed (in order|out of order).
- Users can now define webhooks for rendering specific DataObject types
- Bootstrap Start/Stops are much more robust
- General build fixes for use with EMPIRE
Significant User Visible Changes
- Common and services directories renamed to faodel-common and faodel-services
- Kelpie pools now have a "behavior" that controls how data is copied
- Some components/tools changed directories. Check your includes
Update
During the process of getting FAODEL integrated into Spack, there were some minor bug fixes pushed out in a 1.1811.2 release.
2018-10-10 Wed
faodel hpc io pub
We published an unclassified unlimited release (UUR) technical report.
Publications
- SAND Report Matt Bettencourt et. al., "ASC ATDM Level 2 Milestone #6358: Assess Status of Next Generation Components and Physics Models in EMPIRE" SAND2018-10100
2018-06-11 Mon
faodel pub
We published an unclassified unlimited release (UUR) paper.
Abstract
Composition of computational science applications, whether into ad hoc pipelines for analysis of simulation data or into well-defined and repeatable workflows, is becoming commonplace. In order to scale well as projected system and data sizes increase, developers will have to address a number of looming challenges. Increased contention for parallel filesystem bandwidth, accomodating in situ and ex situ processing, and the advent of decentralized programming models will all complicate application composition for next-generation systems. In this paper, we introduce a set of data services, Faodel, which provide scalable data management for workflows and composed applications. Faodel allows workflow components to directly and efficiently exchange data in semantically appropriate forms, rather than those dictated by the storage hierarchy or programming model in use. We describe the architecture of Faodel and present preliminary performance results demonstrating its potential for scalability in workflow scenarios.
Publications
- IWAC Paper Craig Ulmer, Shyamali Mukherjee, Gary Templet, Scott Levy, Jay Lofstead, Patrick Widener, Todd Kordenbrock, and Margaret Lawson, "Faodel: Data Management for Next-Generation Application Workflows", ScienceCloud'18, June 2018.
we received DOE approval to relase version 1.1803.1 ("Cachet") of faodel on github.
Code
The code is now hosted at github:
2018-02-05 Mon
net systems pub
We presented an unclassified unlimited release (UUR) poster.
Poster