|
FAST 2002 Abstract
Exploiting Inter-File Access Patterns Using Multi-Collective I/O
Gokhan Memik,
EE Dept.
UCLA;
Mahmut Kandemir,
CSE Dept.
Penn State;
Alok Choudhary
ECE Dept.
Northwestern University
Abstract
This paper introduces a new concept called Multi-Collective
I/O (MCIO) that extends conventional collective
I/O to optimize I/O accesses to multiple arrays simultaneously.
In this approach, as in collective I/O, multiple
processors co-ordinate to perform I/O on behalf of each
other if doing so improves overall I/O time. However, unlike
collective I/O, MCIO considers multiple arrays simultaneously;
that is, it has a more global view of the overall
I/O behavior exhibited by application. This paper shows
that determining optimal MCIO access pattern is an NP-complete
problem, and proposes two different heuristics
for the access pattern detection problem (also called the
assignment problem).
Both of the heuristics have been implemented within
a runtime library, and tested using a large-scale scientific
application. Our preliminary results show that MCIO out-performs
collective I/O by as much as 87%. Our runtime
library-based implementation can be used by users as well
as optimizing compilers. Based on our results, we recom-mend
future library designers for I/O-intensive applications
to include MCIO in their suite of optimizations.
- View the full text of this paper in
PDF. Until January 2003, you will need your USENIX membership identification in order to access the full papers.
The Proceedings are published as a collective work, © 2002 by the USENIX Association. All Rights Reserved. Rights to individual papers remain with the author or the author's employer. Permission is granted for the noncommercial reproduction of the complete work for educational or research purposes. USENIX acknowledges all trademarks within this paper.
- If you need the latest Adobe Acrobat Reader, you can download it from Adobe's site.
- To become a USENIX Member, please see our Membership Information.
|
Need help? Use our Contacts page.
