An evaluation of the close-to-files processor and data co-allocation policy in multiclusters

H.H. Mohamed, D.H.J. Epema

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    66 Citations (Scopus)
    95 Downloads (Pure)

    Abstract

    In multicluster systems, and more generally, in grids, jobs may require coallocation, i.e., the simultaneous allocation of resources such as processors and input files in multiple clusters. While such jobs may have reduced runtimes because they have access to more resources, waiting for processors in multiple clusters and for the input files to become available in the right locations may introduce inefficiencies. In previous work, we have studied through simulations only processor coallocation. Here, we extend this work with an analysis of the performance in a real testbed of our prototype processor and data coallocator with the close-to-files (CF) job-placement algorithm. CF tries to place job components on clusters with enough idle processors which are close to the sites where the input files reside. We present a comparison of the performance of CF and the worst-fit job-placement algorithm, with and without file replication, achieved with our prototype. Our most important findings are that CF with replication works best, and that the utilization in our testbed can be driven to about 80%.
    Original languageEnglish
    Title of host publicationProceedings of the IEEE International Conference on Cluster Computing (ICCC'04, San Diego CA, USA, September 20-23, 2004)
    PublisherInstitute of Electrical and Electronics Engineers
    Pages287-298
    ISBN (Print)0-7803-8694-9
    DOIs
    Publication statusPublished - 2004

    Fingerprint

    Dive into the research topics of 'An evaluation of the close-to-files processor and data co-allocation policy in multiclusters'. Together they form a unique fingerprint.

    Cite this