A worldwide flock of Condors : load sharing among workstation clusters

D.H.J. Epema, M. Livny, R. Dantzig, van, X. Evers, J. Pruyne

    Research output: Contribution to journalArticleAcademicpeer-review

    184 Citations (Scopus)

    Abstract

    Condor is a distributed batch system for sharing the workload of compute-intensive jobs in a pool of unix workstations connected by a network. In such a Condor pool, idle machines are spotted by Condor and allocated to queued jobs, thus putting otherwise unutilized capacity to efficient use. When institutions owning Condor pools cooperate, they may wish to exploit the joint capacity of their pools in a similar way. So the need arises to extend the Condor load-sharing and protection mechanisms beyond the boundaries of Condor pools, or in other words, to create a flock of Condors. Such a flock may include Condor pools connected by local-area networks as well as by wide-area networks. In this paper we describe the design and implementation of a distributed, layered Condor flocking mechanism. The main concept in this design is the Gateway Machine that represents in each pool idle machines from other pools in the flock and allows job transfers across pool boundaries. Our flocking design is transparent to the workstation owners, to the users, and to Condor itself. We also discuss our experiences with an intercontinental Condor flock.
    Original languageEnglish
    Pages (from-to)53-65
    JournalFuture Generation Computer Systems
    Volume12
    Issue number1
    DOIs
    Publication statusPublished - 1996

    Fingerprint

    Dive into the research topics of 'A worldwide flock of Condors : load sharing among workstation clusters'. Together they form a unique fingerprint.

    Cite this