A scalable proxy cache for grid data access

C.T. Cirstea, J.J. Keijser, O.A. Koeroo, R. Starink, J.A. Templon

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    1 Citation (Scopus)
    1 Downloads (Pure)


    We describe a prototype grid proxy cache system developed at Nikhef, motivated by a desire to construct the first building block of a future https-based Content Delivery Network for grid infrastructures. Two goals drove the project: firstly to provide a "native view" of the grid for desktop-type users, and secondly to improve performance for physics-analysis type use cases, where multiple passes are made over the same set of data (residing on the grid). We further constrained the design by requiring that the system should be made of standard components wherever possible. The prototype that emerged from this exercise is a horizontally-scalable, cooperating system of web server / cache nodes, fronted by a customized webDAV server. The webDAV server is custom only in the sense that it supports http redirects (providing horizontal scaling) and that the authentication module has, as back end, a proxy delegation chain that can be used by the cache nodes to retrieve files from the grid. The prototype was deployed at Nikhef and tested at a scale of several terabytes of data and approximately one hundred fast cores of computing. Both small and large files were tested, in a number of scenarios, and with various numbers of cache nodes, in order to understand the scaling properties of the system. For properly-dimensioned cache-node hardware, the system showed speedup of several integer factors for the analysis-type use cases. These results and others are presented and discussed
    Original languageEnglish
    Title of host publicationProceedings of the International Confertence on Computing in High Energy and Nuclear Physics 2012 (CHEP 2012), 21-25 May 2012, New York
    Place of PublicationBristol
    PublisherInstitute of Physics
    Publication statusPublished - 2012

    Publication series

    NameJournal of Physics: Conference Series
    Volume396, Pt 3
    ISSN (Print)1742-6588


    Dive into the research topics of 'A scalable proxy cache for grid data access'. Together they form a unique fingerprint.

    Cite this