Abstract
Distributed Web crawlers are extensively used for Web mining nowadays, but their accuracy, dependability and other operational measures have not been fully studied. Distributed Web crawlers are costly and require careful selection of configuration parameters. It is important to have some estimation about the performance, dependability and accuracy of a Web crawler. This paper presents a model-based evaluation of the accuracy and availability of a distributed Web crawler whose architecture is based on UbiCrawler. Stochastic activity networks are used for modelling the crawler. Accuracy and availability of the Web crawler are formally defined, and the effects of environmental failure rates on crawling nodes and on the availability of the whole system are discussed.
| Original language | English |
|---|---|
| Title of host publication | Proceedings - EMS 2008, European Modelling Symposium, 2nd UKSim European Symposium on Computer Modelling and Simulation |
| Publisher | Institute of Electrical and Electronics Engineers |
| Pages | 453-458 |
| Number of pages | 6 |
| ISBN (Print) | 9780769533254 |
| DOIs | |
| Publication status | Published - 2008 |
| Externally published | Yes |
| Event | EMS 2008, European Modelling Symposium, 2nd UKSim European Symposium on Computer Modelling and Simulation - Liverpool, United Kingdom Duration: 8 Sept 2008 → 10 Sept 2008 |
Conference
| Conference | EMS 2008, European Modelling Symposium, 2nd UKSim European Symposium on Computer Modelling and Simulation |
|---|---|
| Country/Territory | United Kingdom |
| City | Liverpool |
| Period | 8/09/08 → 10/09/08 |