Performance modeling of a distributed web crawler using stochastic activity networks

Mitra Nasri; Saeed Shariati; Mohammad Abdollahi Azgomi

doi:10.1007/978-3-540-89985-3_66

Performance modeling of a distributed web crawler using stochastic activity networks

Mitra Nasri, Saeed Shariati, Mohammad Abdollahi Azgomi

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

4 Citations (Scopus)

Abstract

One of the basic requirements of Web mining is a crawler system, which collects the information from the Web. To predict the performance, dependability and other operational measures of a system, it is required to construct and evaluate a formal model of the system. We have constructed a formal model for a distributed crawler, which is based on UbiCrawler, using stochastic activity networks (SANs). The constructed SAN model is used to evaluate some performance measures of the crawler. The results of the evaluation of throughput are same as the published statistics of UbiCrawler. In addition, we have been able to evaluate two other measures that are communication overhead and coverage. In this paper, we will discuss the architecture of the distributed crawler. Then, we will present a SAN model of the crawler and the results of its evaluation.

Original language	English
Title of host publication	Advances in Computer Science and Engineering
Subtitle of host publication	13th International CSI Computer Conference, CSICC 2008 Kish Island, Iran, March 9-11, 2008 Revised Selected Papers
Editors	Hamid Sarbazi-Azad
Publisher	Springer
Pages	535-542
Number of pages	8
ISBN (Print)	3540899847, 9783540899846
DOIs	https://doi.org/10.1007/978-3-540-89985-3_66
Publication status	Published - 2008
Externally published	Yes
Event	13th International Computer Society of Iran Computer Conference on Advances in Computer Science and Engineering, CSICC 2008 - Kish Island, Iran, Islamic Republic of Duration: 9 Mar 2008 → 11 Mar 2008

Publication series

Name	Communications in Computer and Information Science
Volume	6 CCIS
ISSN (Print)	1865-0929

Conference

Conference	13th International Computer Society of Iran Computer Conference on Advances in Computer Science and Engineering, CSICC 2008
Country/Territory	Iran, Islamic Republic of
City	Kish Island
Period	9/03/08 → 11/03/08

Keywords

performance modeling
stochastic activity networks
Web crawler

Access to Document

10.1007/978-3-540-89985-3_66

Cite this

Nasri, M., Shariati, S., & Abdollahi Azgomi, M. (2008). Performance modeling of a distributed web crawler using stochastic activity networks. In H. Sarbazi-Azad (Ed.), Advances in Computer Science and Engineering: 13th International CSI Computer Conference, CSICC 2008 Kish Island, Iran, March 9-11, 2008 Revised Selected Papers (pp. 535-542). (Communications in Computer and Information Science; Vol. 6 CCIS). Springer. https://doi.org/10.1007/978-3-540-89985-3_66

Nasri, Mitra ; Shariati, Saeed ; Abdollahi Azgomi, Mohammad. / Performance modeling of a distributed web crawler using stochastic activity networks. Advances in Computer Science and Engineering: 13th International CSI Computer Conference, CSICC 2008 Kish Island, Iran, March 9-11, 2008 Revised Selected Papers. editor / Hamid Sarbazi-Azad. Springer, 2008. pp. 535-542 (Communications in Computer and Information Science).

@inproceedings{17927ae751d2433783ba376b3ca89d27,

title = "Performance modeling of a distributed web crawler using stochastic activity networks",

abstract = "One of the basic requirements of Web mining is a crawler system, which collects the information from the Web. To predict the performance, dependability and other operational measures of a system, it is required to construct and evaluate a formal model of the system. We have constructed a formal model for a distributed crawler, which is based on UbiCrawler, using stochastic activity networks (SANs). The constructed SAN model is used to evaluate some performance measures of the crawler. The results of the evaluation of throughput are same as the published statistics of UbiCrawler. In addition, we have been able to evaluate two other measures that are communication overhead and coverage. In this paper, we will discuss the architecture of the distributed crawler. Then, we will present a SAN model of the crawler and the results of its evaluation.",

keywords = "performance modeling, stochastic activity networks, Web crawler",

author = "Mitra Nasri and Saeed Shariati and {Abdollahi Azgomi}, Mohammad",

year = "2008",

doi = "10.1007/978-3-540-89985-3_66",

language = "English",

isbn = "3540899847",

series = "Communications in Computer and Information Science",

publisher = "Springer",

pages = "535--542",

editor = "Hamid Sarbazi-Azad",

booktitle = "Advances in Computer Science and Engineering",

note = "13th International Computer Society of Iran Computer Conference on Advances in Computer Science and Engineering, CSICC 2008 ; Conference date: 09-03-2008 Through 11-03-2008",

}

Nasri, M, Shariati, S & Abdollahi Azgomi, M 2008, Performance modeling of a distributed web crawler using stochastic activity networks. in H Sarbazi-Azad (ed.), Advances in Computer Science and Engineering: 13th International CSI Computer Conference, CSICC 2008 Kish Island, Iran, March 9-11, 2008 Revised Selected Papers. Communications in Computer and Information Science, vol. 6 CCIS, Springer, pp. 535-542, 13th International Computer Society of Iran Computer Conference on Advances in Computer Science and Engineering, CSICC 2008, Kish Island, Iran, Islamic Republic of, 9/03/08. https://doi.org/10.1007/978-3-540-89985-3_66

Performance modeling of a distributed web crawler using stochastic activity networks. / Nasri, Mitra; Shariati, Saeed; Abdollahi Azgomi, Mohammad.
Advances in Computer Science and Engineering: 13th International CSI Computer Conference, CSICC 2008 Kish Island, Iran, March 9-11, 2008 Revised Selected Papers. ed. / Hamid Sarbazi-Azad. Springer, 2008. p. 535-542 (Communications in Computer and Information Science; Vol. 6 CCIS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Performance modeling of a distributed web crawler using stochastic activity networks

AU - Nasri, Mitra

AU - Shariati, Saeed

AU - Abdollahi Azgomi, Mohammad

PY - 2008

Y1 - 2008

N2 - One of the basic requirements of Web mining is a crawler system, which collects the information from the Web. To predict the performance, dependability and other operational measures of a system, it is required to construct and evaluate a formal model of the system. We have constructed a formal model for a distributed crawler, which is based on UbiCrawler, using stochastic activity networks (SANs). The constructed SAN model is used to evaluate some performance measures of the crawler. The results of the evaluation of throughput are same as the published statistics of UbiCrawler. In addition, we have been able to evaluate two other measures that are communication overhead and coverage. In this paper, we will discuss the architecture of the distributed crawler. Then, we will present a SAN model of the crawler and the results of its evaluation.

AB - One of the basic requirements of Web mining is a crawler system, which collects the information from the Web. To predict the performance, dependability and other operational measures of a system, it is required to construct and evaluate a formal model of the system. We have constructed a formal model for a distributed crawler, which is based on UbiCrawler, using stochastic activity networks (SANs). The constructed SAN model is used to evaluate some performance measures of the crawler. The results of the evaluation of throughput are same as the published statistics of UbiCrawler. In addition, we have been able to evaluate two other measures that are communication overhead and coverage. In this paper, we will discuss the architecture of the distributed crawler. Then, we will present a SAN model of the crawler and the results of its evaluation.

KW - performance modeling

KW - stochastic activity networks

KW - Web crawler

UR - http://www.scopus.com/inward/record.url?scp=78449236991&partnerID=8YFLogxK

U2 - 10.1007/978-3-540-89985-3_66

DO - 10.1007/978-3-540-89985-3_66

M3 - Conference contribution

AN - SCOPUS:78449236991

SN - 3540899847

SN - 9783540899846

T3 - Communications in Computer and Information Science

SP - 535

EP - 542

BT - Advances in Computer Science and Engineering

A2 - Sarbazi-Azad, Hamid

PB - Springer

T2 - 13th International Computer Society of Iran Computer Conference on Advances in Computer Science and Engineering, CSICC 2008

Y2 - 9 March 2008 through 11 March 2008

ER -

Nasri M, Shariati S, Abdollahi Azgomi M. Performance modeling of a distributed web crawler using stochastic activity networks. In Sarbazi-Azad H, editor, Advances in Computer Science and Engineering: 13th International CSI Computer Conference, CSICC 2008 Kish Island, Iran, March 9-11, 2008 Revised Selected Papers. Springer. 2008. p. 535-542. (Communications in Computer and Information Science). doi: 10.1007/978-3-540-89985-3_66

Performance modeling of a distributed web crawler using stochastic activity networks

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this