Many important applications require local access to extensive portions of the web. . Applications such as these usually access, mine and index a local cache or repository of web pages since performing their analyses directly on the web would be too slow. The repository receives web pages from a crawler, which is the component responsible for mechanically finding new or modified pages on the web. A web hosted repository also offers applications an access interface so that they may efficiently access large numbers of up-to-date web pages.
A web hosted repository stores and manages a large collection of data objects such as web pages. Theoretically it is not that different from other systems that store data objects, however, a web repository is not required to provide a lot of the functionality that the other systems provide, such as transactions, or a general directory naming structure. Therefore, the web repository can be optimized to provide just the essential services, and to provide them in a scalable and efficient way. Due to the size of the web, it is extremely important that the repository scale a vast number of objects.web hosted repositoryonline document managementdocument
↧