A Compression-Based Digital Library

Abstract

The explosive growth in digital information places an ever- increasing burden on the library mechanisms used to store and access it. A wide range of technologies must be developed to manage this flood of data. For example, we need ways of using storage effectively and economically, mechanisms for indexing, and methods for fast content-based querying. In this article we consider these three issues from an implementation point of view, showing how appropriate data compression techniques can be harnessed to provide improved solutions in all three areas. To set a context for the development, we discuss the New Zealand digital library initiative, a web-based tool that provides many different services, the largest of which indexes in excess of 40,000 technical documents drawn from sources around the world.

http://dx.doi.org/10.14429/dbit.17.6.3242