The context-tree weighting method: basic properties

F.M.J. Willems, Yu.M. Shtarkov, T.J. Tjalkens

Research output: Contribution to journalArticleAcademicpeer-review

635 Citations (Scopus)
819 Downloads (Pure)


Describes a sequential universal data compression procedure for binary tree sources that performs the "double mixture." Using a context tree, this method weights in an efficient recursive way the coding distributions corresponding to all bounded memory tree sources, and achieves a desirable coding distribution for tree sources with an unknown model and unknown parameters. Computational and storage complexity of the proposed procedure are both linear in the source sequence length. The authors derive a natural upper bound on the cumulative redundancy of the method for individual sequences. The three terms in this bound can be identified as coding, parameter, and model redundancy, The bound holds for all source sequence lengths, not only for asymptotically large lengths. The analysis that leads to this bound is based on standard techniques and turns out to be extremely simple. The upper bound on the redundancy shows that the proposed context-tree weighting procedure is optimal in the sense that it achieves the Rissanen (1984) lower bound
Original languageEnglish
Pages (from-to)653-664
Number of pages12
JournalIEEE Transactions on Information Theory
Issue number3
Publication statusPublished - 1995


Dive into the research topics of 'The context-tree weighting method: basic properties'. Together they form a unique fingerprint.

Cite this