Mark B. Ring, Ph.D.

I am currently a Visiting Professor at the University of Alberta, working in the RLAI lab with Rich Sutton on Reinforcement Learning. I am also an adjunct faculty member at Chapman University where I occasionally teach a class on Musical Acoustics.

I am also co-owner of Alto Research, the company that developed the vocolo, the musical synthesizer played with the voice. One version of it was manufactured and marketed as "The Saxxy" by Sharper Image in the Fall of 2003 (other versions have recently been licensed to other manufacturers).

Last update: September 29, 2005

Research Interests:

My research tends to revolve around a single focus: Continual Learning—the unending quest to learn and develop, to comprehend and interract with the world increasingly well, and thus to achieve reward increasingly well. My dissertation, Continual Learning in Reinforcement Environments, explored this and related issues in depth. Although many ideas discussed in the dissertation have recently become increasingly fashionable, at the time of their publication, much of the work was breaking new ice in uncharted waters.

The Temporal Transition Hierarchies (TTH) algorithm, when seen from the current moment looking backwards, was the first temporal function approximator that intelligently and incrementally increased history length to resolve contradictions. When seen from the current moment looking forward, TTH's form expectation hierarchies, and as such represent a (perhaps unwitting) predecessor to predictive state representations (PSRs) in that they explicitly encode all future action and observation trajectories as contingencies over intervening actions and observations. The algorithm, which was also presented as an incremental method for learning FSAs (NIPS 5, 1992), was first and foremost intended for use in reinforcement environments (SAB 2, 1992).

Recent papers:

Eddie J. Rafols, Mark B. Ring, Richard S. Sutton, Brian Tanner, Using Predictive Representations to Improve Generalization in Reinforcement Learning, Proceedings of the 19^th International Joint Conference on Artificial Intelligence, 2005. Abstract | Bibtex

[pdf]

Draft: Toward a formal framework for Continual Learning, post-NIPS workshop on Inductive Transfer, 2005.
[pdf]

Older papers :

RCC Cannot Compute Certain FSA, Even with Arbitrary Transfer Functions, from Advances in Neural Information Processing Systems 10 (NIPS 10), 1997. Abstract | Bibtex
[pdf] [ps] [ps.Z]

CHILD: A First Step Towards Continual Learning, Machine Learning Journal, vol. 28, 1997. Also appears as Chapter 11 in Learning to Learn, S. Thrun and L. Pratt, editors. Abstract | Bibtex
[pdf] [ps] [ps.gz]

Finding Promising Exploration Regions by Weighting Expected Navigation Costs, GMD Technical Report, Arbeitspapiere der GMD 987, April, 1996. Abstract | Bibtex
[pdf] [ps] [ps.Z]

Finding promising exploration regions by weighting expected navigation costs, working notes for talk given at the AAAI symposium on Active Learning, 1995. Abstract
[pdf] [ps] [ps.Z]

Continual Learning in Reinforcement Environments, University of Texas at Austin dissertation, 1994. See my dissertation page for more information. Abstract | Bibtex
[pdf]

Sequence Learning with Incremental Higher-Order Neural Networks, University of Texas at Austin AI lab technical report, 1993. Abstract | Bibtex
[pdf] [ps] [ps.Z]

Learning Sequential Tasks by Incrementally Adding Higher Orders, from Advances in Neural Information Processing Systems 5 (NIPS5), 1992. Abstract | Bibtex
[pdf] [ps] [ps.Z]

Two Methods for Hierarchy Learning in Reinforcement Environments, in From Animals to Animats 2: Proceedings of the Second International Conference on Simulation of Adaptive Behavior (SAB '92), 1992. Abstract | Bibtex

[pdf] [ps] [ps.Z]

Incremental Development of Complex Behaviors through Automatic Construction of Sensory-motor Hierarchies, from the proceedings of the Eighth International Workshop (ML91), 1991. Abstract | Bibtex

[ps] [ps.Z]

Contact Information

Mark B. Ring, Ph.D.

683 S. Glenhurst Dr.

Anaheim Hills, CA 92808

Phone: (714) 974-0123

Email: "ring" (use host: cs.utexas.edu)

Mark B. Ring