ReaX—towards a foundation for multimodal, multimedia applications
Solving a universal user interface problem, the ReaX programming language forces the buttons
Modern user interfaces are often needlessly irking to use, simply because the prosaic problem of providing consistent and workable undo and seek mechanisms have been ignored by programmers. Speech enabling applications not already designed with undo leads to offensive designs. Moreover, small buttons on new devices and generally difficult-to-use applications make the lack of a general undo mechanism a basic problem that we show is readily solvable. Our main technical contribution is a new technique for tying declarative reasoning about time and synchronicity to the basic notion of event.
The simpleness of HTML contributed to the explosive growth of the Internet. But what is the content format that will do the same for multimedia content? The format must combine temporal notions for media coordination, such as SMIL (a W3C recommendation), and a reactive programming language, such as Statecharts or a multithreaded version of Ecmascript.
However, a well-known and fundamental semantic conflict exist between a declarative style of specifying temporal relationships and the programming that addresses user and system events. We propose a new multi-threaded, XML-based scripting language—ReaX—that introduces a notion of reaction tree for capturing causality among events. We show how the explicitly tree-oriented semantics of ReaX overcome semantic problems in reactive languages and SMIL.
Our initial work with Jennifer Beckham and Giuseppe di Fabbrizio outlined how a SMIL-like notation, then called ReX, would form a basis for a reactive language with declarative, temporal features. You can see our ideas in our paper.
The document “ReaX: an undoable, timed synchronization language” presents