The audio-processor is a black-box that analyzes incoming sounds and generates OSC messages. A basic set of messages and parameters needs to be specified (those can be extended in the future).
volume / energy
tempo / rhythm
pitch / timbre
frequency spectrum and frequency spread
etc
additional msg (initial and periodical updates) configuration (number of channels, samplerate, etc)
Video-generator has an internal pipeline. e.g. shape-generator, texture-overlay, zoom, filter (eg. blur, brightness/contrast), encode.
every element in the pipeline may have configuration parameters
these parameters are either fixed or can be modified dynamically (interact with input)
dynamic parameters can be set by OSC (either from the sound-analyzer or via user-interaction)
a config file or database (aka preset) describes the pipeline and defines a map of parameters.
User interaction - also sends OSC messages to the video-generator (depending on the config, these are internally forwarded to one or more elements in the pipeline)
interaction can be via HID (mouse, keyboard, etc)
Joystick, Wii-mote, etc
Mixing-desk (faders or knobs)
advanced systems such as camera → processing → OSC