Synthesizer.Storage

Rendering sound effects off-line has its virtue, but really cool is real-time signal generation. For a long time I thought that it is the compiler's responsibility to make list based signal processing fast enough. However, the compiler has to respect correctness first. That is, it cannot do too fancy optimization, since the optimized program must still do the same as the unoptimized program. So, when we write functions that rely on the maximal flexibility, the compiler cannot turn it to something less flexible. Actually, a list as in Synthesizer.Plain.Signal is the best representation of a signal in terms of flexibility: It allows free choice of the element type, even functions, it is element-wise lazy, allowing for short feedback, it allows sharing of computed data. The drawback is, that it is slow and memory inefficient. In most cases we don't need this full flexibility, but the compiler has no chance to find this out automatically. It can merge several operations on a list to a single operation by the fusion technique, however even a single list operation is hard to perform in real-time.

How do real-time software synthesizer achieve real-time performance? They get the popular fast inner loops by processing signals in chunks of raw data. This way, they lose flexibility, because they cannot do quick feedback. We can do the same in Haskell, getting the same restrictions. Additionally, in order to store raw data we must restrict the element types e.g. to the Storable class, since we use StorableVector in Synthesizer.Storable.Signal. With this technique single signal operations are fast, but their combination cannot be optimized in many cases. This is so, again, because top priority in optimization is correctness. Consider mix x (cons 0 x) where cons 0 x means 0:x for our chunky signal data. This expression is a perfect candidate for optimization. But in this case it must not be applied since the chunk structures of x and cons 0 x do not match. In such cases we would not gain anything over SuperCollider and CSound.

Remember that we introduced the chunky signal representation entirely because of efficiency concerns. Actually we are not interested in a special chunk structure, so this should not be a reason for disabling optimization. Of course, we could ignore the correctness and write incorrect optimizer rules that are based on correct ideas. However, experience shows that wrong optimization leads to surprise and infelicities sooner or later. The later the worse, because the later the more code you have written relying on invalid optimization.

What we can try is to use list representation, enjoy the optimization that GHC already provides for it, and then let fusion rules jump in that make lists disappear when they are used in connection with chunky sequences. E.g. Chunky.fromList (List.oscillator freq) could be turned into Chunky.oscillator freq. This approach would be really cool, but works only in theory. In practice it is hard to predict how GHC transforms various operations. Additionally to optimizer rule application it also expands functions to their definitions (known as inlining/unfolding) or specializes functions to fixed types. We cannot rely on our optimizer rules being actually applied. This means however, that in unpredictable cases the optimization fails and the efficiency drops from real-time to non-real-time. This is unacceptable.

The solution is a third signal representation, see Synthesizer.State.Signal. (Already got tired?) It consists of no actual data but it is a function that generates elements. Its type is s -> Maybe (a,s) or short StateT s Maybe a. Given a state of type s it produces Nothing when the list terminates or Just the next element and the updated state. This can be easily converted from and to lists while preserving laziness. We convert to lists by List.unfoldr and from lists using viewL. Actually this signal representation is very close to the list representation used in the streams package. The main differences are: Firstly, we do not use a list storage that is fused away when only used temporarily. Thus we do not need a fusion rule (that could be skipped by the compiler). Secondly, we have no notion of Skip, since operations like filter are uncommon in signal processing. If we write our signal processing in terms of these virtual signals and then convert the result to regular lists or chunky sequences, then only one data structure will be built and GHC does it's best to generate efficient inner loops.

We cannot use these virtual signals for sharing and feedback, because there is no data structure that stores the data. If we try to do so anyway, data will be recomputed. Thus we still need chunky sequences or lists for sharing of interim results and for feedback. Actually, an expression like mix x (reverse x) would definitely benefit from interim conversion to a chunky sequence, but for mix x (cons 0 x) this is overkill.

In order to get processes like the last one efficient we have a new data type (no, not another one!) but this time it is not a signal data type but a signal processor type. It is the result of thinking about which processes allow sharing on a per-sample basis at all. We come to the conclusion that these can be only causal processes, i.e. processes that depend only on current and past data, not on data from the future. So, we already have a good name: Synthesizer.Causal.Process. Causal processes are Control.Arrows, however the higher level variant does no longer fit into the Arrow type class. This means that there are various combinations that turn causal processes into larger causal processes. It needs a bit experience in pointfree coding style in order to use the arrow combinators, but there is no way around it, when you want to use physical dimensions. GHC's arrow notation does only support types of the Arrow class. E.g. the expression mix x (cons 0 x) becomes Causal.mix <<< (Causal.id &&& Causal.cons 0). When you manage this burden you get processes that are warranted to be causal. They can not only be used to make something efficient, but they also allow to process data from the outside world in a streaming way without unsafeInterleaveIO as required e.g. in JACK plugins.

We have now a pretty big set of signal storage types that differ considerably in performance but not in the set of operations. This calls for a type class! You find it in Synthesizer.Generic.Cut and Synthesizer.Generic.Signal.