What is this blog about? What is Spyractable?

 This is an open trial project of a reactable-like synthesizer and a guide of how can anyone make his own, DIY tangible interface synthesizer and adjust it to his own needs. This is also a guide to make your own patches, connect them anyway you like, project them anyway you like, orient  them anyway you like, starting from scratch, and all of these by using free programs.

the two implementations of Spyractable

I am neither a program expert nor a genius, so i am open to any propositions for improvements of what i have started, and more of that, your own versions of "Spyractable" or pieces of code. Please ask anything you want about any line / method / class / of the code or what i had in my mind when i did it (!?), it would be my pleasure to explain and maybe, something good would come out after all...
 Also feel free to comment anything you think about this (even "bad" comments can contribute!).
From this blog you can download all versions of this project. Later on i 'll give instructions of how to add patches...
The truth is that you need to know some processing and puredata to come up with this project. If you have little knowledge of Java, then processing is easy, but i cannot say the same for puredata...

processing (left) & pure data (right) examples

To finish with these first introductions, i must say i have already done and tested the upper implementations and on the next days i 'll handle with the implementation of the tangible interface table and this has no guaranteed results..

Anyway this is it...

In this first video sound creation patches (oscillators)are being demonstrated:

This second video is the introduction of the fx and sound modification patches:

Here it is the lfo usage:

This is the horizontal version:

Requirements:

Processing 1.5 (later versions do not support all the libraries and classes used in the code, such as picking*, Vector, etc.).
and its libraries:

• TUIO_processing
• picking*
• theMidiBus (for those who have midi keyboard..).
• pdlib / pureDataP5 (This is a library that reads pure data files inside processing).
• fullScreen (not necessary. Just for the view....).

PureData - pd_Vanilla (The reason we use this version and not the extended one, is because it is more compatible with the pd - objects that pdlib implements. If you have already the PD-extended, just use it, but check what objects you can use.)

Anyway PD is used in order to extend the code and add more patches. It is not necessary to run Spyractable.
Also i use the pd - libraries: zexy, rjlib (rjdj), soundtouch~ but don't need to download them because they are already in the "Spyractable" packet...

To finish up you will need to have 
portAudio and Jack Audio Connection Kit

OS:
Spyractable is being built with a Mac book 10.6.8 snow leopard and i am not quite sure if other OS can support it. I think linux will be OK but Windows have a little problem with the installation of the pdlib, as i know so far.



Control :
If someone has already a tangible interface table, he/she can use the computer vision program, made for the called reactTIVision that reads fingers and fiducial markers, or any other program with fiducial marker recognition (ex. Community Core Vision). Also please replay if it really works with multiple touch and parallel fiducial handling or bugging, cause so far i' ve been using the reacTIVision simulator (also can be downloaded from the upper site link or from my link, so to have the adjusted to Spyractable versions). 

DOWNLOAD "SPYRACTABLE" 

.

What there is so far...

In Spyractable is based on the architecture of reactable, meaning that there is a connection manager, a visual synthesizer and a sound synthesizer to set the software..

reactable architecture

But all these parts are inside processing, meaning that on part of the code do the connection management, another one the graphics / visual synthesizer (as expected), and also inside processing libpd implements the sound synthesizer, taking orders from the connection manager..

 Right now there have been created 12 patches (tangible objects) that implement these modular synthesizer modules:

1. Amplifier with lfo input
2. A multi-effect with chorus, phaser and mixed   function.
3. A multi-filter with low-pass, high-pass and band-pass function, plus a cut-off frequency generated envelope (GE) with lfo input and a "midi keyboard plug-in that sets the cut-off frequency equal the midi-pitch and a volume GE. This way it could function like a silly Subtractive-synthesizer if you connect it with noisy input..
4. Delay effect
5. Sample player with 8 samples
6. A sine-wave oscillator that also functions as carrier by connection to it a frequency modulator, and also volume sliders for the sine harmonics of the frequency and a GE working when midi keyboard is on.
7. Noise oscillator
8. Microphone input with a midi-keyboard vocoder plug in (fixed chords).
9. A multi-oscillator with sine, saw, square and triangle oscillators, plus midi-keyboard in.
10. lfo (sine, saw, square and triangle oscillators).
11. modulator that takes sine-oscillator patch's frequency and user sets the modulation index and the frequency ratio.
12. Master volume controller.

Everything 's working, but i have to admit that there is a crucial mistake (more like a starting choice)  because the GE sliders and the sine-harmonic sliders appear at a fixed side of the table. This is not wrong at the horizontal version but its no good for the circular version of Spyractable, but I intend  to fix it... 

There are also a setup method that resizes the graphics, according to the table dimensions and the projectors resolution, at the end of "setup_methods" tab at the processing code. But also i don't know if it works because i don't have a multi-touch table yet.. 
.. so anyone 's trying it, please reply me how it works...  

How does "processing" work?

First of all, we have to check the architecture. Based on reactable 's principals we need to separate  the part that is responsible for the connection of the patches (depending on their position), from the part that makes the sound and the part that makes the graphics. In this way, we can improve every part without having to change anything to the other two parts.
This way we can change / improve the pd - objects so to get better sounds (ex. use filter from another pd-library), we can improve the graphics (ex. make better sliders or new controllers that control the same sound parameters) or change the connection options (ex. we have circular and linear reading of the table, but the patches have the same code, or change connectivity settings such as the limit connection distance, angle etc.).
As we have seen the Spyractable's architecture is compiled someway like this:

This means that the hole project is made in "processing", and even though the sound seams to be created outside "processing", it's still controlled from "processing" and in the end it is synthesized by "processing 's" libpd library, that is reading and exchanging data with the pd-patch.
So we have "processing" inletting data, from computer vision program, translating from OSC messages to Java classes data and elements.
Then depending on this data, after calculations, it sets the parameters of connection, as well as the parameters of the visuals and the sound.
"Processing", also, draws the graphics on the screen and exchanges numerical data with the pd-patch, setting, this way, the parameters of the sound and the parameters of the waveforms.
So we firstly have to see how "processing" is working...

The reading of the code in linear, meaning that every complete "command to do something" is executed just when the reading by "processing" is ended. Then the next complete command is executed and so on...
The code is being separated in 5 parts. The first part is the firsts reading lines, so there we can declare global (to our program) objects and the importing of the libraries.
After that "processing" is expecting to read a "setup()" method. What is commanded in "setup()", is only executed one and only time. Also, anything declared in "setup()" can only be seen in "setup()" (as expected from a common method). This method is a good opportunity  to initialize all objects and variables.
If "setup()" ends, "processing 's" 3rd part of code is the standard "draw()" method. "Draw()" method is the one that makes "processing" 'running', since it can be read in a loop, executing again and again the same commands and methods. We just change these commands' and methods' parameters and we take the outcome. If a method commands the picturing of a circle with center (x,y) and every time this command is read, these coordinates change  price, then we see a circle moving...
The 4th part of the code is constituted from some callback methods. These methods are called automatically by "out - processing" factors, such as mouse clicking, midi-in order, joystick sensing. and others. Most of them are libraries orders, in Spyractable TUIO - data from computer vision program and midi orders from keyboard. When the multi-touch table changes its state (finger on, fiducial is being moved / added / removed, we take this data from TUIO library callback methods and used from the connection manager to set up the connections, or used to change the parameters of the classes.
5th and last part, is all the code of  classes and methods (patch methods, visualizing methods, connection manager, etc.).
When "processing" starts the compiling of the code, it does a short reading of the complete code (part 1 to 5), in order to inspect code errors and set the classes and methods in memory (short of).
Then it starts reading again part_1 and execute it.
Then part 2_and execute it
and tHen part_3 in the loop.
Part_4 is executed whenever is called a method and just changing the parameters of the code executed in constant, by part_3 - "draw()" method.