During improvisation, musicians express themselves through live music. This project looks at the relationship between musicians during music improvisation, the processes of expression and communication taking place during performance and possible ways to use musicians’ emotions, to influence a digital instrument – and in consequence improvisation. To illustrate this, a three-layer model is described studying the relationship between band members and the audience as a system, where emotions, expressivity and generation of sound give shape to improvisation. Focus is applied specifically on how individual emotional arousal can be used as input to control as a group a musical instrument: EMjam. The instrument is illustrated, describing the design and implementation as well as the evaluation of it, to subsequently discuss the results.