September 2019

Collaborative Embodied Movement Design Network

Movement-based technologies such as augmented and virtual reality, haptic and robotic interfaces form the cutting edge of human computer interaction (HCI) development. This project has developed new infrastructure to create a national collaborative network of arts/technology researchers, enabling them to work together to optimise the quality of these systems from an embodied perspective, and to create new innovation possibilities for industry, commerce, education, health care and the arts. The network features real-time remote motion capture collaboration between facilities.

Partners

This research was funded by the Australian Government through the Australian Research Council’s Linkage, Infrastructure and Equipment Scheme (Project LE170100066)

Investigators

Kim Vincs (TMT)

John McCormick (TMT)

Troy Innocent (RMIT)

Adam Nash (RMIT)

Simon Biggs (Uni SA)

Bruce Thomas (Uni SA)

Frank Vetere (Uni Melb)

Robert Vincs (Uni Melb)

Saeid Nahavandi (Deakin)

Douglas Creighton (Deakin)

Jordan Vincent (Deakin)

Petra Gemeinboeck (UNSW)

Keith Armstrong (QUT)

Thomas Chandler (Monash)

Scott deLahunta (Coventry, UK)

Related Projects

Enhancing the impact of Australian performing arts: virtual scenography and opera for the 21st century

This project, a partnership with Victorian Opera, explores the artistic and economic potential of 3D virtual scenography through three landmark productions, The Flying Dutchman, Four Saints in Three Acts and The Snow Queen.

Partners

Victorian Opera

This research was funded by the Australian Government through the Australian Research Council’s Linkage Program (Project LP1400100742)

Investigators

Kim Vincs
Richard Mills

Digitally created virtual 3D scenography has the potential to create immersive stage environments that re-imagine and re-tool the artistic language of opera production for the 21st century, and to re-model the economics of live theatre production through reducing touring costs.

In these productions, the performers are integrated within spectacular CG landscapes and characters, which the audience views using 3D glasses. These highly successful works delighted audiences and drew exceptional reviews, awards and nominations, including finalist in the non-game category of Unity’s international Unite awards for The Flying Dutchman.

The Flying Dutchman

This production, created in collaboration with Victorian Opera, explored the potential of 3D stereoscopic technology to provide unparalleled depth to the set and reduce the cost of building and transporting physical sets whilst touring, both vital to a sustainable and visually engaging opera industry.

Original 3D projected scenography, viewed using 3D glasses, was created in the Unity game engine. This enabled us to cue animations in real-time during the live performance, and provided a unique ability to move the camera perspective around the ‘world’ of the opera—in this case a Norwegian Fjord. This methodology provided a new visual dramaturgy for the opera, whereby audiences feel as if they are moving through the set, and experience a visceral sense of depth and presence. Without the need for a physical set, this research also contributed greater economic flexibility to this opera production, increasing its potential to reach regional and remote audiences.

The work was a critical success, demonstrating the power of 3D techniques to refresh the aesthetics of traditional operas such as The Flying Dutchman. The work was a finalist in the Unite (Unity game engine) awards, nominated for eight Green Room Awards (winning three), and nominated for one Helpmann Award. The Flying Dutchman was profiled by the Australian Research Council in their 2015 report as an exemplar of impact.

https://www.victorianopera.com.au/past-productions/the-flying-dutchman

Four Saints in Three Acts

This project, also created in collaboration with Linkage partner Victorian Opera, aimed to develop, implement, and validate the aesthetic opportunities afforded by 3D digital scenography as well as the capacity of virtual, projected sets to remodel the economics of theatre production, extending the artistic and cultural reach of traditional opera.

Building on the success of The Flying Dutchman, Four Saints in Three Acts involved creating 3D stereoscopic imagery using the Unity game engine to augment the surrealist librettos written by Gertrude Stein that form the basis of this opera. When viewed by an audience wearing 3D glasses, radical imagery appears: fish fly, stained glass windows shatter, and portals open to a cosmic space setting. These effects play upon the ambiguous nature of the libretto, whilst camera movements enabled by the game engine, such as orbital shots and sudden zooms, enhance the sense of disjuncture, non-sequitur and play.

The work was created in collaboration with well-known director and theatre-maker Nancy Black, and performed as part of the Victorian Opera’s season at the Merlyn Theatre, Coopers Malthouse in Melbourne. This project brought a little-known yet seminal modernist opera to life for Melbourne audiences, and, without the need for a built set, the production is able to tour at reduced cost. While the opera itself proved challenging to critics, the 3D scenography was an outstanding success and the work was nominated for a Green Room award and the CHASS non-traditional research output prize.

https://www.victorianopera.com.au/season/four-saints-in-three-acts

https://www.victorianopera.com.au/behind-the-scenes/reviews-four-saints-in-three-acts

The Snow Queen

The Snow Queen is the third opera created in collaboration with Victorian Opera as part of this ARC Linkage project, which aimed to design, test and evaluate the artistic and economic value of virtual scenography for touring performing arts companies, and enable opera companies to present large scale productions in previously inaccessible regional and rural areas.

The Snow Queen was specifically designed for regional touring, and premiered at Wodonga, in country Victoria in November 2017. The production simplified camera movement and 3D object placement to successfully accommodate a large on-stage community chorus. This regional staging demonstrated the capacity of 3D virtual scenography to reduce touring costs for opera. The project used Unreal Engine rather than Unity game engine as in the previous two operas. This enabled us to create more visually rich imagery in less time, hence reducing costs.

The work successfully developed and demonstrated technological and artistic compositional approaches that complemented the regional setting of this tour. This is particularly significant for companies such as Victorian Opera, which see regional touring as a critical aspect of their mission to bring new cultural experiences to audiences, and as a core element of their business and funding models.

https://www.victorianopera.com.au/season/the-snow-queen