104829 - VIDEOMAPPING AND INTERACTIVE INSTALLATIONS

Subject type: Optional

Coordinator: Maddalena Fedele

Trimester: Second term

Credits: 6

The profile of the new professional in the audiovisual industry, both from the point of view of creation and management and production, is in many cases due to the incorporation of new methodologies and technologies that support new production formats. . In this subject different techniques, technologies and tendencies are worked with the main objective to offer a technical formation to the student. At the same time, it aims to encourage and guide students in the creation of different audiovisual and / or interactive products in an unconventional format.

In the first unit of the course will work with different videomapping tools in order to learn how to project moving images on non-orthogonal surfaces, taking into account at the same time both technical concepts of projection planning and the possibilities of interconnection of the different tools.

In the second unit of the subject, the study of different sensors for capturing inputs and interaction is proposed, such as capacitive sensors, face-to-face, motion capture, etc. The analysis of different interconnection formats will also be worked on using standard protocols such as MiDi, OSC or DMX.

Finally, during the last unit we will link both the video projections worked on in the first unit and the capture of inputs used in the second unit with in-vehicle software such as Processing and / or TouchDesigner to design complex interactive montages.

In the last sessions to complete the achievement of the contents, students will have to present their own project where different technologies or techniques learned are integrated. To achieve this goal, they will have to prepare both the project presentation document and a functional version of the prototype.

At the end of the course the student must be able to:

• Design an audiovisual to be projected on different surfaces and synchronize it using DMX, OSC and / or MiDi protocols.
• Understand and work with different input devices such as sensors, motion capture devices, etc.
• Learn the operation and simple programming of Arduino boards.
• Understand the use of different libraries with Processing.org and the TouchDesigner tool for interaction with different devices, social networks, etc.

During the development of the last two units, an “inverted classroom” paradigm will be used where students will learn the contents that will be worked on in class through videos and self-assessment questionnaires before attending. In this way the classroom becomes an active workspace. In face-to-face classes, the solution of the problems posed will be worked on directly, or the in-depth work of more complex concepts will be worked on.

The first teaching unit will use a more methodologically standard form. Examples will be projected, practical techniques will be analyzed, the most theoretical part will be briefly reviewed and the operation of the different most common warping software will be studied.

As practical activities, different individual activities have been planned for each of the units, as well as a final final practice that can be carried out in groups.

U1: Video Mapping:

• Understand how the different existing techniques for projection on different surfaces work, and study the different theoretical concepts for projection planning.
• Learn how different warping applications like Milumin or MadMapper work.
• Explore the different possibilities of interaction.
• Planning and production of a projection.

U2: Physical computing and protocols:

• To know the practical operation of the most usual actuators and sensors to be able to carry out simple interactive installations.
• Understand the operation and creation of code for Arduino boards. Integrate different sensors and actuators with these.
•  Interconnect components using the MiDi, OSC and DMX protocols.
• Learn to design graphical interfaces with CSOs.
• Understand the operation of the DMX protocol for light control.
• Understand the operation of different devices such as Kinect or LeapMotion for motion capture.

U3: Creative Computing:

• Acquire the necessary skills to make interactive montages that use audio, image and video both for projection and for the creation of websites or mobile applications.
• Understand the basic concepts of software development and data structuring with a practical and simple prism.
• Experience the operation of a visual platform for making interactives such as Processing or Touchdesigner.

U1: Mapping of projections

Description: This unit will show how to combine art and technology by mapping projections. This is a digital process that distorts and combines images perfectly on any screen irregularly, such as buildings, sculptures, interior spaces or natural landscapes.

Related activities:

• Practice 1: Preparation of a projection from a photograph.
• Practice 2: Warping using specific software. Design of specific animations for projections with After Effects.
• Theoretical exam of the unit (self-assessment test)

U2: Physical computing

Description: This unit will be used to discover physical computing techniques for rapid prototyping. The different applications will be studied, such as interactive prototypes, phisybles, wereables or artistic installations. The most common hardware elements (sensors and actuators) will be analyzed and you will learn how to combine them using Arduino boards. During the development of this unit the student will begin to design what will be his final project.

Related activities:

• Practice 1: TouchDesigner interaction with Kinect.
• Practice 2: MiDi instrument with capacitive sensors. 
• Practice 3: Projection control using proximity sensors. In this practice a projection is synchronized with different proximity sensors.
• Self-assessment test to demonstrate the knowledge acquired

U3: Creative computing

Description: This unit introduces code creation as a creative discipline for generating sounds, images, animations and videos. The technical skills needed to write software that uses audio and graphic images for application to creative projects will be acquired.

Related activities:

• Practice 1: Simple video game using image recognition tools.
• Practice 2: Implementation of a chromesthetic projection with TouchDesigner
• Self-assessment test to certify the knowledge acquired

U4: Final project of the subject

Description: This final project can be developed in groups. This project must involve the realization of the different concepts learned during the course. Both the project report and the final prototype must reflect the different contents covered in the course.

Related activities:

• Preparation of the project briefing document.
• Prototyping and presentation of the project result.

Weighting of the note:

• U1: 25% of the Final mark [5% Test + 20% Practices]
• U2: 25% of the Final mark [5% Test + 20% Practices]
• U3: 25% of the Final mark [5% Test + 20% Practices]
• U4: 25% of the Final grade

For each activity, teachers will report on the particular rules and conditions that govern them. One-to-one activities presuppose the student's commitment to individual performance. All activities to which the student does not comply with this commitment will be considered suspended, regardless of their role. All undelivered activities will be considered suspended. It is up to the teacher to accept deliveries outside the established deadlines, as well as the penalty applied. To pass it will be necessary to pass all three tests with a minimum grade of 5. 

Shiffman, D. (2009). Learning Processing: a beginner's guide to programming images, animation, and interaction. Morgan Kaufmann

Maniello, D. (2015) Augmented reality in public spaces. Basic techniques for video mapping. The thinker

O'Sullivan, D., & Igoe, T. (2004). Physical computing: sensing and controlling the physical world with computers. Course Technology Press.

Shiffman, D. & Fry, F. & Marsh Z. (2012). The nature of code.

Reas, C., & Fry, B. (2007). Processing: a programming handbook for visual designers and artists (Vol. 6812). Mit Press.

Igoe, T. (2007). Making things talk: Practical methods for connecting physical objects. "O'Reilly Media, Inc.".

Tsai, KH (2011). The application of Projection Mapping for Architecture and space.

An Introduction To TouchDesigner - Elburz Sorkhabi - http://book.nvoid.com/#book