See Screens & Scream

See Screens & Scream 2022

Unity, Blender, Raspberry Pi, OSC, TouchDesigner

See Screens & Scream is a speculative Unity experience that gives the viewer the power to control advertisements and billboards in Times Square.

CONCEPT

✿ giving the viewer the power to control the advertisements in a futuristic Times Square

✿ hypothetically, viewer is able to experience a completely different reality of Times Square, replacing the consumerism based experience of being bombarded by various media meant to inflict a need to buy

✿ each billboard in the Unity experience is an artwork, except the eye on the building at the center, which is live camera input from a Raspberry Pi

✿ giving artists the ability to view their own work and others’ in a space typically reserved for massive companies and corporations, leaving little room for any individuality or the ability to challenge this status quo

✿ the camera would be installed in Times Square, as if viewing Times Square in Times Square, to explore the idea of behavior under a capitalist society

✿ this experience would be translated to an AR app where you can point to and change the advertisements you’re forced to consume around you

The concept of this project was to explore the idea of control and manipulation within the context of advertising and commercialism in public spaces. Specifically, the project focused on Times Square, which is known for its iconic billboards and advertisements that dominate the city skyline.

The 3D Unity experience allows users to interact with these advertisements and billboards in a way that is not possible in real life. By giving users the ability to control and manipulate the content of these advertisements, the project aimed to question the power dynamics at play in public spaces and the role of commercialism in shaping our experiences of these spaces.

See Screens & Scream encourages reflection on the relationship between advertising, commercialism, and public space, and prompts critical thinking about the ways in which we engage with and are affected by these forces in our daily lives.

THEMES

✿ identity under capitalism

✿ monopolization of advertisements by massive organizations

✿ consumerism and media

✿ panopticon and effect on behavior by the idea of being observed

The project touches upon several themes that are relevant to our contemporary society, including identity under capitalism, monopolization of advertisements by massive organizations, consumerism and media, and panopticon and its effects on behavior.

Identity under capitalism is a concept that explores the idea that our sense of self is often defined and shaped by the consumption of goods and services, and our ability to participate in the market economy. Through the manipulation of advertisements and billboards in the project, the viewer is confronted with the idea that their identity is constantly being constructed and reinforced by external forces that seek to profit from our desires and needs.

The monopolization of advertisements by massive organizations is another theme that the project touches upon. In real life, only a few large corporations control the majority of advertisements and billboards in public spaces. Through the project, the viewer can manipulate and control the content of these advertisements, which calls attention to the fact that our public spaces are often dominated by a few powerful organizations who control the messages that we see and interact with.

Consumerism and media are also themes that are explored in the project. The interactive nature of the project invites the viewer to actively engage with the advertisements and billboards, which highlights the role of media and advertising in shaping our desires and behaviors. The project prompts the viewer to reflect on their own relationship with consumerism and how their choices and actions are influenced by the media they consume.

Lastly, the project touches upon the concept of panopticon and its effects on behavior. The panopticon is a prison design concept that allows a single observer to monitor all prisoners without them knowing whether they are being observed or not. This idea has been applied to modern society, with the notion that we are constantly being observed and monitored in public spaces. Through the project, the viewer is placed in the position of the observer, and they can control and manipulate the content of the advertisements and billboards. This prompts the viewer to reflect on their own behaviors and actions when they are being observed or monitored.

Overall, the project engages with several themes that are relevant to our contemporary society, and it aims to prompt critical reflection on the power dynamics at play in public spaces and the role of commercialism in shaping our experiences and identities.

CONSUMERISM & CAPITALISM

The consumerism aspect of capitalist society that the project shines a light on is the way in which our desires and behaviors are shaped and influenced by the media we consume, specifically in the form of advertisements and billboards.

In the project, the viewer is placed in a position of control over the content of these advertisements, which prompts them to consider the ways in which they are manipulated and influenced by these messages in real life. By allowing the viewer to actively engage with and manipulate the content of the advertisements and billboards, the project highlights the role of media and advertising in shaping our desires and behaviors, and how our identities are constructed and reinforced by external forces that seek to profit from our consumption.

The project invites the viewer to reflect on their own relationship with consumerism and how their choices and actions are influenced by the media they consume. It encourages critical thinking about the ways in which consumerism is promoted and perpetuated in our society, and how it shapes our values and priorities as individuals and as a society.

Overall, the project shines a light on the pervasive influence of consumerism in our capitalist society, and it aims to encourage critical reflection on the role of advertising and media in shaping our desires and behaviors, and ultimately, our identities.

INSPIRATION

Times Square Live Stream︎︎︎

TECHNICAL PROCESS

3D Modeling with Blender

We 3D modeled each building to be minimalistic, steel-like structures for our futuristic Times Square.

Researched the architecture of Times Square and identified key design elements that would be recognizable to viewers.

Sketched rough models to explore different design options for the buildings. We wanted to create simple, geometric shapes that could still be read as Times Square, with futuristic qualities.

Translated the sketches into Blender and refined the 3D models. We paid close attention to the material properties of steel and glass, and I experimented with different textures and finishes to achieve the desired futuristic look.

Created modular building elements that could be easily duplicated and scaled to create the different buildings in the scene. This helped to save time and ensure consistency in the overall design aesthetic.

Imported the 3D models into Unity and adjusted their positions and orientations to create a cohesive and immersive environment for the viewer.

Throughout the process, we paid close attention to the balance between form and function, and aimed to create a design that was both visually striking and easy to navigate for the viewer. By using a minimalist, futuristic design language, we were able to create a unique and engaging experience that stands out from traditional Times Square visuals, while still paying homage to the iconic architecture of the area.

Live Camera Feed with Raspberry Pi

We used a Raspberry Pi to get live camera feed onto one of the billboards in the Unity scene. Here's a brief explanation of the process:

Connected a camera module to the Raspberry Pi¹ and set it up to capture a live video feed.²

Wrote a Python script to capture the live video feed from the camera module³ and stream it over a local network using the RTSP protocol.

Used a Unity plugin called "EasyMovieTexture" to receive the RTSP stream⁴ and display it as a video texture on a billboard in the Unity scene.

Adjusted the position, size, and orientation of the billboard to fit seamlessly into the 3D environment of the Unity scene.

By using a Raspberry Pi to capture and stream the live video feed, we were able to incorporate real-time content into the Unity scene, which added an additional layer of interactivity and engagement for the viewer.

TouchDesigner <--> Unity

References:

¹ "Camera Module." Raspberry Pi Documentation, Raspberry Pi Foundation, 2021, https://www.raspberrypi.com/documentation/accessories/camera.html.

² "Getting started with picamera." Raspberry Pi Projects, Raspberry Pi Foundation, 2021, https://projects.raspberrypi.org/en/projects/getting-started-with-picamera.

³ Rosebrock, Adrian. "Accessing the Raspberry Pi Camera with OpenCV and Python." PyImageSearch, 30 March 2015, https://pyimagesearch.com/2015/03/30/accessing-the-raspberry-pi-camera-with-opencv-and-python/.

⁴ "MovieTexture." Unity User Manual (version 5.6), Unity Technologies, 2017, https://docs.unity.cn/560/Documentation/Manual/class-MovieTexture.html.