World

This project is set in a stylized world of floating islands. Floating cities or islands are a common concept in speculative fiction. Many of these magical locations can be found in ancient mythology.
Floating islands remain places of wonder and mystery which coincides with the magical ability of Mixed Reality; MR can present fantasy and nonexistent things to people's eyes. Therefore, I choose the floating islands as my magic world's setting.

Aesthetic

I chose the low poly and cartoon stylized design for this project's visual aesthetic.
The sharp edges and abstract shapes of triangular polygons are easily reminiscent of a memory of past emotions, such as computer games in the 1990s. Owing to the unique meaning given by the times, the low poly has a new aesthetic tendency of nostalgia and retro.
‍
Low poly uses advanced rendering, light contrast, and color matching to restore simple objects in an old-fashioned way, giving a more realistic and futuristic feeling. Low poly is a kind of retro-futuristic style design that goes back to the past and goes to the future, finding an aesthetic balance between the swing of the past and the future in digital art. Low poly's retro character just corresponds to my nostalgia factor, and its sense of future is also in line with the scientific and technological nature of the new technology- Mixed Reality.

Mood Board

Island  Stage

After seeing floating islands in the first scene, players can choose to land on any island to experience different interactive stages. For each island's experience content, I did the research and tried to find out what factors can evoke nostalgic memories in most people, not just in the Asia area.

Currently, I have developed three islands - Whack A Mole, Fishing in your Space, and  Build the Snowman. Then I include these nostalgia components into the different islands with the natural environment and let players interact with them.

Besides, when players enter an individual island, the island's area will become square, like a cube room, rather than a complete landscape appearance. I made this modification because I consider that this is an MR experience; the virtual scene in the glasses needs to interact and make a connection with the real-world space. If the presentation of the island is too complex, it will be difficult to fit in each user's physical space, such as interfering with furniture or personal belonging. In contrast, the more simplified cube can be integrated into any corner of the actual space.

Development  Environment

This project is developing in Unreal with Magic Leap 1 glasses using Mixed Reality technology.

I was amazed and inspired by the object's occlusion and interactivity with familiar physical space on Magic Leap. This is one of the main reasons I changed to using MR technology instead of VR for this project ultimately.

When seeing or playing games and things with usual elements, people will involuntarily recall the good memories and times in the past because of the mere-exposure effect and unconsciously have a natural affinity and favor for these games with nostalgic elements.

Users can feel safe, experience, and interact with Neverisland in their familiar physical space instead of a completely virtual environment. Everyone can access the imaginative Neverisland by Magic Leap and have privacy in their own personal space.

By comprehensively combining both physical space and digital renders, Mixed Reality is capable of building new levels of engagement and immersiveness. This helps to create links between human, computer, and environmental interactions.

Technical  Difficulties

I encountered great difficulties when I first started to develop my project. Magic Leap is a relatively new area and device, so there are not many related resources and templates for reference and learning.  Even the tutorials issued by the official channel have only a few videos for development in Unreal. What's worse is that the Magic Leap’s API document and example for Unreal stopped updating in version 4.24 two years ago (the newest version of Unreal Engine is 4.27) when I developed this project. 

Because of the outdated version, the only toolkit template that can be referenced is incompatible and has a lot of errors when I open it with Unreal. The toolkit template cannot execute at all due to several errors in Unreal’s blueprint.

Also, the input part for Unreal to perform the motion controller is empty, which means nothing will happen when users click or move the Magic Leap’s controller.

However, it is unrealistic to change the development game engine in this phase, and I still want to keep the Mixed Reality's advantage of object's occlusion and interactivity with the real-world rather than change to using Virtual Reality devices. There are very few resources to solve problems.

Thus, I decided to fix these errors by myself and make my project  executable using Magic Leap in Unreal.  This is an extremely time-consuming and tough process, but I am glad I did it in the end.

Functionality  Testing

Before I start developing and building the experience content, I want to test what functions and interactions I can achieve based on time limitations. To avoid increasing the challenges of later production because I have already spent too much time solving technical difficulties.

Island  1 - Whack  A  Mole

According to my previous research and topic, for the island one's scene, my overall environment setting is natural in the low poly style, and the season is set in autumn. Thus, the color and material saturation of the whole cube island is low and a little dim to create the atmosphere of autumn.

On the part of the landscape, I deliberately set its height lower because I don't want them to block too many visual areas and physical objects when the virtual cube island is presented in real space inside the Magic Leap glasses.

In order to correspond to my research, I included the elements such as the mole and the hammer to relive the nostalgia and childhood memory from the whack-a-mole game that most people are very familiar with.  The model of the mole and the hammer also apply the low poly style to match the whole project’s aesthetic setting. Whack-a-mole is not only entertaining; it can also relieve the user’s stress and help restore emotions.

Another important reason I choose the Whack-A-Mole is that it is a casual game; players can play in a relaxed mood. The casual game has a straightforward design and goals, repetitive gameplay, and no complex mechanics or previous knowledge required to play- enabling can simply get started playing. Because of these characteristics, causal games can instantly divert the user’s attention and relieve stress .

In the final phrase, I integrated the natural cube island in autumn and whack-a-mole game to island one’s level in Unreal with the soft lighting. Besides, I replaced the ocean part with the dynamic material made by Unreal to enrich the atmosphere of the natural environment.

Lastly, I implement the Place On the floor and Surface Detection from previous function testing to let the cube island spawn on the floor and enable the whack-a-mole module to generate on the actual table surface in the user’s real physical space.

Further, I edited and compiled the whack-a-mole interactive mechanics by Unreal’s blueprint system, which lets the user can hit the mole with the hammer inside the glasses.

Explore and enjoy on Magic Leap headset. During the whole experience, the participant can press the home button on the controller anytime to return to the main menu,  then select and play different islands.

Island  2 - Fishing  in  your  Space

In line with my previous research and topic, for the island two's scene, the overall environment setting is still natural in the low poly style, and the season is set in summer. Thus, the color and material saturation of the whole cube island is relatively high and bright to create the atmosphere of summer.

On the landscape aspect, I also deliberately set its height lower because I don't want them to block too many visual areas and physical objects when the virtual cube island is presented in real space inside the Magic Leap glasses. In addition, I contain the sandcastle in the scene to enhance the nostalgic memory.

In agreement with my research and topic, as I mentioned, I want to include some nostalgia factors that can evoke nostalgic memories in most people globally, not just in the Asia area. After doing research and interviews, I applied the fish model to recall the nostalgia and childhood memory from the magnetic fishing game toy that many people have had contact with.

Then, I recreated the fish toy model into 3Ddigital form by Blender and generated different color textures to assign to thefish model The fish models also use the low poly style tocoordinate the whole project’s aesthetic setting.

Another important reason for choosing a fishing toy as the reference is that it can be perfectly combined with the interactive mechanism of “fishing.” Fishing is unexpectedly associated with happiness and wellbeing.

In an article, psychologists Mark Wheeler and Nick Cooper promoted fishing as an effective method to reduce symptoms of post-traumatic stress disorder. They found that a fishing trip reduced the symptoms of PTSD in war veterans by 30%. For most fishing amateurs, fishing is a way to relieve pressure, it is just an activity to relax, breathe fresh air, embrace nature, and relieve work stress, but it also has an entertaining effect.

In the final step, I integrated the natural cube island in summer and the fishing game to island two’s level in Unreal with the soft lighting. Besides, I also replaced the ocean part with the dynamic material made by Unreal to enrich the atmosphere of the natural environment.

Last, I utilized the Place On the floor and Meshing Real World from previous function testing to let the cube island spawn on the floor in the user’s real physical space. Also, I edited and compiled the fishing interactive mechanics by Unreal’s blueprint system to enable the user can fishing inside the glasses.

Explore and enjoy on Magic Leap headset. During the whole experience, the participant can press the home button on the controller anytime to return to the main menu,  then select and play different islands.

Island  3 - Build  the  Snowman

In accordance with my previous research and topic, the overall environment setting for the island three's scene is also natural in the low poly style, and the season is set in winter. Thus, the color and material saturation of the whole cube island is relatively cold and raw to create the atmosphere of winter.

I intentionally set its height lower on the landscape aspect and deleted unnecessary meshes; because I don't want them to block too many visual areas and physical objects when the virtual cube island is presented in real space inside the Magic Leap glasses.

Corresponding to my research and topic, as stated above, I did the research and interviews; I aimed to find out what factors can evoke nostalgic memories in most people globally so that the user can embrace the nostalgia emotion in this world.

Thus, I applied the snowman to relive those carefree, happy moments of childhood memories with family or friends when building a snowman and also to fit the winter environment setting. The snowman model also applied the low poly style to follow my whole project’s aesthetic setting.

In order to simulate the behavior of building a snowman, I choose the “assembling” to be the primary interaction mechanics on this island. Hence, I edited and compiled the assemble function by Unreal’s blueprint system, enabling the user can assemble and build the snowman with snowballs inside the mixed reality glasses.

For the last stage, I integrated the natural cube island in winter and the snowman interactive module to island three’s level in Unreal with the soft lighting. Besides, I still replaced the ocean part with the dynamic material made by Unreal to enrich the atmosphere of the natural environment.

Last, I implemented the Place On the floor and Meshing Real World from previous function testing to let the cube island spawn on the floor in the user’s real physical space. Also, I edited and compiled the build the snowman interactive mechanics by Unreal’s blueprint system to accomplish the interaction of assembling snowballs inside the glasses.

Explore and enjoy on Magic Leap headset. During the whole experience, the participant can press the home button on the controller anytime to return to the main menu,  then select and play different islands.

Experience  Storyboard

Overall  Storyline

Attraction  Layout

We decided to use the spaghetti layout and one-person ride, so the player could have a better immersive experience and avoid the influence by other players. These curves are designed to leave space for scenes.

There is a strong start; the player will start shooting the monster as long as the ride starts. Then there is a short instruction between scenes one and two to teach and inform the play that they need to collect the five elements during the journey.

Script

Diorama

We built a diorama according to the layout, including every scene and simulation of the ride path.

Unreal  Scene  Build

Since this is a simulation experience of the amusement park ride, I chose to use the RealityScan app (by Epic Games) to do the 3D scan of our diorama and import the real model of each scene into Unreal instead of rebuilding the virtual scene in order to preserve the authenticity of the scenery.

However, the scale of our entire diorama is too large to thoroughly scan the scene in detail.

Therefore, I scanned the scene one by one and then imported them into Blender to remove the excess meshes and finally imported them into Unreal to integrate the whole scene.

In this way, the original appearance and details of diorama be presented in a more complete way and perfectly in Unreal.

Immersive  Experience  Build  in  Unreal

Stewart  Platform

In order to enhance the user's immersion and enrich the authenticity of the whole attraction experience, we apply the Stewart platform to our project, simulating the vibration and movement that the player will feel physically in the actual ride.

We use Max/Msp as a pipeline to connect and transfer data between Unreal and the Stewart platform and use the OSC plug-in in Unreal to send transformation data to Max/Msp so that when the camera is shaking or moving, it will implement on the Stewart platform.

LED  Wall

To level up the immersive experience, we also tried different mediums to present this project. We used the Led wall to create an immersive ride and cinema camera to document the interactive process. By projecting the VR headset's content onto the Led wall, other audiences can also immerse in the scene and enjoy the attraction simultaneously.

•  Ideation

Background

A virtual Social Amusement park during the pandemic; An entertainment environment, a space full of joy, memories, and imagination. It can also be a story that users have experienced or created with their partners before, and users can rebuild that scenario again inside this virtual world.

Concept

Since we are all influenced by the Covid-19 pandemic, going to Disneyland or an amusement park could be an unrealistic dream or a luxury thing to do now( this project was created during the serious Covid pandemic period). What if people would like to experience a traditional theme park, but the parks are almost closed all over the world or are too afraid of getting infected by the virus, so they can’t fully enjoy it? This project provides people with a virtual theme park full of imagination, which you cannot experience in the physical world.
‍

For example, unlike traditional parks, the themed land I built are floating in space. My project is based on Disney theme parks because Disneyland carries a happy memory for almost everyone, but it has been shut down worldwide. In my virtual world, users can hang out together with family, friends, or even strangers and interact with facilities. Each facility will have different ways of interaction, which may be an indoor exploration or simple interactive games so that users can achieve happy memories and fun social life with partners in this virtual Disneyland. I think that “people” is a crucial factor/influencer for the experience in amusement parks, which will bring a different atmosphere, different events, and different sparks.

Reference

Research

To strengthen my concept, I have read a lot of papers, articles, and books about Disneyland and mental health.  In particular, a book called "The Birth of the Magic Kingdom." This book specifies the story behind Disney's Magic Kingdom. From Walt Disney's childhood to the birth of Disneyland, every major event and the turning point in the development of the Disney Kingdom has been recorded and introduced in detail.
‍
An excerpt from this book, "The most interesting is not Disneyland as a game and leisure space, but Disneyland in which many people place a special emotion similar to faith. It wasn't meant to be a playground for children. What Walt Disney aimed at in his mind is the "childlike innocence " hidden in everyone. The "happy place" in Disney's mind is a fantasy world completely detached from daily life."

Book by Masako Notoji

I also created a Google questionnaire before starting the prototyping and integrated these responses to modify and enrich my initial concept.

Themed  Land  Topic

Currently, there are only three themed lands, and more creative islands are still being developed. The choice of the current three topics is based on the spirit or moral of each animation that corresponds to my project's concept.

"No matter how your heart is grieving, if you keep on believing, the dream that you wish will come true."  -Cinderella

In this world, here can “froze” the time to be a timeless space.
Froze happy/wonderful memory here.

“No matter what happens in the world around you, you have the power to create your own reality.”  -Alice

3D  Modeling

Because Cinderella castle is Disneyland park’s most central/signature part, I put a lot of effort into building it by myself.

First, I began modeling based on the 2D castle picture by Blender. I referred to its general architectural style and elements because it doesn’t show the 3D dimensional details, so I built this castle from my perspective and my imagination to model.

Then, I build the whole castle step by step in detail.

Next, refine and apply the texture of my castle on Blender.

And then, add floating landscape.

Last, extend to the entrance and garden part.

Development  Environment

This project is developing in Unreal with Oculus Quest 1 headset using Virtual Reality technology.

VR  Environment  Develop  in  Unreal

After building the models in Blender, I imported them into Unreal. Then I integrated all the themed lands, developed a VR entertainment environment, and added interactions.

Setting for VR functionality.

Create portals that enable the experiencer to travel around different islands. Also, make the basic instructions for each island. The info will appear when the player gazes at the sphere near the portal.

Portal's blueprint and material. This part took me a lot of time to successfully get the character to teleport.

Instruction's blueprint.

Create triggers for interaction with the controller.

Add VFX for interaction.

Each  Themed  Land  in  Unreal

Upload  Apk.  File

In order to let more people experience it and do user testing, I launched and uploaded apk. file to the Oculus headset via SideQuest. This allows my prototype to be installed directly on Oculus without connecting with unreal.

Documentation

Interactive  Step

Simulation  Diagram

Development  Environment

This project is developed in Arduino with Neopixel led, pulse sensor, and temperature sensor.

Arduino  Code

Circuit  Diagram

Making  Process

1. Use 3D pen to produce DNA shape.

2. Paint black acrylic on the wood stick.

3. Drill holes in the wood stick (to put the DNA led strip).

4. Weld Neopixel Led port (total 60 ports).

5. Create pin for 60 ports (enable them to connect to the Arduino board.).

6. Put 20 Led strips into wood sticks.

7. Test each LED strip can be lit up before installation.

8. Drill holes in the wooden box (to put sensors).

9. Engrave symbols on the wooden box with the wood-burning pen.

10. Hide components, Arduino board, and wire inside the box.

11. Organize wires.

Install

Although there is still a lot of room for improvement in this prototype, the final result is still a great achievement. The outcome of the combination of neopixel LED and sensor was presented well, and I received many positive comments throughout the exhibition.

It was an extremely challenging experience to connect 200 LEDs in series, execute successfully, and polish the installation in two weeks by myself. I am very satisfied and proud to have achieved this result!

Documentation

Max/Msp  Development

I use Max/Msp as a bridge between Kinect and P5js sketch (launch JavaScript to socket.io server).

In this part, I apply cv.jit.centroids node to find the object's centroid in front of Kinect's depth camera, and I adjust parameters to capture the most appropriate proportion.

In  next part, I utilize node.js on Max. It allows me to write JavaScript code on my computer, and node.js can find this file.

Then node.js can run and launch code to the Socket.io web server. This max patch lets anyone on the local network can enjoy this interaction. (Open a browser and type "http://localhost:5003")

P5js  Content  Iteration

To create the paper airplane’s trace/trail when players fly it, I have tried many different codes to achieve a more refined outcome.

I used the mouse's position instead of a paper airplane first when I was developing on p5js.

Finally, I modified it to this version, using transparency, size change, and delay time to achieve this effect.

And then, apply the color and add some glowing effect.

Add a dynamic background to enrich the visual effect and add a music. (demonstrate by the paper plane)

Connect to the Socket.io server, and the trace's color and music's speed will change according to the paper airplane's position.

Final Code

Setting  Up

Testing

Final  Result

Players can fold their own paper airplanes and make the best airplane. And then, launch the magic paper plane and try to keep the plane in the air longer; the trace's color and music's speed are affected by the flight of the paper airplane. Players can explore how flight affects the digital content by repeating the simple and enjoyable nostalgic game.

(The main video at the top includes the final result with sound change.)

Concept

In our growth phase, we often find out that there is something similar between our parents and us. Parents shadow our life more or less whenever we make our own choices or behave the exact way we wish to do.
‍
Most parents expect us to be the perfect children they want us to be. They have high expectations, wishing their children to reach the goal they set for them. However, the expectation parents hold is not suitable or even good for every child. The expectation might be shaped by the majority of society; meanwhile, the universal value and standard created by the majority may not be essentially the best. The biggest question is: If we cannot keep our own thoughts, what is the point of creating a world in praise of diversity?

Worldview

In this world, we have a fantasy machine that can control our figures. Parents can use it to control and deform their children. For example, if children do not have the intelligence for learning, parents can use the machine to reform them. After the reformation, parents can find their children to be more intelligent, but the appearance will become freaky; they might have a significantly huge head.

We use this exaggerated way as a metaphor for parents’ control, according to our own experience as children. We find out parent’s control is something rather imperceptible that sometimes we cannot even notice. Hopefully, our animation can form an intense impression in the minds of parents or those who want to forcibly affect or control others. Every person in the world is a different individual, and it is wrong and unnecessary trying to shape another individual.

Character  Design

This animation's title, Ctrl T, is derived from this little boy, which means "control Teagan."

At the beginning of the story, the camera only shows the lower body of the parents; their faces will reveal until the end of the story. The parents' faces are exactly the same as the little boy's scary face at the end.

Storyboard

Animatics

Scene  Setting

The overall style is mainly based on German Expressionism, with distorted/deformed scenes and furniture to symbolize the values of the parallel world.

Color  Script

Technique

We use stop-motion to present our story. Stop motion is a more meticulous and real animation technique in comparison to others. We know that, nowadays, CGI is trying to emulate the qualities of stop motion. But the emotion that stop-motion conveys is impossible to be replaced. We persist in the maker spirit, trying to create everything that appears in our animation on our own, for example, the completely self-made puppet armature.

After the accurate measurement and sufficient research, we went to the maker space and built the metal armature with our own hands. The unique texture and the warmth that stop motion possesses bring out the wishes deep inside the boy. We deliver warmth to the boy's life, yet at the same time, control him as well.

Doll  Making

1. Use Super Sculpey, this material is harder, and it will be solid and not deform after baking, is appropriate for making highly delicate. inner mold.
‍
2. After sculpturing, scoop out the clay of head to be hollow.   (to reduce the weight)

3. Bake to dry.

4. Use silicone to cast outer mould.  (silicone is soft and elastic, easy to remove and take out mold)
‍
5. Cast with poly. (hard, cheap, and does not tend to shrink)

6. After casting replication, use the engraving machine to do engraving, and use mountain meal and clay to do sculpture.

7. After sculpture is done, use sandpaper to rub till smooth, use pen to paint with putty to fill up holes if any.

8. Once the surface gets smooth, spray paint for colors layer by layer, then use the pen to do a detailed drawing.

1. Use Styrofoam balls as a base for the eyes.
2. Coat with hardening clay; then polish and color.
3. Paint with crystal glue. (to create a crystal clear sense)

1. Use iron wire and epoxy putty to make the skeleton.
2. Make the hand model with Super Sculpey.
3. Make a silicone mold and fill it with silicone.

1. Combine a variety of metal mediums to create a unique skeleton.
2. Use the high-density sponge to cover the skeleton as the body.
3. Sewing the exclusive clothes for characters by myself.

Furniture  Making

1. Measure the size.
‍
2. Made of foamboard, styrofoam board, styrofoam ball, thick cardboard, wood sheet, plastic sheet, hardening clay, light clay, etc.

3. Apply putty to the styrofoam surface and the adhesive gap.
4. After the putty is dry, smooth it with sandpaper.
5. Use acrylic paint to color and create an aged appearance.

Shooting  Process

Because the whole setting of the filming process needed to be fixed in the same place for a long time so as to have coherence of photos, we rented a studio as a filming site in order to present the best work.

Dubbing  &  Post  Production

We went to the recording studio to personally dub the soundtrack and add sound effects in order to produce a unique sound that matches our animation.

It was also a time-consuming process during post-production after shooting. In order to present a smoother stop-motion animation, we took up to 7,000 photos and had to remove the dolls' metal support one by one using Photoshop.

Finally, I used After Effects to assemble all the photos, add visual effects, and do the editing.

Logo  Design

Poster