Bodies in Motion - Lab Reports

Week 1 - Rigid Bodies

The first week in the studio lab was spent learning how to setup and calibrate the OptiTrack motion capture system from scratch and how to set up rigid body tracking and streaming to Unreal Engine.

Calibrating the system involves several steps, including masking the tracking area for noise from shiny objects, using the calibration wand to sample the space so that cameras align to each other and the ground plate to align all cameras to the room.

Once the room was tracked we used the readymade rigibbody trackers. These are small objects (either bought or made) that have unique configurations of trackers attached to them. When each group of points is selected in Motive it can be locked as a single rigid body object. We tracked a total of three objects and captured a quick motion scene.

In Unreal Engine, we set up the OptiTrack plugin to receive streaming motion capture data and three rigid-body tracking target. There targets are empty actors that will move any movable objects placed inside them in the scene hierarchy, we tried parenting simple geometric objects as well as more dynamic ones such as motion-reactive particle systems.

to stream the tracking data from Motive all we had to do was play back the scene, and we realized that if we go back to capture mode and press the aptly named “Live” button, live tracking data is sent through to Unreal Engine, allowing realtime performance and simulation feedback.

Week 1 afterthoughts

It was surprising to see how easy it is to set up a tracking system using Motive and streaming to Unreal Engine. It got me thinking of the possibilities for performance with live motion capture data and how to reverse the tracking to not place a person in the scene, but map the scene to a person, a-la “Inori”. I hope the semester will provide an opportunity for more physical-interaction based projects using MoCap data and using it to augment a physical space using projection and lights.

Week 2 - Skeletal Tracking

The second week of the lab was focused on capturing skeletal data and mapping human figures in Unreal Engine. This is a more involved process than dealing with rigid bodies on both the Motive side and Unreal Engine side.

We started, again, by calibrating the studio and then went on to setting up the tracking suits. We mapped each actor with a total of 41 trackers in a predefined tracking setup for posture and toe articulation. Once all the trackers were assigned, all we had to do was set the actors in a T-pose, select all of the floating points in Motive and assign a skeleton, just as we do with rigid bodies. The difficulty was in setting up the suits correctly but Motive has some handy realtime visual guides to show you which points are off.

The Unreal scene was already set for us this week, so all we had to do was set the right naming and ids and a blue demon magically started to move around the scene. The downside was that using skeletal tracking requires setting up a control blueprint instead of using the actors, and both can’t be used at the same time, so our attempts to also track some chairs failed for now.

Once both actors were rigged and tracked we had some time to mess around, we used the (surprisingly smooth) office chairs as moving platforms and experimented in recreating scenes of dance, fighting, explosions, flying and swimming. The simulation renders still like bad green screen keying from an 80’s superhero movie but hey, it’s only the second week!

Week 2 afterthoughts

I’ve been learning how to use Unreal Engine blueprints and set up a simple targeting setup to control multiple actors based on motion data. I want to try and create an interface blueprint that will act as an abstraction between Unreal and the tracking hardware, so that the same project could be run with OptiTrack rigs as well as Kinects or Notch trackers.

I’m also working on motion-reactive particle systems that Amit based on distance and speed rather than time. I like the ideas of being able to trace movement without having an actual figure appearing on the screen, on the transient artifacts of motion.

My favorite drawing tools (right now)

The pasts few days I’ve been reviewing all the drawing tools I use and tried out different ones. I found that I actually use a very wide range of tools for digital painting, drawing, modeling, and other forms of computer based general art-making.

These are my current favorites:

(straight up) Adobe Photoshop

Photoshop is great, and I’ve been using it for year and years, though mainly for photo editing and composition. I picked up a Wacom digital pen for the first time and tried to draw a quick sketch in photoshop. While my drawing skills leave a to be desired, I was impressed with Photoshop’s range of brushes, quality of pressure and articulation when using a digital pen and how easy it was to get a feel for expression and control. It was much better than tablet based systems I also tried this week like the Apple Pen and the Microsoft Surface Pen. I’m also very familiar with Photoshop’s photo manipulation and composition features, which will hopefully allow me to incorporate digital drawing into a larger digital art context.

 Butt ugly flower for reference, drawn with a Wacom Cintiq on Photoshop CC

Butt ugly flower for reference, drawn with a Wacom Cintiq on Photoshop CC

TouchDesigner & GLSL

Most of my recent generative visuals work has been in TouchDesigner using GLSL shaders. I love the visual node-based programming environment, how quickly it allows me to sketch ideas and how it all works in real-time with no render times and potential live control from audio and input signals. I use TouchDesigner for more abstract work and real-time composition but maybe it could complement assets made in Photoshop.

 Internal State (Barak Chamo, 2018). GLSL Shader in TouchDesigner.

Internal State (Barak Chamo, 2018). GLSL Shader in TouchDesigner.

Oculus Medium VR

For an unrelated Art Toy character design project I tried drawing and sculpting in VR using Oculus Medium. I found the spatial modeling tool to be surprisingly intuitive albeit very basic. I was able to sketch ideas quickly, modify characters and paint them by manipulating them in space using the Oculus controllers, it was an unparalleled easy modeling experience. I even had the chance to export the models for 3d printing, which was a breeze.

Even though not technically a drawing tool like Quil or TiltBrush, Medium felt less toyish and more practical in asset creation, a lot like a VR Z-Brush. I like it as a space to explore and mess around and potentially even export rough sketches that can be iterated on.


VR Character design experiments

I’ve been experimenting with different approaches to character design, from more abstract form to kit bashing and procedural modeling to create different poses, convert different emotions and make characters that are highly stylized but are still relatable.

I used Oculus’ Medium, a VR sculpting tool, to create different forms and play around with different aspects of character design. In particular, I wanted to see if the key parameters I identified in previous weeks, namely scale and posture, are enough to create a relatable figure even before any features are painted and textures are applied.

I started with simple cubes and spheres and attempted to make a “cute” character, the result was “Block” and “Blob”, two characters made of cubes and spheres. What was interesting to me was how, even though the different parts are fixed, the figures seemed to convey different emotions when observed from different angles. “Blob” seemed both shocked and asking for a hug and “Block” wanted a hug as well, but also looked a bit down, depending on where you thought the head is pointing.

This reminded me of something I red in “Understanding Comics”, on how the more a character is stylized and simplified, the more it relates to a broad audience as they can see themselves in it. The trick, I suppose, is in conveying particular emotions with a limited degree of motion in the design.

Next I tried a few more modeling tools Medium had to offer, like stamps and clay-like modeling. Using stamps I created a “kit-bashed” character that somewhat reminds me of the “Bike Mice from Mars”, one of my favorite shows as a kid, or just a mecha-steam-punk-mickey-mouse. The second character was a sort of devil figure with his tongue out and eyes rolling, I tried Medium’s painting tools with this one too.

Medium has very handy export features so I was able to go from Medium to Cura and straight to the 3d printer in minutes. I tried different settings for printing and learned a lot about how to improve designs for a 3d-printing manufacturing pipeline. While Cura provides automated supports calculation for free-hanging parts, the filament still drips in places and the supports leave rough edges and require careful removal and cleaning. Still, pretty decent for a first attempt.

The final step was to give all the prints a coat of matte grey primer, this really brought out the features as lights and shadows became much more visible (I used clear filament which was very tricky to see through). Priming 3d-printed filament was tricky but with patience, a steady hand and several layers I think I managed a fairly clean coat. Unfortunately, the primer also brought out all the small imperfections of 3d printing.

All in all, I like this quick prototyping pipeline. I’m now working on refined designs in Houdini and Cinema4D and will try to print them on higher quality printers and give them some more finishing love.

Art Toy Concept & Turnaround

This week I explored different forms of art toys, and how figure and posture is used to imply mood and emotion.

I started by sketching familiar characters I like, like Marvin, the robot from “The Hitchhiker’s Guide to the Galaxy”, Danbo the cardboard boy, Uamou & Boo and others. All these characters share similar traits of styled and exaggerated physical features and postures that capture their mood and convey an emotion, even though the painted detail is very minimal.

I then started sketching my character. I knew I wanted to feature the oversized head also found in Dunnys and Munnys but in a posture that is not menacing, rather curious and melancholic - a lot like Marvin the robot, one of my favorite motion picture characters.


Finally, I placed the character in a turnaround sheet. It’s very basic and mainly used to get a sense for the scale of the head compared to the small body and the tilt of the head.

With this character I’m planning to focus on the shape, phisique and posture as I think it’ll be a cool blank candidate.


The Temporary Expert

Part 1 - Energy Field Guide

After reading through Steve Easterbook's Systems analysis of GMO protests in the UK, I decided to approach the system mapping of Caloric Energy from multiple perspectives, understanding the different points of view and boundaries that could be applied to this topic.

Caloric Energy - Systems

Caloric energy can be described in several different ways and, expanding on those descriptions and their perspective, several systems can be mapped:

1. A system of units, measurements and conversion of energy

Scientists across fields established defined units of measurement that quantify and describe anything from time, distance, scale and magnitude. Energy, for example, is measured in Joules and various kinds of energy and energetic substances can be compared and standardized. Caloric energy, in this context, is defined as the amount of energy needed to heat up water by one degree celsius.

The international system of units does not, however, exist in isolation. It is subject to the progress and politics of science, such as the standardization of the metric and imperial measurement systems that are mutually exclusive and incompatible. The process of measurement itself is also dependent on factors such as scientific progress and environmental conditions, the process of measuring calories involves heating water, which requires different amounts of energy depending on local climate and barometric pressure.

2. A system of food, nutrition, nutritional values and physiological energy

Different foods, depending on their nutritional content, contribute differently to a balanced diet, sustaining the human body in both energy and micro and macronutrients. The way in which a certain food's nutritional quality is measured is usually via it's nutritional density, the ratio of caloric energy to nutritional value, calories that come from foods with low nutritional value are considered empty calories.

The calorie intake of an average human varies by country and is used as a benchmark for the average amount of food we must eat daily. Commercially sold foods must be labeled for their caloric value as well as nutritional value. Only some nutrients are detailed in the label and these depend on the governing health and nutrition policy of the local government.

3. A system of marketing, food industry, capitalism, labor and food politics

The food industry, and health foods as well as fast or junk food in particular, are focused on marketing nutritional value, and the impression of such, in their food products. Terms such as "low calorie", "diet" and "zero calorie" are frequently used to persuade consumers of health benefits or positive nutritional balance in these products. Such terms are now regulated in many countries around the world, defining strict standards for what constitutes a "low calorie" food.

Caloric value also plays a role in the actual ingredients of such foods, as economies of scale made it so that "empty calories" or calories from fats and simple carbs are cheaper. The fact that not all calories are equally priced are at the core of a health and obesity epidemic that impacts low income households the most, as they are often unable to afford high nutritional value food and opt for "junk food" that has more marketing value than nutritional one.

Caloric energy is an interesting subject for systems analysis, as different perspectives and system boundaries relate to science, health, politics, business and marketing. The systems described above are only three top-level and distinct ones, though many crossing systems can probably be identified as well.

A Taxonomy of The Science and Business of Food

In creating a taxonomy of The Science and Business of food (measurement), I decided to set a particular system boundary and perspective that will allow me to focus my guide and direct further research. I want to shine a light at how misleading marketing, use of nutritional terminology and buzzwords, fuzzy food science and an overwhelming range of food quality indices work in favor of large food corporations and against the individual trying to maintain a balanced and healthy diet.

In exploring this topic and forming a taxonomy it became ever clearer that the number of nutritional variables that are “scientifically established” are far more than can be expected of a casual shopper to memorize and consider. At the same time, I was surprised to discover how many terms have become regulated by the FDA due to abuse, misinformation and blatant disregard for consumer health by food manufacturers, these includes “low fat”, “zero sugar”, “low calories” and even “artisanal” for crying out loud!

The Science and Business of Food - A Visual System

 An initial taxonomy of Food Measurement Science and Business

An initial taxonomy of Food Measurement Science and Business