Computer Graphics World- Edition 2

Crafting Daemons

Framestore played a key creative and collabo- ensure emotional, realistic interactions be- show meant Framestore drew on skills from
rative role in the worldbuilding process for His tween human actors and fully-CG charac- across the entire company, with the work
Dark Materials, delivering all of the first sea- ters, first-pass takes were conducted with split between offices in London and Mon-
son’s VFX and served as a key creative advisor puppets, which helped provide visual and treal. “It was a team effort on a grand scale,”
from the earliest stages of production. interactional cues. The puppets were then says Fiona Walkinshaw, Framestore’s global
removed from set, and clean plates featur-
This saw Framestore build a produc- ing just the actors in situ were filmed. It was  managing director, film. “We’re no stranger
tion-side visual effects team at Wolf Stu- then up to Framestore’s artists to conjure
dios in Cardiff led by senior VFX supervisor forth the show’s distinctive daemons.  to huge projects and tight deadlines, but His
Russell Dodgson, on-set VFX supervisor Rob Dark Materials really feels like a landmark in
Duncan, and VFX executive producer James “On the one hand, there was the terms of both the quantity and quality of
Whitlam. They provided guidance, advice, challenge of creating so many photoreal VFX shots TV audiences witnessed. It’s a
and input as to how best to bring Philip Pull- creatures,” says Dodgson of the process. perfect example of how film and high-end
man’s unique world to life – from feedback “You’re building each one of these char- TV are converging: There’s a heightened de-
on the production notes to creating the acters from the ground up, which means mand from clients and audiences alike for
show’s lush environments and developing crafting their skeletons and musculature, VFX work that would look as good in your
the nuanced relationships between the and that’s before you even get on to the living room as it would at the IMAX.” 
actors and their daemons.  intricacies of feathers, fur, eyes, and claws.”
Bringing these characters to life was not As well as allowing the company’s artists
The result is one of the most ambitious just a question of volume, however. “It was to flex their creative and technical muscles,
feats of storytelling yet seen on TV, with obviously important to imbue each daemon the work also tapped another of Frame-
over 2,000 shots of spectacular CG and with its own personality,” says Dodgson. “A store’s core skills: the ability to get inside
animation work bringing the richly-imagined deeper concern, however, was how they a piece of existing IP and breathe life into
universe and its colorful denizens to life. “His then interacted with their humans, and how complex creatures, characters, and settings
Dark Materials demanded an extraordinarily their gestures and behavior augmented, that exist in the hearts and minds of a great
high VFX baseline,” says Dodgson.   reflected, or concealed what their human many loyal fans. 
was thinking or feeling.”
Crafting a visual spectacle equal to “This isn’t a world we’re building from
Pullman’s beloved source material drew The scope and scale of Season 1 of the scratch – it’s one that millions of readers
on a wide range of talent, and presented feel deeply invested in,” Framestore CEO
challenges in terms of scope, scale, and William Sargent points out.
storytelling. “Every human character having
his or her own daemon means, theoretically,
every shot that features an actor could also
require VFX,” explains Dodgson. “This level
of work and detail is unparalleled when it
comes to television – the sheer volume of
shots requiring the inclusion of a pine mar-
ten, a hawk, a monkey, or a snow leopard
meant our lighters, animators, and CG art-
ists really had their work cut out for them.”

Framestore created more than 50
distinctive daemons as well as the bear-
like panserbjørn, effectively bolstering the
show’s overall cast with its CGI creations. To

things in previs that we probably couldn’t to get realistic about what’s achievable in how it’s going to be used,” May adds.
make for real, ultimately. But doing those terms of time and money.” Meanwhile, the series recently received
techniques and finding discoveries were
still useful,” says May. “Even though we In addition to His Dark Materials, May two BAFTA awards, including one for visual
were limited by our imaginations in Unreal, and Painting Practice are using the virtual effects. 
we had budgets, health and safety, and a pre-production process on other shows,
million other things to consider. You have which are under NDA at the moment. Karen Moltenbrey is the chief editor of CGW.

“I think we’re still in the early stages about

4 9e d i t i o n 2, 2020 c gw

Spinning a Yarn Images ©2020 Netflix.

NETFLIX’S THE WILLOUGHBYS – NOT YOUR TYPICAL TALE

BY KAREN MOLTENBREY

50 cgw e d i t i o n 2, 2020

Throughout our lifetime, we have parents only have love for each other; they do lovebirds freeze to death. But the ungrate-
read fairy tales of all kinds, nearly all not extend it to the children, who they see as ful pair continue their selfish ways and
of which have happy endings. The nuisances. Better not seen and not heard. commandeer the flying contraption the
Willoughbys, an animated film based on the kids built, abandoning the children on the
book by the same name, is not one of them. The children are mistreated. They are mountain top to a frozen fate.
neither fed nor clothed. Never is a kind word
“If you love stories about families that offered to them. One day the children find But maybe the story has a bit of a happy
stick together, and love each other through an orphan outside the gate to their home. ending, after all, which has nothing to do
thick and thin, and all end happily ever after, They want to keep it, but their parents forbid with the parents. Instead, Nanny and the
this isn’t the film for you,” says the narrator, it. The kids find a seemingly perfect place for Commander rescue the children, who are
a world-weary cat, in the opening of the the baby: at a candy factory. Convinced they truly orphans now, and they, along with baby
animated film. “This family’s story is weird, would be better off as orphans themselves, Ruth, the infant at the gate, become a family
hidden away from the modern world in their the siblings come up with a plan: entice – a perfectly imperfect family.
old-fashioned home.” their self-absorbed parents to take an exotic
vacation to dangerous locales around the This anti-tale, based on the book by
Indeed, the cat’s perception of the story world. As the pièce de résistance, they add Lois Lowry, was brought to life in Burnaby,
is accurate. However, when looking at the the caveat “no children allowed” to their fake Canada, at a greatly expanded Bron Anima-
film as a whole, “unique” seems to be a brochure, ensuring their parents’ interest. It tion, a division of Bron Studios, a Canadian
more fitting description than “weird.” works, and off the parents go. motion-picture company. The film is written,
produced, and directed by Kris Pearn (Open
Unique Story Happy to be “orphans,” the children’s Season, Surf’s Up, Cloudy with a Chance of
staycation is interrupted by a nanny (Maya Meatballs). In fact, Pearn’s varied back-
Inside the big, old-fashioned home filled Rudolph), who the parents have hired to ground in the animation industry (animator,
with antiques is a history that goes back watch the kids. Eventually, the children story artist, character designer, writer, and
a long way, a family legacy of tradition, inven- warm up to her, that is, except for Tim, who director) made him an ideal candidate for
tion, creativity, and courage, as depicted in remains mistrusting. this project, which took on a homemade
the portraits hanging on the walls. The family aesthetic, due to his experience in 2D,
magnificence has been passed down from Feeling guilty about tricking their parents stop-motion, and 3D animation.
generation to generation (like their promi- (even after Mom and Dad sell the house so
nent red, bushy facial hair) – that is, until the they can afford to continue their journey), Producer Luke Carroll optioned the book
current one. the children enlist Nanny and a generous in 2014 after listening to an audio version on
candymaker, Commander Melanoff (Terry a family vacation. Pearn signed on, and work
The house now sits in the middle of a Crews), to help them build a candy dirigible began in earnest two years later and finished
modern, bustling city, closed off from the so they can rescue their parents before just about two weeks before North Amer-
rest of the world. Living amid all the books, tragedy befalls them. However, the children ican went into lockdown due to COVID-19.
paintings, and heirlooms is the latest family sneak off in the airship by themselves and “Taking the tropes of animated films and
of Willoughbys: Mother (Jane Krakowski) and fly off to the “Unclimbable Alps,” the last having the opportunity to play with them to
Father (Martin Short), who are selfish and stop in their parents’ trip. The children tell a story that I think is different in a way
lovestruck. They share the space begrudg- succeed in their mission, following a trail that celebrates the independence of kid
ingly, their children: Tim (Will Forte), Jane of Willoughby-red yarn up the mountain logic and hopefulness was an interesting
(Alessia Cara), and the Barnaby twins (Sean and arriving in the nick of time before the challenge,” says Pearn.
Cullen). While love is an amazing thing, the

The Willoughbys, directed by Kris Pearn, is based on book by Lois Lowry.

5 1e d i t i o n 2, 2020 c gw

The Willoughby children long to be rid of their parents.

Unique Design erything in stop motion is handmade, so we Textures were key to achieving the tactile
started wondering, what if everything in this aesthetic of the characters. The way Pearn
The artwork from the book is sparse, giving world was handmade? Hence, countless used the cameras also provided a stop-mo-
production designer Kyle McQueen a clean frames [are] overflowing with textures and tion feel. To this end, McQueen devised an
palette to work from. The design he and the toy-like components.” environment that could be captured without
filmmakers chose is certainly atypical, one a lot of digital moving  cameras, and Pearn
that feels homemade. “The stop-motion Unique Animation Approach decided to use minimal motion blur, citing
influence in The Willoughbys came from this Hal Ashby’s ​Harold &  Maude​and three-cam-
idea of an old-fashioned story – kids growing As big a role as stop motion played in era sitcoms as inspirations (see “Inspiration,”
up on books rather than the Internet – and defining the film’s look, it is Pearn’s 2D page 55). “When we do move a camera, you
so we really wanted to dive into that from animation background, with its hand-drawn feel it,”  says Pearn. “We tried to pick cameras
a visual standpoint and create something approaches and pose-to-pose animation, and do setups that gave us a lot of width. We
that didn’t feel digital,” says McQueen. “We that held sway when it came to making the use a lot of depth of field so that your eye
wanted a world that really felt tactile and characters move. always knows where to go. It does kind of feel
visceral, like turning the pages of a book and like miniature  photography. And that’s a big
smelling the ink.” “Early on we looked at 2D pose-to-pose part of the stop-motion appeal.”  
animation, and I challenged the animators
Adds Pearn, “I wanted the audience to to create a movement style for each char- Creatively, the lack of camera motion also
think they could go to Michael’s [craft store] acter based on who they were,” says Pearn. gives the sense that the children are stuck in
and buy all the materials to make this movie.” For example, Tim is very still, almost like a the house. “I wanted to shoot the house like
grown-up. Jane, meanwhile, represents the it was a practical set, like a sitcom. Locking
The characters are disproportionate in future, she just wants to escape. She’s like a the camera allowed us to give the audience
shape. But what stands out the most are bird, fragile but tough, which played into the that three-camera setup feeling where
the textures. The Willoughby family, as way she moved through the use of arcs.” there are long takes, you trust the acting
generations before them, have big red hair and let the composition do the work,” Pearn
that looks as if it is made of yarn, a notion While the film is informed by a stop-mo- explains. “On the flip side, when we unpin
that is emphasized by the fact that Mother tion aesthetic and 2D animation principles, the camera, it drives a story point. When
is obsessed with knitting. Nanny also sports the world of The Willoughbys is neither in the children leave the house, we let the
a big head of hair, only hers is dark brown, actuality. Rather, it is 100 percent CGI. And, camera dolly up, because that telegraphs to
curly, and heart-shaped. “tricking” the computer into believing oth- the audience that we’re now loose and out
erwise during the creation process was the
Likewise, all the clothing looks handspun, biggest challenge Pearn and the crew faced.
too, like you would see on a stop-motion
character – making them look like minia- The Willoughby history of greatness skipped Dad’s generation.
tures. That is, except for Tim, who is wearing
a too-small explorer-like outfit he outgrew
many years ago.

“When I pitched the idea to Kyle Mc-
Queen, he immediately got excited by the
notion of boiling down the shapes to simple
cartoon principles with heightened textures,
almost in the vein of stop motion,” says
Pearn, who spent part of his career at Aard-
man Animation, known for its stop motion
productions such as Shaun the Sheep. “Ev-

52 cgw e d i t i o n 2, 2020

Meet The Characters

Jane (Alessia Cara)  Nanny (Maya Rudolph)  on his doorstep. Once the siblings start
tinkering in his high-tech factory, Colonel
Jane Willoughby, the family artist, is a joyful A huge-hearted optimist, Nanny is hired to Melanoff has softened into a generous spirit
and curious 12-year-old. Though outnum- take care of the Willoughby children while and doting dad.   
bered by her three brothers, she is full of their parents travel. Once she realizes that
ideas, opinions, and daydreams, and is eager they’ve had a sheltered upbringing, Nanny
for her voice to be heard. introduces the kids to new food, music,
and views, becoming their tour guide to
modern life. Always bursting into melodies
and giving out hugs, Nanny emerges as the
comedic backbone of the story.  

Tim (Will Forte)   M& oFtahtehrer(J(aMnae rKtirnakSohwosrkt)i ) The Cat (Ricky Gervais) 

As the eldest The selfish Willoughby parents pay little Since the adventure unfolds from this four-
Willoughby child, attention to anyone except each other. legged narrator’s viewpoint, the filmmakers
Tim fancies him- Hidden away in a mansion full of antiques, designed the elements on-screen to resem-
self their leader. Mother and Father entertain themselves ble a collection of colorful, handmade toys.
He has firm with old-fashioned crafts like knitting and At first, The Cat is a jaded onlooker prowling
beliefs about how building  ships in bottles —​ until their kids through the Willoughby home, pausing to
his family should trick them into booking a vacation. On that lick himself and grumble about the latest
act to maintain human happenings. Yet he roots for the
its legacy, which  trip, they kids and
causes some ar- learn increasingly
guing with Jane. A about the feels like part
brave, energetic, Internet of their
and scholarly  and family, so
14-year-old, Tim has endless potential ​— ​if decide to much so that
he can give up being in charge, listen to his sell their he creates
siblings, and have a little fun. home a diversion
from afar. to reunite
Barnaby A & B (Seán Cullen)  them with
C(Toemrrmy CanredwersM)  elanoff Nanny. 
Whether finishing each other’s sentences,
solving complex math problems, or building A world-famous candymaker whose face Ruth​(Brian Drummond) 
gadgets from scratch, the young twins beams from bus and TV ads, Commander
always act as one Willoughby. Clever go-get- Melanoff is actually quite lonely. New ac- Pink-cheeked baby Ruth will scale any sur-
ters, Barnaby A & B love learning and follow- quaintances tend to be intimidated by his face and gobble every snack. After Mother
ing instructions word for word. They admire size, uniform, and baritone bellow, but not and Father refuse to take her in, the giggly
their older siblings very much, often acting Nanny or Ruth —​ ​the infant he discovers youngster brings the Willoughby siblings
as their referees.    and Nanny into Commander Melanoff’s life.
She also sends the kids on their first trek
into the city, foreshadowing their eventual

globe-trotting
journey. 

5 3e d i t i o n 2, 2020 c gw

Cat’s Tale The film contains a homespun aesthetic throughout.

Director Kris Pearn wanted The Wil- into the world. When the children make the the family’s signature red hair as well as the
loughbys to be a fairy tale. To this end, choice to come back and stay at the house, cotton candy elements.
he decided to have the cat (voiced by the camera is locked again.”
Ricky Gervais) narrate, enabling the “By controlling the movement as opposed
director to push the animation a little To further implement the less-is-more to straight simulation, we could make the
further over the top. approach of simple, graphic character de- hair feel designed, and we could respond to
signs, the  crew coined a term related to T​ he what we were learning from the textures,”
“The movie starts off from the Willoughbys’​ cinematography: variable frame explains Pearn. “Establishing this style of
point of view of this external charac- exposure. Today, standard features are filmed movement also gave us cost savings in
ter, the cat, who’s looking through a at 24 frames per second, with every frame various conditions, such as rain, snow, and
window. It’s the cat’s tale, told from displaying a different image. With variable wind. The graphic quality of the yarn and the
its point of view, so it might not be frame exposure, the filmmakers showed rich texture added to the handmade feeling
100 percent real,” Pearn explains. certain images for one,  two, or three frames. by not overreacting to the FX elements,
“That gave us a sense this could be “You might be giving them fewer images per which we wanted to feel comp’d over the
a once-upon-a-time sort of world. second, but it never came at the cost of less movement, rather than embedded.”
That let us pivot into the world of information,” maintains Aniket Natekar, one
metaphor. And because we had of the film’s lead animators. Unique Effects
these heightened textures, we had
permission to then push the choices Pearn explains this further. “Instead of While the imagery in the movie’s foreground
we would make in terms of posing splining the animation so that it’s full-frame is 3D, the backgrounds were matte paint-
and pushing our characters toward range, 24 fps, we would do it on twos for ings, again supporting that homemade feel.
that almost broad Cartoon 101/ the characters, and then we would do our “I loved the idea that behind a certain layer
Chuck Jones approach with simple effects on threes and fours. The feeling that of depth, there’s a painting,” says Pearn.
silhouettes.” I was trying to achieve was similar to when “Also, we had to commit to the locked
[filmmakers] would use film and shoot their camera idea so it felt like a sitcom. I found
Nevertheless, the animators primary layer and roll it back, and do the ef- that soundstage feeling very attractive to
always maintained gravity, the sense fects on a pass-through. I wanted things to the story.”
that the world was real. If a character feel like they had a little bit of an off-timing.
fell, he or she would hit the ground Of course, the computer really likes it when A good portion of the film takes place
and likely react accordingly. things make sense. So, trying to figure out inside the house, where Pearn found the
how to work with the software and the pipe- children’s library to be an especially difficult
“We wanted to have that phys- line to give us the robustness of the textures environment due to the extreme detail and
icality,” says Pearn. “I wanted it to that we were after, and the timing we want- the difficulty in lighting it so the viewers’
seem like there’s always bones in ed, was really a challenge. But, we had some eyes knew where to look. “I wanted the
the characters, but there are bones really smart people working on the film. Russ room to feel rich. It’s sort of a metaphor;
that have a lot of loose joints when Smith [VFX supervisor] was able to get in it’s full of all the ideas bubbling in the twins’
they hit big poses. And yet, they are there and add scripts and break the widgets, heads,” he says. Grading was also challenges
not rubbery.” and get [the software] to do what we were to ensure the characters stood out among
creatively chasing. Plus, it was a small studio, all the detail.
54 cgw e d i t i o n 2, 2020 which enabled us to be really collaborative.”
The snow near the end of the film was
Bron Animation was already established also challenging in terms of establishing the
as an Autodesk Maya-based studio, which right combination of 2D and 3D. It took a lot
the Willoughbys team used as well. The of trial and error, Pearn says, to get the snow
artists also used Pixologic’s ZBrush for mod- to feel like it was applied to the scene, but
eling. In addition, they used Maya’s XGen also gave the audience the sense that the
geometry instancer for populating polyg- world is very cold.
onal mesh surfaces with primitives, either
randomly or uniformly placed, to generate “We had this amazing artist, Helén
Ahlberg, who was our effects lead. She was
a 2D animator by training, but had that

The animated film was crafted at Bron Animation. Inspiration

superpower where she could step between fours, instead of the normal ones for effects, The Willoughbys is a CG-animated
the 2D and 3D worlds,” Pearn adds. so they wouldn’t stand out too much.” film, though it looks as if it is a hybrid
production of 3D, 2D, and stop
According to Ahlberg, effects are usually In addition to fire, the group tackled other motion. Also, it uses techniques from
accurate calculations and simulations, but elements. For smoke or dust or clouds, they dual mediums, infusing the camera
for this feature, the filmmakers wanted the tried to make them look like cotton or even style and more used on various tele-
effects to feel homemade, to fit with the spun cotton candy. And for water, Ahlberg vision sitcoms within this feature-film
overall aesthetic. “That was definitely a big didn’t feel like fluids would really fit the style production.
and extreme challenge for our team,” she of this film. “So, we thought more about the
says. “We would use the 2D mind-set of material and played  around with reflec- “Visually, one of the things early
building them by hand, but we would do it in tive Mylar strips, just deforming them and  on I really wanted to achieve was
the computer.” breaking them apart,” she says. “Splashes the collision of filmmaking styles. I
would break into little pieces of paper wanted the time that the kids spend
The production designer wanted fire instead of droplets.” in the house to feel like a sitcom,
to look like cutout paper and snow to feel where we really lock the camera and
like 2D confetti flying around. To this end, There is also a rainbow made out of have three-camera setups and long
Ahlberg and the visual effects artists drew cotton candy. The effects department also takes. And, I wanted the rat-tat-tat
inspiration from a number of places for the handled some of the cloth, such as a carpet of the dialog to remind people of that
work, from stop motion, to paper cutouts, or flag. And, the group was in charge of Chuck Lorre kind of time,” explains
to children’s illustrations, focusing on the paper electricity, bubbling soda acid fields, Kris Pearn, director. “And then when
craftsmanship and then mimicking that and lava. they leave the house, the camera
in the computer using Maya and SideFX’s unpins and the colors change; the
Houdini. For rendering the FX, the group Unique Ending idea is that outside of the gate is a
used Mantra within Houdini. (The lighting/ movie. That kind of collision between
comp team used RenderMan within Found- In most coming-of-age stories like this, those two worlds was something we
ry’s Katana for rendering and Foundry’s there would be a fairy-tale ending. The really got excited about early on in
Nuke for compositing.) parents would see the error of their ways, the process.”
and the family would reunite and live hap-
Their approach varied somewhat pily ever after. Well, that was not the case Pearn was particularly influenced
depending on what was needed, but it all here, obviously. by TV shows he grew up watching,
started by first sketching out the effects such as Three’s Company, Cheers, and
by hand for timing. “We had to really think Still, the kids do get a happy ending, but All in the Family, whereby the char-
creatively and go outside the box a bit,” says not the one the audience expects. And the acters might be wrong about their
Ahlberg. “We didn’t come up with any new parents, well, that’s another story with… and opinions of the world, but there’s an
groundbreaking technology. It was a matter abrupt ending. underpinning of optimism.
of taking a unique approach that would
give us very strong silhouettes and graphic Likewise, viewers probably did not expect As for the mixed aesthetic, Pearn
shapes on all the effects.” the unique aesthetic of this film. “We really was also influenced by his stop-­
have tried to create a hybrid world that looks motion background, but also by his
For instance, when crafting fire, Ahlberg like a  cross between CG and stop motion. 2D animation experience. “I still love
wanted something iconic that felt like fire This is a movie that looks like nothing else the choices that come from pose-to-
but didn’t have big, fluid movement like the out there,” Natekar says. pose animation, like when you watch
element usually does. “We played around Chuck Jones or the old Disney hand-
with triangular shapes and would hand- Adds McQueen: “So much love and care drawn films like 101 Dalmatians or The
sculpt them to look like they were made went into this film, and I really hope that au- Jungle Book, that feeling that there’s
out of paper. And then we put a light that diences find it and love it as much as we do. a hand behind the pencil,” he says.
flickers inside to symbolize the chaos that It’s such a weird, funny, emotional film, and I
comes from fire,” she adds. “But when you think it’s very unique in the animation space.” Despite the stop-motion and 2D
look at it, you know it’s fire. We’d even have influences, Pearn never considered
some things that were on twos or threes or We couldn’t agree more.  creating The Willoughbys as anything
but a CGI film. In this vein, practicality
Karen Moltenbrey is the chief editor of CGW. reigned, as Bron Animation already
had a 3D pipeline in place and was
able to draw on the abundance of 3D
talent in the Vancouver area to create
this film.

5 5e d i t i o n 2, 2020 c gw

Classroom Disrupted

TEACHING THE NEXT GENERATION OF ANIMATORS DURING COVID

BY KAREN MOLTENBREY

In this era of COVID-19, educational in-person when on-ground courses resume. into the university’s award-winning studios,
institutions have had to rethink the way “Our successful spring and our ambitious, classrooms, and historic buildings while
they teach. In the spring, schools had to adhering to health and safety protocols.
move to online/remote learning with just a thoughtful plans for fall are not really about A SCAD task force, in close consultation
moment’s notice. With many animation and following some mythical ‘playbook.’ This is with outside health and safety experts, has
visual effects schools/departments current- simply SCAD at its finest: inventive, strategic, created plans for maintaining appropriate
ly on summer break, institutions have been positive, collaborative, transformative,” social distancing, with enhanced cleaning of
using the time to look beyond their earlier says SCAD president and founder Paula high-touch areas, as well as recommenda-
Band-Aid approach and more thoroughly re- Wallace. “Very soon, our beloved campuses tions on PPE requirements where appropri-
search and evaluate their teaching methods will reopen and students will swarm joyfully ate, especially in situations where full social
and directions as the fall semester quickly through our doors and classrooms once distancing is challenging.
approaches. again. It is a new day in academia, and SCAD
is poised to shine on and keep shining, SCAD classes are already small in keeping
We spoke to a number of animation/ thanks to the resilience, light, and love of this with the university’s mission to ensure all
visual effects schools to find out what types heroic learning community.”  students receive individual attention in a
of alterations they had to make to finish up positively oriented environment. The average
the spring semester, and how that affected In March, SCAD launched a new ini- class size is 21 students, and lecture-style
the students and curriculum. We received tiative, Guests and Gusto, a Zoom series classes are capped at 35 students. 
various responses. We also asked what kind programmed exclusively for SCAD stu-
of changes they are looking to adopt for the dents during the university’s virtual Spring Real-time virtual instruction, what most
fall and what kind of impact this ongoing quarter 2020. The dynamic series, which will SCAD students experienced during the
crisis might have on student learning. continue, features new weekly content that spring quarter, will continue alongside on-
represents all of SCAD’s 40-plus top-ranked ground learning this fall. 
Indeed, plans are under way to provide degree programs through illuminating con-
the best experience under the circumstanc- versations, surprise drop-ins, and interactive SCAD eLearning allows students to work
es. (Keep in mind, the schools were queried games.  Guests have included:  author and independently, explore content, participate
in early July, just about the time when severe New York Magazine art critic Jerry Saltz; in online discussions, and move at their own
outbreaks were re-occurring. So, schools award-winning producer and songwriter pace on a Monday-through-Sunday sched-
may be forced to alter their plans once again Tricky Stewart; TEDtalk’s head of television ule. SCAD eLearning offers 24 graduate and
before fall semester begins, as the pandem- Juliet Blake; film director and producer undergraduate degrees in everything from
ic situation remains fluid.) Morgan Neville; actor Alan Cumming; and animation to interior design.
GQ creative director and author Jim Moore.
  Students also engaged with industry leaders  “Because SCAD has always been com-
in their individual classes, which led to many mitted to the ‘what’s next’ philosophy and
SCAD students receiving job offers and internships.  execution in professional practice, I see our
students excelling in this digital and virtual
Savannah College of Art and Design (SCAD) Meanwhile, SCAD commencement 2020 wave. Twenty years ago, we took art and
is on the quarter system, not semesters, was a high-energy, live-streamed event. design education online; over a decade ago,
with its traditional academic year span- SCAD celebrated its newest alumni with a SCAD was one of the first universities to use
ning Fall, Winter, and Spring quarters, with personal digital commencement experi- REVIT as a required part of the curriculum,
Summer quarter optional. Earlier this year, ence, featuring speaker Alicia Keys and host now it’s an industry standard,” says Anthony
two weeks before the Spring quarter started, Rachel Brosnahan. J. Cissell, AIA, chair of Architecture and
the university announced the need to move Urban Design.
all courses and instruction online due to the  For the Fall 2020 quarter, SCAD will offer
virus. SCAD wanted to be sure students had real-time on-ground instruction, real-time Virtual reality and the online collabora-
that time, as well as the first week of class, to virtual instruction, and anytime online tive studio is going to be a standard of the
decide whether they wanted to take classes instruction. These methods all individual- architectural practice of the future, and
in a completely online environment, or drop ize the acquisition of knowledge and the this is where our students, who are digital
courses and receive a full tuition refund. advancement of student outcomes. natives, are going to be professionally ready.
“Here at SCAD, we are already working in the
Furthermore, if a student was dissatisfied SCAD will resume real-time on-ground virtual reality realm through our programs
with an online course experience this past instruction at SCAD Atlanta and SCAD in Motion Media, Digital Effects, and Game
spring, SCAD has offered the option to re- Savannah this fall, inviting students back Design. Now we’re exploring how to bring
take that course for no additional tuition and

56 cgw e d i t i o n 2, 2020

those cutting-edge resources and strategic and options to online and on-site students, submission deadlines were extended also.
partners to our Architecture students to po- including face-to-face contact via Zoom with “Students were still able to receive the
sition them as leaders in this virtual model of instructors and other students, Tate notes.
professional practice,” says Cissell. experience they expected, which was
School of Visual Arts incredible given the circumstances,” says
Academy of Art University Anna Hodge, training & education manag-
School of Visual Arts (SVA) suspended er, Rising Sun Pictures. “With protocols put
Academy of Art is but one of the many in-person classes beginning Wednesday, in place, students said they felt safe at-
institutions that swiftly responded to March 11, and began remote learning on tending classes during the pandemic. They
address the pandemic issue this past spring Monday, March 16. “In the midst of this also enjoyed the amendments that were
by locking down its San Francisco campus global health crisis, it was critical for us to made to the course due to social distanc-
and moving classes online and virtually. It continue on with the semester so that stu- ing, and were grateful for the experience
may seem like such a drastic shift in learning dents’ progress towards their degrees went and opportunity.”
environments, but the truth is that it was uninterrupted,” says Jimmy Calhoun, chair
a transition that required no fanfare on the of Computer Art, Computer Animation, and RSP wrapped up the semester with
school’s part because it was an environment Visual Effects. students presenting their work in the RSP
already set up and functioning in place. theater to key artists. Pre-COVID, the FX &
SVA, like every single free-standing, Lighting and Compositing & Tracking stu-
The university’s online education program, accredited college of art and design, made dents viewed one another’s presentations
established in 2002, allows its students and the best of it via Zoom and the school’s on- together. However, with restrictions placed
faculty to carry on as normal in their classes, line learning platform, Canvas. Once remote on room numbers, the groups were split into
albeit with certain modifications. Make no learning began, the school used RemotePC two separate sessions. Zoom was set up
mistake, our online courses are designed to to provide students with remote access so the students in the other session could
be just as rigorous and comprehensive as to the facility. In addition, many software still watch the presentations, albeit from
those taken on campus, says Catherine Tate, companies graciously provided the stu- another room.
director of the School of Animation and dents at SVA with licenses for the remain-
Visual Effects. der of the semester. “We’re very grateful for Two weeks after the students complet-
how generously and quickly they provided ed scheduled classes, RSP invited other
“It will require just as much effort and support,” says Calhoun. local visual effects companies to come
commitment to make the grade. To put it and view the students’ work and provide
simply, our students haven’t missed a beat.... Calhoun also applauds the dedication individual feedback. According to Hodge,
It’s a challenge that puts a lot of systems of SVA’s faculty to unwaveringly deliv- the students valued the networking experi-
to the test, but it’s a challenge that we’ve er their course content, even for studio ence and appreciated the feedback about
long been prepared to ensure our students’ disciplines, and who will continue to meet their projects. Already, she has received
education is not interrupted,” she adds. For and engage with students virtually until follow-up emails from employers making
instance, the school’s Spring Show was held restrictions are lifted. queries about the graduates.
virtually this year and was a success. 
The Fall 2020 semester at SVA is sched- LA Film School
For now, Academy of Art University will uled to run September 28 through Decem-
continue to offer its remote and virtual ber 22, spanning 12 weeks, with extended The Los Angeles Film School was able to
classes for the Fall 2020 semester. Howev- class durations. The spring 2021 semester is maintain its graduation ceremonies earlier
er, the school will be opening its labs and scheduled to start on the previously estab- this year, albeit in a virtual setting with the
workshops for studio course homework lished date of January 11, and is to run for the plans to have a larger in-person graduation
along with its residence halls and dorms. full 15 weeks.  when it is safe to do so in the future. 
“We will be taking precautions to protect our
campus community by utilizing virtual class- Upon reopening, SVA will follow govern- Meanwhile, campus classes were re-
es for lecture, demo, and critique (which we ment guidelines and best practices in higher vamped to be 100 percent online as of this
feel is an unnecessary group risk to perform education for social distancing, which could interview, in addition to the school’s existing
in-person), and allow students to reserve include some or all of the following measures: fully online programs, and will remain this
facility time for on-site labs, studios, and limiting class size, wearing masks while inside way until it is safe for students to return
workshops,” says Tate. campus buildings, and limiting the size of to campus in line with the CDC and local
campus functions and activities in accor- governmental guidelines. According to a
In addition, Academy of Art has been dance with state and local requirements.  school representative, it is also studying the
forced to accelerate the StudioX process, guidelines of the DGA, major studios, and
a student-run, faculty-mentored experi- Rising Sun Pictures the local IA and SAG AFTRA, and will make
ence that emulates a real-world animation adjustments to the curriculum to support
and VFX production environment where There was minimal disruption to the courses the new reality of making films in a post-
students work with real clients, on real films and the curriculum when COVID-19 hit at COVID-19 world. 
and games, with real deadlines – and real Rising Sun Pictures Education in Australia.
problems to solve. The situation also is It took some time to pivot, but the time lost Karen Moltenbrey is the chief editor of CGW.
enabling the school to offer more flexibility to set up the new classrooms was added to
the end of the course, plus a few extra weeks;

5 7e d i t i o n 2, 2020 c gw

Ahead of the Curve

ANIMATION MENTOR HAS LONG PRACTICED ONLINE LEARNING FOR ANIMATION STUDENTS

BY KAREN MOLTENBREY

While many animation and visual available to listen and help find solutions for personal than a brick-and-mortar school,
effects colleges and education struggles the students may have encoun- and we wanted people to make friends and
institutions have had to quickly tered. This included providing assistance for connections they’d have for life,” says Beck.
pivot to virtual/online learning due to the issues related to financial struggles, or even “The cool thing is that those friendships
novel coronavirus, not so for Animation Men- lack of motivation in the face of uncertainty. could be across continents and come from
tor. The highly regarded animation school, in They also instituted extra resources on a different cultures, bringing their voices and
fact, was founded on the premise of distance weekly basis, including an office hours-style their stories to the world of animation.”
learning 15 years ago, pairing up its all-remote live session with animator and mentor Paul
student body with mentors, experienced Allen. This gave students an additional way The founders are as confident in their
working professionals in the industry. to connect, feel supported, and talk through decision now as they were 15 years ago, as
any issues (creatively or otherwise) they the advantages to this model are many. For
Since its founding in 2005 by three may be facing. instance, it allows people from anywhere in
working animators – Bobby Beck, Shawn the world to learn alongside equally passion-
Kelly, and Carlos Baena, from Pixar Anima- Nevertheless, there were some changes/ ate peers, and it allows people who are more
tion Studios and Industrial Light & Magic precautions taken by the school to keep introverted to have their voices be at the
– Animation Mentor has developed and the education process running smooth- same level as those with louder voices. In
perfected a curriculum and infrastructure ly. For instance, Animation Mentor has addition, students can see other students’
specifically tailored to remote students.   been using Zoom for all its online teacher/ work and mentor feedback on the virtual
student meetings for years, but recently campus, regardless of the class they’re
The structure of the school’s courses and implemented additional security features to enrolled in.
the nature of its community have always ensure that the learning experience wasn’t
been virtual, so the online transition that interrupted. “Zoom became the go-to for “An open campus means they can learn,
many other educational institutions were online meetings this spring, and with that not just from their own class group, but
forced into earlier this year did not alter additional scrutiny, instigators bombarded from other students as well. This facilitates
Animation Mentor’s own teaching process- the Zoom platform. The security features a sense of mentorship, with the advanced
es. The school believes in smaller class sizes we enabled ensured that only our students classes helping newer students gain their
(which it limits to nine students per instruc- and mentors could join each meeting (for confidence and footing,” says Beck.
tor) and experienced, tech-savvy mentors. Q&A),” Beck says.
It also runs on its own proprietary platform, Not only do students join Animation
which keeps students and mentors engaged Out of the gate, Animation Mentor Mentor to learn a new creative skill like
and encourages its community to make embraced online instruction. Ambitious, animation, modeling, storyboarding, visual
friends and connections. yes, but the founders believed it to be the development, and so on, but they also
right move, even amid many warning against come out with so much more, according to
At a time when many schools are an Internet business at a time when the the school. They build confidence in them-
scrambling to shift its education model from dot-com bubble had just burst. “Despite selves, and they make incredible, lifelong
in-person learning to online learning due to the bubble, we knew that we wanted to be friendships and connections, both with
COVID, it was pretty much business as usual online. We were part of awesome creative their mentors and the Animation Mentor
at Animation Mentor, at least from a curricu- communities like CGChar, and we wanted staff. There have even been over two dozen
lum and platform standpoint. “We made to take things to the next level. We wanted marriages from students who have met
sure our students knew we were available if to bring animation to people in their homes through Animation Mentor. 
they were in need of help, and we also made and bring the top talent in the animation
sure to put additional resources in place to industry directly to people so they didn’t Of course, there are challenges to online
help students manage their focus time,” have to move across the globe to access learning, as many other colleges are just
says Beck, CEO and co-founder of Anima- them,” says Beck.  now discovering, such as accountability
tion Mentor. and potential feelings of isolation. Never-
The founders also liked the fact that there theless, Animation Mentor recognized and
Even though Animation Mentor wasn’t weren’t any models for online animation addressed these early on, and from the
negatively affected by stay-at-home orders schools, allowing them to define what that beginning, focused on making the school
like other schools, some of its students’ could look and feel like. more than just a place to learn, but a place
lives were heavily impacted by the virus. To to come together and connect as well. As
this end, the school staff made themselves “We wanted to make online feel more a result, students feel engaged, even when

58 cgw e d i t i o n 2, 2020

they’re spending the majority of their time Work by Antonio Vieira, from the 2019 Animation Mentor showcase.
alone in front of their computers.
Work from Cinthia Mussi, also from the same showcase.
The school also stresses the importance
of deadlines and following the mentor’s Work from Daniel Vargas, from that showcase as well.
(director’s) instructions. More than just
learning the art of animation and other try go from a very low percentage of remote over the last six months. “People are giving
artistic subjects, the school also teaches workers to 100 percent remote workers in online learning a real chance, and they want
students good work ethics and habits. “We record time. It’s something we’ve always to see if it really is ‘as good’ as their brick-
want students to be prepared for the studio believed in, as it gives artists the opportunity and-mortar school. We hope that people
environment, and having a strong work ethic to have more balance and gives the industry are pleasantly surprised. It’s not something
is a big part of that,” says Beck.  the opportunity to tap into talent that they we had to piece together overnight,” Beck
may have had to pass on previously due to points out.  
The core teaching model at Animation visa restrictions,” he says. 
Mentor has remained the same over the Karen Moltenbrey is the chief editor of CGW.
years – generally a small class size to each As far as Animation Mentor is concerned,
mentor, so that students get more indi- it has seen an increase in the student body
vidualized attention. The curriculum and
technology, however, are always evolving.
The school is constantly looking at how its
classes are performing, and make routine
adjustments to the curriculum to ensure the
best learning experience. 

Furthermore, the school is constantly
evolving its technology and platform, creating
custom-developed tools to help support its
teaching model. This includes feedback tools
where mentors (and any other community
members) can directly draw over and com-
ment on student work. Animation Mentor
also built a Web-based pipeline for students
so that they can begin learning studio work-
flows while in class, as well as gaining a host
of other useful features from that system.
The core teaching model at Animation
Mentor typically involves some pre-recorded
material that covers the key learning goals
for the week. That’s followed by a live Web
meeting between the mentor and students.
Students get an assignment to complete by
the end of the weekend, and then the mentor
will record a video critique of the assignment,
noting what’s working and what could be im-
proved. This continues each week as student
learning and growth continue.

Animation Mentor offers four terms
per year – Winter, Spring, Summer, and
Fall – and all of the courses and workshops
start at the beginning of each term. Since
the stay-at-home orders began in March,
Animation Mentor has seen an increase in
enrollment, something it attributes to the
fact that people are at home with additional
time to focus on animation.

Will this trend toward remote education
and even remote working continue across
the industry? Beck believes so. “Firstly, it’s
been interesting to see the animation indus-

5 9e d i t i o n 2, 2020 c gw

Hybrid Drama

THE BLACKLIST’S OUT-OF-THE-BOX PRODUCTION STRATEGY DURING THE PANDEMIC

BY KAREN MOLTENBREY

If necessity is the mother of invention, problem was, that episode was only halfway and integrated into the live-action that had
then adversity can be considered its through filming in New York. already been shot.
father. Both are fitting descriptions of the
tack producers of the NBC crime thriller The Nevertheless, by thinking outside the box Creative Problem-Solving
Blacklist took after filming abruptly ceased and using CG animation to fill in the missing
when stay-at-home orders were instituted segments, the showrunners were able to The Blacklist stars James Spader as Ray-
because of COVID-19. accomplish their goal with a very unique and mond “Red” Reddington, a former US Navy
suspenseful episode that leaves audiences officer-turned-high-profile criminal who
Unlike some television series that had counting the days until the series returns. agrees to work for the FBI to capture (or kill)
already completed production and even the world’s most dangerous operators and
post-production for the season, The Blacklist This is not the first time that CGI has terrorists on the government’s “Blacklist”
still had a number of episodes to go before saved the day for films and TV series that (in exchange for immunity) – all the while
its Season 7 finale. In fact, The Blacklist was in have encountered unforeseen obstacles following his own secret agenda. The twist:
the midst of shooting Episode 19 of 22 when during the middle of shooting, mostly involv- This antihero will only work with the young
the coronavirus lockdown went into effect. ing actors who have unexpectedly passed agent Elizabeth Keen (Megan Boone), who
away before filming was completed (Paul may or may not be his daughter.
What to do? The network already had Walker in Furious 7, Oliver Reed in Gladiator,
renewed the series for an eighth season, and Brandon Lee in The Crow, and Nancy March- Season 7 Episode 19, called “The Ka-
audiences expect an edgy show, especially and in The Sopranos, to name a few). But, zanjian Brothers,” finds the FBI Task Force
one as dramatic as The Blacklist, to close what happens when the situation adversely investigating an accountant with criminal
out the season in a thrilling, suspenseful affects not just one person, but the entire clientele, and to find him, they try to locate
way. Realizing that completing the planned cast – and crew? The solution was CGI. Only a pair of brothers who are hired to protect
season-ending Episode 22 was out of the this time, entire sequences were animated him. Meanwhile, the cat-and-mouse game
question, the producers believed Episode 19 in a comic-book/graphic novel style – between Liz and Red is coming to a head,
would make the second-best ending. The including actors as well as environments – as is Liz’s relationship with her mother, the

60 cgw e d i t i o n 2, 2020

dangerous Katarina Rostova, as Liz is forced series, the showrunners had an aesthetic in recent years. But, it was not quite ready for
to choose sides in the deadly game between mind; they also pointed to animated films this prime-time application. To this end,
Red and Katarina. such as those by Richard Linklater (Waking the Proof team in Atlanta began develop-
Life, A Scanner Darkly). “They were fond of ing special shaders for the imagery within
Shooting in New York was only partially rotoscope-style animation, and were familiar Autodesk’s Maya.
complete when the cast and crew were sent with the work at Proof,” says Matt Perrin,
home. During brainstorming sessions, series senior visualization supervisor at Proof in Lon- “Because we were pitching for final
creator Jon Bokenkamp and executive pro- don. “That was kind of the jumping-off point delivery on a show for broadcast rather than
ducer John Eisendrath mulled over a number for us, and we developed the look from there, for previews, we had to redevelop our toon
of options, including a live cast reading to fill creating our own toon look that was more shader to add more detail to the models and
in the missing shots. Then they pondered sophisticated [than what we use for previs] to get a closer likeness to the actors with
whether a hybrid live-action/animation and more in line with what they wanted.” hatching on the faces,” says Coglan.
approach would work. After all, The Blacklist
has been made into graphic novels in the The work was indeed a departure from Animation Meets Live Action
past – not such a large leap given the show’s the typical animations Proof usually does for
pulp-fiction feel to the various blacklisters visualization work. Yet, most of the anima- Due to time restrictions – the team of 30-35
and even Red himself, who dons a signature tors there, including Perrin and Coglan, had artists and TDs (20-plus in London and
trench coat and fedora. Eisendrath knows at one time or another worked on projects 10-15 in Atlanta) had just over five weeks
Ron Frankel, owner of visualization company for final delivery. And as both point out, to complete the work – there was no time
Proof, Inc., and approached him to see if the although the visualization work is intend- to devise an entirely new process. “We had
idea was doable and to seek general advice ed as a pre-production planning tool, the to rely on what we had done all along, and
and information due to their unfamiliari- quality has grown quite sophisticated in use that as a model and adapt as we went
ty with the medium. After a great deal of along,” adds Coglan.
discussion, Bokenkamp and Eisendrath
decided to proceed, and hired Proof to The toon-style animation meshed well with the series’ spy drama theme.
help them end the season with Episode 19,
which would be fleshed out with animation Proof generated CG versions of the actors with unfinished parts in the episode.
by Proof, whose own staffs in Atlanta and
London were under stay-at-home orders
and working from home.

According to Proof producer Patrice Avery,
all the characters with unfinished parts
became animated, which included all of
the main and featured casts. “It was pretty
much, ‘Here is what we’ve shot so far, and
this is what we need.’ So, sometimes we were
transitioning from a close-up of Liz to an
animated Liz,” she explains. “It was a matter
of making it work with what we had [filmed].
It was an interesting ride, that’s for sure.”

The episode begins with an introduction
by the cast explaining the reason for the
hybrid direction. Then throughout, there is
an animated morph from the live-action
into the CGI. And within the animation itself,
comic-book chyrons are sometimes used
to provide context and to emphasize subtle
emotional beats.

The use of the two methods is split nearly
50/50, with approximately 22 minutes of live
action and 21 minutes of animation.

“We all knew the aesthetic Jon Boken-
kamp was going for and found it fairly
straightforward to adapt their ideas,” adds
Adam Coglan, visualization supervisor at
Proof in London.

Because of the comic-book/graphic novel

6 1e d i t i o n 2, 2020 c gw

In addition to characters, Proof also matched live-action environments. near the deadline,” Perrin points out.
Initially, the animators weren’t going to
One immediate adaptation involved from Allegorithmic) for texturing and sculpt-
a remote workflow due to the at-home ing. They also used Adobe’s After Effects for incorporate much lip sync, but Coglan and
restrictions. As a result, the animators had the warped transitions from live-action to Perrin determined they could use Proof’s
to get used to communicating remotely animation. An Adobe Photoshop filter was previs rigs to do some basic work in that
through Zoom and WhatsApp. They also used to toon-ify the backgrounds. regard. “The rigs were not set up to do full-
relied heavily on Teradici technology to log on facial lip sync, but the animators were
in to the Proof studio network after the Characters with Character doing the lip sync as they were going along
OpenVPN they had been using was unable anyway, so all we had to do was switch it
to handle the added workload. “The process To create the characters, an asset builder on in the end, which pleased the Blacklist
went extremely well. The IT guys rebuilt our at the London location pulled as many of group,” says Coglan.
infrastructure to enable us to work from the bodies, clothing, and other items from
home, something we never really did before, its generic character libraries as possible, Meanwhile, the audio files were recorded
so we could work on some other shows we and then built the remainder within Maya. by the actors themselves from their homes.
had going prior to The Blacklist,” says Coglan. While that was ongoing, proxy versions of
the characters (grayscale stand-in mod- The artists also had to toon-ify the back-
The team also had to ensure that the final els that didn’t resemble the actors) were grounds. Although, the hybrid production
output was void of all the shot information employed for the animation, to get a feel for enabled the producers to be more flexible
that appears on visualizations. “Logging the scenes, since “we were literally building with the planned environments. “When John
and tracking that information without the everything from the ground up,” says Perrin. and JB (Jon Bokenkamp) committed to do-
HUD that we usually depend on was a new There were no storyboards – the animators ing the comic-book [style], they committed
process for us,” says Perrin. Additionally, they had to do all the staging themselves. completely; it inspired them to get a little
used Apple’s ProRes video compression more exotic with some of their locations
format for the final export, which is not very Later, using detailed cast information and play with things they perhaps hadn’t
common in the feature-film world. (photos, body measurements, and so forth), thought about when they had the restraints
the modelers continued the character builds of shooting [the DC-based show] in New
One of the biggest changes came in the so they resembled the respective actors. York,” says Perrin. “All of a sudden, a scene
workflow, as the project schedule for The The finished models were then sent to the that took place in a dimly lit apartment was
Blacklist had to be greatly compressed, Atlanta-based studio to be toon-ified and now at the Washington Mall, which was a
which led to more overlap among the hatched; the hatching was often augmented big build for us.” There were more crowd
departments – asset building, R&D, layout, with hand-drawn lines on the main char- scenes, too, and even one sequence wherein
animation. “There was no time to wait for acters’ faces, as it’s a look that requires a a helicopter lands in a forest clearing as the
one department to finish before we put it degree of finesse, notes Perrin. accountant attempts his getaway.
through to the next,” explains Coglan. “So,
for a long time, the producers had to look Afterward, the models were passed While some scenes enabled the artists
at grayscale characters that didn’t look back to the London team, where the client to flex their creative muscle, other scenes
anything like their respective actors.” signed off on them and they were placed that had already been filmed required them
into the scenes. to translate those physical locales into CGI,
Because the imagery for The Blacklist was which they did with the help of blueprints,
so heavily stylized, that meant redeveloping The scenes progressed at different rates, drawings, and even references in past
the asset pipeline, and all of the Maya-based with some in the animatics stage while episodes. For instance, the artists built the
shaders and lighting rigs the group typically others reached final animation. “And [the medical facility where Liz’s grandfather lies
used had to be revised to achieve the toon producers] understood that. Often they in a coma using set drawings from the crew.
look and the hatch lines. The artists still used were still looking at their hero characters “They had already filmed some shots in that
Maya for the animation and most of the ren- in grayscale, but they trusted that every- location, so it was important for us to match
dering, along with Maya, Pixologic’s ZBrush, thing would look right at the end. They the continuity as we transitioned to the
and Adobe’s Substance Painter (formerly knew what to look for – shot design, story comic-book world,” says Perrin.
progression – with the details to be filled in
It’s not easy to transition production on
dime, but that is exactly what the actors,
producers, crew, and Proof animators did
when faced with the sudden shutdown.
And, they were able to find a successful
solution thanks to the use of computer
graphics, even embracing the flexibility the
medium provided in terms of action and
storytelling. 

Karen Moltenbrey is the chief editor of CGW.

62 cgw e d i t i o n 2, 2020

Quest for Reality

A PRIMER ON LIGHT FIELD TECHNOLOGY

W e increasingly have been hearing metallic or speculative surfaces will appear Dan Ring
about light field technology as it far more ‘real.’
pertains to simulating the “ulti- Practical digital light field capture is a com-
mate” visual experience. But, what exactly Which segments of the industry will plex engineering problem as well as a com-
is light field technology, and why should we benefit most from light field tech, and in plex computational problem. The explosion
care about it? what way? of available compute, storage, and network-
ing over the last 20 years has brought the
In the purest sense, a light field is “just” Firstly, light field tech could benefit live-ac- field to the cusp of being viable.
a mathematical function to describe light tion on-set capture, where the additional
flowing in all directions through all points in a data could be widely exploited in post-pro- Visualizing light fields had been a problem
volume. When applied to cameras, where a duction, particularly in visual effects where up until around 2012, when techniques for
conventional image is a captured represen- it could allow for easier or higher-quality compressive sensing of light fields on the
tation of light intensity, a light field image integration of elements. GPU were developed and used by MIT,
captures both the intensity and the direction MPI & Nvidia, for example. This enabled
of that light. When applied to display devic- Secondly, if in the future display devices more content to be delivered to multi-layer
es, the combined function of intensities and were to become available at scale, post-pro- display devices at higher spatial and depth
directions means the captured subject can duction could pass the data through to resolutions, and started putting light fields
be visualized from any perspective. the final display to create an unequaled under people’s eyes and driving interest in
viewing experience. Well before this became the technology.
Here, Dan Ring, head of research at practical or affordable for home use, it could
Foundry, provides a primer on this technol- be a compelling part of premium cinema, in How important are light fields to achiev-
ogy in a Q&A with Computer Graphics World a more persistent way than stereo 3D has ing a realistic visualization experience?
Chief Editor Karen Moltenbrey. managed. Light field displays could also
have an impact if they were used instead Simply put, if you could fully capture the
Why is light field technology important? of LED panels in virtual production envi- complete light field of a space and fully
ronments, as they would enable natural display it back later to a viewer, it would be
A conventional lens-and-sensor system parallax, lighting, and focus. visually indistinguishable from the reality.
collapses all of that dense light field data In practice, of course, there will be limits on
down into a two-dimensional image. This What about any segments outside of resolution and dynamic range, which make
loses information and effectively makes final our industry? it an imperfect experience, but in principle,
choices at capture time (about things like it still has the potential to deliver a visual
focus, parallax, and even camera posi- Light fields already have uses in industrial experience that can’t ever be achieved by
tion) that cannot later be manipulated in applications, where the extra data can be two-dimensional images.
post-production, and also cannot be re-cre- used for detailed inspection (http://raytrix.
ated for display to viewers later. de/inspection/, for example). Visualization A key part of a ‘realistic visualization
of products in the industrial and automotive experience’ is obviously the actual experi-
But, if the richer data could be captured design space will particularly benefit from ence. Light field display technology offers
and displayed to the user, it would provide a light fields, where the value of more physi-
viewer with a natural sense of parallax from cally correct light transport is important.
head movements, and also allow a viewer’s
eye to converge and focus naturally into Light field technology is not new, is it?
different depths within the scene.
The mathematical and physical concept of
Deferring these photography decisions a light field dates at least back to the work
on capture allows greater freedom of Michael Faraday in 1846 and has been
when visualizing. continuously expanded on since then. The
practical applications in photography are
The viewing experience would then be also not new and have been used experi-
much closer to ‘being there,’ in a way that mentally for at least 100 years.
stereo 3D displays have only ever approxi-
mated. In addition to the viewing position It seems that suddenly we are hearing
and ability to focus on different depths, the more about light field tech – why is that?​
added visual richness afforded to the viewer
is in view-dependent surfaces. For example,

6 3e d i t i o n 2, 2020 c gw

a different way of reproducing a captured
or approximated light field than, say, a
VR headset.

With the current volumetric display de-
vices, the visualization is delivered through
a physical display in the same world as the
viewer – they’re not being asked to be taken
out of their current world.

What are some of the specific obstacles A light field array built by the Saarland Informatics Campus.
to achieving light fields, and why are
they important? is commercially viable, there’s a chance that could be very compelling, adding a sense of
niche can expand. We may not need to solve immersion without necessarily needing a full
The volume of data is orders of magnitude capture to post-production to viewer all in wrap-around experience.
greater than conventional imagery, and the one go.
weight and size of capture equipment is sig- In either case, the ambition would be to
nificant compared to current camera tech- When we finally do overcome these is- move away from stereo display technology,
nology. While there aren’t any compelling sues, what will the result look like? which is a very poor approximation for a
reasons to support the technology (benefits human visual system.
on set, benefits in post-production, or I’d start from the answer to the above
benefits to a viewer), it is highly unlikely that question, ‘How important is light fields to Is there any other information about
adoption will happen any time soon. The achieving a realistic visualization experi- light fields you would like to highlight?
movement of such large amounts of data ence?’ If we have that, then there are some
is impractical, and the energy requirements first things we can think about that might be Foundry has been developing light fields re-
(and heat output) of prototype displays are practical enterprise applications before the search as part of SAUCE, a collaborative EU
very large. tech could one day evolve to home use: Research and Innovation project between
Universitat Pompeu Fabra, Foundry, DNeg,
Assuming the above problems of storage,  Large displays used for on-set back- Brno University of Technology, Filmakad-
computation, and transport can be solved, drops, replacing LED walls with environ- emie Baden-Württemberg Animationsin-
there also needs to exist the production ments that have the correct perspective stitut, Saarland University, Trinity College
pipelines for working with light fields, from from multiple viewpoints and allow natural Dublin, and Disney Research to create a
capture to post-production and delivery. On focus pulls into synthetic environments. step-change in allowing creative industry
the post-production side, deep images – companies to reuse existing digital assets
which are an image-based, camera-view-de-  Large displays used for future cinema, for future productions.
pendent projected volume approximation to maintain a premium viewing experience
– have been used for VFX and animation for for big-screen entertainment. The goal of SAUCE is to produce, pilot,
a while now. Even though the data rates are and demonstrate a set of professional tools
significantly lower than light fields, they are  Large displays used for live events. and techniques that reduces the produc-
still substantial enough to make working with tion costs of enhanced digital content – in
deep images challenging. Why do light fields have the potential of particular, targeting the creative industries
eliminating viewing gear in VR? by increasing the potential for repurposing
Again, the investment required and the and reusing content as well as providing
timescales to produce are huge. Digital CAVE environments aren’t new but significantly improved technologies for digi-
could be greatly improved with light field tal content production and management.
How does one go about solving display panels, removing the need for head
these issues? gear while greatly improving the experi- For more insights into light fields and
ence. This isn’t likely to be viable in a home related topics visit Foundry’s Insights Hub at
Like all technology advances, we see a lot environment, but we think that the switch https://bit.ly/insightslightfields. 
of dogged persistence by innovators in from headsets to an immersive wall display
capture technology and in display technol-
ogies. Once one of these finds a niche that

2020, Issue 2: COMPUTER GRAPHICS WORLD (USPS 665-250) (ISSN-0271-4159) is published bi-monthly with special additional issues in January and July resulting in 8 issues
per year by COP Communications, Inc. Corporate offices: 620 West Elk Avenue, Glendale, CA 91204, Tel: 818-291-1100; FAX: 818-291-1190; Web Address: [email protected]. Pe-
riodicals Postage Paid at Glendale, CA, 91205 & additional mailing offices. COMPUTER GRAPHICS WORLD is distributed worldwide. Annual subscription prices are $72, USA; $98,
Canada & Mexico; $150 International airfreight. To order subscriptions, call 847-559-7310.

© 2020 CGW by COP Communications, Inc. All rights reserved. No material may be reprinted without permission. Authorization to photocopy items for internal or personal use, or
the internal or personal use of specific clients, is granted by Computer Graphics World, ISSN-0271-4159, provided that the appropriate fee is paid directly to Copyright Clearance
Center Inc., 222 Rosewood Drive, Danvers, MA 01923 USA 508-750-8400. Prior to photocopying items for educational classroom use, please contact Copyright Clearance Center
Inc., 222 Rosewood Drive, Danvers, MA 01923 USA 508-750-8400. For further information check Copyright Clearance Center Inc. online at: www.copyright.com. The COMPUTER
GRAPHICS WORLD fee code for users of the Transactional Reporting Services is 0271-4159/96 $1.00 + .35.

POSTMASTER: Send change of address form to Computer Graphics World, P.O. Box 3296, Northbrook, IL 60065-3296.

64 cgw e d i t i o n 2, 2020