Zeglin, G., Walsman, A., Herlant, L., Zheng, Z., Guo, Y., Koval, M. C., et al. (2014). HERB’s Sure Thing: A rapid drama system for rehearsing and performing live robot theater (pp. 129–136). Presented at the 2014 IEEE Workshop on Advanced Robotics and its Social Impacts (ARSO), IEEE. http://doi.org/10.1109/ARSO.2014.7020993
Notes: Video of the play at
p.129: Theatrical drama involving robot and human actors provides an opportunity to explore techniques for seamless physical and verbal collaboration. — Highlighted Feb 5, 2017
p.129: emphasis on conversational gesturing — Highlighted Feb 5, 2017
p.129: This process uncovered a rich set of questions as we extended our software infrastructure to support dramatic performance. We also gauged audience reactions to understand contemporary expectations of robot performance. — Highlighted Feb 5, 2017
p.129: human operator — Highlighted Feb 5, 2017
p.129: Our goal was to replace one human actor with a robot within a conventional play and theater. — Highlighted Feb 5, 2017
p.129: develop a motion graph approach which balances flexibility with the limitations of real-time operator input. — Highlighted Feb 5, 2017
p.129: improvisatory tools — Highlighted Feb 5, 2017
p.129: Our ultimate aim is not to replace actors but to understand how body movement and prosodic speech create an understanding of intention within a narrative — Highlighted Feb 5, 2017
p.129: integrating expressive behavior into robot daily life. — Highlighted Feb 5, 2017
p.129: This can guide robot designers toward techniques for infusing the practical motions of everyday robot life with expressive gestures which convey the intention of the robot. — Highlighted Feb 5, 2017
p.130: depends on expressive dialogue delivery with comedic timing for an effective performance — Highlighted Feb 5, 2017
p.130: text-to-speech — Highlighted Feb 5, 2017
p.130: motivation in common is the use of theater as a means of exploring human-robot social interaction — Highlighted Feb 5, 2017
p.130: renderings lack the prosodic nuances required for effective drama. — Highlighted Feb 5, 2017
p.130: goal of developing the performance via rehearsal with humans — Highlighted Feb 5, 2017
p.130: essentially the speech equivalent of motion capture: the prosodic structure of an utterance can be rerendered with the opportunity for manipulating parameters — Highlighted Feb 5, 2017
p.130: initial goal was defined as replacing a human actor within a conventional dramatic performance — Highlighted Feb 5, 2017
p.130: approach was to create a set of expressive motions in advance using a combination of direct pose capture on the robot and animation trajectory editing. The arms and head of the robot are easily positioned by hand when running a low-impedance gravity-compensation controller, allowing an animator to apply their physical intuition and real-world observation to choose effective poses using the robot itself as a master control — Highlighted Feb 5, 2017
p.130: The goal for the operator interface was to reduce the required input to a single cue action per line of dialogue while still allowing for flexible control in the event of a major mistake — Highlighted Feb 5, 2017
p.130: From a dramatic standpoint, the primary anthropomorphic elements of HERB are the pair of arms, the presence of a head, and basic mobility. — Highlighted Feb 5, 2017
p.130: constrain the overall movements to animated transitions between prespecifid poses in a motion graph. This keeps the amount of operator input to a feasible level and eliminates the need for real-timemotion planning — Highlighted Feb 5, 2017
p.130: emphasizes the elements of timing and tactics over detailed motion variation. — Highlighted Feb 5, 2017
p.130: Our general approach was to emphasize coordinated arm and head movement over driving around the stage. This approach assumes that affective body language can provide effective dramatic expression despite the lack of an articulated face, an idea well-supported in puppetry. — Highlighted Feb 5, 2017
p.131: idle Each of the 91 transitions is a hand-animated trajectory beginning and ending at a reference pose. — Highlighted Feb 5, 2017
p.131: fliptow This design is a balance between realtime puppeteering and scripted motion. The overall dramatic outcome emphasizes the choice and timing of gestures as the basis for expression. — Highlighted Feb 5, 2017
p.131: handflipleftsubtleThe first and most heavily used category was keep-alive motion [24] designed to add subtle speech cues and prevent the robot from appearing too static — Highlighted Feb 5, 2017
p.131: Referencing gestures — Highlighted Feb 5, 2017
p.131: most of our gestures, in and of themselves, should not suggest any particular emotion. — Highlighted Feb 5, 2017
p.131: specifically draw the audience attention to a particular item or physical location. — Highlighted Feb 5, 2017
p.131: A gesture represents different intent in different contexts — Highlighted Feb 5, 2017
p.131: “specific meaning” gestures — Highlighted Feb 5, 2017
p.131: such as a head scratching motion to indicate confusion — Highlighted Feb 5, 2017
p.131: common technique in crowd and video game animation — Highlighted Feb 5, 2017
p.132: Prior to this project most of the motion executed on the HERB robot was purely functional and produced by automatic planners — Highlighted Feb 5, 2017
p.132: resulting trajectories often do not convey intent or exhibit predictable timing — Highlighted Feb 5, 2017
p.132: iteratively refined during testing on the robot and with the actor. — Highlighted Feb 5, 2017
p.132: We found that using the actual robot as a master to specify animation poses was far less time-consuming and produced superior results compared to using the animation interface alone — Highlighted Feb 5, 2017
p.132: During the animation refinement we found most adjustments involved correcting the timing or the speed of the motions rather than the poses themselves — Highlighted Feb 5, 2017
p.132: In collaboration with the director, many gestures were retimed to achieve more pronounced inflection points to fit with the dialogue. — Highlighted Feb 5, 2017
p.132: To streamline the keyframing process we developed a plugin for Blender enabling the creation of animation poses via manipulation of the physical robot in a low-impedance gravity-compensated mode — Highlighted Feb 5, 2017
p.132: Many motions that seem benign on humans or in simulation are easily perceived as threatening when played on the actual robot with its large arms — Highlighted Feb 5, 2017
p.132: This provides direct manual control of the joints which is much easier than controlling a virtual character, and unlike the Blender animation interface, does not require any special skills or training — Highlighted Feb 5, 2017
p.132: Working with the actual robot provides immediate visual feedback on pose and scale and reduces the iterations required to discover a satisfactory result. — Highlighted Feb 5, 2017
p.132: In general, during testing on HERB we found most of the preliminary animations too dramatic and scaled them gradually into less pronounced movements, as well as creating additional subtle versions of many poses — Highlighted Feb 5, 2017
p.133: the relatively flat delivery would not be satisfying for performance — Highlighted Feb 5, 2017
p.133: When a change was requested, the director’s feedback was typically very abstract, such as “can you say that more hesitatingly?” While these complex instructions are easily understood by a human, it is a difficult task to deconstruct the abstract concept of hesitancy down to the level of which syllable to elongate or diminish. As such, many iterations of small changes followed by listening and evaluation of the resulting waveform were needed to reach a satisfactory output. — Highlighted Feb 5, 2017
p.134: Although robots are less spontaneous than human actors, one advantage is repeatability. The consistency helped the human actress rehearse since she knew what the robot wasgoing to do and could rely on the same reaction every time — Highlighted Feb 5, 2017
p.134: It is important to note that the audience’s expectations were very low in the first place, which could be explained by low exposure to robots in similar social settings — Highlighted Feb 5, 2017
p.135: robot’s timing. There was some confusion as to whether the occasional pauses before the robot delivered his lines were intentional for dramatic effect or mistakes — Highlighted Feb 5, 2017
p.135: repeated theme of assumed capability. Because the audience could see what appeared to be functional mechanical or body parts, they expected that all of the parts would be involved in the performance — Highlighted Feb 5, 2017
p.135: 4Relation Rating4Robot = Human Rating Producing an effective performance requires prioritizing artistic needs, and the demands of a live show place an emphasis on simplicity and reliability. — Highlighted Feb 5, 2017
p.135: 2Relation Rating2Robot = Human Rating Ceding artistic authority to the director and frequently running performance tests provided early feedback on features and allowed dramatic goals to take priority over technical aspirations — Highlighted Feb 5, 2017