A Digital Essay by Melinda White, University of New Hampshire
Computer labs and computers as tools of expression are hardly new to students, however there are new ways of reading, new ways of creating meaning and narrative through multimodal storytelling that students, particularly creative writers, may not have considered. It is only natural that generations that, as Ted Nelson predicted, “live in media, as fish live in water” (3)—who have grown up with or adapted to computers, cell phones, cable television, music downloading, and social networking (sometimes all at once)— would utilize the tools at hand to compose creative and scholarly work1, and communicate with others. As Anne Frances Wysocki argues in Writing New Media, “when someone makes an object that is both separate from her but that shows how she can use the tools and materials of her time, then she can see a possible self—a self positioned and working within the wide material conditions of her world, even shaping that world—in that object” (21). In the digital creative writing course I developed at the University of New Hampshire, students read and compose multimedia texts, requiring them to think beyond the page and consider more modes besides linguistic, particularly including visual and audio modes,2 in storytelling. They develop a crucial understanding of how, and more importantly why these texts are composed using the medium, and then proceed to shape the world with their own words and conscientious employment of multimodal textual elements.
The pedagogical foundation of the course is based on the significance of a text that needs the medium, that N. Katherine Hayles defines in her essay “Electronic Literature: What is it?” as “‘digital born,’ a first-generation digital object created on a computer and (usually) meant to be read on a computer” (Hayles). Just as in any creative writing course, form and theory play a major role. We read multimodal examples of creative nonfiction and discuss the repercussions and unique techniques of rhetorical analysis needed for these types of works. This includes the affordances3 of each mode—what elements are best communicated with different media—for instance, oral readings of work are often more emotional than written text and visual language is often more ambiguous. They learn how the meaning of texts cannot be separated from their medium. As Cheryl E. Ball offers in a clever Tweet included in Writer/Designer: A Guide to Making Multimodal Projects: “What you say cannot be divorced from how you say it. It’s the same with multimodal texts. #WriterDesigner” (42).
Although the course is digitally focused, we still cover the basics of creative nonfiction and read both print and digital texts. The first assignment is print, but written about a work of art; digital assignments include a snapshot (short video) essay, a map essay, and a hypertext essay. Throughout the course, students also choose one work from the course readings and give a close-reading presentation. Instructor conference, peer workshop, and revision are also main components of each assignment. The course description reads:
Focused on creative uses of multimedia in composition, this course will cover traditional nonfiction elements such as sensory details, narrative, and expressing the human condition, while also including visual, audio, and electronic text to engage readers. Like an artist’s paintbrush, the computer can be a creative tool in the writing process. Exploring methods, forms, and functions of works of both print and digital nonfiction will provide students with context and the foundational skills to express themselves through multimedia writing projects such as video, Google Maps, and the web. Writers will become composers, telling their stories with digital media.
As an introductory course in creative nonfiction, we cover what it is (and, rather, isn’t), what it means to write in the genre, talk about style and form, sensory details and dialogue, and adhere to Anne Lamott’s suggestion to “listen to your broccoli” (110), meaning trust your instincts. Although we read print and multimedia texts, with the digital texts we stress throughout the semester the “digital born” aspect and why these texts need (or don’t) their medium and modes. This takes some getting used to but the computer allows students to read, interact with, analyze, and compose multimodal texts that are meant for the medium.
In 1974, when Ted Nelson wrote Computer Lib/Dream Machines, he knew that technology would play a huge role in our lives and predicted that “many [people] want to use them to communicate artistic visions” (3). For digital natives, this doesn’t seem to be too much of a stretch, however throughout the course students (ideally) additionally become more conscious of the rhetorical choices of various modes in both reading analysis and composition. As Troy Hicks expresses in Crafting Digital Writing: Composing Texts Across Media and Genres:
I want to see our “digital generation” live up to their potential as conscientious citizens and creative producers of text in all forms. Sure, they can post status updates quicker than most of us adults can pull out our phones. Yet inviting them to be intentional about the craft of digital writing is perhaps the best way to help them realize their potential in academic, social, political, and community contexts. When we talk and teach thoughtfully about the elements of digital writing—words, images, sounds, videos, links, and other media elements—we are helping them be purposeful and, in turn, helping them be creative. (19)
Because students are familiar with technology—texting, posting photos on Instagram, reposting to Tumblr, updating Facebook, uploading videos to YouTube—this doesn’t preclude that some students are not comfortable with various technology, and some might even be unsure of their abilities on the computer or with new software in general. Aesthetic design is not the main focus of the course4 but must be given attention as it contributes to (or can take away from) meaning. As Hicks suggests, it would be easy to say that writing courses should not concern themselves with design modes when teaching digital writing, and focus on the text, but, of course, we are already aware that the modes and the meaning cannot be separated. Hicks expands: “With digital writing, we need to think with words, of course; yet we also need to begin thinking like artists, web designers, recording engineers, photographers, and filmmakers. In other words, intentional choices about craft can lead to creative work in a variety of writing media” (18-19).
There are more intentional choices with multimodal writing and while some students struggle with the technology in some or all portions of the course, for the most part the digital composition process is almost second nature to them. As technology becomes more complicated and provides students more options, the creativity and experimentation tends to flourish as well. Daniel Anderson discusses the benefits of introducing students to unfamiliar technologies in his article, “From a Pedagogy of Multiliteracies: Designing Social Futures”: “engaging technical things comes with a bigger payoff. Experimenting with unfamiliar technologies can facilitate a sense of creativity that can lead to motivation” (363). This is particularly significant in creative writing, where exploring the creative affordances of the media, seems to extend to the words, the topics, and the stories as themselves. Students learn technical skills as they compose, often from each other. We also have a great resource, the Parker Media Lab, where students can check out equipment, as well as work on and get assistance with digital projects.
Besides our course textbook (for print and graphic essays), Creating Nonfiction, a Guide and Anthology by Becky Bradway and Doug Hess, digital course readings are linked on the course website, along with the schedules and assignment descriptions, and a course gallery of student work. Each unit has a topic/theme for both the assignments and readings: Art, Identity, Place, and Connections. These units move from discussing art to visual rhetoric and linear to nonlinear writing.
The Art of Writing (and writing about art)
In the first unit we focus on the ambiguous definition of creative nonfiction as a genre and integrating art into writing, a step to talking about both written description and incorporating visual elements (without actual images), that we need as the semester progresses. We first focus on the idea of Truth and writing sensory description and dialogue. For this unit we read descriptive essays, two excerpts from ekphrastic print works: Mark Doty’s Still Life with Oysters and Lemon and Terry Tempest William’s Leap, a book based on Hieronymus Bosch’s Garden of Earthly Delights. During our reading discussion, we look at the artwork from each of these essays and discuss how the author’s descriptions match (or don’t) what we’re seeing and how seeing the art might influence our interpretations. We also read/view our first video essay, Brian Bouldrey’s “Dead Christ” and discuss why it needs the medium, how it would “look” as a print essay, how his voice enhances the emotion behind the words.
We then take a trip to the UNH Art Museum where we participate with the curator in a rhetorical analysis of some of the artworks, students do a collaborative descriptive words poetry activity and then individually choose a work of art to trigger their first assignment, a personal essay that incorporates extensive description of the work and integrates it into their memory or event. Ekphrastic writing requires writers to consider visual narratives and pull their own stories from visual media. Students are already aware of visuals and structural experimentation, as they can integrate the art descriptions and the scenes of their essay in a variety of ways. With Bouldrey’s essay, they also begin to see the affordances of the visual mode, and recognize the linguistic work they must do to make up for the reader not seeing their artwork.
Identity (a picture is worth a thousand words)
This leads us into a discussion of visual rhetoric, images as text, and video essays for the second unit, on identity. We read excerpts of John Berger’s Ways of Seeing and discuss visual rhetoric, how visuals can be text and convey emotions, often more directly, but sometimes in a more fluid, ambiguous, subjective way (and how this complicates things for both author and reader). As an in-class assignment, students work in groups to compose visual narratives to practice telling a story or making an argument with only images, no alphabetic text. This illustrates for them both the difficulty of telling stories without words, but also the challenge of conveying an unambiguous message to an audience this way.
We read essays in both print and digital form that embody some expression of identity, including the video essays, That Kind of Daughter by Kristen Radtke and Baptism by Marilyn Freeman, which, besides Bouldrey, is the introduction to the form. Both essays prompt a conversation about visual style, need for the medium, visual and audio modalities, and identity. That Kind of Daughter is a creative example of how the visual mode can be an artistic representation of the linguistic, rather than the author attempting a direct correlation, and, further, how that representation can change the meaning of the text. With the affordance of audio, this is a good place to discuss voice and tone; although Radke’s essay is personal and the visuals have an intimate delicacy about them, her voice is even and monotone, perhaps purposefully to contrast the content and show her emotional distance. Baptism is also especially good for discussing how medium can convey message. Not only does Freeman focus on identity, through her name and how it does or does not fit her, she uses an interesting film technique to visually express the metaphor of misplaced identity, by having various others mouth her own spoken words throughout the beginning of the essay. We hear Freeman’s voice, but are never sure who is mouthing the words in the video and, moreover, are left wondering which person, if any, is our narrator; her identity is not revealed until much later in the essay, which enhances the purpose.
For this unit, students compose a short video essay (called a Snapshot essay) on identity, their own or someone else’s, with voice narration (and sometimes background music or sounds, although this is another discussion). The challenge is to not let go of descriptive writing because they can now use images to “show” their narrative. Students often hate hearing their own voice so that is something they have to overcome but the audio illustrates the emotional nature of the text and the sound of language, a significant element in creative writing. These essays are linear in structure though, which is familiar for them, even if they experimented with structure in the art narrative essay. Because of the theme of identity and the narration, these are often the most emotionally moving essays of the semester.
In “An Exclusive Interview with Joey Kenny, Kenny Brothers Band,” Joseph Kenny writes about his own identity through music in a “Before They Were Famous”-type interview style. One other student has done their essay in an interview style but this one is unique in that it has a fictional premise, where the student is reflecting back on the beginning of his music career, yet it is still autobiographical. Joseph’s combination of music, images, and video clips really reflect his writing style. He conscientiously chose a style that fits reader expectations of the genre and affordances of both visual and audio modes.
Whitney Carrier’s innovative essay, “A Letter from a Mother to a Daughter,” was the first one a student recorded using someone else as the narrator. In this case, Whitney has her daughter read the essay, a letter to her daughter, and then adds conversational questions throughout the video. The form of a letter makes the essay more personal and images of the daughter growing up (most with mother and daughter together) add an important visual layer. Perhaps most significantly, the daughter’s voice makes it even more sentimental and expressive, as it adds to the meaning of the author’s identity as it was formed and impacted by the identity of her daughter. Speaking to her daughter throughout the essay illustrates the mother/daughter relationship in a way we could only be witness to through the affordance of audio. Whitney not only considered the best linguistic approach to this essay, but also one that would benefit from multiple modes, affording multiple layers of meaning.
Place (where we have been & where we are going)
In unit 3, we move to the role of place and places in shaping identity. Place and travel writing play a large role in creative nonfiction. Although these may seem like straightforward, more factual forms, the author’s identity and unique style are what separate creative nonfiction from travel journalism. For this unit, we discuss the idea of place while also continuing with how message needs medium and an introduction to nonlinear writing with the creation of a map essay. Although place writing could be about the author’s hometown (and some of the essays we read are), Bradway and Hesse offer a broad definition of place, which I use in my assignment description for this unit:
Writers have long been writing about the paths they encounter, whether these are found on the other side of the world or in the backyard. It’s easy to make the assumption that place writing is synonymous with nature writing. Not so. A NASCAR track, the Apollo Theater, and the riverbank are all places, worthy of the same degree of absorption (23).
Because the map essay also provides an ideal forum for student travel and study abroad experiences, we also talk about travel writing and how it differs from more informative writing on travel destinations (including, conveniently, something about the author’s identity). This unit includes several place and travel-themed readings that include more of a sampling of interactive digital literature. In groups, students analyze works from the National Film Board of Canada’s (NFB) HYPERLOCAL essays, six interactive pieces on place that provide comparisons in both the variants of this topic and also varying degrees of interactivity, use of voice, image, narration, and video work. This also starts a discussion of navigation in digital work, something that becomes vital in the final unit of the semester. We also read one map essay to prepare for the assignment, “Mr. Plimpton’s Revenge” by Dinty W. Moore, which works well in a discussion of writing and medium (many students think this essay would work just as well in print). Students then compose an essay on place or travel using Google’s My Maps, where they can add writing, images, and video to points on a map. The goal for the map essay is for students to both write about place or travel that is meaningful to them, and utilize a new, potentially nonlinear, medium to convey their experience of place. Some students at this point find it difficult to give up on the idea of a firm linear structure to their essays and will employ numbering or other ways of influencing the order the reader chooses to interact with their essays (including introductions in the map description field and subtitles like, “read this one last”). Although giving up on authorial control is sometimes frustrating, these essays are often the students’ favorite—the technology is fairly simple and it’s an effective introduction to the idea of nonlinear writing.
Students are often drawn to writing about their homes, some literally write about their own backyards, and some explore mapping journeys or a collection of significant moments that are tied to place. Since in this essay, medium is also already chosen, the subject matter is influenced by the affordances of the visual of the map. They also put thought into the navigation, some visual design, including whether or not to include pictures and/or video and, if so, of what. In Emma Giordano’s essay, “My Other Home,” she writes about her family experiences tied to their summer cabin in Vermont. Andrew Teagno’s essay, “Travels to 90,000’,” documents his experience climbing Cotopaxi in Ecuador. This essay fits with our discussion of creative travel writing, which, according to Bradway and Hesse, “has a strong narrative voice and a writer deeply and directly involved in the events of the place. The narrator is usually going somewhere new; a sense of adventuring into the unknown is nearly always the dominant tone, often inflected by physical and/or emotional risk” (24). Both of these essays express something about their authors, whether it is through family scenes or a travel adventure.
Connections (listen to your (multimodal) broccoli)
The final unit of the semester is all about connections, as we delve even deeper into interactive and nonlinear works of electronic literature and students compose a hypertext essay with images and links, published online through the students’ web space. For foundational theory, we discuss Marshall McLuhan’s concept of “the medium is the message” (19), Ted Nelson’s coining of the term “hypertext,” and his Project Xanadu, and travel even farther back to Vannevar Bush’s article, “As We May Think” (from 1945)—intriguing ways to discuss hypertext and organic thought processes, as well as early predictions of the Internet. Students often relate to the media’s effect on message and meaning by this point in the semester and find ideas of connection in the Internet and the way we think, read, and learn. As Vannevar Bush says of the organic associations of the brain: “Man cannot hope fully to duplicate this mental process artificially, but he certainly ought to be able to learn from it” (44). During this unit we delve deeper into digital-born electronic literature, looking at work that is more game-like or “computer generated,” and use these to prompt larger discussions of literature, genre conventions and authorship.
We read some more structurally straightforward hypertexts, such as Shelley Jackson’s “My Body—A Wunderkammer,” an excellent example of a fairly straightforward hypertext in execution with complex themes of fragmented self-image and connection through pieces of memoir on various parts of the narrator’s body. We also read more interactive and dynamic works where the interaction creates the metaphor, such as “Fitting the Pattern” by Christine Wilks, that connects stages of sewing to the narrator’s memoir where each task has to be performed by the reader to move the story forward. Some students find they prefer a more straightforward navigation in an electronic text and others are intrigued by intuitive interfaces and the added meaning and/or metaphor that learning to read a text can provide. We look at some, too, where too many modes might detract from meaning; this helps students with rhetorical decisions in their own texts, for instance, navigation, audio, movement, scrolling, images, collage, font, color, etc.
This influences their design and interface choices in storyboarding their final project, where students are provided with the theme of connections and challenged to compose a hypertext essay with multiple pages and links. Students are introduced to Dreamweaver5 and FTP software to compose and upload their “digital born” texts. This is the most technically challenging project of the semester, although I’ve noticed in the student’s reflective memos, it is often the most rewarding. Anderson reminds us that, “The challenge of composing with unfamiliar forms opens pathways to creativity and motivation” (364).
This is the capstone of the semester, including a reflective memo, where students comment on the culmination of projects and how their essay “needs” the medium. For Gabrielle Greaves’, “The Lovely Bodies,” she started with an online survey, asking other students on campus what they both liked and disliked about their bodies. She then integrated their answers into pop-up windows connected to words of her poetry and included some researched sections on different “bodily” concerns. The medium allowed her essay to be a forum of collaborative voices, information, and poetry on body image, particularly in the African American community. This is an ideal environment for a multi-vocal, multi-layered work.
“Jenna’s Dream Canvas” by Jenna Ward is a more personal essay that recounts the author’s dreams and includes music and dream interpretations for each one. “The Enjoyment of Music” by Geneth Chin also uses music to represent each instrument in the narrative, while the text was “concrete,” carefully formed into the shape of each instrument. While Jenna and Geneth have pretty straightforward navigation, knowing where to click for the pop-up windows on Gabrielle’s homepage is a bit more intuitive. Adding the music also adds the audio mode, something each student must decide on (some students also include video or animation). Some students have integrated Flash or animated GIFs into their essays for movement, but due to the timeframe for this assignment, some of the things they want to execute would require more technology and most are limited to links for the level of reader interaction. These essays really do take advantage of the medium and students are able to reflect on how and why they composed and designed them the way they did.
As fish in water, it is not difficult for students to understand McLuhan’s idea of media as “extensions of ourselves” (19), yet it is crucial that they understand the materiality of a text as it relates and contributes to meaning-making in order for them to compose “digital born” works of their own that are more than bells and whistles, but thoughtful and creative digital projects. The emphasis will continue to be on why these texts are made, why they must be digital born, and how students can best work with multiple modes of communication to express themselves creatively and, of course, it’s always a work in progress. As Wysocki states: “We can only see ourselves within the texts we make and give to others if we understand those texts (and how and where and with what we work as we produce them) to be connected to us through our various material relations” (18). Reading multimodal texts, from print to video to hypertext essays and learning new ways of talking about them (and analyzing them), better prepares students for relating text to medium and the affordances of multiple modalities, how they can work together to express something that may be expressed differently in a print medium. Many of our conversations of a text focus on how it needs the medium, what the design, video and audio modes, and interactive elements add to (or sometimes detract from) the meaning, and what this text would have looked like as a printed text (or if that would even be possible in many cases). This helps them understand why their own essays need the digital medium, as well as the affordances of multiple modes for creative expression; these are the tools of their time. Through this understanding, learning to talk about and create media and text in new ways, these students are breaking down the barriers of traditional print texts and using the tools of their time to shape their world and the future of storytelling.
- I would just like to take a moment, as some of the contributors to Multimodal Composition: A Critical Sourcebook did, to acknowledge a bit of irony in composing a linear article for a discussion of multimodal narratives. I felt the chronological trajectory of the course fit best in this format and hopefully have preserved a smidgen of my multimodal cred. with the course content (and a website!).
- Design modes identified by the New London Group are: Linguistic, Visual, Audio, Gestural, Spatial and Multimodal ((198), presented in their essay, “From a Pedagogy of Multiliteracies: Designing Social Futures” in Multimodal Composition: A Critical Sourcebook. These are also used in Writer/Designer: A Guide to Making Multimodal Projects, to discuss multimodal texts.
- Also addressed in Writer/Designer: A Guide to Making Multimodal Projects. “These different strengths and weaknesses of media (video, writing, pictures, etc.) and modes are called affordances” (15).
- As this is a creative writing course and my background is not in media design, I don’t contend that we go in-depth on these topics. Some students, of course, have a natural aesthetic or have taken art or design courses, but for the most part, we cover some basic design principles and ask questions about why this color, font, grouping, alignment, etc., and consider the rhetorical implications of the modes as much as possible.
- I am always considering new software for the hypertext portion of the class and will likely use Twine next semester as an option. UNH has an Adobe license, and I have a background in Dreamweaver and web design, so this is feasible for us, but I realize would not be for everyone. The main benefit (besides my own ability to teach and trouble shoot) is that it offers students a blank canvas and, thus, does not limit their creativity with the design the way a template might.
Anderson, Daniel. “The Low Bridge to High Benefits: Entry-Level Multimedia, Literacies, and Motivation.” Multimodal Composition: A Critical Sourcebook. Claire Lutkewitte, ed. Boston: Bedford/St. Martin’s, 2014.
Arola, Kristin L., Jennifer Sheppard, and Cheryl E. Ball. Writer/Designer: A Guide to Making Multimodal Projects. Boston: Bedford/St. Martin’s, 2014.
Berger, John. Ways of Seeing. London: Penguin Books, 1990.
Bouldrey, Brian. “Dead Christ.” TriQuarterly. Issue 146. Summer/Fall 2014.
Bradway, Becky and Doug Hess. Creating Nonfiction, a Guide and Anthology. Boston: Bedford/St. Martin’s, 2009.
Bush, Vannevar. “As We May Think.” In The New Media Reader, ed. Noah Wardrip-Fruin and Nick Montfort. Cambridge, MA, and London: The MIT Press, 2003. 35-47.
Carrier, Whitney. “A Letter from a Mother to a Daughter.” English 501, Fall 2015. Course Gallery. <http://multimodalmel.com/501/pages/visual%20identity%20-%20Small.mov>.
Chin, Geneth. “The Enjoyment of Music.” English 501, Fall 2015. Course Gallery.
Doty, Mark. Still Life with Oysters and Lemon. Boston: Beacon Press, 2001.
Freeman, Marilyn. “Baptism.” Blackbird. Spring 2010. Vol. 9, No. 1. <http://www.blackbird.vcu.edu/v9n1/gallery/ve-freeman_m/baptism-video.shtml>.
Giordano, Emma. “My Other Home.” English 501, Fall 2014. Course Gallery.
Greaves, Gabrielle. “The Lovely Bodies.” English 501, Fall 2014. Course Gallery.
Hayles, N. Katherine. “Electronic Literature: What Is It?” 2009. Aug. 2010.
Hicks, Troy. Crafting Digital Writing: Composing Texts Across Media and Genres. Portsmouth, NH: Heinemann, 2013.
HYPERLOCAL. National Film Board of Canada. NFB website.
Jackson, Shelley. My Body—A Wunderkammer. Electronic Literature Collection Volume 1.
Kenny, Joseph. “An Exclusive Interview with Joey Kenny, Kenny Brothers Band.” English 501, Spring 2015. Course Gallery.
Lamott, Anne. Bird by Bird: Some Instructions on Writing and Life. New York: Anchor Books, 1995.
McLuhan, Marshall. Understanding Media: The Extensions of Man. Cambridge, MA:
The MIT Press, 1994.
Moore, Dinty W. “Mr. Plimpton’s Revenge.”
Nelson, Ted. Computer Lib/Dream Machines. Rev. ed. Richmond, WA: Tempus Books of Microsoft Press, 1987.
The New London Group. “From a Pedagogy of Multiliteracies: Designing Social Futures.” Multimodal Composition: A Critical Sourcebook. Claire Lutkewitte, ed. Boston: Bedford/St. Martin’s, 2014.
Radtke, Kristen. “That Kind of Daughter.” TriQuarterly. Issue 141. Winter/Spring 2012.
Teagno, Andrew. “Travels to 90,000’.” English 501, Fall 2015. Course Gallery.
Ward, Jenna. “Jenna’s Dream Canvas.” English 501, Fall 2015. Course Gallery.
Wilks, Christine. Fitting the Pattern. Electronic Literature Collection Volume 2.
Williams, Terry Tempest. Leap. New York: Pantheon Books, 2000.
Wysocki, Anne Frances. “Opening New Media to Writing: openings & justifications.”
Writing New Media: Theory and Applications for Expanding the Teaching of Composition. Logan, Utah: Utah State UP, 2004.