Effective viewing behavior and viewing strategies: Can we teach foreign language learners to effectively use subtitles?

An interview with PhD candidate Sjoerd Lindenburg

How do language learners read subtitles? Can we use eye-tracking to determine effective viewing strategies? Can we teach students effective viewing behavior to improve language learning? These are some of the central questions in Sjoerd Lindenburg’s PhD project, funded by an NWO Promotiebeurs voor leraren (https://vakdidactiekgw.nl/author/sjoerdlindenburg/) and carried out in collaboration between Stichting Alberdingk Thijm Scholen and Leiden University (ICLON & LUCL).

Q: What does your research focus on primarily?

A: My research focuses on second language learners’ behavior. Obviously, you process the information in a certain way, but there is a difference when it is in a foreign language rather than in your first language, especially in the use of subtitling in that context.

Q: What sparked your interest?

A: I completed a Master’s in Translation here at Leiden University and I took one of Lettie Dorst’s courses, Multimodal Translation, which I thought was really interesting. As I wanted to get my teacher’s degree as well, I looked for ways to merge those two interests together. Then it turned out that a lot of research had been done on the use of subtitles for language learning, mostly during the 1970s and 1980s, but with a recent resurgence in popularity from around the 2010s.

Q: Does your research focus on other languages than English?

A: It focuses on English and French, because we want to implement our findings at the secondary school I work at (teaching English). As English has a rather unique position in the Netherlands – most children start learning it very early on, watching English audiovisual content on YouTube or television – we are also looking at French to even this exposure effect out. Most pupils don’t start learning French until they are in high school and most do not watch a lot of audiovisual content in French or listen to French music.

Q: What are your plans for the future? How do you plan on carrying out this project?

A: It is a five-year PhD-project, consisting of three phases: phase one is gaining insight into how people view audiovisual materials with subtitles. That is, whether they are good at the languages that we have, so English and French, and also Japanese for this first part, to see where they look at. To do this, we are using the eye-tracker in the LUCL lab. When do language learners look at certain parts and why do they look at them? How do people watch subtitled material? Part two is hopefully going to be next year, and we will be doing the same thing with secondary school students, to see if they behave differently when watching TV series, as they might not be as interested in learning a language as someone who chose to study it. Then part three is using the information we have gathered to create lessons to teach students how to watch those videos in a way in which they can maximize their language learning without taking away the entertainment videos provide us. Of course, watching Netflix is fun, and it is also educational, but if you make it too educational, will they still do it?

Q: What would you say are the main difficulties? Are there specific difficulties per age group, or is it more difficult if someone is obliged to learn the language rather than if they choose to do so voluntarily?

A: I think that right now, the biggest difficulty has nothing to do with participants, but with the experimental set-up. Technical difficulties have thus far been the worst, because we use such a very specific machine for this research. Another difficulty is with participants. Right now, people view parts of a TV show and afterwards the participant and I re-watch them together to try and get to the bottom of why they did what they did, and to see if they notice anything in their own behavior. As a researcher I have the data there, but I do not know the reasons behind their behavior. However, participants always understand less of their own behavior than I would have liked, even when re-watching their own recordings.

Q: Our last question: Did the pandemic really change the way you work or the duration of the experiment? Were there any delays?

A: No, I had only just started my PhD, and the technical difficulties meant that I could not conduct the experiment anyway at that point. The only issue I had with Covid-19 was that I had it three weeks ago, which then postponed it for a little bit but only very marginally.

After the interview, Sjoerd walked us to the eye-tracking laboratory. There, we could see how the experiment works and we actually took part in it. We took turns in the soundproof booth where a computer screen streamed bits of different TV shows in three different languages. Outside, Sjoerd could see our eyes’ movements on another screen, and he recorded them, creating a sort of map showing which areas of the screen our eyes were focusing on the most. Each one of us had to see three different videos, but only with the last two our eye movements were recorded. The first video had Dutch subtitles, while in the second one the subtitles matched the foreign language that was being spoken, namely English, then Japanese, then French. After each session, participants are usually dismissed, so that Sjoerd can export the video and download the data. Then, the participants are called back into the lab, and they are asked questions regarding their understanding of the plot of the TV show they saw and their awareness of their own behavior when looking at the screen.

We are glad we took part in this experiment, and we hope that we made a small contribution to Sjoerd’s research.

Elisa, Elsa and Maria Vittoria


ONLINE EVENT: Expand your knowledge of language and culture through films and subtitling

Leiden Translation would like to bring to your attention the following ONLINE EVENT, co-organized and funded by EUNIC and the Directorate-General for Translation of the European Commission:

Expand your knowledge of language and culture through films and subtitling

23 February 2021, 19:30 to 21:00 | OBA Online

How to expand your knowledge of language and culture by watching (subtitled) films? What exactly is involved in the art of subtitling films? How much of the original language and culture really comes across in subtitled films? And what are the effects of subtitles on our approach, interpretation, reception and appreciation of films and cinema?

Professional subtitler Peter Bosma, MA talks with Dr. Marie-Aude Baronian about choosing the right words and finding the right nuances when subtitling foreign films. Without providing a definitive answer to the question of whether film is “translatable” at all, they examine what is important in translating film and where translating words becomes translating culture. Together they investigate how subtitles contribute to a better understanding of foreign languages and cultures, or how they hinder them.

There is the opportunity to ask questions after the discussion.

With fragments from the films Calendar (1993) by Canadian-Armenian filmmaker Atom Egoyan and BlacKkKlansman (2018) by American filmmaker Spike Lee, and a video message by Michael Haneke. Recording images from these film clips is not allowed.

Marie-Aude Baronian is an Associate Professor of Film and Visual Culture at the University of Amsterdam. Her most recent monographic book is Screening Memory: The Prosthetic Images of Atom Egoyan (Royal Academy Belgium, 2017).

Peter Bosma studied English Language and Literature in Groningen and has been working as a subtitler for television and cinema since 1995. He is also a freelance editor and a board member of the Subtitles Department of the Dutch “Auteursbond”.

View the program via this Zoom LINK. Online from 23 February, 7:30 PM

This event is co-organized and funded by EUNIC and the Directorate-General for Translation of the European Commission.

19:30 introduction by the European Commission
19:40 conversation between Marie-Aude Baronian and Peter Bosma, moderated by Emma Hartkamp (European Commission)
20:15 question and answer session
21:00 end

Source: https://www.oba.nl/agenda/online/verrijkjekennisvantaalencultuurdoorfilmenondertiteling.html