Interview with dr. Eline Sikkema

Telops and language learning – Experiences and insights from conducting a PhD study

Eline Sikkema is a language teacher in the School of Asian Studies (SAS) at Leiden University. She holds a PhD in Applied Languages from Dublin City University. For her doctoral research she used a multimodal transcript, eye-tracking technology, questionnaires and field notes to explore Japanese language learners’ perceptions of a Japanese variety show that featured intralingual (same-language) text. Her research interests include Japanese language learning, pedagogy, Audiovisual Translation and multimodality.

Q: Could you tell us a bit about your research on Telops?

A: The research focused on the learners’ perception of an authentic Japanese show that featured same language text. The approach regarding this subject was based on two concepts: multimodal perception and visual attention, we were trying to cross-check findings on these two concepts by finding out how students look at such materials, what they think is useful for learning Japanese, and also to check their behavior while watching such program in order to understand how they make sense of that. And based off of that the study tries to give suggestions on how you can use these materials for the Japanese language classroom.

Examples of telops from Sikkema (2020), page 19.

Q: You used eye-tracking for this project, how did you start being interested in eye-tracking?

A: Actually, at the beginning I wasn’t necessarily thinking of using eye-tracking. My PhD research was part of a larger project. This larger project used eye-tracking technology in order to research Japanese variety shows and telops, so it was inspired by that study to use that technology, but it also made sense for this kind of perceptional research. All data sets have limitations so you need to consider very early on what you want to research and what kind of data you need in order to answer the research questions. So, people can also report on themselves but it can be influenced by their perception, so they may say things that are not always true. And the same happens with eye-tracking, for example, during the pilot study there were students saying: “you won’t see me looking at the telop” and then you look at the data and they were constantly looking at the telop, so for perception studies it is very good to have the eye-tracking data to cross-check findings. So, the original interest for the eye-tracking technology was sparked by the kind of data that we needed to answer our questions and to make sure that our findings were based on the data itself. I read in another interview on your blog (Sjoerd Lindenburg interview: https://leidentranslation.wordpress.com/2022/04/11/effective-viewing-behavior-and-viewing-strategies-can-we-teach-foreign-language-learners-to-effectively-use-subtitles/?msclkid=1cd789e8c6f511ec92ce2512919b1afe) something that is relevant to this discussion, and which also needs to be considered for any type of eye-tracking study: when you look at the eye-tracking data, it’s pure data, you can’t really say why they are looking at that, there could be a reason or they could just be staring at the screen. This shows that eye-tracking data also has its limitations and that e.g. self-reports can give insights into someone’s viewing behavior.

Q: Where you expecting the results that you got?

A: I was quite surprised by the importance of the number of inputs. Even though it was not completely out of the blue since communication is based to a great extent on non-verbal communication, I was still surprised at how the students could structure their own understanding of the program based on these components, that was surprising. At first, I thought that students at the beginner level would struggle with understanding the program, but they were able to catch up on these components and I thought that was very interesting and it changed my way of looking at subtitles.

Q: Do you think that telops could be used in reality shows in Europe and produce the same effect as they do in Japan?

A: That’s a difficult one because telops very much cater to the Japanese viewing audience. If you look at the research done on telops, there’s a lot of discussion regarding the viewer audience; in Japan telops are also used because people usually look at a program while doing something else, they are not 100% focused on the television, and the telop is a way for people to hear or understand what is being said even if they are not paying that much attention. So, in addition to making the program more interesting it’s also a way for them to stand out against other programs. So, there is a certain usage behind telops that is more catered towards the Japanese audience, and another important thing to consider is that telops don’t necessarily follow the same standards (e.g. display rate, color usage) as those used in Europe. So, I’m not sure it would catch on with the general viewer in Europe but I think that for learning purposes it can be helpful. However, when using them for learning purposes we need to be mindful of the color usage of telops, as telops can consist of more than one color. While the study showed that typography was helpful in understanding affective aspects of the program, it is not clear to what extent these colors can be helpful for understanding linguistic contents as they may not follow meaning or grammar rules.

So, we would need to be careful when introducing them to the general audience as it may take time for viewers to get used to them but it would be interesting to see how students use them for learning purposes.

Examples of telops from Sikkema (2020), page 21.

Q: do you think that telops could be more effective than “normal” subtitles, especially for the hearing impaired? Due to their size, effects, and colors?

A: In the conclusions of my thesis, I do mention SDH, subtitles for deaf and hard of hearing, as they use colors to differentiate the speakers. Telops use techniques that are reminiscent of SDH but they do not seem to use these in a systematic way. In the video I chose for the study there is only one particular telop style that is always used for a particular guest on the show, but all the others aren’t Considering that telops can also employ effects such as moving, shaking etc. I think their design may be more confusing to the hearing impaired and therefore less effective than SDH for this particular viewer audience.

Q: What are your projects and goals for the future?

 A: An interesting path would be to introduce Japanese variety shows and telops in class to see how students react to them. With this feedback we can then start thinking about what kind of learning tasks we can use for such materials. 

Dr. Sikkema’s PhD thesis can be downloaded here: Sikkema, E.C. (2020) Telops for language learning: Japanese language learners’ perceptions of authentic Japanese variety shows and implications for their use in the classroom. PhD thesis, Dublin City University. doi: https://doras.dcu.ie/24018/1/Hardbound%20Thesis.pdf.

Interview and blog post by: Elisa, Elsa, Maria Vittoria, Annalisa and Martina

Advertisement

Effective viewing behavior and viewing strategies: Can we teach foreign language learners to effectively use subtitles?

An interview with PhD candidate Sjoerd Lindenburg

How do language learners read subtitles? Can we use eye-tracking to determine effective viewing strategies? Can we teach students effective viewing behavior to improve language learning? These are some of the central questions in Sjoerd Lindenburg’s PhD project, funded by an NWO Promotiebeurs voor leraren (https://vakdidactiekgw.nl/author/sjoerdlindenburg/) and carried out in collaboration between Stichting Alberdingk Thijm Scholen and Leiden University (ICLON & LUCL).

Q: What does your research focus on primarily?

A: My research focuses on second language learners’ behavior. Obviously, you process the information in a certain way, but there is a difference when it is in a foreign language rather than in your first language, especially in the use of subtitling in that context.

Q: What sparked your interest?

A: I completed a Master’s in Translation here at Leiden University and I took one of Lettie Dorst’s courses, Multimodal Translation, which I thought was really interesting. As I wanted to get my teacher’s degree as well, I looked for ways to merge those two interests together. Then it turned out that a lot of research had been done on the use of subtitles for language learning, mostly during the 1970s and 1980s, but with a recent resurgence in popularity from around the 2010s.

Q: Does your research focus on other languages than English?

A: It focuses on English and French, because we want to implement our findings at the secondary school I work at (teaching English). As English has a rather unique position in the Netherlands – most children start learning it very early on, watching English audiovisual content on YouTube or television – we are also looking at French to even this exposure effect out. Most pupils don’t start learning French until they are in high school and most do not watch a lot of audiovisual content in French or listen to French music.

Q: What are your plans for the future? How do you plan on carrying out this project?

A: It is a five-year PhD-project, consisting of three phases: phase one is gaining insight into how people view audiovisual materials with subtitles. That is, whether they are good at the languages that we have, so English and French, and also Japanese for this first part, to see where they look at. To do this, we are using the eye-tracker in the LUCL lab. When do language learners look at certain parts and why do they look at them? How do people watch subtitled material? Part two is hopefully going to be next year, and we will be doing the same thing with secondary school students, to see if they behave differently when watching TV series, as they might not be as interested in learning a language as someone who chose to study it. Then part three is using the information we have gathered to create lessons to teach students how to watch those videos in a way in which they can maximize their language learning without taking away the entertainment videos provide us. Of course, watching Netflix is fun, and it is also educational, but if you make it too educational, will they still do it?

Q: What would you say are the main difficulties? Are there specific difficulties per age group, or is it more difficult if someone is obliged to learn the language rather than if they choose to do so voluntarily?

A: I think that right now, the biggest difficulty has nothing to do with participants, but with the experimental set-up. Technical difficulties have thus far been the worst, because we use such a very specific machine for this research. Another difficulty is with participants. Right now, people view parts of a TV show and afterwards the participant and I re-watch them together to try and get to the bottom of why they did what they did, and to see if they notice anything in their own behavior. As a researcher I have the data there, but I do not know the reasons behind their behavior. However, participants always understand less of their own behavior than I would have liked, even when re-watching their own recordings.

Q: Our last question: Did the pandemic really change the way you work or the duration of the experiment? Were there any delays?

A: No, I had only just started my PhD, and the technical difficulties meant that I could not conduct the experiment anyway at that point. The only issue I had with Covid-19 was that I had it three weeks ago, which then postponed it for a little bit but only very marginally.

After the interview, Sjoerd walked us to the eye-tracking laboratory. There, we could see how the experiment works and we actually took part in it. We took turns in the soundproof booth where a computer screen streamed bits of different TV shows in three different languages. Outside, Sjoerd could see our eyes’ movements on another screen, and he recorded them, creating a sort of map showing which areas of the screen our eyes were focusing on the most. Each one of us had to see three different videos, but only with the last two our eye movements were recorded. The first video had Dutch subtitles, while in the second one the subtitles matched the foreign language that was being spoken, namely English, then Japanese, then French. After each session, participants are usually dismissed, so that Sjoerd can export the video and download the data. Then, the participants are called back into the lab, and they are asked questions regarding their understanding of the plot of the TV show they saw and their awareness of their own behavior when looking at the screen.

We are glad we took part in this experiment, and we hope that we made a small contribution to Sjoerd’s research.

Elisa, Elsa and Maria Vittoria

Erasmus+ exchange @ Leiden Translation

Our First Month in Leiden

Hi everyone, it’s Elisa, Elsa, and Maria Vittoria, three exchange trainees from Italy working at Leiden University at the Center for Linguistics with Dr. Susana Valdez and Dr. Lettie Dorst. We wanted to introduce ourselves, as we will be posting some articles in this blog in the upcoming months. We are from Genoa University, and we have just graduated in Translation and Interpreting. Since we arrived, we got the chance to get to know the city that will be our home for the next four months. In this first month, we have had some time to get used to Dutch culture and university life. The campus is amazing; the buildings are beautiful and we especially love working at PJ Veth, at the Arsenaal building and at the Law Faculty.

But what do we do exactly for work? The three of us are taking part in an Erasmus+ Traineeship program divided into two distinct branches: Elsa is focusing on medical translation, while Elisa and Maria Vittoria are studying metaphor in machine translation. As you can tell, we are especially interested in studying translation, whether it occurs in specific fields or with the aid of specific tools. Along this research, we are also carrying out various other tasks: namely, we will be organizing Leiden Translation Talks and we will be interviewing prominent personalities studying and working at Leiden University in the realm of foreign languages and translation. As you can tell, we just love learning languages and getting in touch with different cultures! We are Italian native speakers, but, among us, we can speak a variety of foreign languages including English, Russian, German, Spanish, and French. Unfortunately, we have not gotten around to learning Dutch yet – but we are working on it!

Our experience at university is going smoothly, but it began with some technical problems which were promptly resolved by the University staff at the Plexus and at the Lipsius buildings. If you are looking for some technical support, these are definitely the places to go to. Another great service that the university provides is the library and the possibility of retrieving a book anywhere on campus. When we needed to borrow a book, it just took a few taps on our phones, and we could get the book delivered to the building nearest to our home. It was so convenient and helpful! When we are not working, we enjoy taking long walks along the canals and, as we are cat people, we love spotting the numerous cats that live around here. Leiden is such a picturesque town! We climbed up the Burcht, which is a castle on the hill. We were delighted when we reached the top and came across a scenic view of the whole city. Luckily for us, we took this stroll on a bright sunny day, which was really appreciated after being caught in the rain several times. The weather was the main shock for us, because it is so different from what we are used to! It changes so abruptly, and we are always caught off guard. On the very first rainy days we would always try and use umbrellas, but we were the only ones in the streets doing so and we quickly found out why: the wind blows so strong that it is really no use trying to fight it with an umbrella, and so we just gave up – you know what they say: when in Rome…

One thing we have not tried out yet is riding a bike. It may sound silly, but the number of bikes on the streets and the speed at which people ride them kind of confuse us! We come from a mountainous Italian region and people do not really ride bikes there; we do ride scooters, though, but we are not used to the lack of helmets. So, we mostly walk around the town. During our various aforementioned walks, we also stopped for food, and we tried the typical Dutch delicatessen, that is, fries, bitterballen, and Dutch pancakes. We also enjoy sipping a refreshing Heineken sitting by the water, but we are not going to lie: our favorite drink is coffee, and we usually get it at the cafeteria at the Lipsius building. Nothing like a good roast to give our days a kick start! During our lunch breaks we usually head to the Us Bertus bakery, just a few steps away from campus, which offers fresh-made and filling sandwiches, perfect to grab something quick before getting back to work.That is all from us today; we hope our experiences and suggestions will be useful for other exchange students! We will be posting articles and interviews covering a number of topics, mainly regarding translation and language studies, so stay tuned. Bye!

Elisa, Elsa and Maria Vittoria

The Europeana XX: Subtitle-a-thon Challenge Amsterdam

Do you speak Dutch and English and are you an audiovisual archives enthusiast? Then join us for the Europeana XX: Subtitle-a-thon Challenge Amsterdam! Share your language and subtitling skills, receive a participation certificate and win prizes!

The Netherlands Institute for Sound and Vision, The European Union National Institutes for Culture (EUNIC) Cluster The Netherlands, and the Leiden University Centre for Linguistics are pleased to invite you to a subtitle-a-thon focused on audiovisual heritage.

The kick-off of the subtitle-a-thon takes place on Sunday, September 26th at 15.00 CET as part of one of the activities celebrating the European Day of Languages 2021. The event will start with a two-hour introductory session and will run online for seven days with a closing session on Saturday, October 2nd at 17.00 CET. The event will be held in English. Registration opens on September 7th  at 15.00 CET at www.subtitleathon.eu.

Making AV heritage more accessible

The Subtitle-a-thons are four sprint-like online events organised by the Deutsches Filminstitut & Filmmuseum – DFF (Germany), The Istituto Luce Cinecittà (Italy), the National Film Archive – Audiovisual Institute – FINA (Poland) and, the Netherlands Institute for Sound and Vision – NISV (The Netherlands). During the events, people with different language skills work together toward a common goal: to create and add different subtitles to archival media fragments from various European collections.

The aim of the subtitle-a-thon initiative is to engage language learners and students in a playful way with multilinguality and audiovisual archival material, while raising awareness about the value of multilingual access to AV archival footage. The events also hope to promote European national languages and to stimulate discussions around the value of language, and thereby increasing the interest of local communities in 20th-century cultural heritage. The subtitle-a-thons are educational activities highlighting the profession of translators. The subtitles produced in the event do not intend to compete with nor to replace professional level translation.

Join the Amsterdam Challenge

The event targets youngsters and adults interested in audiovisual heritage, language and multilingualism. And more specifically, students, translators, language teachers and followers of advanced language courses (C1 and above). It is eligible only for participants fluent in Dutch, Flemish, and English. Participants will be invited to work individually, using a specialised media player with a subtitle editor, to subtitle the available audiovisual material. The winners of the Subtitle-a-thon Challenge Amsterdam will receive online prizes and all entrants will receive a participation certificate.

The participants will be subtitling short clips coming from the Netherlands Institute for Sound and Vision archival collection, as well as from the British Broadcasting Corporation.

We invite you to:

  • Create subtitles for a variety of short audiovisual clips, such as (but not limited to) newsreels, short documentaries and clips about art and culture.
  • Test a new tool and your language skills in a friendly and fun environment.
  • At the same time join the community of language enthusiasts and audiovisual heritage fans.
  • Win a prize (ecomondo gift cards) for becoming one of the best Europeana XX subtitlers!

Learn more about the event and register today for free here: https://www.subtitleathon.eu/event/amsterdam/

About Europeana XX

Europeana XX: Subtitle-a-thon Challenge Amsterdam is hosted by the Netherlands Institute for Sound and Vision under Europeana XX: Century of Change, an EU-funded project dedicated to opening up 20th-century digital heritage collections, highlighting the changes that shaped Europe and still impact our world and lives today. Europeana XX is part of the Europeana family of initiatives supporting European cultural heritage institutions in sharing their digitised collections. Millions of digital objects from all across Europe are showcased on Europeana.eu, the trusted portal for digital cultural heritage in Europe.

***

Explore the collections and stories from 20th-century history and culture through the Europeana XX editorials at www.europeana.eu/en/highlights-from-the-20th-century

For more information about the Europeana XX project please visit: https://pro.europeana.eu/project/europeana-xx

ONLINE EVENT: Expand your knowledge of language and culture through films and subtitling

Leiden Translation would like to bring to your attention the following ONLINE EVENT, co-organized and funded by EUNIC and the Directorate-General for Translation of the European Commission:

Expand your knowledge of language and culture through films and subtitling

23 February 2021, 19:30 to 21:00 | OBA Online

How to expand your knowledge of language and culture by watching (subtitled) films? What exactly is involved in the art of subtitling films? How much of the original language and culture really comes across in subtitled films? And what are the effects of subtitles on our approach, interpretation, reception and appreciation of films and cinema?

Professional subtitler Peter Bosma, MA talks with Dr. Marie-Aude Baronian about choosing the right words and finding the right nuances when subtitling foreign films. Without providing a definitive answer to the question of whether film is “translatable” at all, they examine what is important in translating film and where translating words becomes translating culture. Together they investigate how subtitles contribute to a better understanding of foreign languages and cultures, or how they hinder them.

There is the opportunity to ask questions after the discussion.

With fragments from the films Calendar (1993) by Canadian-Armenian filmmaker Atom Egoyan and BlacKkKlansman (2018) by American filmmaker Spike Lee, and a video message by Michael Haneke. Recording images from these film clips is not allowed.

Marie-Aude Baronian is an Associate Professor of Film and Visual Culture at the University of Amsterdam. Her most recent monographic book is Screening Memory: The Prosthetic Images of Atom Egoyan (Royal Academy Belgium, 2017).

Peter Bosma studied English Language and Literature in Groningen and has been working as a subtitler for television and cinema since 1995. He is also a freelance editor and a board member of the Subtitles Department of the Dutch “Auteursbond”.

View the program via this Zoom LINK. Online from 23 February, 7:30 PM

This event is co-organized and funded by EUNIC and the Directorate-General for Translation of the European Commission.

Programme:
19:30 introduction by the European Commission
19:40 conversation between Marie-Aude Baronian and Peter Bosma, moderated by Emma Hartkamp (European Commission)
20:15 question and answer session
21:00 end

Source: https://www.oba.nl/agenda/online/verrijkjekennisvantaalencultuurdoorfilmenondertiteling.html