Testifying While Black -- New media coverage

[EDIT 3: Pod Save the People talked about our research this week! Start listening at 11min 58seconds into the episode. Its a really good discussion of the research and its implications. And so cool to have this group of people discussing what we did!]

[EDIT 2: Taylor was on Radio Times with Marty Moss-Coane on WHYY Philly this morning. The other guests were Cassie Owens who wrote about our research in the Philadelphia Inquirer and Kami Chavis, a Professor of Law and Director of the Criminal Justice Program at Wake Forest School of Law. They had an awesome discussion about the research and its implications — with a nice little shout out to me and other co-authors at the top of the program! You can listen to the whole episode here.]

[EDIT: Taylor went on the CBC Radio One program As It Happens to talk about the research two days ago. You can listen here. Also, either he or both of us will be on Radio Times on WHYY Philly on Friday morning.]

We have been getting a lot of media attention for our forthcoming paper in Language about court reporter mistranscription of African American English. (See my previous blog post, or this excellent post from my co-author, Taylor Jones, to read a little about what the research was and what we found. )

Here is the Philadelphia Inquirer’s coverage of the research.

And here is the coverage by John Eligon at the New York Times. The article made the Sunday Times print edition!

We are still getting inquiries, so watch this space for more media related to this research.

Testifying While Black: (Mis)transcription of African American English in the Court Room

Following four years of work [EDIT: it’s been 6 years], a project I did with several colleagues has just been accepted for publication in the journal Language. Together with Taylor Jones (U Penn Linguistics, www.languagejones.com), Ryan Hancock (Philadelphia Lawyers for Social Equity, WWD), and Robin Clark (U Penn Linguistics), I conducted a study to test Philadelphia court reporter’s transcription accuracy and comprehension of African American English (see Taylor Jones’ explainer on this language variety).

In order to work as an official court reporter in Philadelphia, court reporters have to be certified by the court system at 95% accuracy. In other words, they have to be able to correctly transcribe 95% of all words they hear in a test at specific speeds (words per minute) based on the type of speech — there are different required speeds for testimony versus question and answer, etc. However, based on informal interviews with court reporters, we determined that their certification tests and training are based on Mainstream American English spoken by lawyers, judges, and broadcasters. We know from previous research that there are issues of cross dialect comprehension between speakers of AAE and MAE (some examples outlined here) so we decided to test court reporters’ ability to accurately transcribe AAE as spoken by native speakers from the Philadelphia area.

We recruited 9 native speakers of AAE (5 men and 4 women of varying ages) from West Philadelphia, North Philadelphia, Harlem, and Jersey City and recorded them reading 83 sentences in AAE that were chosen from actual heard speech (we did not create the sentences from imagination). These sentences included syntactic features of AAE both alone and in combination. We then randomized the voices and sentences. We played the audio for 27 court reporters — one third of the official court reporting pool in Philadelphia. Reporters were given a 220Hz warning tone followed by a sentence repeated twice and then ten seconds of silence. The sentences were played at 70-80 decibels at 10 feet — more than loud enough for the court reporters to hear — and at speeds much slower than their certification tests. We asked the court reporters to transcribe the sentence and then paraphrase the sentence in “classroom” English. While we were aware that paraphrasing is not part of their normal job, we were curious if miscomprehension contributed to mistranscription.

The results showed that the court reporters in our sample could not transcribe spoken AAE at their required level of accuracy. When measured at the sentence level - was the sentence right or wrong - our sample transcribed 59.5% of sentences accurately on average. When measured at the word level, on average our court reporters transcribed 82.9% of the words accurately. In 31% of transcriptions, errors change who, what, when, or where. Accuracy was not related to race, age, where they got their training, or the number of years on the job. Additionally, court reporters paraphrased the sentences correctly only 33% of the time. Surprisingly, reporters’ individual paraphrase and transcription accuracy were not systematically related.

From post-test conversations with the study participants, it was clear that these court reporters’ wanted the tools to perform better and did not hold explicit malice towards the speakers or the individuals they transcribe in court. They did, however, express the opinion that speakers like the ones we played for them were speaking incorrectly and that the difficulties they had with transcription were the fault of the speaker. Rather, in our paper, we contend that court reporters are not given appropriate training related to other varieties of English they are likely to encounter in their day-to-day work. Given that the court reporter is responsible for the official court record and that the official record has consequences in terms of cross-examination, appeals, etc, it would behoove the court system to ensure that certification standards are related to the task at hand.

You can read more here on the study, its implications, and what we think comes next.