Auckland Transport launches AI sign language avatars on buses, trains and ferries as Deaf advocates urge caution

A Kiwi technology company has begun deploying real-time AI sign language avatars on Auckland’s trains, buses and ferries in what is being described as a world first for live, non-pre-recorded avatar translation. The rollout has been welcomed by some in the Deaf community as a long overdue accessibility step, but advocates are also urging caution about accuracy and the risks of using machine generated signing in high stakes settings.

Kara Technologies, working in partnership with the Deaf and Hard of Hearing Foundation, says its system uses motion capture suits and multiple cameras to record human signers, then assembles individual signs into complete sentences using artificial intelligence. The company is building a library of more than 10,000 individual signs and has carried out three rounds of consultation with Deaf communities in Auckland, Wellington and Christchurch.

Co-founder Farmehr Farhour told 1News that the avatars were designed to complement rather than replace human interpreters. “Our focus is to be there where human interpreters cannot be, especially in emergencies where you need information instantly in an accessible format,” she said.

Auckland Transport is the first major deployment partner. Live updates on train, bus and ferry services will now appear in New Zealand Sign Language alongside spoken and written announcements. The system is intended to plug a gap in real time information for the more than 24,500 New Zealanders who use NZSL, including roughly 4,500 Deaf people who rely on it as their first language.

The launch comes as the Deaf community marks 20 years since NZSL became an official language under the New Zealand Sign Language Act 2006. World Federation of the Deaf president Joseph Murray told 1News that “Aotearoa has been very leading in its promotion of sign language” and that the country was being used as a template internationally. Even so, officials have acknowledged that gaps remain, particularly in healthcare and education access.

Kara Technologies NZSL expert Jon Tai-Rakena said the system also incorporates te reo Māori concepts used by Turi Māori, the Deaf Māori community, with a separate round of Māori consultation under way to ensure those signs are properly represented. Deaf and Hard of Hearing Foundation chief executive Natasha Gallardo has described community feedback as “fabulous” and said organisers were committed to “listening to them and that they have a voice in how this technology is brought to life”.

Not everyone is convinced. Lara Draper, general manager for adults and seniors at Deaf Aotearoa, told the New Zealand Herald that translation back from sign language into spoken or written language remained “too complex at the moment” and warned that systems without human oversight risked “flattening” the language by failing to capture the multi-dimensional grammatical role of facial expressions and shoulder movements.

Draper said it mattered “who was creating these tools and who would be responsible for the implications if they were incorrect,” and cautioned that external developers too often treat Deaf people as “an afterthought,” producing technologies that “inevitably fail because they haven’t considered Deaf people from the outset”. She argued AI should not be used in legal, health or mental health settings without human verification because the consequences of misinterpretation in those environments were too high.

The accuracy debate has practical implications. Kellye Bensley, who leads the Ministry of Disabled People’s Sign Language team, has previously asked, “If there’s an emergency and an interpreter isn’t there, how can that person make a decision about their healthcare?” That question sits at the heart of Auckland Transport’s case for the avatars, which will appear during service disruptions when interpreter access is unrealistic.

For now, the rollout will begin with low risk environments. Live transport announcements represent the kind of high frequency, low complexity use case where the technology is considered relatively safe. Higher stakes uses, in courtrooms, hospitals and mental health services, are not part of the initial launch and remain dependent on human interpreters.

The pressure on those interpreters is one of the reasons Kara Technologies and the Deaf and Hard of Hearing Foundation argue the avatars are needed at all. New Zealand has a small pool of qualified NZSL interpreters relative to demand, and Deaf people have long described missing out on real time information in everyday situations such as airport announcements, station updates and emergency alerts when no interpreter is available.

The technology itself is anchored in human signing rather than purely synthetic generation. Each of the 10,000 plus signs in the library is captured from a human signer wearing a motion capture suit, and Kara says Deaf language experts review the assembled sentences before they appear on public screens. The company has positioned that human in the loop approach as a direct response to ethical concerns about AI generated language.

Education advocates say the avatars should sit alongside, rather than ahead of, broader reforms to NZSL learning. Canterbury resident Shelley Bakker, who has hearing loss, has argued that NZSL should be taught in schools because it “enables community connection and inclusion”.

The next test for the rollout will be how reliably the avatars perform on disruption alerts, where signs need to be assembled quickly from unscripted text and where errors could cause genuine confusion for travellers with sensory disability or for Deaf passengers trying to make split second decisions about an alternative route. Auckland Transport has not yet released data on accuracy rates or how complaints will be handled.

Have you used NZSL on public transport, or do you have views on where AI accessibility tools should and should not be used? Share your thoughts in the comments below.