Web Translation API

[domenic]

Request for Mozilla Position on an Emerging Web Specification

• Specification title: Web Translation API
• Specification or proposal URL (if available): Web Translation API WICG/proposals#147
• Explainer URL (if available): https://github.com/WICG/translation-api
• Proposal author: @domenic

Other information

This is an early-stage idea. However, we'd love early engagement from other implementers to make sure what we're designing fits well with their architecture.

[tomayac]

(TAG review in w3ctag/design-reviews#948.)

[saschanaz]

FYI, we have published our own Translation API explainer: https://github.com/mozilla/explainers/blob/main/translation.md

[saschanaz]

We are negative on this proposal. While we plan to describe the reasons further in our proposal, briefly it's about:

1. The spec assumes a certain form of translation backend, exposing information about model availability, download progress, quotas, and usage prediction. We'd like to minimize the information exposure so that the implementation can be more flexible.
2. DOM tree translation is a powerful feature that is nontrivial to polyfill. We'd like to provide first-level support for the page translation use case.

[domenic]

Looking forward to hearing more!

1. The spec assumes a certain form of translation backend, exposing information about model availability, download progress, quotas, and usage prediction. We'd like to minimize the information exposure so that the implementation can be more flexible.

I don't think this is accurate. The specification is clear that all of these signals are not required, e.g. you could always return "available" and there's no download progress, or it explicitly allows infinite quotas with zero usage.

2. DOM tree translation is a powerful feature that is nontrivial to polyfill. We'd like to provide first-level support for the page translation use case.

Do you have evidence of developer demand for DOM tree translation? We've not seen developers interested in this, since browsers have this functionality built in. Instead, they're interested in translating information that's specifically not in the DOM tree.

[sandstrom]

We are negative on this proposal. While we plan to describe the reasons further in our proposal, briefly it's about [...]

Use-case

We have a SaaS B2B application built in Javascript, using a single-page app framework. In this application, people can write notes to each other (among other things). For these, it would be useful if we could offer translation. We also have some text that's in images (not in DOM) that we've extracted and would like to translate and then overlay the images.

Since it's an SPA, we're operating within a framework and not directly on the DOM, so just having JS-based API we can call would be perfect. Tying it close to the DOM would actually make things more complicated for us, because we cannot easily edit the DOM directly (using a template language from the framework, etc).

For us, the proposal under webmachinelearning/translation-api would be a good fit.

I realize there are many other use-cases, so obviously just one data point.

[collimarco]

I have made a similar proposal here:
WICG/proposals#215

[xM8WVqaG]

FYI, we have published our own Translation API explainer: https://github.com/mozilla/explainers/blob/main/translation.md

Another context where a JS based API would be useful is in WebExtensions.

The Mozilla rejections are primarily focused on performing translations on content within the DOM - from the context of a regular webpage. However, browser extensions doesn't necessarily operate strictly within that environment. They may not have a GUI or write to the current page's DOM at all.

Making access to translation models dependant on having a valid DOM element to call means WebExtensions that currently operate "headless" may need to be a bit creative to use this API.

IMO it may be worth implementing the proposed JS API for usecases where the JS interface is a better experience, and then Mozilla could extend the proposal with an additional DOM-based interface for the scenarios Mozilla is concerned about. I don't think it makes sense to block this proposal on the current reasoning as the two usecases are mutually exclusive.

[saschanaz]

We still think many of the spec is assuming cloud backend based on AI which we are not interested in. We can be positive if the spec can be split and anything non-basic features can go to a separate extension spec, like what happened to File System and File System Access. The non-basic features include model availability, download progress, quotas, and usage prediction as described above.

That means for now the proposed position is not changing.

[xM8WVqaG]

While I agree that the current proposed spec is very cloud LLM/model centric, several of the non-basic features you just highlighted are still relevant to the offline NMT based Bergamot engine used in Firefox such as model availability and download progress. The quota for offline translation is unlimited and usage prediction is zero.

Given Firefox already supports calling out to cloud-hosted solutions in the AI Chatbox sidebar, it's not too difficult to imagine enabling external translation providers as an option for languages that aren't supported in Bergamot, or for complex translation (such as mixed language content) that Bergamot doesn't handle.

The current workaround is using the trialML API to run MT models which, given their relatively small size to run in the browser, give poor results in comparison.