IME handling guide

This document explains how Gecko handles IME.

Introduction

IME is an abbreviation of Input Method Editor. This is a technical term from Windows but these days, this is used on other platforms as well.

IME is a helper application of a user’s text input. It handles native key events before or after focused application (depending on the platform) and creates a composition string (a.k.a. preedit string), suggests a list of what the user attempts to input, commits composition string as a selected item off the list and commits composition string without any conversion. IME is used by Chinese, Japanese, Korean and Taiwan users for inputting Chinese characters because the number of them is beyond thousands and cannot be input from the keyboard directly. However, especially on mobile devices nowadays, IME is also used for inputting Latin languages like autocomplete. Additionally, IME may be used for handwriting systems or speech input systems on some platforms.

If IME is available on focused elements, we call that state “enabled”. If IME is not fully available(i.e., user cannot enable IME), we call this state “disabled”.

If IME is enabled but users use direct input mode (e.g., for inputting Latin characters), we call it “IME is closed”. Otherwise, we call it “IME is open”. (FYI: “open” is also called “active” or “turned on”. “closed” is also called “inactive” or “turned off”)

So, this document is useful when you’re try to fix a bug for text input in Gecko.

Composition string and clauses

Typical Japanese IME can input two or more words into a composition string. When a user converts from Hiragana characters to Chinese characters the composition string, Japanese IME separates the composition string into multiple clauses. For example, if a user types “watasinonamaehanakanodesu”, it’s converted to Hiragana characters, “わたしのなまえはなかのです”, automatically (In the following screenshots, the composition string has a wavy underline and the only one clause is called “raw input clause”).

When a user presses Convert key, Japanese IME separates the composition string as “わたしの” (my), “なまえは” (name is) and “なかのです” (Nakano). Then, converts each clause with Chinese characters: “私の”, “名前は” and “中野です” (In the following screenshot each clause is underlined and not connected adjacently. These clauses are called “converted clause”).

If one or more clauses were not converted as expected, the user can choose one of the clauses with Arrow keys and look for the expected result form the list in the drop down menu (In the following screenshot, the clause with the thicker underline is called “selected clause”).

Basically, composition string and each clause style is rendered by Gecko. And the drop down menu is created by IME.

Each clause is represented with selection in the editor. From chrome script, you can check it with nsISelectionController. In native code, you can access it with either nsISelectionController or mozilla::SelectionType (the latter is recommended because of type safer). And editor sets these IME selections from mozilla::TextRangeType which are sent by mozilla::WidgetCompositionEvent as mozilla::TextRangeArray. The following table explains the mapping between them.

Selection types of each clause of composition string or caret

	nsISelectionController	mozilla::SelectionType	mozilla::TextRangeType
Caret	SELECTION_NORMAL	eNormal	eCaret
Raw text typed by the user	SELECTION_IME_RAW_INPUT	eIMERawClause	eRawClause
Selected clause of raw text typed by the user	SELECTION_IME_SELECTEDRAWTEXT	eIMESelectedRawClause	eSelectedRawClause
Converted clause by IME	SELECTION_IME_CONVERTEDTEXT	eIMEConvertedClause	eConvertedClause
Selected clause by the user or IME and also converted by IME	SELECTION_IME_SELECTEDCONVERTEDTEXT	eIMESelectedClause	eSelectedClause

Note that typically, “Selected clause of raw text typed by the user” isn’t used because when composition string is already separated to multiple clauses, that means that the composition string has already been converted by IME at least once.

Modules handling IME composition

widget

Each widget handles native IME events and dispatches WidgetCompositionEvent with mozilla::widget::TextEventDispatcher to represent the behavior of IME in the focused editor.

This is the only module that depends on the users platform. See also [Native IME handlers] section for the detail of each platform’s implementation.

Note

Android widget still does not use TextEventDispatcher to dispatch WidgetCompositionEvents, see bug 1137567.

mozilla::widget::TextEventDispatcher

This class is used by native IME handler(s) on each platform. This capsules the logic to dispatch WidgetCompositionEvent and WidgetKeyboardEvent for making the behavior on each platform exactly same. For example, if WidgetKeyboardEvent should be dispatched when there is a composition is managed by this class in XP level. First of use, native IME handlers get the rights to use TextEventDispatcher with a call of BeginNativeInputTransaction(). Then, StartComposition(), SetPendingComposition(), FlushPendingComposition(), CommitComposition(), etc. are available if BeginNativeInputTransaction() return true. These methods automatically manage composition state and dispatch WidgetCompositionEvent properly.

This is also used by mozilla::TextInputProcessor which can emulates (or implements) IME with chrome script. So, native IME handlers using this class means that the dispatching part is also tested by automated tests.

mozilla::WidgetCompositionEvent

Internally, WidgetCompositionEvent represents native IME behavior. Its message is one of following values:

eCompositionStart

This is dispatched at starting a composition. This represents a DOM compositionstart event. The mData value is a selected string at dispatching the DOM event and it’s automatically set by TextComposition.

eCompositionUpdate

This is dispatched by TextComposition when an eCompositionChange will change the composition string. This represents a DOM compositionupdate event.

eCompositionEnd

This is dispatched by TextComposition when an eCompositionCommitAsIs or eCompositionCommit event is dispatched. This represents a DOM compositionend event.

eCompositionChange

This is used internally only. This is dispatched at modifying a composition string, committing a composition, changing caret position and/or changing ranges of clauses. This represents a DOM text event which is not in any standards. mRanges should not be empty only with this message.

eCompositionCommitAsIs

This is used internally only. This is dispatched when a composition is committed with the string. The mData value should be always be an empty string. This causes a DOM text event without clause information and a DOM compositionend event.

eCompositionCommit

This is used internally only. This is dispatched when a composition is committed with specific string. The mData value is the commit string. This causes a DOM text event without clause information and a DOM compositionend event.

Table of event messages

	meaning of mData	who sets mData?