Handwriting recognition is the ability of a computer to receive intelligible handwritten input. The image of the written text may be sensed "off line" from a piece of paper by optical scanning (optical character recognition). Alternatively, the movements of the pen tip may be sensed "on line", for example by a pen-based computer screen surface.

Handwriting recognition principally entails optical character recognition. However, a complete handwriting recognition system also handles formatting, performs correct segmentation into characters and finds the most plausible words.

On-line recognition

On-line handwriting recognition involves the automatic conversion of text as it is written on a special digitizer or PDA, where a sensor picks up the pen-tip movements X(t),Y(t) as well as pen-up/pen-down switching. That kind of data is known as digital ink and can be regarded as a dynamic representation of handwriting. The obtained signal is converted into letter codes which are usable within computer and text-processing applications.

The elements of an on-line handwriting recognition interface typically include:

• a pen or stylus for the user to write with.
• a touch sensitive surface, which may be integrated with, or adjacent to, an output display.
• a software application which interprets the movements of the stylus across the writing surface, translating the resulting curves into digital text.

Handwriting recognition is commonly used as an input method for PDAs. The first PDA to provide written input was the Apple Newton, which exposed the public to the advantage of a streamlined user interface. However, the device was not a commercial success, owing to the unreliability of the software, which tried to learn a user's writing patterns. By the time of the release of the Newton OS 2.0, wherein the handwriting recognition was greatly improved, including unique features still not found in current recognition systems such as modeless error correction, the largely negative first impression had been made. Another effort was Go's tablet computer using Go's Penpoint operating system and manufactured by various hardware makers such as NCR and IBM. IBM's ThinkPad tablet computer was based on Penpoint operating system and used IBM's handwriting recognition. This recognition system was later ported to Microsoft Windows for Pen Computing, and IBM's Pen for OS/2. None of these were commercially successful.

Palm later launched a successful series of PDAs based on the Graffiti® recognition system. Graffiti improved usability by defining a set of pen strokes for each character. This narrowed the possibility for erroneous input, although memorization of the stroke patterns did increase the learning curve for the user.

A modern handwriting recognition system can be seen in Microsoft's operating system running on Tablet PCs (notably Windows XP Tablet PC Edition and Windows Vista). It is based on CalliGrapher, a newer version of the same handwriting recognition software that was used on Newton OS 2.0, which in 1999 Microsoft licensed from ParaGraph International (now PhatWare Corp).

A Tablet PC is a special notebook computer that is outfitted with a digitizer tablet and a stylus, and allows a user to handwrite text on the unit's screen. The operating system recognizes the handwriting and converts it into typewritten text. Windows Vista includes personalization features that learn a user's writing patterns and/or vocabulary for English, Japanese, Chinese Traditional, Chinese Simplified and Korean. The features include a 'personalization wizard' that prompts for samples of a user's handwriting and uses them to retrain the system for higher accuracy recognition. This system is distinct from the less advanced handwriting recognition system employed in its Windows Mobile OS for PDAs.

In recent years, several attempts were made to produce ink pens that include digital elements, such that a person could write on paper, and have the resulting text stored digitally. The success of these products is yet to be determined.

Although handwriting recognition is an input form that the public has become accustomed to, it has not achieved widespread use in either desktop computers or laptops. It is still generally accepted that keyboard input is both faster and more reliable. As of 2006, many PDAs offer handwriting input, sometimes even accepting natural cursive handwriting, but accuracy is still a problem, and some people still find even a simple on-screen keyboard more efficient.

Off-line recognition

Off-line handwriting recognition involves the automatic conversion of text in an image I(x,y) into letter codes which are usable within computer and text-processing applications. The data obtained by this form is regarded as a static representation of handwriting.

The technology is successfully used by businesses which process lots of handwritten documents, like insurance companies. The quality of recognition can be substantially increased by structuring the document (by using forms).

The off-line handwriting recognition is comparatively difficult. As different people have different handwriting styles, so it becomes difficult to recognize the handwriting by computer. Nevertheless, limiting the range of input can allow recognition to improve. For example, the ZIP code digits are generally read by computer to sort the incoming mail.

Research

Handwriting Recognition has an active community of academics studying it. The biggest conferences for handwriting recognition are the International Workshop for Handwriting Recognition (IWFHR), held in even-numbered years, and the International Conference for Document Analysis and Retrieval (ICDAR), held in odd-numbered years. Both of these conferences are organized under the auspices of the IEEE. Active areas of research include:

• Online Recognition
• Offline Recognition
• Signature Verification
• Postal-Address Interpretation
• Bank-Check Processing

See also

• optical character recognition
• scriptnetics

Related websites

• online handwriting recognition and potential applications Vision Objects' website.
• CalliGrapher handwriting recognition technology and SDK PhatWare Corporation website.
• International Workshop for Frontiers in Handwriting Recognition Homepage of the 2006 workshop. Upgraded to a conference starting 2008.
• International Conference for Document Analysis and Recognition Homepage of the 2007 conference.
• Lipi Toolkit An Open Source toolkit for Online Handwriting Recognition