Overview of Graphical User Interfaces

Most user interfaces created these days are called Direct Manipulation User Interfaces. A direct manipulation user interface is an interface in which a computer application provides concrete, pictorial representations for objects and users can directly manipulate them. By doing so, the user is able to change the state of the application. Such interfaces have also been dubbed WYSIWYG (wiz-ee-wig) interfaces, where WYSIWYG stands for What You See Is What You Get. Examples of direct manipulation interfaces include such commmonly used applications as word processors, video games, operating systems (Mac, Windows), spreadsheets, and databases.

A graphical user interface consists of two components:

  1. a presentation component that provides a pictorial interface to the user, and
  2. a behavior component that determines how the interface reacts and/or responds to the user.

Presentation Component

The presentation components of an application may consist of:

  1. Widgets: Pre-defined (canned) pieces of graphics (look) and behavior (feel) that allow the user to interact with an application by: An application may also use a widget to display application data values.

    Widgets are standard interaction techniques that are commonly used, and which are duplicated from application to application. Consequently, widgets are often called "application-independent". Examples of widgets include type-in text boxes, radio buttons, check boxes, and menus.

  2. Graphics that represent application objects: These graphics are typically composed from primitive graphical objects provided by a window system, such as rectangles, lines, bitmaps, and text. Since application objects vary greatly from one application to another, it is typically impractical to create standard widgets for each application. Instead, a graphic artist designs new graphical objects for each application. The graphical objects representing application objects are often called "application-dependent".

    Because the graphics for application objects must be created from scratch for each application, creating this portion of the user interface typically requires one to two orders magnitude more effort than creating the widgets. Similarly, the behaviors of application objects typically varies from application to application, so it is difficult to have pre-defined input-handlers. Thus the behavior of application objects (i.e., input-handlers) typically must be created from scratch.

    Icons are bitmaps that represent application objects, such as game pieces, gates in a circuit application, or documents in a text editor. Icons are probably the most frequently used type of application object, because 1) a graphic artist can create very detailed pictures using bitmaps; and 2) a toolkit can draw them very efficiently on the display. However, icons have a very limited ability to change themselves in response to changes in an application's data, and thus are relatively poor choices for displaying an application's output.

    Behavior Component

    The behavior of interfaces is controlled via events. An event represents some action that is performed by an input device, such as a keyboard, mouse, pen, trackball, or data glove. Most window packages contain a few types of events and the developer of an input device must map the allowable actions of the input device to one of the types of events provided by the window system. When you attach a new input device to your computer for the first time, you typically must load a disk that contains software which tells the OS how to perform this mapping. Common event types are mouse events, motion events, and keyboard events. Predictably most interaction devices will have their events mapped to motion or mouse events.

    When an input device generates an event, the OS receives the event and through some interaction with the window system, converts the event to an event recognizable by the window system. The OS then passes the event to the window system, which typically places the event into a serial event queue. The window system removes events in a FIFO order from the event queue and dispatches them to the appropriate window.

    A window manager has priority over an application in processing the events that are sent to windows. The window manager is a piece of software that sits on top of the window system and manages an application's windows. The icons that appear across the top of many windows are parts of the window manager's interface and frequently include expand, minimize, and close window icons. A toolbar is often considered to be part of the window manager and so are the scroll bars around a window.

    When an event is dispatched to a window, the window manager first checks to see whether it should handle the event and if so it performs the appropriate action and then returns control to the window system. Otherwise it passes the event to the application which typically invokes a callback procedure to handle the event. A callback procedure is a piece of application code that performs some processing and then passes the results back to the user interface. Typically these results are made known to the user by modifying various graphical objects in the graphical interface. This type of feedback is called semantic feedback because it is application-specific.

    Widgets come with their own, pre-defined callback procedures but programmers can frequently add an action procedure that will call the appropriate application routine. The pre-defined callback procedures provided by the widgets provide syntactic feedback to the user. Syntactic feedback represents an immediate response that acknowledges the user's input and is typically application-independent. For example a button object might press down down while a text entry field will echo the keystroke.

    Programmers must explicitly create syntactic feedback and event handling procedures for custom graphics. For example, the programmer might create a dashed line box and move it around the screen in response to an attempted move operation by the user. Sometimes the line between semantic and syntactic feedback becomes blurry. If I try to drag an icon around the screen both Windows machines and Macs will try to highlight an application icon if the dragged icon is moved over it and the application associated with the application icon can do something intelligent with the file represented by the dragged icon (e.g., place it in a folder, throw it away, etc.). This type of feedback requires both syntactic feedback, namely a ghost-like icon that gets dragged, and semantic feedback, namely the highlighting of an application icon when it can handle the dragged icon.

    The type of control exerted by a user over a GUI is called external control because the user controls the execution of the program and the program must immediately respond to user input, even if it only initially provides syntactic feedback. In terms of implementation, the application is broken into many small callback procedures instead of one monolithic program. Hence there are many possible threads of control in a direct manipulation GUI as opposed to the single thread of control common in most programs.