GUI Building Blocks
I. Image Models
A. Stroke Model: An image is comprised of a series of strokes.
1. The stroke model is the one we naturally think of when we are
drawing images on a piece of paper.
2. A stroke has the following properties:
a. shape (e.g., straight line or curved arc)
b. thickness
c. color
d. shape-specific properties (e.g., for a circle this would be
a radius, for an arc this would be a radius plus starting and
ending angles)
B. Pixel Model: An image is comprised of a series of pixels
1. Key aspects of pixel resolution
a. Number of rows
b. Number of columns
c. Number of bits per pixel
2. Pixel models
a. Bitmap: 1 bit per pixel which is either on or off. Good for
black and white images
b. Gray-Scale: Several bits per image indicating a shade of
grayness between white and black. NeXT was an early
example of a gray-scale system.
c. Full Color: Bits are used to specify color values for
red, green, and blue. The combination of these color
values produces the desired color. Most existing systems
allow 8 bits per color for a total of 3 bytes per color.
d. Color Map: Color values are an index into a table of fully
specified (i.e. 24 bit) colors. Typically 8 bits are
allocated for the color value giving a possibility of
256 colors.
i. An application can use its own color map or use the
system color map. If it uses its own color map, it
will often look very funny when the mouse leaves the
application's window because the system's color map
will replace the application's color map, resulting
in a new, and usually incongruous appearance for the
application.
ii. Advantages of color maps over full color
1) less space
2) less computation to manipulate pixels
3) adequate for normal interaction--full color
is preferred if photographic quality is desired
3. Anti-aliasing: Pixels drawn in gray-scale or less color intensity
at the edges of an image to reduce the jagged quality that can be
produced by discrete pixels.
4. Workstations use the pixel model and printers convert images
to the pixel model for printing
C. Region Model: Stroke objects are used to outline the region to be filled
1. The region model differs from the stroke model in that the region
model permits filling the region whereas the stroke model does not
2. The region model requires much less memory than the pixel model
to represent a filled shape.
3. Printers often use the region model since it allows an image
to be compactly represented and hence more quickly transferred.
Postscript is a region-based model.
II. Coordinate Systems
A. Device Coordinates: coordinates of the actual display device
1. Typically pixels
2. Window coordinates are typically in pixels, since they are
treated as virtual devices
3. The origin for windows is the upper left corner of the window
B. Physical Coordinates: Coordinates expressed in physical units, such
as inches or centimeters
1. Advantage over device coordinates: The size of a pixel varies
from computer to computer so images expressed in pixels will
have different sizes on different computers. Expressing an
image in physical coordinates ensures that the image will have
the same size on all platforms
2. An image can be translated from physical to device coordinates
by knowing the pixels per physical unit for that device.
C. Model Coordinates: Coordinates are expressed in the measuring system
used by the application
1. Example: A landscape system might want to express coordinates
in feet or yards
2. Advantage over physical coordinates: The programmer can use
coordinates that match the model rather than having to keep
translating between model and physical/device coordinates.
A scaling transformation can be used in the drawing methods
that transforms model coordinates to physical/device coordinates.
D. Interactive Coordinates: The translation of an input point from
device coordinates to model coordinates:
(InputPoint - WindowOrigin)
Model Point = --------------------------------
PhysicalToPixel * ModeltoPhysical
1. Example: PhysicalToPixel = 100 pixels per inch
ModeltoPhysical = 1 inch per 10 feet
InputPoint.x = 823
WindowOrigin.x = 623
1 in 10 ft
ModelPoint.x = (823 - 623) pix * ------- * -----
100 pix 1 in
= 20 ft
III. Human Visual Properties
A. Update Rates
1. 20-30 frames per second is required to gain the illusion of
smooth, continuous motion. Since human perception varies, 30
frames per second is the figure to shoot for.
2. 5 frames per second is the lower acceptable bound for dragging
interactions
3. Non-continuous interactions
a. 1-2 second delays are acceptable for retrieving information
or finding information in a document
b. After 10-15 seconds the illusion of interactivity is lost
c. Percent-done indicators or some other indicator that gives
an estimate of remaining time or at least an indication that
something is happening are useful when a delay is necessary
B. Color Sensitivity
1. Human retina can only distinguish about 64 levels of intensity
==> we can represent all of the colors that can be sensed by the
retina using 6 bits for each primary color
2. Human ocular system can sense somewhat more intensities. 256
levels of intensity for each primary color is more than adequate
for human visual needs which is why computer displays represent
colors with 24 bits.
3. Humans are 10 times more sensitive to variations in intensity than
they are to variations in hue or the color wavelength of light
==> contrasting intensities of colors should be carefully
considered when superimposing colors on a display (e.g.,
dark on light or vice versa works well).
4. Humans are much less capable of distinguishing intensity at
the fringes of the spectrum (deep red and violet) than in the
center where the yellows and greens are found.
==> The reds and blues are harder for the eye to resolve than
greens and yellows
5. About 1% of the population is color blind to one or more of
the primary colors--this does not affect the ability to
sense intensity.
==> discrimination of colors should *not* be based soley on
hue--also use intensity to discriminate colors
C. Color Models
1. RGB Model: Colors formed from the additive primaries of red, green
and blue
a. This model is used most often by drawing packages
2. HSV Model: Colors formed from hue, saturation, and value
a. Hue = primary wavelength of light
b. Saturation = pureness of the primary wavelength-high saturation
means a relatively pure color whereas low saturation means a
lot of white or gray is mixed in
c. Value = intensity or brightness
d. This model is what most "ordinary" people are used to
3. CMY Model: Colors defined by the subtractive primaries of cyan,
magenta, and yellow
a. Each color corresponds to the absence of one of the additive
primaries
Example: Cyan is the absence of red, magenta is the absence
of green, and yellow is the absence of blue
b. This model is most familiar to people in the visual arts
IV. Hardware
A. The frame buffer holds the current display image
B. The graphics package in the CPU modifies pixels in the frame buffer
C. The display controller continuously reads pixels from the
frame buffer and refreshes the display
D. Good displays can refresh themselves 60 times per second
V. Drawing
A. Canvas: An abstract drawing surface on which graphics are
drawn
1. Typically represented as a class
2. Properties include current font, color, coordinate system,
line width, etc.
a. Must be accessed via methods so that sets can be monitored
and perform any required actions
3. Methods include accessor methods for properties, drawing
methods for graphics, text drawing methods, and clipping methods
4. A canvas is an object that is typically attached to a window and
is the widget controller for that window
5. Windows are usually organized as a hierarchy of trees with each
window containing a single event controller (see Olsen Figure 2.3
for an example)
a. If a window's only objects are subwindows, then the controller
directs events to the appropriate subwindow
b. If the window contains a widget, such as a menu or text box, then
the controller handles all events associated with that widget and
the user does not have to write any event handlers
c. If the window is a canvas on which custom graphics are drawn,
then the user needs to write custom event handlers to deal with
mouse and keyboard events
B. Drawing
1. Design of drawing packages
a. Most drawing packages provide a context independent graphics
object through which an application can interface with a
drawing surface. The drawing surface could be a bitmap, the
display, a pdf document, or some other file format
i. The object typically provides methods for drawing objects,
setting graphical attributes, and querying clipping regions
b. Most drawing commands are limited to geometric information
i. location
ii. size
c. Drawing commands may either refer to specific objects, such
as drawLine, or take the object to be drawn as a parameter,
such as draw(lineObject).
i. Advantage of shapes as objects: The view can store geometric
or visual information with the objects rather than spreading
the information across different data structures
1) Typically the graphics object draw method would call the
object's draw method--each object must know how to draw
itself
d. Visual properties, such as line width, color, texture, or
font are frequently stored as properties of the graphics
object (Java) or in a bundle in a separate object, such as
a pen (line stroking) or brush (region filling) (C#).
i. Graphics object: When a shape is drawn, the drawing package renders
the shape by combining the geometric information contained in
the drawing command with the current visual properties
of the graphics object
ii. Bundling: The drawing command takes both a bundle and
a shape and draws the shaping by combining the geometric
information in the shape with the geometric information in
the bundle
2. Types of shapes
a. Path objects: 1D objects drawn in a 2D space. They *cannot*
be filled.
i. line
ii. circle
iii. arcs: a portion of a circle
iv. ellipses and elliptical arcs
v. splines: curved paths
vi. polylines: piecewise path objects
b. Filled (closed) Shapes: A shape that can be filled.
i. Distinction between paths and filled shapes: Filled shapes
can be filled whereas path objects cannot. A filled shape is
defined by a path object, typically a closed path object.
All points on the interior of the path object can be filled
C. Text
1. Font Selection
a. Major families
i. fixed-space (typewriter) fonts: every font has the same
width
e.g., courier
ii. serif font: each vertical stroke that reaches the
baseline has a little foot, or serif, on the bottom
e.g., times roman
ii. sans serif: vertical strokes do not have curves
e.g., helvetica
b. Font size: Usually represented as point size, with each point
being 1/72 of an inch. Sometimes font size may be specified
as pixel size, but then the size of your font becomes dependent
on the resolution of your monitor. It is better to use
point size when possible, so that your text shows up as the
same size on all monitors. Note that point size and pixel
size are not the same, and that a point size of say
20 will typically translate to a different number of pixels,
which depends on the screen's resolution.
c. Font face: bold, underline, italics
d. Considerations in font selection
i. proportionally spaced fonts (serifs and san serifs) are
generally more pleasing to read
ii. proportionally spaced fonts are more space efficient
iii. serif fonts are easy to read on paper because when
characters are strung together, the serifs tend to form
a line that facilitates tracking across the line
iv. on workstations, sans serifs may be preferable because
at low resolutions or point sizes, the serifs may
make serif characters harder to discriminate
2. Font Information: Necessary for positioning text
a. height: indicates how much vertical space to allocate to
a line of characters. Height is fixed for most, if not all
fonts.
b. leading: space between lines of text: measured from the
descent of one line to the ascent of the next line
i. may or may not be included in the height of a font--varies
from system to system. It is included in Java's definition
of a font's height.
c. baseline: where text will be positioned in the y coordinate. In
Java, when you draw a string, the y coordinate specifies the
baseline.
d. ascent: distance between the baseline and the highest extent
of any character in the font
e. descent: distance between the baseline and the lowest extent
of any character in the font
f. width: not fixed for proportional fonts
i. graphical drawing packages provide calls that given
a string and a font return the width of the string.
3. Drawing text
a. Drawing command generally expects location and a text string
b. To draw mixed font faces, sizes, or families on a line, the
string must be broken into pieces
c. To draw multiple lines, use the height of the font to
compute the second and succeeding lines.
first line = y
second line = y + height
third line = y + (2 * height)
...
4. Outline vs. Bitmapped Fonts
a. Bitmapped fonts are more efficient but
i. only provide discrete sizes
ii. consume a good deal of space
b. Outline fonts are closed shapes described by mathematical
equations.
i. they can be arbitrary size although this will require
computation that is more expensive than simply blitting
a bitmap to the screen
ii. they consume less space
iii. they are more flexible--for example they can be rotated
5. Text Selection: The problem can be stated as follows:
Given several lines of text and an (x,y) position for the
mouse, find the selected character and put the cursor after
that character.
a. In Java, you can use a TextLayout object to do custom text
editing
i. A TextLayout object is an immutable object but you can
use it to place a caret in an object and allow the
user to move the caret back and forth
ii. Keep a back-end StringBuilder object for handling edits
to the text. Each time the user edits the text, create
a new TextLayout object and provided it with the
newly updated string.
b. Algorithm for finding the correct line (you should know this
algorithm in case you are using a toolkit that does not
provide a text layout object like Java).
let left = the pixel position of the left side of the text
let top = the pixel position of the top of the text
line_number = (y - top) / (height + leading)
i. You may not want the mouse to select a line if it
is in the leading between lines. You can check
if the mouse is in the leading with the
following conditional:
if (y-top) > ((height + leading) * (line_number+1) - leading)
line_number = -1; // deselect line
b. Algorithm for finding the correct character in the line: This
algorithm places the cursor to the right of a character if
the x position falls in the right half of the character and to
the left of a character if the x position falls in the left
half of the character. The return value is the index of where
the cursor should appear with 0 denoting before the first
character. For example:
a b c
cursor index = 0 1 2 3
let line = the line of characters that has been selected
let CharWidth be a function that returns the width of a
character in the current font
let StringWidth(line[0:end] be a function that returns the width of
a string in the current font that starts at index
position 0 and ends at index position (end-1).
let x be the x position of the mouse
let left be the leftmost boundary of the text and
right be the rightmost boundary of the text
// make sure x falls within a reasonable distance of
// the text where threshold is the amount by which
// x can be outside the textbox
if (x < (left-threshold)) or (x > (right+threshold))
cursorIndex = -1
// translate x so that it is relative to the start
// of the textbox
x = x - left;
for (cursor_index = 0; cursor_index < line.length;
cursor_index++)
if (x < StringWidth(line[0:cursor_index+1])
- CharWidth(line[cursor_index])/2)
break;
The above algorithm can be modified to also return the pixel position
at which the cursor should be placed.
6. Fonts in Java
a. Font class: Each instance contains a family, size, and style
i. important methods
1) Font(String name, int style, int size): Creates a new
font from the specified name, style, and point size.
a. Predefined fonts are "Monospaced", "Serif", "SansSerif",
"Dialog", and "DialogInput".
b. style can be Font.{PLAIN, BOLD, ITALIC, or BOLD+ITALIC}
2) deriveFont(float size) or (int style) or
(int style, float size): derives a new
font from the existing font with the specified
style and size information
b. If you want to find the list of fonts that are supported on
your platform, you can use the following code:
String fonts[] = GraphicsEnvironment.getLocalGraphicsEnvironment().getAvailableFontFamilyNames();
for (int i = 0; i < fonts.length; i++) {
System.out.println(fonts[i]);
}
c. Obtain the attributes of a string, such as length, width, etc.
using the FontMetrics class
1) You need a Graphics object to obtain an instance of a
FontMetrics class. The Graphics class has two methods
that allow you to obtain a FontMetrics object
a. getFontMetrics(): returns a FontMetrics object for
the currently set font
b. getFontMetrics(Font f): returns a FontMetrics object
for the specified font
2) All measurements will be in integers
3) Useful FontMetrics methods
a. getAscent(): returns ascent of the font and is useful for
vertically centering numbers since numbers do not have
a descent
b. getHeight(): Returns the standard height of a line of
text in this font. This is the distance between
the baseline of adjacent lines of text. It is
the sum of the leading + ascent + descent of the
font.
get the individual heights of characters. getHeight()
c. stringWidth(String str): returns the width of the string
in the given font
4) Regardless of which approach you use, you will be
unable to obtain the actual height of a character
string since getAscent and getDescent will always
return the average ascent and descent for the font.
5) If you want to vertically center a single line of text,
use ascent + descent, unless you know that
all characters in the string have no descent (e.g.,
a string of numbers), in which case just use
ascent
VI Redrawing
A. Reasons For
1. A portion of the application changes
2. A window is re-sized
3. A window de-iconified (i.e., made visible)
4. A portion of a window that was previously obscured becomes visible
because of some operation, such as bringing it to the front, or
moving an obscuring window
B. Redraw command: Most widgets implement a redraw(Graphics G) command
that is called when a window must be redrawn
C. Clipping: Limits drawing to a particular region of the screen
1. Clipping regions are usually either:
a. rectangular, or
b. rectilinear: a collection of rectangles
2. A region is typically represented as an abstract class that
implements the following operations:
a. Union(Region)
b. Intersection(Region)
c. Difference(Region)
d. IsEmpty: returns true if the region is empty and false
otherwise
e. Bounds: returns the smallest rectangle that completely
encloses the region
f. IsInside(Point): returns true if the point is inside of
the regions and false otherwise
g. MakeRegion(basic primitive shape): Constructs a region
from a basic primitive shape
3. Canvases typically have a set of operations that support regions.
Example operations might include:
a. BoundingRegion:returns the regions that define the outside of the
canvas
b. VisibleRegion: returns the region of the canvas actually
visible through all other canvases that might be overlapping
c. SetClipRegion(Region): Sets the clipping region to be the
intersection of the argument and the visible region.
d. GetClipRegion: Returns the clipping region
4. Simple redraw algorithms
a. Redraw the entire window and ignore the clip region: Appropriate
for applications that contain relatively few objects.
i. Use this algorithm first and only use more complicated
algorithms if flickering is noticeable
b. Using the clip region: Before drawing an object, check to see
whether or not it intersects the clip region:
redraw(Graphics G) {
for each obj in canvas
if (clip.intersects(obj.getBounds()))
obj.draw()
}
i. Why this algorithm is more efficient than simple redraw: The
draw method will always only render that portion of an
object which falls within the clip region. However, the
draw method uses an expensive test to determine what portions
of an object intersect the clip region. It pixelizes each
horizontal line of the object and then determines what portions of
that line intersect the clip region. The intersection test
above is a much more efficient way of determining whether
any portion of an object intersects the clipping region
and saves the pixelization effort if the object does not
intersect the clipping region
C. Transparency: The fraction of the light that should show through from
behind the object being drawn
1. 0 = none: the object is opaque
2. 1 = 100%: the object is completely transparent
3. Formula for computing a transparent color for a particular pixel
B = color of that pixel in the background
O = color of the pixel for the object being drawn
T = transparency value of the object being drawn
N = the pixel color
N = B*T + O*(1.0-T)
4. The RGBA color model adds an alpha channel (A) for transparency:
An alpha value of 0 is completely transparent and a value of 1 is
completely opaque