GUI Building Blocks

I. Image Models

    A. Stroke Model: An image is comprised of a series of strokes. 

        1. The stroke model is the one we naturally think of when we are 
	   drawing images on a piece of paper. 

	2. A stroke has the following properties:
	    a. shape (e.g., straight line or curved arc)

	    b. thickness

	    c. color

	    d. shape-specific properties (e.g., for a circle this would be
	        a radius, for an arc this would be a radius plus starting and
		ending angles)

    B. Pixel Model: An image is comprised of a series of pixels

        1. Key aspects of pixel resolution

	    a. Number of rows

	    b. Number of columns

	    c. Number of bits per pixel

	2. Pixel models

	    a. Bitmap: 1 bit per pixel which is either on or off. Good for
	         black and white images

	    b. Gray-Scale: Several bits per image indicating a shade of
	         grayness between white and black. NeXT was an early
		 example of a gray-scale system.

	    c. Full Color: Bits are used to specify color values for
	         red, green, and blue. The combination of these color
		 values produces the desired color. Most existing systems
		 allow 8 bits per color for a total of 3 bytes per color.

	    d. Color Map: Color values are an index into a table of fully
	         specified (i.e. 24 bit) colors. Typically 8 bits are
		 allocated for the color value giving a possibility of
		 256 colors.

		 i. Advantages of color maps over full color

		     1) less space
		     2) less computation to manipulate pixels
		     3) adequate for normal interaction--full color
		         is preferred if photographic quality is desired

		ii. GIFs use a 256 color map. 

		iii. JPEGS use an arbitrary color map, but will not represent
			every color in an image precisely, especially ones
			near the boundaries

	3. Anti-aliasing: Pixels drawn in gray-scale or less color intensity
            at the edges of an image to reduce the jagged quality that can be 
            produced by discrete pixels.

	4. Workstations use the pixel model and printers convert images
	    to the pixel model for printing

    C. Region Model: Stroke objects are used to outline the region to be filled

        1. The region model differs from the stroke model in that the region
	   model permits filling the region whereas the stroke model does not

	2. The region model requires much less memory than the pixel model
	   to represent a filled shape.

	3. Printers often use the region model since it allows an image
	   to be compactly represented and hence more quickly transferred.
	   Postscript is a region-based model.

II. Coordinate Systems

    A. Device Coordinates: coordinates of the actual display device

        1. Typically pixels

	2. Window coordinates are typically in pixels, since they are
	     treated as virtual devices

        3. The origin for windows is the upper left corner of the window
		(Mac Quartz system starts from lower left)

    B. Physical Coordinates: Coordinates expressed in physical units, such
        as inches or centimeters

	1. Advantage over device coordinates: The size of a pixel varies
	   from computer to computer so images expressed in pixels will
	   have different sizes on different computers. Expressing an
	   image in physical coordinates ensures that the image will have
	   the same size on all platforms

	2. An image can be translated from physical to device coordinates
	   by knowing the pixels per physical unit for that device.

    C. Model Coordinates: Coordinates are expressed in the measuring system
        used by the application

	1. Example: A landscape system might want to express coordinates
	    in feet or yards

	2. Advantage over physical coordinates: The programmer can use
	    coordinates that match the model rather than having to keep
	    translating between model and physical/device coordinates.
	    A scaling transformation can be used in the drawing methods
	    that transforms model coordinates to physical/device coordinates.

    D. Interactive Coordinates: The translation of an input point from
        device coordinates to model coordinates:

	                 (InputPoint - WindowOrigin)
	Model Point = --------------------------------
	              PhysicalToPixel * ModeltoPhysical

	1. Example: PhysicalToPixel = 100 pixels per inch
	            ModeltoPhysical = 1 inch per 10 feet

		    InputPoint.x = 823
		    WindowOrigin.x = 623

		                                      1 in      10 ft
		    ModelPoint.x = (823 - 623) pix * ------- *  -----
		                                     100 pix    1 in

				 = 20 ft

III. Human Visual Properties

    A. Update Rates

        1. 20-30 frames per second is required to gain the illusion of
	   smooth, continuous motion. Since human perception varies, 30
	   frames per second is the figure to shoot for.

	2. 5 frames per second is the lower acceptable bound for dragging

	3. Non-continuous interactions

	   a. 1-2 second delays are acceptable for retrieving information
	      or finding information in a document

	   b. After 10-15 seconds the illusion of interactivity is lost

	   c. Percent-done indicators or some other indicator that gives
	      an estimate of remaining time or at least an indication that
	      something is happening are useful when a delay is necessary

    B. Color Sensitivity

        1. Human retina can only distinguish about 64 levels of intensity

	    ==> we can represent all of the colors that can be sensed by the 
	        retina using 6 bits for each primary color

        2. Human ocular system can sense somewhat more intensities. 256
	    levels of intensity for each primary color is more than adequate
	    for human visual needs which is why computer displays represent
	    colors with 24 bits.

	3. Humans are 10 times more sensitive to variations in intensity than 
	    they are to variations in hue or the color wavelength of light

	    ==> contrasting intensities of colors should be carefully 
	        considered when superimposing colors on a display (e.g.,
		dark on light or vice versa works well).

	4. Humans are much less capable of distinguishing intensity at
	    the fringes of the spectrum (deep red and violet) than in the
	    center where the yellows and greens are found.

	    ==> The reds and blues are harder for the eye to resolve than
	        greens and yellows

	5. About 1% of the population is color blind to one or more of
	    the primary colors--this does not affect the ability to
	    sense intensity. 

	    ==> discrimination of colors should *not* be based solely on 
	        hue--also use intensity to discriminate colors

   C. Color Models

       1. RGB Model: Colors formed from the additive primaries of red, green
          and blue

          a. This model is used most often by drawing packages

       2. HSV Model: Colors formed from hue, saturation, and value 

          a. Hue = primary wavelength of light
          b. Saturation = pureness of the primary wavelength-high saturation
             means a relatively pure color whereas low saturation means a
	     lot of white or gray is mixed in
          c. Value = intensity or brightness
          d. This model is what most "ordinary" people are used to

        3. CMY Model: Colors defined by the subtractive primaries of cyan, 
           magenta, and yellow

	   a. Each color corresponds to the absence of one of the additive 

	      Example: Cyan is the absence of red, magenta is the absence
	        of green, and yellow is the absence of blue

	   b. This model is most familiar to people in the visual arts

IV. Hardware

    A. The frame buffer holds the current display image

    B. The graphics package in the CPU modifies pixels in the frame buffer

    C. The display controller continuously reads pixels from the
         frame buffer and refreshes the display

    D. Refresh rates
	1. High end LCD TV displays can refresh themselves up to 600 times per
	2. Many existing computer displays can refresh themselves up to 120
	   times per second
        3. All displays should be able to refresh themselves 60 times per second

V. Drawing

    A. Canvas: An abstract drawing surface on which graphics are 

	1. Typically represented as a class 

	2. Properties include current font, color, coordinate system,
	    line width, etc.

	    a. Must be accessed via methods so that sets can be monitored
	       and perform any required actions

	3. Methods include accessor methods for properties, drawing
	    methods for graphics, text drawing methods, and clipping methods

        4. A canvas is an object that is typically attached to a window and
            is the widget controller for that window

        5. Windows are usually organized as a hierarchy of trees with each
            window containing a single event controller (see Olsen Figure 2.3 
            for an example)
            a. If a window's only objects are subwindows, then the controller
               directs events to the appropriate subwindow

            b. If the window contains a widget, such as a menu or text box, then
               the controller handles all events associated with that widget and
               the user does not have to write any event handlers

            c. If the window is a canvas on which custom graphics are drawn,
               then the user needs to write custom event handlers to deal with
               mouse and keyboard events

    B. Drawing

        1. Design of drawing packages

            a. Most drawing packages provide a context independent graphics
               object through which an application can interface with a
               drawing surface. The drawing surface could be a bitmap, the
               display, a pdf document, or some other file format

               i. The object typically provides methods for drawing objects,
                  setting graphical attributes, and querying clipping regions

	    b. Most drawing commands are limited to geometric information 

	       i. location
	       ii. size

            c. Drawing commands may either refer to specific objects, such
               as drawLine, or take the object to be drawn as a parameter,
               such as draw(lineObject).

               i. Advantage of shapes as objects: The view can store geometric
                  or visual information with the objects rather than spreading
                  the information across different data structures

                   1) Typically the graphics object draw method would call the
                      object's draw method--each object must know how to draw 

	    d. Visual properties, such as line width, color, texture, or
	       font are frequently stored as properties of the graphics 
               object (Java) or in a bundle in a separate object, such as
               a pen (line stroking) or brush (region filling) (C#).

	       i. Graphics object: When a shape is drawn, the drawing package renders 
                  the shape by combining the geometric information contained in
	          the drawing command with the current visual properties
		  of the graphics object

              ii. Bundling: The drawing command takes both a bundle and
		  a shape and draws the shaping by combining the geometric
		  information in the shape with the geometric information in
		  the bundle
        2. Types of shapes

	    a. Path objects: 1D objects drawn in a 2D space. They *cannot*
	       be filled.

	       i. line
	       ii. circle
	       iii. arcs: a portion of a circle
	       iv. ellipses and elliptical arcs
	       v. splines: curved paths
	       vi. polylines: piecewise path objects

	    b. Filled (closed) Shapes: A shape that can be filled.

	       i. Distinction between paths and filled shapes: Filled shapes 
	          can be filled whereas path objects cannot. A filled shape is 
		  defined by a path object, typically a closed path object.
		  All points on the interior of the path object can be filled

    C. Text

        1. Font Selection

	    a. Major families
	        i. fixed-space (typewriter) fonts: every font has the same

		   e.g., courier

		ii. serif font: each vertical stroke that reaches the
		     baseline has a little foot, or serif, on the bottom

		     e.g., times roman

		ii. sans serif: vertical strokes do not have curves

		     e.g., helvetica

	    b. Font size: Usually represented as point size, with each point
	 	being 1/72 of an inch. Sometimes font size may be specified
		as pixel size, but then the size of your font becomes dependent
		on the resolution of your monitor. It is better to use
		point size when possible, so that your text shows up as the
		same size on all monitors. Note that point size and pixel
		size are not the same, and that a point size of say
		20 will typically translate to a different number of pixels,
		which depends on the screen's resolution.

	    c. Font face: bold, underline, italics

	    d. Considerations in font selection

	        i. proportionally spaced fonts (serifs and san serifs) are 
		   generally more pleasing to read

		ii. proportionally spaced fonts are more space efficient

		iii. serif fonts are easy to read on paper because when
		     characters are strung together, the serifs tend to form
		     a line that facilitates tracking across the line

		 iv. on workstations, sans serifs may be preferable because
		     at low resolutions or point sizes, the serifs may
		     make serif characters harder to discriminate

	2. Font Information: Necessary for positioning text

	    a. height: indicates how much vertical space to allocate to
	        a line of characters. Height is fixed for most, if not all

	    b. leading: space between lines of text: measured from the
	        descent of one line to the ascent of the next line

		i. may or may not be included in the height of a font--varies 
		   from system to system. It is included in Java's definition
		   of a font's height.

	    c. baseline: where text will be positioned in the y coordinate. In
                Java, when you draw a string, the y coordinate specifies the

	    d. ascent: distance between the baseline and the highest extent
	        of any character in the font

	    e. descent: distance between the baseline and the lowest extent
	        of any character in the font

	    f. width: not fixed for proportional fonts

	        i. graphical drawing packages provide calls that given
		   a string and a font return the width of the string.

	3. Drawing text

	    a. Drawing command generally expects location and a text string

	    b. To draw mixed font faces, sizes, or families on a line, the
	        string must be broken into pieces

	    c. To draw multiple lines, use the height of the font to
	        compute the second and succeeding lines.

		first line = y
		second line = y + height
		third line = y + (2 * height)

	4. Outline vs. Bitmapped Fonts
	    a. Bitmapped fonts are more efficient but
	       i. only provide discrete sizes
	       ii. consume a good deal of space

	    b. Outline fonts are closed shapes described by mathematical

	       i. they can be arbitrary size although this will require
	          computation that is more expensive than simply blitting
		  a bitmap to the screen

	      ii. they consume less space

	      iii. they are more flexible--for example they can be rotated

	 5. Text Selection: The problem can be stated as follows:

	     Given several lines of text and an (x,y) position for the
	     	mouse, find the selected character and put the cursor after
		that character.
	     a. In Java, you can use a TextLayout object to do custom text
		i. A TextLayout object is an immutable object but you can
			use it to place a caret in an object and allow the
			user to move the caret back and forth
		ii. Keep a back-end StringBuilder object for handling edits
			to the text. Each time the user edits the text, create
			a new TextLayout object and provided it with the
			newly updated string.

	     b. Algorithm for finding the correct line (you should know this
		algorithm in case you are using a toolkit that does not 
		provide a text layout object like Java).

	         let left = the pixel position of the left side of the text
		 let top = the pixel position of the top of the text

		 line_number = (y - top) / (height + leading)
		 i. You may not want the mouse to select a line if it
			is in the leading between lines. You can check
			if the mouse is in the leading with the 
			following conditional:

			if (y-top) > ((height + leading) * (line_number+1) - leading)
			    line_number = -1; // deselect line

	    b. Algorithm for finding the correct character in the line: This
	    	algorithm places the cursor to the right of a character if
		the x position falls in the right half of the character and to
		the left of a character if the x position falls in the left 
		half of the character. The return value is the index of where
		the cursor should appear with 0 denoting before the first
		character. For example:

		                 a b c
		 cursor index = 0 1 2 3

	         let line = the line of characters that has been selected
		 let CharWidth be a function that returns the width of a
		     character in the current font
		 let StringWidth(line, 0, numChars) be a function that returns the width of 
		     line's string in the current font that starts at index
		     position 0 and extends for numChars characters
		 let x be the x position of the mouse
		 let left be the leftmost boundary of the text and 
                     right be the rightmost boundary of the text

                 // make sure x falls within a reasonable distance of
                 // the text where threshold is the amount by which
                 // x can be outside the textbox
                 if (x < (left-threshold)) or (x > (right+threshold))
                    cursorIndex = -1
                 // translate x so that it is relative to the start
                 // of the textbox
                 x = x - left;
		 for (cursor_index = 0; cursor_index < line.length;
	 	     if (x < StringWidth(line, 0, cursor_index+1) 
                              - CharWidth(line[cursor_index])/2)
	The above algorithm can be modified to also return the pixel position
	at which the cursor should be placed.

        6. Fonts in Java
	    a. Font class: Each instance contains a family, size, and style

	        i. important methods

		    1) Font(String name, int style, int size): Creates a new
		    	font from the specified name, style, and point size.
			a. Predefined fonts are "Monospaced", "Serif", "SansSerif",
				"Dialog", and "DialogInput".
			b. style can be Font.{PLAIN, BOLD, ITALIC, or BOLD+ITALIC}

		    2) deriveFont(float size) or (int style) or 
		    	  	 (int style, float size): derives a new
				 font from the existing font with the specified
				 style and size information

	    b. If you want to find the list of fonts that are supported on 
	       your platform, you can use the following code:
	    String fonts[] = java.awt.GraphicsEnvironment.getLocalGraphicsEnvironment().getAvailableFontFamilyNames();
            for (int i = 0; i < fonts.length; i++) {

	    c. Obtain the attributes of a string, such as length, width, etc.
		using the FontMetrics class 

		    1) You need a Graphics object to obtain an instance of a
		       FontMetrics class. The Graphics class has two methods
		       that allow you to obtain a FontMetrics object

		       a. getFontMetrics(): returns a FontMetrics object for
		       	   the currently set font

		       b. getFontMetrics(Font f): returns a FontMetrics object
		       	   for the specified font

		    2) All measurements will be in integers

		    3) Useful FontMetrics methods
			a. getAscent(): returns ascent of the font and is useful for
				vertically centering numbers since numbers do not have
				a descent
			b. getHeight(): Returns the standard height of a line of 
				text in this font. This is the distance between 
				the baseline of adjacent lines of text. It is 
				the sum of the leading + ascent + descent of the 
			c. stringWidth(String str): returns the width of the string 
				in the given font	

		     4) Regardless of which approach you use, you will be
			unable to obtain the actual height of a character
			string since getAscent and getDescent will always
			return the maximum ascent and descent for the font.

		     5) If you want to vertically center a single line of text,
			use ascent + descent, unless you know that
			all characters in the string have no descent (e.g.,
			a string of numbers), in which case just use
VI Redrawing

    A. Reasons For
        1. A portion of the application changes
	2. A window is re-sized
	3. A window de-iconified (i.e., made visible)
	4. A portion of a window that was previously obscured becomes visible 
	    because of some operation, such as bringing it to the front, or
	    moving an obscuring window
    B. Redraw command: Most widgets implement a redraw(Graphics G) command
	that is called when a window must be redrawn

    C. Clipping: Limits drawing to a particular region of the screen

        1. Clipping regions are usually either:

	    a. rectangular, or

	    b. rectilinear (in which case it is represented as a region which
		is defined in the next bullet): a collection of rectangles

	2. A region is typically represented as an abstract class that
	    implements the following operations:

	    a. Union(Region)
	    b. Intersection(Region)
	    c. Difference(Region)
	    d. IsEmpty: returns true if the region is empty and false
	    e. Bounds: returns the smallest rectangle that completely
	        encloses the region
	    f. IsInside(Point): returns true if the point is inside of
	        the regions and false otherwise
	    g. MakeRegion(basic primitive shape): Constructs a region
	        from a basic primitive shape

	3. Canvases typically have a set of operations that support regions.
	   Example operations might include:

	   a. BoundingRegion:returns the regions that define the outside of the

	   b. VisibleRegion: returns the region of the canvas actually
	      visible through all other canvases that might be overlapping

	   c. SetClipRegion(Region): Sets the clipping region to be the
	      intersection of the argument and the visible region.

	   d. GetClipRegion: Returns the clipping region

        4. Simple redraw algorithms

	   a. Redraw the entire window and ignore the clip region: Appropriate
		for applications that contain relatively few objects. 

		i. Use this algorithm first and only use more complicated
		    algorithms if flickering is noticeable

	   b. Using the clip region: Before drawing an object, check to see
		whether or not it intersects the clip region:

		redraw(Graphics G) {
		  for each obj in canvas
		    if (clip.intersects(obj.getBounds()))

		i. Why this algorithm is more efficient than simple redraw: The
		     draw method will always only render that portion of an
		     object which falls within the clip region. However, the
		     draw method uses an expensive test to determine what portions
		     of an object intersect the clip region. It pixelizes each
		     horizontal line of the object and then determines what portions of
		     that line intersect the clip region. The intersection test
		     above is a much more efficient way of determining whether
		     any portion of an object intersects the clipping region
		     and saves the pixelization effort if the object does not
		     intersect the clipping region

    C. Transparency: The fraction of the light that should show through from
	behind the object being drawn

	1. 0 = none: the object is opaque
	2. 1 = 100%: the object is completely transparent
	3. Formula for computing a transparent color for a particular pixel
	    B = color of that pixel in the background
	    O = color of the pixel for the object being drawn
	    T = transparency value of the object being drawn
	    N = the pixel color 

	    N = B*T + O*(1.0-T)
        4. The RGBA color model adds an alpha channel (A) for transparency:
	    An alpha value of 0 is completely transparent and a value of 1 is
	    completely opaque