The display model we have used to this point in the course involves a total redraw of the screen. While that works for applications that have only a few dozen objects, it may not be fast enough to provide interactive performance for applications that display hundreds of objects, and it will not be fast enough to provide interactive performance to applications that display thousands of objects. For such applications, it is necessary to use incremental redisplay algorithms, which only redraw the damaged portion of the display. The damaged portion of the display is the portion of the display occupied by changed objects. Such objects may have changed location, changed size, or changed a visual property, such as their fill color or border style. The damaged portion of the display includes both the old and new locations of the objects. These notes will discuss a number of different incremental display algorithms, as well as how a number of special cases, such as window exposure events and feedback objects, are handled.


Double Buffering

A common technique used to avoid flicker in windows is to draw objects to an offscreen buffer, called a double buffer, and then blit the buffer to the screen. While this technique actually slows down the rendering process, because you first render to a buffer and then render the buffer to the screen, it is often perceived by the user as speeding up the rendering process because it eliminates flickering. When you directly render to the screen, you typically first blank a portion of the screen, and then draw to it. The user's brain can often distinguish between the blanking and redraw operations, thus causing the image to appear to flicker. By default double buffering is turned on in Java and with today's graphics chips for accelerated drawing, there is rarely a reason why you would turn it off. Java handles double buffering for you, so you simply call the draw method and Java redirects the rendering to a buffer. When your paintComponent returns, Java's virtual machine then blits the buffer to the display.


Feedback Objects and Interfaces with NonOverlapping Objects

  1. The simplest type of interface to redraw is one in which no two objects may overlap on the display. In this case it is ok to update an object by erasing its old image and displaying its new one.

  2. XOR can provide a way to quickly draw and erase objects. The first time an object is drawn in XOR mode it appears on the screen. The second time it is drawn in XOR mode it disappears. Hence you can erase an object by simply redrawing it.

  3. Feedback objects can be efficiently drawn and erased with XOR, even if an interface has overlapping objects. Feedback objects, like rubberband boxes, are typically drawning on top of other objects, so the display manager can make them appear by drawing them in XOR mode, and can make them disappear by drawing them a second time in XOR mode.

  4. Many windowing systems provide a way to specify the color with which we wish to draw an xor object against a normal background. The system then computes the color that needs to be xor'ed with the background color in order to achieve the desired color in which to draw the object. Formally, if C is the desired foreground color and B is the background color, than C = XOR(D,B) and D is the color the system computes to produce the foreground color when xor'ed with the background color. If the system does not provide a way to compute D, it can be easily computed as D = XOR(C,B). In Java, the Graphics class provides two methods for setting the drawing mode:

    1. setXORMode(Color c): objects will now be drawn in XOR mode and when drawn on the background, will appear as the color c.
    2. setPaintMode(): objects will be drawn opaquely so that they overwrite whatever is beneath them.

    By default the graphics context is set to paint mode.

  5. xor.java illustrates the use of xor in java. The below applet shows xor.java in action. You should click on the applet to get the two xor'ed red rectangles to appear and disappear. Note that the code specifies that an xor'ed rectangle should appear red when drawn over a white background. When it is drawn over the black and yellow backgrounds, the resulting color is unpredictable.

  6. When using xor, the display manager needs to be notified when an object changes, and should keep a list of changed objects. It should also save the object's old location, so it can erase the object by drawing it a second time at its old location. When the display manager is called to update the display, it runs through the list of changed objects and draws them at both their old and new locations:
         update() {
            for obj ∈ changed_objs {
              draw obj at old location
              draw obj at new location
            }
         }
         
    The display manager will need to be notified whenever a visual feature of an object changes, such as its location, size, or color. Often the display manager is implemented as an observer and the objects as observees. For example, the display manager might implement a notifyChanged method as follows:
           notifyChanged(obj) {
             // save the changed object and its old location
             changed_objs = changed_objs ∪ ( obj, obj.getLocation() )
           }
         

  7. To implement the redrawing of feedback objects, the display manager might have a "feedback" mode in which it only renders feedback objects, and a non-feedback mode in which it renders the rest of the interface. It would be in "feedback" mode if it had a non-null list of feedback objects to draw. Whether in feedback mode or not, it would first go through a list of saved feedback objects and draw them, in order to erase them from the screen. If in feedback mode, it would then draw each object on the feedback list and save that object and its location so that it could be erased on the next display cycle. It would then exit to short circuit the rendering process. If not in feedback mode, the display manager would proceed to draw the objects in the interface. Here is an example algorithm:
         update(Graphics g) {
            // erase old feedback objs
            g.setXORMode(some color)
            for each obj ∈ saved_feedback_objs
                 draw obj at its old location
            saved_feedback_objs = NULL
            if (feedback_objs != NULL) { 
                 for each obj ∈ feedback_objs {
                     draw obj at its current location
                     saved_feedback_objs = saved_feedback_objs ∪ ( obj, obj.getLocation())
                 }
                 g.setPaintMode()
            }
            else {
               g.setPaintMode()
               // draw objects in interface
            }
         }
         
    Since this feedback drawing algorithm works with many types of display managers, it could be encapsulated in its own class and invoked by any display manager that wanted to handle feedback objects by xor'ing them.

Interfaces with Overlapping Objects

XOR does not work with overlapping objects because two objects that overlap one another will partially erase one another. You also cannot simply erase and redraw each object, for two reasons:

  1. If you erase an object's old image, it may erase parts of other objects as well

  2. If you redraw an object, it will appear on top of all other objects. If it is not supposed to be the top object, that would be a bug.

Hence we need a more sophisticated algorithm which redraws parts of other objects which are erased and draws the object in the proper stacking order. The stacking order is the order in which objects should be drawn from back to front. Objects that should appear under other objects should be drawn first, while objects that should appear on top of other objects should be drawn last.

The total redraw solution is to redraw all objects using the appropriate stacking order, but this may be too expensive when there are hundreds or thousands of objects. A more efficient solution is to only redraw damaged parts of screen. This is called incremental redisplay, and it requires that we maintain data structures to keep track of damaged objects.

Our strategy is as follows:

  1. Keep track of all objects that have a changed visual property, such as position, size, color, or stacking order.

    1. As objects are changed, merge their old position into an "old" bounding box. This bounding box keeps track of the original locations of the changed objects.
    2. A bounding box is the smallest rectangle that encloses an object or set of objects.

  2. When an update command is issued (i.e., a command to redraw the interface)

    1. Compute the new bounding box of the changed objects

    2. For each displayed object, determine whether the displayed object intersects either the new or old bounding box.

      1. Redraw the objects that intersect either the new or old bounding box in the appropriate stacking order

      2. Only redraw the portion of the object that changes--most windowing systems provide a clip region that facilitates redrawing only a portion of the object.

    3. A number of optimizations are possible:

      1. For composite objects, check the composite object's bounding box before checking its individual children. If the composite object doesn't intersect either of the bounding boxes, then none of its children can, so none of its children must be checked.

      2. Check to see how much overlap there is between the new and old bounding boxes. If the overlap is considerable, then merge them. This will prevent objects from having to be rendered and clipped twice. Often there will be considerable overlap betweeen the two bounding boxes, because often an object has either changed in place (e.g., a color change), has moved a short distance, or been resized.

    4. Algorithm

      1. Data structures and Variables: For each window you should maintain the following variables:

        1. objs_changed: The set of objects that have changed
        2. old_bbox: The bounding box for the changed objects' old positions
        3. new_bbox: The bounding box for the changed objects' new positions

      2. Update algorithm
        	     -all variables prefixed by self refer to instance variables
        	     -if a variable is not prefixed by self then it is a local
        	     	variable
        		
        	     Win::Update()
        
        		new_bbox = empty
        		old_bbox = self.old_bbox
        		for each obj ∈ self.objs_changed do
        		    new_bbox = new_bbox U obj's position
        	        // clear the damaged part of the display
        	        win.clearRect(old_bbox)
        	        win.clearRect(new_bbox)
        	        // redraw the damaged part of the display
        		for each obj in self.children do
        		    if intersect(obj, old_bbox) or intersect(obj, new_bbox)
        			obj.update(old_bbox, new_bbox)
        		self.old_bbox = empty
        		     
        	     Container::Update(old_bbox, new_bbox)
        		
        		for each obj in self.children do
        		    if intersect(obj, old_bbox) or intersect(obj, new_bbox)
        			obj.update(old_bbox, new_bbox)
        
        	     Primitive_Obj::Update(old_bbox, new_bbox)
        		self.draw()
        
      3. Notification algorithm: The notification algorithm for the display manager must now record both the changed object and merge its old bounding box into the bounding box of old locations being maintained by the display manager:
        	    notifyChange(obj)
        		self.objs_changed_objs = self.objs_changed U obj
        		self.old_bbox = self.old_bbox U obj's position
        

  3. Java Notes

    1. Java's repaint method merges bounding boxes so you cannot use an old and a new bounding box--you must change the above update procedure so that it uses a single bounding box instead
    2. If you really want to use an old and new bounding box, there are two alternative things you can do:

      1. call paintImmediately. paintImmediately will immediately queue a paint event rather than aggregating a number of repaint calls and then generating a single paint event. However, you must be careful about calling paintImmediately because if there are several changes to the display, you could end up calling it several times. One scheme that could work with a display manager would be to add a second notification procedure called UpdatesCompleted. UpdatesCompleted would call paintImmediately twice, once with the old bounding box and once with the new bounding box.

      2. In the event handler, call repaint with no arguments so that the clip region gets set to the entire component. Then in paintComponent, call the graphics object's setClip method to first set the clipping region to the old clip region and clip/draw all objects against this clip region. Then call the setClip method to set the clipping region to the new clip region and clip/draw all objects against this clip region.

    3. You could simply set a clip region and draw objects without first checking that they intersect the clip region. However, this would be much more expensive because Java will first scan convert each object, then draw those pixels that intersect the clip region. It will not first check if the object intersects the clip region, because it assumes that you've already done that if you wanted to.

  4. Separation of display manager from windows: The previous algorithm hard-codes the display algorithm into the window. A better approach is to allow display managers to be attached to windows so that different display algorithms can be used depending on the application.

    1. Interface: Use the observer pattern
      		interface DisplayManager {
      		  public:
      		    void update(GraphicsContext);
      		    void addObject(Shape);
      		    void removeObject(Shape)
      		    void notifyChange(Shape)
                          // the following method might not be needed depending on
                          // the window manager
                          void updatesCompleted(); 
                 	 }
           
    2. Variables: Each window should maintain a pointer to a display manager object

    3. Methods:
      	    Win::paintComponent(g) 
      		displayManager.update(g);
      
      	    Container::draw(g) {
      		for each obj &isin self.shapes do
      		    if intersect(obj, g.getClip())
      			obj.draw(g)
      	    }
      
                  DisplayManager::Update(g)
      		new_bbox = empty
      		old_bbox = self.old_bbox
      		for each obj in self.objs_changed do
      		    new_bbox = new_bbox U obj's position
      	        // clear the damaged part of the display
      	        win.clearRect(old_bbox)
      	        win.clearRect(new_bbox)
      	        // redraw the damaged part of the display
      	        g.setClip(old_bbox)
      		for each obj in registeredObjects do
      		    if intersect(obj, old_bbox)
      			obj.draw(g)
      	        g.setClip(new_bbox)
      		for each obj in registeredObjects do
      		    if intersect(obj, new_bbox)
      			obj.draw(g)
      		self.old_bbox = empty
      

Quadtrees

When there are thousands of objects on a display, it may be too expensive to test every object against the clip region. A solution is to subdivide the display and only examine objects that fall within the subdivisions that intersect the damaged region. Quadtrees provide a way of performing this subdivision. They divide a region into quadrants. Each quadrant contains a list of all the objects that intersect its bounding box. Each node of a quadtree has four children, corresponding to a northeast, northwest, southeast, and southwest quadrant. A redisplay algorithm starts at the root and compares the clip region with each quadrant. It recursively descends to each quadrant that intersects the clip region, until reaching the leaves. In each quadrant it merges objects that intersect the clip region into a master list of objects that must be redrawn. When it is finished, it redraws the objects on the master list.

  1. Splitting a quadrant: When the number of objects in a quadrant exceeds a threshold, the quadrant is subdivided into four quadrants.

  2. Alternative Quadtree implementations:

    1. Only leaf nodes contain a list of objects. Objects can appear in multiple leaf nodes if they overlap multiple quadrants.

    2. All nodes contain a list of objects. An object appears in the topmost quadrant that is large enough to completely contain that object.

  3. If objects can overlap, then there must be a way to construct the stacking order. Since objects are no longer kept in a single list, it is not possible to simply traverse a list from front to back in order to draw the objects. A simple solution is to assign each object a z number (corresponding to the z-axis in 3D graphics). The list of objects to be redrawn is constructed by merging the objects from each quadrant that intersect the clip region. Then the objects are sorted by z-order and drawn. The z numbers are typically doubles. If you must insert an object between two other objects in the stacking order, you typically compute its z number as the average of the two z numbers of the objects it is between. Very, very rarely you might have to re-assign z numbers because you could ultimately lose precision if too many numbers are assigned in the same "space" (e.g., between 1 and 1.2).

Window Exposure Events

When a portion of a window is uncovered or a window is moved, many window managers never notify the application. Instead they maintain an offscreen image for the window, and simply blit to the screen the portion of the double buffer associated with the newly exposed region. This image may be different from a double buffer, since Java will blit an image to the screen even if you are not using double buffering.

By contrast, when you resize a window or deiconfy a window, the window manager will normally fire a window event and the application will be asked to redraw the window. In Java your application's paintComponent method will be called, typically with a clipping region set to the size of the entire window. This can be annoying when the window is asked to resize itself, because often nothing needs to change if the window becomes smaller, and only the portion of the newly exposed window needs to be drawn if the window becomes larger. However, sometimes you will have objects pinned to the sides of a window, and then it is necessary to redraw those objects. If you have a large application where redrawing the entire window on each resize event is prohibitively costly, you can save the old window size and then set the clip region to the newly exposed portions of the window using the graphics object's setClip method. You can set the clip region multiple times within the paintComponent method, so for example you could set the clip region to the horizontally exposed region, clip and draw all the objects to this clip region, then set the clip region to the vertically exposed region and repeat the clip and drawing of the objects. In older versions of Java the setClip method was buggy, and would only allow you to further restrict the clip region, but now it appears you can both expand and restrict the clip region. In this case you are restricting the clip region.