The display model we have used to this point in the course involves a total
redraw of the screen. While that works for applications that have only
a few dozen objects, it may not be fast enough to provide interactive
performance for applications that display hundreds
of objects, and it will not be fast enough to provide interactive
performance to applications that display thousands of objects. For
such applications, it is necessary to use incremental
redisplay algorithms, which only redraw the damaged portion of
the display. The damaged portion of the display is the portion of
the display occupied by changed objects. Such objects may have changed
location, changed size, or changed a visual property, such as their
fill color or border style. The damaged portion of the display includes
both the old and new locations of the objects. These notes will discuss
a number of different incremental display algorithms, as well as how
a number of special cases, such as window exposure events and feedback
objects, are handled.
Double Buffering
A common
technique used to avoid flicker in windows is to draw objects to an
offscreen buffer, called a double buffer, and then blit the
buffer to the screen. While this technique actually slows down the
rendering process, because you first render to a buffer and then render
the buffer to the screen, it is often perceived by the user as speeding
up the rendering process because it eliminates flickering. When you
directly render to the screen, you typically first blank a portion of
the screen, and then draw to it. The user's brain can often distinguish
between the blanking and redraw operations, thus causing the image to
appear to flicker. By default double buffering is turned on in Java and
with today's graphics chips for accelerated drawing,
there is rarely a reason why you would turn it off. Java handles double
buffering for you, so you simply call the draw method and Java redirects
the rendering to a buffer. When your paintComponent returns, Java's virtual
machine then blits the buffer to the display.
Feedback Objects and Interfaces with NonOverlapping Objects
- The simplest type of interface to redraw is one in which no two objects
may overlap on the display. In this case it is
ok to update an object by erasing its old image and displaying its
new one.
- XOR can provide a way to quickly draw and erase objects. The first
time an object is drawn in XOR mode it appears on the screen. The
second time it is drawn in XOR mode it disappears. Hence you can
erase an object by simply redrawing it.
- Feedback objects can be efficiently drawn and erased with XOR, even
if an interface has overlapping objects. Feedback objects, like
rubberband boxes, are typically drawning on top of other objects, so
the display manager can make them appear by drawing them in XOR mode,
and can make them disappear by drawing them a second time in XOR
mode.
- Many windowing systems provide a way to specify the color with
which we wish to draw an xor object against a normal background.
The system then computes the color that needs to be xor'ed with
the background color in order to
achieve the desired color in which to draw the object.
Formally, if C is the desired foreground
color and B is the background color, than
C = XOR(D,B) and D is the color the system computes to produce the
foreground color when xor'ed with the background color. If the system
does not provide a way to compute D, it can be easily computed as
D = XOR(C,B). In Java, the Graphics class provides two
methods for setting the drawing mode:
- setXORMode(Color c): objects will now be drawn in XOR
mode and when drawn on the background, will appear as
the color c.
- setPaintMode(): objects will be drawn opaquely so that
they overwrite whatever is beneath them.
By default the graphics context is set to paint mode.
- xor.java illustrates the
use of xor in java. The below applet shows xor.java in action. You
should click on the applet to get the two xor'ed red rectangles to
appear and disappear. Note that the code specifies that an xor'ed
rectangle should appear red when drawn over a white background. When
it is drawn over the black and yellow backgrounds, the resulting
color is unpredictable.
- When using xor, the display manager needs to be notified when an object
changes, and should keep a list of changed objects. It should also save
the object's old location, so it can erase the object by drawing it a
second time at its old location. When the display manager is called to
update the display, it runs through the list of changed objects and draws
them at both their old and new locations:
update() {
for obj ∈ changed_objs {
draw obj at old location
draw obj at new location
}
}
The display manager will need to be notified whenever a visual feature
of an object changes, such as its location, size, or color. Often the
display manager is implemented as an observer and the objects as
observees. For example, the display manager might implement a
notifyChanged method as follows:
notifyChanged(obj) {
// save the changed object and its old location
changed_objs = changed_objs ∪ ( obj, obj.getLocation() )
}
- To implement the redrawing of feedback objects, the display manager might
have a "feedback" mode in which it only renders feedback objects, and
a non-feedback mode in which it renders the rest of the interface. It
would be in "feedback" mode if it had a non-null list of feedback objects
to draw. Whether in feedback mode or not, it would first go through a
list of saved feedback objects and draw them, in order to erase them
from the screen. If in feedback mode, it would then draw each object
on the feedback list and save that object and its location so that it
could be erased on the next display cycle. It would then exit to short
circuit the rendering process. If not in feedback mode, the display
manager would proceed to draw the objects in the interface. Here is
an example algorithm:
update(Graphics g) {
// erase old feedback objs
g.setXORMode(some color)
for each obj ∈ saved_feedback_objs
draw obj at its old location
saved_feedback_objs = NULL
if (feedback_objs != NULL) {
for each obj ∈ feedback_objs {
draw obj at its current location
saved_feedback_objs = saved_feedback_objs ∪ ( obj, obj.getLocation())
}
g.setPaintMode()
}
else {
g.setPaintMode()
// draw objects in interface
}
}
Since this feedback drawing algorithm works with many types of display
managers, it could be encapsulated in its own class and invoked by
any display manager that wanted to handle feedback objects by xor'ing
them.
Interfaces with Overlapping Objects
XOR does not work with overlapping objects because two objects that overlap
one another will partially erase one another. You also cannot simply erase
and redraw each object, for two reasons:
- If you erase an object's old image, it may erase parts of other
objects as well
- If you redraw an object, it will appear on top of all other
objects. If it is not supposed to be the top object, that
would be a bug.
Hence we need a more sophisticated algorithm which redraws parts of
other objects which are erased and draws the object in
the proper stacking order. The stacking order is the order
in which objects should be drawn from back to front. Objects that should
appear under other objects should be drawn first, while objects that should
appear on top of other objects should be drawn last.
The total redraw solution is to redraw all
objects using the appropriate stacking order, but this
may be too expensive when there are hundreds or thousands of objects.
A more efficient solution is to only
redraw damaged parts of screen. This is called incremental redisplay,
and it requires that we maintain data structures
to keep track of damaged objects.
Our strategy is as follows:
- Keep track of all objects that have a changed visual property,
such as position, size, color, or stacking order.
- As objects are changed, merge their old position into
an "old" bounding box. This bounding box keeps track
of the original locations of the changed objects.
- A bounding box is the smallest rectangle that
encloses an object or set of objects.
- When an update command is issued (i.e., a command to redraw the
interface)
- Compute the new bounding box of the changed objects
- For each displayed object, determine whether the displayed
object intersects either the new or old bounding box.
- Redraw the objects that intersect either the new or
old bounding box in the appropriate stacking order
- Only redraw the portion of the object that changes--most
windowing systems provide a clip region that
facilitates redrawing only a portion of the object.
- A number of optimizations are possible:
- For composite objects, check the composite object's
bounding box before checking its individual children. If the
composite object doesn't intersect either of the bounding
boxes, then none of its children can, so none of its children must
be checked.
- Check to see how much overlap there is between the new
and old bounding boxes. If the overlap is considerable,
then merge them. This will prevent objects from having
to be rendered and clipped twice. Often there will be
considerable overlap betweeen the two bounding boxes,
because often an object has either changed in place (e.g.,
a color change), has moved a short distance, or been
resized.
- Algorithm
- Data structures and Variables: For
each window you should maintain the following variables:
- objs_changed: The set of objects that have changed
- old_bbox: The bounding box for the changed objects'
old positions
- new_bbox: The bounding box for the changed objects'
new positions
- Update algorithm
-all variables prefixed by self refer to instance variables
-if a variable is not prefixed by self then it is a local
variable
Win::Update()
new_bbox = empty
old_bbox = self.old_bbox
for each obj ∈ self.objs_changed do
new_bbox = new_bbox U obj's position
// clear the damaged part of the display
win.clearRect(old_bbox)
win.clearRect(new_bbox)
// redraw the damaged part of the display
for each obj in self.children do
if intersect(obj, old_bbox) or intersect(obj, new_bbox)
obj.update(old_bbox, new_bbox)
self.old_bbox = empty
Container::Update(old_bbox, new_bbox)
for each obj in self.children do
if intersect(obj, old_bbox) or intersect(obj, new_bbox)
obj.update(old_bbox, new_bbox)
Primitive_Obj::Update(old_bbox, new_bbox)
self.draw()
- Notification algorithm: The notification algorithm for
the display manager must now record both the changed object
and merge its old bounding box into the bounding box of old
locations being maintained by the display manager:
notifyChange(obj)
self.objs_changed_objs = self.objs_changed U obj
self.old_bbox = self.old_bbox U obj's position
- Java Notes
- Java's repaint method merges bounding boxes so you cannot use an old and
a new bounding box--you must change the above update procedure so that
it uses a single bounding box instead
-
If you really want to use an old and new bounding box, there are
two alternative things you can do:
- call paintImmediately. paintImmediately will immediately
queue a paint
event rather than aggregating a number of repaint calls and then
generating a single paint event. However, you must be careful about
calling paintImmediately because if there are several changes to
the display, you could end up calling it several times. One scheme
that could work with a display manager would be to add a second
notification procedure called UpdatesCompleted. UpdatesCompleted
would call paintImmediately twice, once with the old bounding box
and once with the new bounding box.
- In the event handler,
call repaint with no arguments so that the clip region gets
set to the entire component. Then in paintComponent,
call the graphics object's
setClip method to first set the clipping region to
the old clip region and clip/draw all objects against this
clip region. Then call the setClip method to set
the clipping region to the new clip region and clip/draw all
objects against this clip region.
- You could simply set a clip region and draw objects without first
checking that they intersect the clip region. However, this would
be much more expensive because Java will first scan convert each
object, then draw those pixels that intersect the clip region. It
will not first check if the object intersects the clip region,
because it assumes that you've already done that if you wanted to.
- Separation of display manager from windows: The previous algorithm
hard-codes the display algorithm into the window. A better
approach is to allow display managers to be attached to windows
so that different display algorithms can be used depending
on the application.
- Interface: Use the observer pattern
interface DisplayManager {
public:
void update(GraphicsContext);
void addObject(Shape);
void removeObject(Shape)
void notifyChange(Shape)
// the following method might not be needed depending on
// the window manager
void updatesCompleted();
}
- Variables: Each window should maintain a pointer to
a display manager object
- Methods:
Win::paintComponent(g)
displayManager.update(g);
Container::draw(g) {
for each obj &isin self.shapes do
if intersect(obj, g.getClip())
obj.draw(g)
}
DisplayManager::Update(g)
new_bbox = empty
old_bbox = self.old_bbox
for each obj in self.objs_changed do
new_bbox = new_bbox U obj's position
// clear the damaged part of the display
win.clearRect(old_bbox)
win.clearRect(new_bbox)
// redraw the damaged part of the display
g.setClip(old_bbox)
for each obj in registeredObjects do
if intersect(obj, old_bbox)
obj.draw(g)
g.setClip(new_bbox)
for each obj in registeredObjects do
if intersect(obj, new_bbox)
obj.draw(g)
self.old_bbox = empty
Quadtrees
When there are thousands of objects on a display, it may be too expensive
to test every object against the clip region. A solution is
to subdivide the display and only examine objects that
fall within the subdivisions that intersect the damaged region.
Quadtrees provide a way of performing this subdivision.
They divide a region into quadrants.
Each quadrant contains a list of all the objects that intersect
its bounding box. Each node of a quadtree has four children,
corresponding to a northeast, northwest, southeast, and southwest
quadrant. A redisplay algorithm starts at the root and compares the
clip region with each quadrant. It recursively descends to each quadrant
that intersects the clip region, until reaching the leaves. In each quadrant
it merges objects that intersect the clip region into a master list of objects
that must be redrawn. When it is finished, it redraws the objects on the
master list.
- Splitting a quadrant: When the number of objects in a quadrant
exceeds a threshold, the quadrant is subdivided into four
quadrants.
- Alternative Quadtree implementations:
- Only leaf nodes contain a list of objects. Objects can appear
in multiple leaf nodes if they overlap multiple quadrants.
- All nodes contain a list of objects. An object appears in the
topmost quadrant that is large enough to completely contain
that object.
- If objects can overlap, then there must be a way to construct
the stacking order. Since objects are no longer kept in a single
list, it is not possible to simply traverse a list from front to
back in order to draw the objects. A simple solution is to assign
each object a z number (corresponding to the z-axis in
3D graphics). The list of objects to be redrawn is constructed by
merging the objects from each quadrant that intersect the clip
region. Then the objects are sorted by z-order and drawn. The
z numbers are typically doubles. If you must insert an object between
two other objects in the stacking order, you typically compute its
z number as the average of the two z numbers of the objects it is
between. Very, very rarely you might have to re-assign z numbers
because you could ultimately lose precision if too many numbers are
assigned in the same "space" (e.g., between 1 and 1.2).
Window Exposure Events
When a portion of a window is uncovered or a window is moved,
many window managers never
notify the application. Instead they maintain an offscreen image for
the window, and simply blit to
the screen the portion of the double buffer associated with the newly
exposed region. This image may be different from a double buffer, since
Java will blit an image to the screen even if you are not using double
buffering.
By contrast, when you resize a window or deiconfy a window,
the window manager will normally fire a window event and the application
will be asked to redraw the window. In Java your application's paintComponent
method will be called, typically with a clipping region set to the size
of the entire window. This can be annoying when the window is asked to
resize itself, because often nothing needs to change if the window becomes
smaller, and only the portion of the newly exposed window needs to be drawn
if the window becomes larger. However, sometimes you will have objects pinned
to the sides of a window, and then it is necessary to redraw those objects.
If you have a large application where redrawing the entire window on each
resize event is prohibitively costly, you can save the old window size and
then set the clip region to the newly exposed portions of the window
using the graphics object's setClip method. You can set the clip
region multiple times within the paintComponent method, so for example you
could set the clip region to the horizontally exposed region, clip and
draw all the objects to this clip region, then set
the clip region to the vertically exposed region and repeat the clip and
drawing of the objects. In older versions of Java the setClip method was
buggy, and would only allow you to further restrict the clip region, but now
it appears you can both expand and restrict the clip region. In this case
you are restricting the clip region.