Heuristic evaluation of user interfaces is a practical approach for identifying potential usability problems (Nielsen, 1993). In such an evaluation, a usability expert reviews the design systematically using a checklist of common usability problems and issues. This chapter presents a heuristic evaluation of Argo/UML's cognitive support features.
Each of the following sections evaluates one proposed cognitive support feature in detail using a variation of the cognitive walkthrough method. A cognitive walkthrough is a theory-based user interface usability evaluation that breaks down an interaction into detailed steps and evaluates each step in terms of the following four criteria (Warton et al., 1994):
The programming walkthrough (Bell, Rieman, and Lewis, 1991), a modification of the cognitive walkthrough, considers mental steps performed by the user and refines a model of the knowledge and skills needed by users. This model is called a doctrine. An initial doctrine for Argo/UML is shown in See Initial doctrine for Argo/UML . It is assumed that typical Argo/UML users will have the knowledge described in the initial doctrine. This doctrine, along with the cognitive theories described in Chapter 2, is used to justify the usability of each step in the user interface tasks below. In cases where the initial doctrine is insufficient to justify a step, a new piece of knowledge is added to the doctrine. Each addition to the doctrine is examined again to see if it poses an unreasonable barrier to usage. The final section of this chapter empirically verifies the reasonableness of some additions with a user survey.
I have modified the programming walkthrough method to better suit the evaluation of Argo/UML's cognitive support features. First, the evaluation that follows pays special attention to the knowledge needed to perform each step and the knowledge gained in each step. Second, the steps for each user task, or use case, are presented concisely in a table that describes both the required user actions and the tool's reactions. In those tables, minor task variations are handled as alternative steps, while major variations are addressed in separate use cases. Also, optional steps are marked as such. Third, fairly straightforward steps are addressed briefly; the four questions of the cognitive walkthrough method are used implicitly on each step and only identified problems are discussed. Fourth, the level of detail is chosen to fit the specificity of the proposed cognitive support feature. For example, non-modal wizards are a generic approach to providing procedural support for fixing identified design problems; the evaluation of non-modal wizards will focus on those aspects that differentiate them from standard wizards rather than the specifics of an individual non-modal wizard.
The standard cognitive walkthrough method emphasizes the need for the user to recognize progress on an explicit task. This is needed in part because users often abandon partially completed tasks if they feel that they are on the wrong path. Many of the use cases considered below aid designers in achieving explicit goals. Several of the steps, however, support challenges endemic to the design task without requiring that the designer form an explicit goal before beginning the interaction. Also, many steps are optional because they offer additional support in cases where the normal interaction fails to provide enough to guide the designer. For example, interactions with design critics via clarifiers can help designers to form goals related to improving the design, and when simple cues are not enough to prompt the designer to fix identified problems, additional information can be accessed in optional steps. In fact, mixed initative on the part of the designer and the tool is essential to design support systems that try to augment the designer's own decision-making.
Argo/UML users can access the feedback provided by critics via clarifiers or via the "to do" list. Tables See Steps for using a clarifier and the "to do" item tab and See Step for using the "to do" list and the "to do" tab show the steps for each of these two use cases.
Read "to do" headline or item text and understand the issue; form the intention and plan to fix the problem |
Read the "to do" item text and understand the issue; form the intention and plan to fix the problem |
The first use case begins with an action (A-1) that Argo/UML users perform very often during their work. The tool's reaction to this action includes drawing the standard selection handles found in other direct-manipulation diagram editors, but also includes drawing clarifiers. The wavy-red underline is the most common clarifier in Argo/UML, and it is a familiar indication of trouble to users of other recent desktop applications. This leads the user directly to step A-2. If the user has encountered the identified error before, the clarifier may be enough to prompt the designer to fix the problem without further interaction with the clarifier. Otherwise, in step A-3, the designer requests additional information via a tool tip. Tool tips are common user interface elements that users often access when presented with a new user interface element. Even if the user does not suspect the availability of a tool tip at first, he or she is likely to accidentally trigger one while working with the diagram and thus learn of its availability. The designer is next expected to read the critique headline tool tip and understand, to some degree, the issue raised. If the designer understands the issue described in the headline well, he or she may jump to step A-7, resulting in a plan to fix the problem without further interaction. Alternatively, the designer can access the details of the criticism by opening a popup menu (step A-5), selecting the feedback item from the "Critiques" sub-menu (step A-6), and reading the "to do" item text displayed in the "to do" item tab. The expected interaction with clarifiers suggests the following additions to the Argo/UML doctrine:
The second use case begins with the designer taking the initiative to review and resolve identified problems. Step B-2 requires the designer to look at items in the "to do" pane in the lower-left quadrant of the main Argo/UML window. The designer eventually takes an interest in a particular "to do" item (step B-3) and selects it (B-4). This causes the text of the "to do" item to be displayed in the "to do" item tab. From there, the designer must understand the issue and form a plan to fix it. This interaction with the "to do" list suggests two new doctrine additions:
See Steps for non-modal wizards shows the use case for non-modal wizards. This use case assumes that the "to do" item text has already been accessed and understood, as described above. This starting point is needed because wizards are only presented after a "to do" item description has been presented.
Update the design document and wizard progress bar on each step |
||
Optionally, leave the wizard to work with other features or even other wizards |
||
The first step of using a non-modal wizard is choosing to use it. Then, the designer must click "Next>" to move from the problem description to the first step of the wizard. "Next>" and "<Back" buttons are familiar to anyone who has experience with standard wizards. Indeed, the availability of these two buttons is a strong cue to the user to apply his or her knowledge of standard wizards. In step C-3, the designer interacts with the widgets within a particular wizard, while the tool makes immediate updates to the design. Although the usability of each particular wizard could be evaluated with a separate walkthrough, I am more concerned with the usability of the aspects of the non-modal wizard approach that distinguish it from the standard wizard approach. In particular, the immediate update of the design is unique to non-modal wizards. Immediate updates are part of the direct-manipulation paradigm and are encouraged by the guidelines proposed by Shneiderman (1998) and others.
The last two steps in See Steps for non-modal wizards are both optional, and it is expected that the designer will work on other tasks between steps C-4 and C-5. In step C-4, the designer may leave the wizard to use other features of the tool, such as the diagram editor, menu commands, other items in the "to do" list, or even other wizards. At first, designers using Argo/UML may apply their knowledge of standard wizards and assume that they must finish the wizard or cancel it. After a little experience with non-modal wizards, however, designers will encounter a cue card wizard that explicitly directs them to use other tool features. Also, the fact that non-modal wizards do not overlap the main editing window is a strong indicator that other tool features are always available. Designers must desire working with other features and they must feel sure that they will not lose work by leaving a wizard. The cognitive theory of opportunistic design indicates that designers will deviate from prescribed processes, such as the steps of a wizard, when needed information must be looked for elsewhere. Designers who have become familiar with Argo/UML's "to do" list will know that items are never removed without being resolved. This leads to step C-5, where the designer may return to a partially completed wizard. This is done by selecting the "to do" item via a clarifier or the "to do" list as described above. Designers may return to a partially completed wizard simply by investigating the clarifier or "to do" list as they did originally, or they may explicitly seek out a particular "to do" item that they wish to finish. The walkthrough for the former case is presented above. In the latter case, designers should be able to scan down the "to do" list and identify partially completed wizards when they see blue progress bars. This interaction suggests the need for two additions to the Argo/UML doctrine:
The user interface steps for using context sensitive checklists are much more simple than those needed for accessing criticism via clarifiers or the "to do" list. The mental steps, however, can be more difficult because checklist items can address broader design issues and are less closely linked to the state of the design. See Steps for using context sensitive checklists summarizes the steps for using context sensitive checklists.
Form interest in addressing common problems related to the selected element |
||
The first step requires the designer to select a design element as he or she normally would do. In addition to drawing selection handles, Argo/UML also fills the checklist tab with a checklist appropriate to the selected design element. If the checklist tab is already showing, this change will be seen immediately; otherwise, the designer will have to select the checklist tab (step D-3). The designer must desire information on common design problems (step D-2) before he or she can be expected to select the checklist tab. This leads to the following addition to the Argo/UML doctrine:
In step D-4, the designer must read a checklist item and consider the design issue it raises. Whether this step is reasonable or not depends on the wording of specific checklist items. Two factors that increase the effectiveness of checklist items are (1) the fact that checklist items are context sensitive and contain specific design terms rather than vague pronouns, and (2) the availability of on-line help for many checklist items. In step D-5, the designer may access additional information that explains the issue raised in more detail. In the final step, the designer may check off items as they are considered. This serves as a reminder of progress and prompts the designer to go through the checklist systematically. The option to check off items suggests that the following should be added to the Argo/UML doctrine:
Designers can access information about past design decisions using Argo/UML's "History" tab as described in See Steps for using design history .
Display list of history items related to selected design element |
||
Display details of that item and list the design elements involved |
||
First, the designer must form the desire to review the history of design choices. The cognitive theory of reflection-in-action indicates that designers do reflect on the state of the design, but it is not obvious that they would seek out design history. Once they have formed the desire to review past design decisions, they must recognize the "History" tab as providing that information (step E-2). In steps E-3a or E-3b, the designer selects global history or chooses to view a subset of the history related to the selected design element. This step seems reasonable because the labels are clear and because the designer can safely experiment with these two settings and understand their effects immediately. Step E-4 is a difficult step that requires the designer to read a one line description of a history item and form an understanding of its meaning. Specifically, designers must recognize history items that relate to current design decisions. Once the designer recognizes a relevant history item, he or she can view its details by selecting it (step E-5). These details include the full text of the history item and a list of the design elements that were involved in the criticism or modification. The designer may optionally follow a trail of history items by selecting one of the listed design elements to see its history (step E-6).
See Steps for using the opportunistic search utility shows the steps needed to use Argo/UML's opportunistic search utility. This feature is an improvement on the standard search utilities found in many commercial UML tools and other widely used desktop applications. This walkthrough emphasizes the aspects that differ from standard search utilities.
Designers commonly use search utilities in programming and design tools as a means of navigation and to get an overview of a certain aspect of the design. In step F-1, the designer opens the search window via a menu command or keystroke. Argo/UML makes finding the search command easy by placing it under the Edit menu, which is the same place that it is found in many other tools. As soon as the search window is displayed, the "Help" tab is displayed in the lower half of the window. The help tab explains the non-standard aspects of the search utility. In step F-2, the designer enters search criteria, such as a wild-card expression for the names of design elements to find or the type of element to find. Most search criteria fields are standard and straightforward, but some may be difficult to understand or use. The emphasis here, however, is on the cognitive support provided when the designer reviews the search results. In step F-3, the designer presses the Search button and Argo/UML creates a new tab with the search results. The new tab is labeled with a concise summary of the search criteria. The designer then proceeds to review the search results (step F-4). Results are displayed in a table with one result in each row. A designer familiar with standard desktop tools should have no trouble recognizing or understanding the search results, and he or she will naturally click on a search result to learn more about it (step F-5). When the designer selects a search result, the opportunistic search utility also displays related design elements in a second table in the lower half of the results tab. The fact that the items in the lower table are related to the selected result should be clear since they appear after a selection is made. However, the nature of the relationship could be unclear. Step F-6 in See Steps for using the opportunistic search utility requires the designer to form an interest in a related item. Note that the exact nature of the relationship need not be understood, so long as the designer forms an interest. In fact, it is expected that designers may see coincidental relationships in addition to the rules used by Argo/UML to generate these items. Finally, the designer may review a related result in context by double clicking on it. This step should also be natural to designers who have used other search utilities. This use case suggests the following additions to the Argo/UML doctrine:
The walkthrough of Argo/UML's opportunistic table views consists of a single use case with several optional steps. The use case is described in See Steps for using opportunistic table views .
A designer using Argo/UML can edit any semantic aspect of the design via either a diagram view and properties tab or via a tabular view. The diagram view is the default and more familiar of the two. Making the choice to use a tabular design view may require meta-cognitive insight on the part of the designer: i.e., the designer must realize when there is an advantage to using the tabular view.
Once the designer has formed the desire to use a table view, he or she selects the "As Table" tab (step G-2). Choosing the "As Table" tab should be easy because the label is clear, it is centrally located, and users know that they can safely explore tabs. The most commonly useful table perspective is shown by default. Designers may optionally switch to an alternative table perspective from a choice widget (step G-3). Next, the designer may adjust the highlighting mode to better fit his or her systematic scanning behavior (step G-4). Thinking of one's scanning behavior in advance definitely requires meta-cognitive awareness that the designer may not have. However, even if step G-4 is not carried out, the default row-by-row highlighting mode is the one most likely to be useful. In step G-5, the designer uses the table as normal. This includes selecting and editing table cells to accomplish some editing task. Since it is assumed that the designer specifically chose the table view instead of the diagram view, it is likely that he or she will perform a task that takes advantage of the tabular nature of the table view. For example, the designer is likely to systematically scan the design or make a series of edits to corresponding parts of the design. While the designer is carrying out that task, he or she may need additional information from some other part of the design. The theory of opportunistic design indicates that designers are likely to switch tasks to work with other parts of the design to gather needed information. Upon returning from such a design excursion, the designer may use the "Instant Replay" button to request a replay of the most recent activity in the table view (step G-6). The intent of the replay feature is to help the designer see where he or she left off and thus re-establish a part of the mental task context. However, the effectiveness of the instant replay animation needs to be verified.
The expected interaction with opportunistic table views suggests the following additions to the Argo/UML doctrine:
Designers use Argo/UML's navigational perspectives by performing the steps described in See Steps for using navigational perspectives .
As with several of Argo/UML's cognitive support features, the designer must first realize that he or she needs a certain type of information. In the case of navigational perspectives, the designer must form a question about a design structure (step H-1). The theory of reflection-in-action and other cognitive theories suggest that raising and answering questions about the design is a key part of the design process. In step H-2, the designer must choose a navigational perspective from the choice widget above the navigation tree. To accomplish this step, the designer must first recognize that the choice widget is the proper affordance. Then, the designer must choose a perspective that will help answer the question raised based on the name of the perspective. Some of the perspective names are very clear because they use standard terms like "inheritance," while others may not be so clear. If this is the first use of the selected perspective, it will be completely collapsed and must be expanded (step H-3). Expanding the tree should be natural because, at this point, the tree has obviously changed, and expanding is the normal operation on a collapsed tree. In step H-4, the designer must understand the design structure that is shown. As with the opportunistic search utility, there is the possibility that the designer will see a different structure than the one intended. This could be useful, or it could be misleading. This use case has identified the following doctrine additions:
See Steps for using the broom alignment tool shows the steps needed to use Argo/UML's broom alignment tool. This walkthrough is specified at a lower level of detail than most of the others because the broom uses a novel interaction that does not make use of standard widgets.
The cognitive theory of secondary notation indicates that designers will desire diagrams that make effective use of visual properties such as alignment and spacing. Based on this theory, steps I-1 and I-2 should occur naturally. Since it is assumed that designers are experienced with diagramming tools, step I-3 should not be a problem. In fact, designers are likely to move diagram elements into rough alignment as a way to quickly evaluate the visual impact of a given layout. In step I-4, the designer can use the toolbar button for the broom mode (step I-4a) or use control-drag (step I-4b). The toolbar button provides a visual affordance to start broom mode, but the designer may have difficulty recognizing the broom mode icon. The control-drag option is less likely to be discovered, but it might be learned from the documentation. Once the designer has changed modes in the diagram editor, he or she is likely to click or drag in the diagram area (step I-5). Furthermore, a message in the status bar of the main Argo/UML window briefly explains the broom and prompts the designer to drag. The status bar message is, "Push objects around. Return toggles pulling. Spacebar distributes." If the designer merely clicks, a blue plus-sign is shown until the user eventually drags the mouse. Dragging produces an immediate visual effect that indicates that something is happening. The shape of the broom tool provides a pushing affordance that should lead designers to step I-6. Even if the shape of the broom does not immediately suggest its behavior, the designer should naturally discover and understand the broom's push-to-align action. In step I-7, the designer may optionally undo the movement of diagram elements by moving the broom in the opposite direction as the original drag. Again, the existence of this feature is not obvious, but it is likely to be invoked accidentally, and once it has been seen its usefulness will be obvious. Alignment by itself does not completely achieve the goal of forming visual groups; even spacing is also needed. Designers may evenly space the diagram elements on the broom by pressing the spacebar (step I-8). The availability of this action is not obvious, but it is mentioned in the status bar whenever the broom is in use. Once the designer has pressed the space bar, its effect is immediately visible, and the message "space evenly" is shown behind the broom. The final step (I-9) is simply to release the mouse button.
Expected usage of Argo/UML's proposed model-based layout feature consists of the two use cases described in See Steps for using grid-based layout and See Steps for using region-based layout .
Show tab, including attribute fields, preview pane, and "Layout Diagram" button |
||
Layout diagram using grid constraints and simple geometric rules |
Show tab, including constraint field, region drawing pane, and "Layout Diagram" button |
||
Layout diagram using constrained regions and simple geometric rules |
Both use cases begin with the desire to automatically redo the layout of a diagram. This desire occurs naturally during design and is commonly found and used in other CASE tools. Likewise, the second step is also reasonable. The final step in each use case is also very straightforward. Even if all the other steps are skipped, the designer can simply press the "Layout Diagram" button to produce a layout that is as good as that produced by other CASE tools. Even if a designer does not initially desire model-based layout, it is likely that they will discover this cognitive support feature eventually through normal usage.
In step J-3, the designer must select the "Grid-Based" tab. Accessing a clearly visible tab should not be a problem because exploring tabs is always a safe operation. Next, the user must choose attributes to control the layout. Since this aspect of Argo/UML's model-based layout is not found in other tools, designers may not understand what is being requested. Furthermore, designers may have difficulty choosing the particular model element attributes needed to achieve the desired layout. Once the attributes are selected, value ranges must be defined. For some attributes, this can be trivial or automatic; for others, determining appropriate value ranges may require trial-and-error. The layout preview pane quickly shows a rough approximation of the layout to help designers explore alternatives. In step J-6, the designer may reorder the rows or columns or move a given attribute from a row to a column or from a column to a row. Reordering should be easy to accomplish via the up and down buttons to the right of the attribute list. The first use case suggests the following addition to the Argo/UML doctrine:
In step K-3, the designer selects the "Region-Based" tab. Then, in steps K-4 and K-5, the designer must define geometric regions for the layout and assign constraints to them. It is expected that steps K-4 and K-5 will be done repeatedly as the designer refines the layout. The following additions to the Argo/UML doctrine are needed for the designer to successfully define and refine the region-based layout:
The walkthrough of Argo/UML's selection-action buttons consists of one use case with three alternative conclusions.
Designers naturally form the desire to add new nodes and edges to the diagram as part of the diagram construction process. New diagram elements may be the initial elements in new clusters of related elements, or they may elaborate on existing clusters of classes or states. The standard toolbar interface must be used for initial diagram elements. However, if the designer wants to elaborate on existing diagram elements by adding new related elements (step L-1), he or she can use the selection-action buttons.
Selection-action buttons are implemented as an enhancement to normal diagram element selection. As such, the designer will encounter them during the normal course of selecting diagram elements (step L-2). Steps L-1 and L-2 may be reversed if the selection-action buttons serve as a prompt for the designer to consider adding new elements.
The designer can choose three alternatives for the final step. In alternative L-3a, the designer simply clicks the selection-action as he or she would click a toolbar button. This has the immediate effect of producing a new diagram node and edge of the appropriate type in a default position near the originally selected node. In alternative L-3b, the designer drags from the selection-action button to the desired position of the new node. This alternative is not immediately obvious because buttons are usually not dragged. However, once the designer is familiar with alternative L-3a, he or she may be dissatisfied with the default position and seek to specify the position via dragging. Alternative L-3b might also be discovered accidentally if the designer tries to click and quickly move the mouse to select the new node. While the designer is dragging, a rubberband line is drawn to give immediate feedback and the status bar shows the message, "Drag to define an edge (and a new node)." The final alternative, L-3c, requires the designer to drag from a selection-action button to an existing node. This results in a new edge between the two nodes. Designers may attempt the alternative interaction simply because it seems to be an obvious way to accomplish this goal. The status bar message also suggests that it is possible to define a new edge without defining a new node. Furthermore, the rubberband line feedback mechanism is the same one used when adding an edge between nodes in many diagramming tools, so it may prompt the designer to assume that the same functionality is available in this context.
See Steps for creating multiple design elements by pattern and See Steps for creating multiple design elements by form show the steps needed to use Argo/UML's create multiple feature.
Show a list of available pattern names, description and parameter panes, summary pane, and "Create" button |
||
As with the model-based layout use cases, both of the use cases for create multiple begin with forming a desire, followed by opening a secondary window, and end with pressing a button to confirm the operation. These three steps should not be a problem because they are familiar to users who have experience with other desktop applications.
In step M-3, the designer must select the "By Name" tab. This step seems reasonable because the designer cannot do anything in the help tab, and because exploring tabs is always a safe operation. Next, the user must select a pattern by name from a list. This could be a difficult step because the designer may not recognize the meanings of the pattern names. Even if designers do not initially understand every pattern, they can be expected to explore some of the patterns in the list because list selection is always a safe operation. In step M-5, the designer may optionally request more information about the selected pattern. This step should not be difficult because the "More Info" button is clearly visible. In step M-6, the designer supplies names for new design elements to be created and the names of existing elements onto which the new elements will be grafted. The reasonableness of this step depends on the specific prompts provided by each design pattern configuration panel.
Step N-3 in the second use case requires the designer to select the "By Form" tab. Again, this should not be a problem. Next, the designer must select a form by name from a list. It is not expected that the designer will be able to choose the correct form on his or her initial attempt, instead the designer is expected to explore various forms until he or she sees one that looks relevant. A designer must gauge the relevance of a particular form to his or her immediate needs based on the brief description of the form and the design structures that are visually evident in the form. Once a form is selected, the designer may optionally click on one of the design pattern links to access more information on the intent and applicability of that design pattern. Clicking on an underlined link should not be a problem for users that are familiar with web browsers. In step N-6, the designer fills in the form with the names of new design elements and the names of existing elements to graft the new elements onto. The process of entering the names should not be a problem because it only involves using standard text entry widgets. However, choosing which widgets to fill in and which to leave blank requires the designer to understand the grafting rules. These two use cases suggest the following doctrine additions:
Performing the cognitive walkthroughs of the proposed features has been a productive step in my feature generation method. Breaking down the user's expected interaction with each feature has forced me to think through all the details and has highlighted several potential problems and refinements. One of the main dangers of user interface design is assuming that the user knows everything that the user interface designer knows. Several of the elements of the Argo/UML doctrine may be barriers to usage. I have conducted a set of user surveys to probe the knowledge of Argo/UML users and determine which elements of the doctrine are, in fact, problematic. Removing these problems will require refining the cognitive support features to eliminate the need for certain knowledge or skills. The surveys and refinements are discussed below.
One straightforward way to determine if Argo/UML users possess a given piece of knowledge is simply to ask them questions that require that knowledge. In August 1999, I conducted an anonymous survey of registered Argo/UML users. The survey consisted of three questionnaires with ten to twelve questions each. Most of the questions presented a screenshot and asked the user for his or her thoughts on the purpose of each particular widget. An example question is shown in See A survey question on clarifiers . Some questions used three parts to test the degree of knowledge support provided by a feature: the first part asked how the designer would address a given problem, the second part asked the designer to explain the purpose of particular feature, and the third part described the support provided by the feature and asked again how the designer would address the problem in light of that support.
For each questionnaire, I constructed a set of web pages with one question per page. Each page included a text field or multiple choice widget and a "Submit" button. Subjects were instructed give their first thoughts if they did not know the answer, and they were told not to return to previously answered questions. For each questionnaire, an email message with the web page address was sent to a subset of registered Argo/UML users. Approximately one hundred and fifty messages were sent for each questionnaire, and they were sent to users in alphabetical order. This method of validating the cognitive walkthroughs yielded very rapid results: for each questionnaire, I received more than a dozen responses within the first two days.
The results of particular questions are described above. Tables See Questionnaire results for clarifiers, "to do" list, wizards, and checklists through See Questionnaire results for broom and selection-action buttons summarize the survey results. The first questionnaire probed the issues raised by the walkthroughs of clarifiers, the "to do" list, non-modal wizards, and checklists. The second questionnaire probed the issues raised by the walkthroughs of Argo/UML's opportunistic search utility and table views. The third questionnaire probed the issues raised by the walkthroughs of the broom alignment tool and selection-action buttons.
See Questionnaire results for clarifiers, "to do" list, wizards, and checklists shows that the majority of the assumptions added to the Argo/UML doctrine are reasonable or marginal. One possible problem is that a fair fraction of users confused the "to do" list with the checklist tab. Also, the count of items at the top of the "to do" list was clear, but the use of a plus or minus sign to indicate the trend of the size of the "to do" list was not clear to anyone. The non-modal wizard progress bar was recognized much more easily than I would have expected.
See Questionnaire results for opportunistic search and table views shows the results of the questionnaire on the opportunistic search utility and opportunistic table views. The search utility seemed understandable to the majority of users. However, a fair number of users had difficulty with the related results table. When asked how they would systematically scan all the associations in the design, the most common answers were to use the search utility or navigational perspectives. This indicates that many designers will not realize that tables are an effective user interface for systematic design review. However, it may have been biased by the fact that the proceeding questions all dealt with the search utility. When the surveyed users were asked how they would access and use the table views, the majority gave answers indicating that they would be able to use these features. The "Instant Replay" button was not very recognizable, but it was assumed to be safe, and about half of the surveyed users understood its purpose once its animation was seen. The animation question probably would have been answered more positively if an actual animation had been shown in the question rather than a sequence of four small screenshots. In general, the survey results are pessimistic because they ask questions about usage without allowing the users to actually work with the tool.
See Questionnaire results for broom and selection-action buttons shows the results of the questions on using the broom and the selection-action buttons. First, the surveyed users shows a marked preference for diagrams that used alignment as a secondary notation as opposed to misaligned diagrams. Few users could recognize the broom icon, and the tool tip "broom" did not provide any help. However, once users saw the broom in the diagram pane, almost two-thirds were able to predict its behavior. Once they were shown screenshots of the broom in action, all but one user recognized its usefulness. A second user had trouble understanding that reversing the broom direction served as a kind of undo, otherwise it was clear to all users. Most users recognized the selection-action buttons as variants of the toolbar buttons and could predict that pressing a selection-action button would create new design elements that were related to the current selection. When asked how they would create a new class at a desired position on the diagram, most users said that they would first use a selection-action button to create the class in its default position and then move the class to the desired position. More users were able to guess the possibility of creating a new edge by dragging from the selection-action button of one class to another existing node. However, several users said that they would prefer to use the standard tool bar buttons to create edges between existing classes. Again, the survey is pessimistic in its evaluation because users are asked to explain what they would do rather than actually use the tool. For example, if users were actually using Argo/UML, the difficulty of using the standard tool bar buttons would encourage them to use the selection-action buttons.