Computer Science notes ⇒ Design of Interactive Systems

Designing interactive systems on computer systems for non-experts has only been developed for 25 years, whereas something like teapots has had 4000 years to be perfected (and still has dripping spouts!). Books have had 500 years to develop systems containing the best design features (page numbers, table of contents, index, chapter numbers, headings, etc) and books can be used as interactive systems.

Affordance

Are Current Systems Sufficiently Usable?

This style is not widely used, but is useful to understand it to consider more recent and long-term developments. This is the first interaction style that appeared on PCs, taking over from mainframe systems.

The conversational metaphor applies here, where you're talking to the computer through your keyboard and it reacts. However, the language you speak in must be correct to the last dot and in the correct order, much like speaking a foreign language. The system doesn't give you any clues on what to do, so you must remember (or use a crib sheet), the syntax of the language. These commands can get quite long and complex, especially when passing lots of options and (in UNIX), piping one command to the other.

You also get limited feedback about what is happening, a command such as rf may return you directly to the command line, after deleting 0 or 100 files. The later versions of DOS took the feedback step too far, however, asking for a confirmation of every file by default. This is programmed by the interaction designer, and they have to remember to do this and get the level of interaction right.

If you do get the command correct, however, this can be a very efficient way of operating a complex system. A short, but powerful, language allows you to acheive a great deal and people are willing to invest the time to learn this language to get the most efficient use of the system.

However, although computers are good at dealing with cryptic strings, complex syntax and an exact reproduction of the syntax every time, humans aren't. This interaction system stemmed from the fact that processing power was expensive, so humans had to adapt to the way computers needed to interact, not vice versa. This is no longer the case.

Although the command line style was good for experts, it wasn't for novice of infrequent users, so an interaction style was developed which is almost the complete opposite of command line dialogue in terms of strengths and weaknesses - menus.

Menus are simple, as not much needs to be remembered as the options are there on screen and the physical interface corresponds directly to the options available. Feedback is immediate - selecting an option will either take you to another screen of menus or to perform the selected task.

Feedback is natural and built in -you can see whether you are going the right way, because either something relevant occurs - much like handling objects gives you instant built in feedback.

Selections from objects (such as radio buttons) can be thought of as a menu, even though the selection method is different.

Menu systems should be usable without any prior knowledge or memory of previous use, as it leads you through the interaction. This is excellent for novice and infrequent users (hence their use in public terminals of many varieties). However, a complex menu structure can complicate matters where particular features will be found (e.g., IVR systems)

Expert and frequent users get irritated by having to move through the same menu structure every time they do a particular task (hence shortcut keys in applications such as Word), and menu systems also present a lack of flexibility - you can only do what menu options are present for you, so menu driven systems only work where there is a fairly limited number of options at any point. Items also need to be logically grouped.

A form is provided for the user to fill in. Perhaps not the most revolutionary or interesting of interaction styles, but it is based on another real world metaphor - filling out a paper form. This also illustrates another problem with metaphors, not everything transfers well from one medium to another.

Forms present lots of little problems:

• How many characters can be typed in the field - this is often not clear, and there is no inherent indication of this (needs to be explicitely built in by the interaction designer) - this violates the feedback principle.
• What format is supposed to be used - particularly for things like dates
• Lack of information - what if all information known by the form isn't available immediately. The design may not let you process beyond the dialogue box

However, this system does have strengths. If the system is well-designed, any novice can use them and it has a strong analogy to paper forms, and the user can be led easily through the process. However, they are easy to design badly, and hence confuse and irritate the user.

The heart of modern graphical user interfaces (GUIs, sometimes referred to as WIMP - Windows, Icons, Mouse/Menus, Pointers) or "point and click" interfaces is direct manipulation (note, GUI and WIMP themselves are not interaction styles).

The idea of direct manipulation is that you have objects (often called widgets) in the interface (icons, buttons, windows, scrollbars, etc) and you manipulate those like real objects.

One of the main advantages of direct manipulation is that you get (almost) immediate natural feedback on the consequences of your action by the change in the object and its context in the interface. Also, with the widgets being onscreen, you are provided with clues as to what you can do (doesn't help much with items buried in menu hierarchies). The nature of widgets should tell you something about what can be done with them (affordance), and with widgets, many more objects can be presented simultaneously (more than menus).

However, interface actions don't always have real world metaphors you can build on (e.g., executing a file) and information rich interactions don't always tend to work well with the object metaphor and direct manipulation becomes tedious for repetitive tasks, such as dragging, copying, etc...

The ultimate goal of the conversational metaphor is for us to have a dialogue with the computer in our own natural language. This might be written, but spoken seems easier. Voice in/voice out systems have been developed, where the computer recognises what you say (using speech recognition technology) and produces a response, spoken via text-to-speech (TTS).

This kind of system should be "walk up and use", just speak your own language and this should suit both novices and expert users. This isn't yet technologically feasible, as the AI is largely fake and can only recognise key words. If speech recognition fails, it is easy to get into nasty loops, where it is not clear as to how you get out again (this is the feedback principle). There is also the problem of different accents and recognising older voices, and privacy and security issues are introduced.

See MSD.

The advantages of the waterfall model is that each stage produces a set of specifications that can be handed on to the next stage, and the work is compartmentalised into clear chunks. There is some feedback from each stage back to the previous one, but this slows down the whole process.

With the waterfall model, it is very problematic to change the requirements as the project develops, and is difficult to consider alternate designs. Any change in design ideas are difficult to implement, as the system must be recoded.

See MSD.

This model never really gained acceptance in the long-term in software engineering, or in human-computer interaction.

Here, evaluation is at the centre of the design process; we must be doing evaluation at all stages of the design process, and we move around the star from analysis clockwise.

We can consider two modes of design activity:

• Analytic mode - top-down, organising and formal; working from the systems view towards the user's view
• Synthetic mode - bottom-up, free-thinking, creative and ad-hoc; working from the user's view to the systems view

Interface designers need to flip between these two modes.

The star lifecycle captures something important about the real design process - that we need to work from both ends of the problem, from the user's perspective and from the perspective of the technology, but it doesn't tell us very much about the ordering and nature of the different processes involved in design and how to move a design forward.

This method was developed by Jack Carroll and colleagues and has the advantage of there being numerous opportunities to include user requirements and alter specifications as new knowledge is developed from studying prototypes. You are not committed to genuine coding until a satisfactory design has been prototyped.

The problem with this method is that you do not always know when to break out of the cycle. Evaluating a prototype may not be accurate and may not reveal all of the problems in the prototype.

The UE lifecycle takes a combination of HCI and software engineering methods, and is more useful for big systems. It is not as much a new concept (despite being recent - Mayhew, 1999), but it brings together lots of previous ideas and takes the best bits from previous methods.

Here, the lifecycle is divided up into three phases.

Requirements Analysis

Design/Testing/Development

Installation

This method has the advantage of stressing lots of user testing and involvement and divides the development into clear phases of an increasingly realistic design, but it is difficult to change requirements as you learn about the user's understanding of, and interacting with the system.

This was based on further development by Carroll and Rosson in 2002 and evolved from the difficulties of using multi-disciplinary teams for system development. How can all these people work together and understand each others visions and problems? This method believes you can do everything up until the implementation phase.

This system is based around scenarios, or stories, which elaborate the design and then proposed design solutions are written for the problems. These start looking quite simple, but increase in complexity as the design is articulated.

All scenarios have characteristic elements:

• Setting - situation elements that explain or motivate goals, actions and relations to the actors
• Actors - humans interacting with the technology or other setting elements or personal characteristics relevant to the scenario
• Task goals - effects on the situation that motivate actions carried out by the actors
• Plans - mental activity directed at converting a goal into a behaviour
• (User) Evaluation - mental activity directed at interpreting features of the situation
• Actions - observable user behaviour
• Events - external actions or reactions produced by the computer, or other features of the settings; some of these may be hidden to the actors, but important to the scenario

The first step is to write a problem scenario, which is representative of the current situation. There may be many of these to cover a variety of actors/situations, etc. They are not called problem scenarios because they emphasise problematic aspects of current practices, but because describe activities in the problem domain.

The next step is claims analysis, where each feature is identified and then the positive and negative effects listed. Claims analysis may lead to elaboration of scenarios. To move forward from the claims analysis, you should only have positive features.

The next step is to write activity scenarios - what and how an activity is going to address a problem is the focus of activity scenarios. Only high levels of abstraction need to be considered, no specific details. The design team first introduces concrete ideas of new functionality and new ways of thinking about users' needs. As in other steps in the process, a claims analysis is generated to help identify tradeoffs as you move forward with prototypes.

Further levels involve rewriting the scenarios at higher levels of detail (or lower levels of abstraction, depending on how you want to look at it), and dealing with new issues which are caused. Information and interaction design scenarios specify representations of a task's objects and actions that will help make users perceive, interpret and make sense of what is happening.

The goal of interaction design is to specify the mechanisms for accessing and manipulating the task information and activities.

See MSD.

After all of these stages, we can move onto UML, etc and treat it like a classic design problem.

This does have the advantage of allowing quite detailed development of the design without committing to coding or prototyping and makes it easier to change requirements and consider alternative designs, however, we must consider the disadvantages of not giving the users hands-on experience with even lo-fidelity prototypes which may inform the views of the designers. All prototypes are created after the design has been set. It is difficult to measure against usability goals until late in the process.

Questionnaires are produced in a written format that requires very careful work (piloting and refining a questionnaire) to ensure that the questions are absolutely clear to all respondents and that you are collecting all the information you need.

Questionnaires are good for when the issues you want to address are well-defined (e.g., finding out problems users are having with a current system to be improved) and you need to get information from a lot of people in a way that can be relatively quickly analysed. Conversely, they are less good for situations where the questions are not very well defined (e.g., when developing a very novel system) and the response rates for questionnaires are very low - 40% is considered good.

The better a questionnaire is, the more likely it is to have a higher return rate, and your data will also be of better quality. Following up questionnaires with reminder letters, phone calls and e-mails, as well as offering incentives such as offering raffles all improve response rates also. The length of the questionnaire should be commensurate with the importance of the topic, but 20 minutes is too long for any questionnaire.

There are three types of questions on questionnaires:

• Open-ended - a completely free response. This is good when you want to elicit all kinds of information, want respondents to be creative (many enjoy this) and you don't know what they might say. It is more difficult to analyse, however, particularly if you have lots of respondents - you need to develop your own categories to group the answers, and the problem is then to decide whether two differently worded answers are the same category. This is often biased, so two people often need to look at the answers.
• Closed questions - These are yes/no-style questions with a limited set of options, often with "Don't Know" or "Other" as an option (these answers are desired in less than 10% of cases, however). The options for closed questions can often be decided from doing a pilot open-ended question and then using the answer grouping from that to decide the closed question answers. Closed questions are easy for respondents to answer and good for getting lots of data that's easy to process and understand. It is fundamentally categorically in nature, however - answering "what" type questions, not on how people found something to use. These quantitative questions are very useful in designing systems - you may want to compare different designs, or compare one design as it evolves over time. Eliciting extreme information from people ("What did you like most/least about X?") is also good.
• Likert scales - These are a very simple and neat way of measuring "how much" type questions. The Likert (or rating) scale gives a scale from 1-5, where 1 is agree and 5 is disagree (or could use 5, 7 or 9 gradations, depending on how much you want the respondents to discriminate). Likert scales are very useful and should be used for everything (according to Helen). They can be combined with open-ended comments where users can justify their ratings. The disadvantage from Likert scales is that people can express no strong preference either way. When you're comparing multiple examples and want to know the best one, asking people to rank their options in order can often be more useful.

There is a checklist to creating a good questionnaire:

• Make questions clear - avoid double-barreled questions (e.g., "Do you think it'll be useful to make the icons or buttons larger?" - does the answer refer to the icons or the buttons?).
• Make questions specific - the generalising should be done by the analyist, not the respondent. (e.g., "How many hours do you use the Internet a week?" - the respondent here is trying to average in their head. It's better to ask just about yesterday and then multiply by 7 and take into factors like weekends - this is the same as what the respondent is doing, but you're likely to do it better with a better process).
• Think about the ordering of the questions - Ask easy questions first and get the respondent relaxed and lulled into a false sense of security. They are more likely to complete a questionnaire once they have started it. Personal questions should be asked last (people don't like answering these, but if they're completed the rest of the questionnaire, they probably will). Does answering one question lead you into answering another one in a certain way?
• Make scales intuitive, clear and consistent - if you use numbers, 1 = low agreement and 5 = high. This is intuitive. Also, when asking closed questions, questions should be either positive or negative - mixing them up occasionally does keep respondents on their toes, however!
• Avoid technical or HCI jargon (e.g., when navigating about a website, a lot of people link navigating with ships and compass).
• Provide clear instructions on how to complete the questionnaire and specific questions (people will ignore you, but at least you tried). Examples often help.
• Clear layout is important and helpful - enough room is required for open questions, but a balance is required between a lot of whitespace (which can give an intimidatingly long questionnaire) and room.

Interviews can range from the very formal, rather like a questionnaire given face-to-face, to very informal, where the interviewer has a set of topics, but has a very free ranging conversation with the interviewee. A lot of the principles from questionnaire design still apply to interviews - clear question design, ordering of questions, etc, etc.

Interviews allow you to develop a relationship with the interviewee, and because you are talking, you can elicit much more information, as well as explaining things the interviewee might not understand and being able to tailor the questions to the interviewee much more easily.

Interviews are much more time consuming than questionnaires and the interviewee might feel a bit intimidated and may not reveal personal information. This method is also more prone to "researcher bias" - with the interviewee telling the interviewer what they want to hear, and not necessarily what they don't wish to hear.

Normally, 5-7 people are brought together, usually from a particular user group to discuss a particular system/problem. They are facilitated by a researcher who has a list of questions/topics to be covered and who needs to discreetly guide the discussion. Focus groups normally last for about 2 hours and are a fairly time-efficient way of eliciting information and bouncing ideas around - people can spark ideas off each other. It is less good at eliciting personally sensitive data, however.

There are three basic types of observational studies:

• "Quick and dirty" observation (good for initial user requirements elicitation) - right at the beginning of the design process, you might want to get a general idea of how people are using a current technology (and the problems they are having), or how they might use a new technology - "quick and dirty" observation is a good way to accomplish this.
• Observation in a (usability) lab - if you want to study in detail how users interact with technology, you want to have them in a controlled laboratory setting. This allows us to observe how users undertake a range of tasks which are thought to be vital, to gauge performance and "affective" reactions (how they feel).
• Observation in field studies - for a variety of reasons you might need to take a really detailed look at the current use of technology (a "quick and dirty" observation may not reveal the problems). Also, when an early prototype is available, this method can be used for an evaluation use, where the prototype is used in a real world context.

There are also different levels of observer involvement; "insider-outsiderness". The most extreme level of this is that of total outsider, where the outsider does not participate in the situation at all (doesn't help people who get stuck or ask them about their experience), and the other extreme of this is marginal/complete participation, for example where an observer becomes part of a company and uses their system for a while, participating in the processes of the system. In the middle there are observers who also participate, by creating certain configurations to see how participants react.

Marginal/complete participation in observation is referred to as ethonography - a term from anthropology where anthropologists go and live in a different culture and participate in their lives in order to understand their customs. In both anthropology and HCI, this is a big undertaking, with researchers sometimes participating in the use context of a situation for months, if not years. It can be done on a small scale, however, with observing and interviewing users and the researchers trying out the technologies for themselves.

Observation can be considered as varying on a scale from hypothesis-driven (where you have a clear idea what you are looking for, what the problem is and what your theory is) to holistic (where you are not sure what's going on, what the problem is and what you are going to find), and this will influence exactly how your observations are done. With hypothesis-driven, it is clear what specific behaviours you should be observing in classrooms, whereas with holistic, you will need to observe everything and make sense of it later - there is a serious danger of drowning in the information you collect, however.

There are different methods to observe:

• Notes plus still camera - Taking handwritten notes seems the most basic way of noting down observations, but it is very difficult to write and observe at the same time, and very difficult to write quickly enough to give much useful detail later. A dictaphone could be used in a public situation to make up for this, and combining it with something like a hands-free kit can make it look like you're on a mobile phone, decreasing your obviousness. Taking still pictures of situations can be very useful, but it may upset observees in public situations (and has ethical implications). This method is okay for use in labs, and is possible, if difficult, in public.
• Checklists of behaviours - A list is made up of behaviours of interest, and all you need to do is simply check off whether and how frequently they occur. This is good is the observation is hypothesis-driven, but you may need a pilot study to work out the checklist. Once the checklist is working, it saves an enormous amount of effort collecting and later analysing the data.
• Audio recording plus still camera - Small, inconspicuous digital tape recorders are useful, but this only works if the participants are talking, and even then it is difficult to work out what is happening without visual record; still shots can help. If you choose to transcribe the whole conversation, this can be very time consuming, but it may not be necessary - relevant points and problems can be picked out and a summary generated.
• Video - This is good for capturing all visual and aural information, but it can be intrusive and make people feel self-conscious (although they usually forget about the camera when they are focussing on the technology). Video is very time-consuming to analyse (1 hour video = 100 hours analysis), but as with audio recording, it may be summarised with only important segments analysed in detail. Additionally, videoing public places may not capture all available information.

Frameworks have been developed for doing observation, as when performing observation, especially holistic studies, there is a lot of information to record and the appropriate information needs to be captured.

A very simple framework for field observation is the who/where/what framework. Who is the person using the technology (age, gender, type, etc) and are others involved? Where are they using it (the place) and what are the important characteristics of that environment? What is the thing/task of what they are doing?

A more complex framework has been developed based on the simple one above which considers more factors:

• Space: What is the physical space and how is it laid out?
• Actors: What are the roles and relevant details of those involved?
• Activities: What are the actors doing and why?
• Objects: What physical objects are present, such as furniture?
• Acts: What are specific individuals doing?
• Events: Is what you are observing part of a special event?
• Goals: What are the actors trying to accomplish?
• Feelings: What is the mood of the group and individuals?

In laboratory observation, one of the frustrating things is that you can get a lot of detail (particularly from video analysis) and still not know what is going on. In particular, we have no insight into the user's mental model of what is happening. A variation of pure observational technique has been designed to deal with this situation - this is called the "think aloud" or "concurrent verbal" protocol, where the person is asked to say out loud everything that they are thinking and trying to do, so that their processes (and mental model) are externalised. It is usually very straightforward to get the person to do this; they can be gently prompted if they go silent.

Alternatives to this include getting them to work in pairs and describe what they are doing to each other. In some situations, you can get one person to teach another how to do something.

For some tasks, it can be disruptive to get a person to talk whilst doing the task (e.g., something that requires a lot of concentration, or when talking is part of the task), so a variation on the above called retrospective verbal protocol can be used. Here, you video the person and immediately (while it's still fresh in their minds) get them to watch the video and talk through what they are thinking at the time. However, you often lose a lot of the finer detail of the thoughts going through your head, and it's often embarrasing to watch a video of yourself.

In some instances, actually observing people may be too intrusive. We can ask people to keep diaries of their behaviour, but this often falls off as people lose interest, and structure and incentives are needed. Another method is interaction logging, taking some kind of automatic log of the key presses, mouse movements and clicks, which gives us on a somewhat more meaningful level the menu items chosen, web pages and links visited, etc... This information by itself is not that useful - did the user find that page helpful/interesting/etc..., so it needs to be combined with video or audio recording from user, preferably with verbal protocol data.

Logging is a completely unobtrusive measure - the user does not need to know it is happening. Psychology has long liked the idea of unobstrusive measures like this, measures which do not interfere with the behaviour at all. HCI needs more useful unobtrusive methods.

If we involve people in research, we have an ethical obligation to inform them that we are collecting data from them and using it. This is often a problem with observational data, particularly if it is collected in a public place and this ethical obligation is ignored. Unobtrusive methods are attractive specifically because they do not upset the naturalness of behaviour - informing people would definately do so. There is no easy way out of this, but the best rule of thumb is to get people's consent whenever you can. Collected research should be anonymous.

Heirarchial tasks analysis (HTA) is concerned with observable behaviour and the reason for this behaviour. It is less detailed than some techniques, but is a keystone for understanding what users do.

HTA represents tasks as a hierarchical decomposition of subtasks and operations, with associated plans to descibe sequencing:

• tasks/subtasks - activities to achieve particular goals/subgoals
• operations - lowest level of decomposition, this is the level defined by the stopping rule
• plans - these specify the sequencing of activities associated with a task and the conditions under which the activities are carried out

A HTA can be represented by a structued, indented text, or using a structured chart notation.

A text variant of the above structured chart may be:

• 0. To photocopy a sheet of A4 paper:
  1. Enter PIN number on the photocopier
  2. Place document face down on glass
  3. Select copy details
     1. Select A4 paper
     2. Select 1 copy
  4. Press copy button
  5. Collect output

• Plan 0: Do 1-2-4-5 in that order; when the defaults are incorrect, do 3
• Plan 3: Do any of 3.1 or 3.2 in any order; depending on the default settings

We can consider different types of plan:

• fixed sequence (e.g., 1.1 then 1.2 then 1.3)
• optional tasks (e.g., if the pot is full then 2)
• wait for events (e.g., when the kettle boils, 1.4)
• cycles (e.g., do 5.1-5.2 while there are still empty cups)
• time-sharing (e.g., do 1 and at the same time, do 2)
• discretionary (e.g., do any of 3.1, 3.2 or 3.3 in any order)
• mixtures - most plans involve several of the above

Is waiting part of a plan, or a task? Generally, task if 'busy' wait - you are actively waiting or a plan if the end of the delay is the event, e.g., "when alarm rings", etc...

To do HTA, you first need to identify the user groups and select representatives and identify the main tasks of concern. The next step is to design and conduct data collection to elicit information about these tasks:

• The goals that the users are trying to achieve
• The activities they engage in to achieve these goals
• The reasons underlying these activities
• The information resources they use

This steps can be done with documentation, interviews, questionnaires, focus groups, observation, ethnography, experiments, etc...

The data collected then needs to be analysed to create specific task models initially. Decomposition of tasks, the balance of models and the stopping rules need to be considered. The specific tasks models then should be generallised to create a generic task model - from each task model for the same goal, produce a generic model that includes all the different ways of achieving the goal. Models should then be checked with all users, other stakeholders, analysts and the process should then iterate.

To generate a hierarchy, you need to get a list of goals and then group the goals into a part-whole structure and decompose further where necessary, applying stopping rules when appropriate. Finally, plans should be added to capture the ordering of goal achievement. Some general things to remember about modelling are:

• Model specific tasks first, then generalise
• Base models on real data to capture all the wonderful variations in how people do tasks
• Model why people do things, as well as how
• Remember, there is no single, correct model
• Insight and experience are required to analyse and model tasks effectively and to then use the models to inform design

When you are given an initial HTA, you need to consider how to check/improve it. There are some heuristics that can be used:

• paired actions (for example, place kettle on stove can be broken down to include turning on the stove as well)
• restructure
• balance
• generalise

There are limitations to HTA however. It focuses on a single user, but many tasks involve the interaction of groups to people (there is a current shift towards emphasising the social and distributed nature of much cognitive activity). It is also poor at capturing contextual information and sometimes the 'why' information and can encourage a focus on getting the model and notation 'right', which detracts from the content. There is a danger of designing systems which place too much emphasis on current tasks or which are too rigid in the ways they support the task.

There are many sources of information that can be used in HTA. One such is documentation - although the manuals say what is supposed to happen, they are good for key words and prompting interviews, and another may be observation (as discussed above) and interviews (the expert: manager or worker? - interview both).

GOMS is an alternative to HTA that is very different. It is a lot more detailed and in depth and is applied to systems where timing and the number of keystrokes are vital. It is more complex to use than HTA, and it is not common. GOMS is an acronym for:

• Goals - the end state the user is trying to reach; involves heirarchial decomposition into tasks
• Operators - basic actions, such as moving a mouse
• Methods - sequences of operators or procedures for achieving a goal or subgoal
• Selection rules - invoked when there is a choice of methods

Conceptual design is taking the requirements and turning them into a description of the proposed system in terms of a set of integrated ideas and concepts about what it should do, how it should behave or look in a way that will be understandable by users and other stakeholders. It is not yet a real detailed, fully specified design, but is the "general idea".

At this early stage of design, we should be considering alternate design ideas, so:

• Keep an open mind but never forget the users, their tasks and environments
• Discuss ideas with all the stakeholders as much as possible
• Use lo-fi prototyping to get rapid, but well-informed, feedback from stakeholders
• Iterate, iterate, iterate

The big decisions that need to be made in the conceptual design stage is what interaction mode/style to use (which will be the most suitable for all types of users, tasks, environment, etc...), what interface metaphor to use and the interaction paradigm (desktop, wearable, mobile). We also need to consider what functions will the system perform - task allocation, what will the system and user do, and how will these functions relate back to each other - temporal ordering - and make themselves available to the user.

After working with the conceptual design, hopefully evaluating it, considering alternatives and refining it, eventually you are happy, or you run out of time, the next stage of design is required - physical design.

Physical design deals with the detailed issues of both the physical interface and the conceptual interface, but what will the detailed screen layout, etc, look like? The number of screens and icons and their general functionality will have been decided at the conceptual design stage. In the physical design stage, you also need to consider the temporal arrangement of actions and their reactions. High fidelity prototypes can be used at this stage to see how different users react to different presentations and different detailed dialogue arrangements.

Determine the goals - what do you really want to find out, which determines the method you use, the people you involve, etc...

Explore the questions - often a high level question (e.g., "why don't people like this system?") needs to be broken down to find concrete things to ask or measure - this is called operationalising the problem.

Choose the evaluation paradigm and techniques - a number are available, and we will look at this in more detail later.

Identify practical issues - e.g., appropriate users need to participate in any user-based evaluation (selecting the right level of expertise, age range, gender mix, etc...). You need to consider the tasks the user will be doing with the system, the facilities and equipment available, any schedule or budget constraints to be considered, and the expertise of the evaluation team.

Decide how to deal with ethical issues. It is very important to consider this and to deal with participants involved in evaluations ethically. The British Pyschology Society's "Ethical Principle for Conducting Research with Human Participants" is a good guide to consult before conducting experiments with humans.

• You must inform participants approximately what they will be asked to do before getting their consent (usually by signing a consent form).
• All information is confidential and participants should be told this - you can report results, but not in a way that identifies individuals.
• Participants should not be subjected to undue stress (tasks that are too difficult to do), boredom (evaluations should be interesting and informative) or fatigue (sessions should not be too long).
• Participants need to know that they can leave an evaluation at any time if they are unhappy.
• Participants should be reimbursed appropriately for their time, but they shouldn't be bribed to act against their better judgements.

Evaluate, interpret and present the data - is your data reliable, i.e., would you get the same results if you collected them on another day, with another group of users (probably not the exact same results, but the conclusions should be the same). The validity of the data also needs to be checked; is it really measuring what you really want to measure?

There are the "gold standard" of evaluations - the key is the control you hold over the situation. Usually conducted in a usability lab, but this is not essential. Any controlled situation (where the information available to the users, the tasks and the data collected are controlled) will do, so long as the situation and results are shown to be replicable.

This is artificial, but it is so for a reason. By keeping everything constant apart from the one thing you are interested in, you get rid of as much extraneous variation as possible, and concentrate on what you are interested in. This form of evaluation does need to be complemented by evaluation in realistic situations.

Control is needed due to confounding hidden variables - a problem using correlational, natural data as you do not know what other relationships exist that are not explicit.

Basic Setup

Experimental Setup

Grice's conversational maxims (not exactly rules, as they can be broken very easily, but intuitively we know it's odd when someone does so) are principles that govern conversations.

• Maxim of Quantity - Make your contribution to the conversation as informative as necessary, but do not make your contribution to the conversation any more informative as necessary.
• Maxim of Quality - Do not say which you believe to be false or that for which you lack insufficient evidence.
• Maxim of Relevance - Be relevant (i.e., say things related to the current topic of conversation).
• Maxim of Manner - Avoid obscurity of expression or ambiguity. Be brief, orderly and appropriately polite.

Pitt and Edwards have developed many speech-based interfaces and propose the additional guidelines:

• Limit the number of choices available at each stage of the interaction to an absolute minimum (like menu design)
• Keep each utterance as short as possible (Grice - be brief)
• Try to anticipate what the user is most likely to need at each stage of the interaction and present that information first (often called front-loading information)