FFP Score - Measuring Fitness for Purpose

A Practical Definition of Fit for Purpose

Fitness for purpose is the measure by which usability and legislative compliance are established. For the purpose of this guidance a practical definition would imply that a design is fit for purpose if the kiosk:

• Performs in the manner intended by the designer.
• Performs in the manner expected by the user.
• Is accessible and usable for all its target users.
• Is so designed that its size and shape facilitates good fit and reasonable comfort
  for the user.
• Presents no undue risk to the user.
• Presents no undue risk to other people who interact with it, including members of staff, members of the public, children, installers and maintainers.
• Allows for reasonably foreseeable misuse (e.g. it is not intended to be used as a ‘climbing frame’ but could be by an unattended child and, therefore, should not topple over under such circumstances).
• Continues to operate satisfactorily for items 1-7 above throughout its intended
  life cycle.

Fitness for purpose can be assessed in a descriptive, piecemeal way but it can be difficult to determine the impact of improvement measures. Inclusive design – and the requirements of fitness for purpose – demands consideration of a host of factors. It is not uncommon in such multi-factorial situations that an improvement in one aspect can have a detrimental effect in another.

For example, providing headphones on a lead to support screen reader output for the visually impaired may increase the risk of strangulation of children or other vulnerable members of the public. In an attempt to resolve this difficulty and to manage and monitor the fitness for purpose criteria throughout the design cycle an approach called ‘FFP Score’ can be used.

FFP Score is based on input from all those involved in a project, from the originator to the end-user. It draws upon information and data which is available through the literature and project-specific trials to assess issues such as dimensions and ranges of adjustment, usability of controls and displays, maintenance provision and suitability for use.

FFP Score has five stages; the first three establish the criteria against which the equipment will be assessed, the fourth is the appraisal itself and the fifth stage is intended to enable monitoring of the design as it evolves, is implemented and used in-service.

FFP Score - Measuring Fitness for Purpose

Stage 1 : Creating a Stakeholders’ Wish List

It is generally accepted that if a kiosk design is to succeed then the stakeholders should be involved throughout the design or selection cycle. The first stage of the FFP Score process is, therefore, to ask representatives of all those involved in the project (designers, IT, accessibility experts, customers, ergonomists etc.) to provide a ‘wish list’ of all the features they would like to see in the kiosk. Each representative is also asked to use the following scale to rate how important they consider each criterion:

5 - Essential (non-negotiable, must be included)
4 - Very Important
3 - Important
2 - Slightly Important
1 - Not Important (but would like to be included anyway)

The mode (most frequently occurring) rating is identified for each criterion and a single prioritised wish list representing the consensus view is presented back to the stakeholder representatives for comment and review.

It is most convenient and beneficial to conduct this stage in a group setting as it allows for interactive discussion, which is particularly helpful when reviewing the consensus wish list to confirm priorities in the case of ‘surprises’ and to decide whether additional items should be added to the list.

As an example of output from Stage 1, the table below shows the prioritised consensus stakeholder wish list produced as part of the development of the interactive information kiosk.

Stage 2 : Developing a Human Factors Specification

The consensus wish list is by its very nature somewhat basic and in need of development if it is to be of value to the project in considering and monitoring individual design factors. The second stage of FFP Score is, therefore, for the ergonomist with their specialist knowledge of usability and accessibility to categorise and develop the list into more specific criteria. As an example, drawing from the earlier prioritised wish list, ‘comfortable use of input devices’ can be broken down to include, criteria such as:

• Keyboard should have a QWERTY layout for familiarity.
• Keyboard should be designed according to ergonomics principles.
• Non-keyboard input device (e.g. track ball) should be designed according to ergonomics principles.
• Touch screen should enable ease and accuracy of use.
• There should be a speech activation facility.

Progressive redescription can continue until each design aspect is clearly specified to the level required. For example the ergonomics principles for each aspect of keyboard design could be listed separately (key spacing, key depression force, material finish etc.). It is, however, usually sufficient to only break down the wish list to the first level. Furthermore, a very large and unwieldy set of criteria is likely to result from further redescription.

To decide on the relative importance of each of the specific criteria, the ergonomist applies the same rating scale as before. For example, if a QWERTY keyboard is considered to be an essential part of the requirement for ‘comfortable use of input device’, it would receive a rating of 5.

The product of the stakeholder wish list rating and the ergonomist’s breakdown list rating provides an overall weighting of importance on a scale of 1-25; 1(1x1) being not important, 25 (5x5) being essential.

The conclusion of this second stage is for the weighted breakdown list to be verified by the stakeholder representatives. When the list is finally agreed it forms the ‘human factors specification’ (HFS) for the kiosk. From this point, all design and selection decisions should derive from or be compared to this HFS.

Stage 3 : Specifying HFS Criteria

The third and final stage, of preparing FFP Score ready for appraisal is to identify from the literature or through trials any specific criteria against which designs should be created or compared (e.g. actual dimensions for clearances).

Stage 4 : Conducting Appraisals

At the appraisal stage, any concept, prototype, end product or in-service kiosk is evaluated against each of the specific criteria of the HFS using the following bipolar rating scale:

7 - Very Good (equivalent to human factors best practice)
6 - Good
5 - Slightly Good
4 - Minimum Criteria Satisfied
3 - Slightly Poor
2 - Poor
1 - Very Poor (equivalent to failing most human factors criteria)

To determine the performance of the design and to make accurate comparisons of each criterion it is necessary to multiply the weighting value with the appraisal rating. As this yields big numbers, it is simplest to first divide each weighting by the sum of all the weightings – it then becomes a fraction of a total weighting value of 1. From this there are several ways in which the data can be used, for example:

• Totalling all the multiplied scores will provide a score out of 7 which indicates the overall fitness for purpose.
• The multiplied scores can be viewed in sub-sets (e.g. ‘input devices’, ‘safety’, ‘maintenance’) to determine how well each is performing.
• The multiplied values can be shown as a chart which shows the overall and specific criterion performance.

The chart below is an example of the FFP Score appraisal output:

• The straight green line toward the top of the chart represents ‘ergonomics best practice’ (rating 7).
• The straight amber line toward the middle of the chart represents ‘minimum standard’ (rating 4).
• The straight red line toward the bottom of the chart represents ‘fails most ergonomics criteria’ (rating 1).
• The undulating thicker black line represents the appraisal of each criterion shown on the x-axis.

Stage 5 : Monitoring Kiosk Development

In a selection task to appraise different kiosks, FFP Score will serve to definitively compare overall scores and the performance of key factors. However, in a design process, a further stage of use offers additional benefit. FFP Score can be conducted at every major stage of the design cycle to monitor the effectiveness (or otherwise) of design changes and ultimately to assess how the kiosk actually performs in the ‘real world’ in its intended environment.

all content © david hitchcock limited, 2010.