Eindhoven University of Technology MASTER Problem solving behaviour of microscopists : a case study Spierenburg, Marc

(1)

Eindhoven University of Technology

MASTER

Problem solving behaviour of microscopists : a case study

Spierenburg, Marc

Award date:

1992

Link to publication

Disclaimer

This document contains a student thesis (bachelor's or master's), as authored by a student at Eindhoven University of Technology. Student theses are made available in the TU/e repository upon obtaining the required degree. The grade received is not published on the document as presented in the repository. The required complexity or quality of research of student theses may vary by program, and the required minimum study period may vary in duration.

General rights

Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.

• Users may download and print one copy of any publication from the public portal for the purpose of private study or research.

• You may not further distribute the material or use it for any profit-making activity or commercial gain

(2)

Institute for Perception Research PO Box 513, 5600 MB Eindhoven

Rapport no. 881

Problem solvin~ behaviour of microscopists: a case study M.M. Spierenburg

08.12.1992

(3)

Problem solving behaviour of microscopists: a case study

Eindhoven University of Technology

Graduation project at the

Institute of Perception Research (IPO) March 16 - November 30, 1992 Student:

Marc Spierenburg Mentors:

Dr. Ir. M.D. Brouwer-Janse Philips Research Laboratories,

Institute of Perception Research (/PO) Drs. R. Gobits

Eindhoven University of Technology Dr. H. de Ridder

Institute of Perception Research (/PO)

(4)

Problem solving behaviour of microscopists: a case study I

Summary

Plan

At Philips Electron Optics, a project is started to design a new user interface for the CM series Scanning Transmission Electron Microscopes (STEM). A major goal of the project is (1) to reduce the training time and to assist users who have a moderate level of

experience, (2) to gain optimal usage and benefit of the technical capabilities of the instrument. Another issue to address is whether different domain disciplines can be satisfied with the same user interface. A starting point in the approach to achieve these goals, is to investigate the problem solving behaviour of microscopists in the context of use. The results will be supportive to the design of the new user interface. An user

interface which matches the problem-solving behaviour of users facilitates the operation of a STEM.

A case study is performed to investigate the problem-solving behaviour of users of a STEM. Four subjects perform a representative task with the STEM. They have different backgrounds and different levels of experience in using electron microscopes. The four subjects have the following backgrounds: (1) material science expert having two years of STEM experience, (2) material science expert who worked as a student with a STEM for one month six years ago (3) material science expert having ten years of STEM experience and (4) clinical chemist having fifteen years of STEM experience and who is an expert in the area of biological specimens. During the experiment subjects are asked to 'think aloud'. Subject's simultaneous verbalizations while performing the task are assumed to come as close as we can get to their actual underlying thinking processes. Each task takes about two hours to perform. The tasks are videotaped.

Task

Subject 1 (material science expert with two years of experience) investigated a wool sample. Subject 3 (material science expert with ten years of experience) investigated two different materials; Silicon Nitride (a difficult task) and Gallium Arsenide (a routine task).

Subject 2 (non-expert STEM user) started with a tutorial session. Afterwards she operated the STEM on her own, but she could use the advise of an expert when she reached a dead-lock situation. Subject 4 (STEM expert in the area of biological specimens) investigated a liver specimen. During the experiment he only worked for part of the time at his task.

(5)

Problem solving behaviour of microscopists: a case study II

Analyses.

A verbatim transcript is made of the verbal output that was recorded on videotape. An analysis is made of the problem-solving strategies of subjects 1 and 3 (material science experts having two respectively ten years of experience). For that purpose the raw protocol is divided into (1) propositions, simple sentences occurring in the context of the connected discourse and (2) episodes, propositions which are grouped together for contextual reasons.

The propositions are coded corresponding to a theoretical problem-solving model that is domain independent. The propositions are classified according to problem-solving

strategies (see appendices B and D). Apart for the coding, we made a structure diagram to give an overview of the different actions the subjects performed at the electron microscope (see appendices A and C).

Data of the experiment with subject 2 (material science expert, STEM non-expert) is used to analyze when dead-locks are encountered. Data of subject 4 is used to make an

overview of the specific differences in use of STEMs between material scientists and biologists.

Results and discussion.

An experienced electron microscopist performs his task having a total overview of the problem-solving process. Goals and sub-goals are gone through in a structured manner.

A less experienced microscopist solves problems with a limited overview of the problem- solving process. He attacks each appearing problem directly. The broad outline of the problem-solving process is not always recovered after a local problem has been solved.

To solve problems successfully with an electron microscope, a certain level of overview is necessary. When the level of overview over the problem-solving process is insufficient, users get stuck. Changes made by a novice microscopist who has reached a dead-lock situation do not help to resolve the problem. On the contrary they make it worse. One cannot operate a STEM by knowing rules by heart. An electron microscopist has to have insight in what is done and how it is done when operating a STEM.

Material scientists use STEMs at a clearly different way than biologists/ pathologists.

Material scientists use the ultimate specifications (e.g. maximum enlargement and resolution) of an electron microscope. They are interested in the physical operation of a microscope and the way ultimate specifications can be achieved. Material scientists examine specimens for four hours or more.

Subject 4 states in a retrospective report that the majority of biologists do not use the maximum specifications of a STEM. Resolution is limited because of the specimen used, not because of STEM specifications. Enlargements used are ten times less than feasible.

Biologists prefer an easy to use electron microscope with little or no alignment: a microscope which does not require technical knowledge for its operation. Specimens are

examined for less than half an hour typically. Special features they like to be incorporated in a new STEM are: high throughput (time needed from specimen to diagnosis) and the possibility to compare different specimens simultaneously.

(6)

Problem solving behaviour of microscopists: a case study III Conclusions and recommendations.

A beginning microscopist lacks a total overview during the problem-solving. He will reach dead-lock situations. To avoid these dead-locks, a microscopist has to be able to solve detailed problems in a structured manner. The total problem-solving process has to be evaluated regularly.

To operate a STEM effectively, the microscopist has to use high level problem-solving strategies, for instance: Selection, Hypotheses, Pattern Extraction and Evaluation. He needs to have a good insight in the physical processes which take place in the STEM in order to use these high level strategies. A dynamic visualisation of the physical processes which take place in the STEM can be advantageous in forming a good insight of the physical processes. Physical processes in a STEM are however very complex. A

visualisation has to be comprehendible but at the same time realistic. Appropiate represen- tations at a level of abstraction that matches the mental model of the user is of crucial importance.

(7)

Problem solving behaviour of microscopists: a case study IV

Preface

This report is written in the scope of my final project at the Eindhoven Technical University. The project is performed at the Institute for Perception Research (IPO) in cooperation with Philips Electron Optics. It concerns a case study to investigate the problem solving behaviour of microscopists.

v

During this project, I received help from a lot of people I would like to thank for their support. First I would like to thank Maddy Brouwer-Janse for her interest, inspiration and help during this project. I would like to thank Rudy Gobits and Huib de Ridder, my faculty mentors, for their support.

People at Philips Electron Optics were very supportive and co-operative. I appreciated the patient explanations on electron microscope matters and interest of Auke van Balen, Ulrich GroB, Filip GroB, Hugo van Leeuwen and Geurt Wisselink. Without the kind co-operation of Judith Brock, Wim Busing, Michael Felsmann, Marjoleine Haakma and Max Otten the experiments wouldn't have been possible.

Finally I would like to acknowledge Anita Heister for her patience and support when she had to cope with a burned-out boyfriend.

Eindhoven, December 7, 1992

Marc Spierenburg

(9)

Problem solving behaviour of microscopists: a case study 1

CHAPTER 1 Introduction

Operating a scanning transmission electron microscope (STEM) is a difficult task.

Beginning microscopists follow a course of one week. After that, it takes than about half a year before they are able to operate the microscope independently. A STEM is a precision instrument. Enlargements up to one million times can be made and the resolution can be less than 0.2 nm. To accomplish these values the microscopist has to compensate

mechanical inaccuracies of the STEM. He has to align the microscope by judging the properties of the images.

At Philips Electron Optics, a project is started to design a new user interface for the CM series STEMs. A major goal of the overall project is (1) to reduce the training time and to assist users who have a moderate level over experience, (2) to gain optimal usage and benefit of the technical capabilities of the instrument. Another issue to address is whether different domain disciplines can be satisfied with the same user interface.

In this case study, we take the microscopist as a starting point for a new user interface design. We want to investigate the differences in problem solving behaviour between novice microscopists and experts. We want to know the differences in use of a STEM between different scientific fields. Specific questions arise when a new user interface is developed, e.g. how much a user needs to know about the physical processes in the microscope? What functions may be shielded off?

We performed a case-study where four microscopists do a representative task. We investigated the problem solving behaviour of four microscopists. The results of the case study are supportive to the design of the user interface.

In chapter 2 the basic functions of a STEM are explained. A broad overview of the physical operation of a STEM and the main modes is given. Chapter 3 explains about the design of the experiment. During the experiments, the underlying thinking processes of the subjects are revealed by using the 'thinking aloud' protocol. In chapter 4 the organization of the analyses is explained. A verbal transcription of the audio data is labelled to problem solving strategies. In chapter 5 the results of the case study are presented and discussed. In chapter 6 recommandations for the improvement of the STEM user-interface are given.

(10)

CHAPTER 2 Electron microscopy

§ 2.1 Principles of electron microscopy

The resolution of a microscope depends on the wavelength that is used. Par- tides smaller than the shortest

wavelength cannot electron gun

be made visible. _anode

Hence, the maxi-

beam centering coils mum enlargement

of a light micros- vacuum lock cope is not more

1st condensorlens than approximate

3000 times. 2nd condensorlens

beam tilting coils

In 1926, H. Busch _ION

found that electrons 2nd condensor aperture GETTER

can be diffracted by objective lens ^PUMP

rotation symmetric specimen chamber magnetic fields in a selected area aperture similar manner as diffraction lens light rays are dif-

intermediate lens fracted by glass

lenses. The wave- 1st projectorlens length of electrons 2nd projectorlens can be made much

shorter than visible binoculair (12 x) light. When an elec-

tron beam illumi- vacuum lock

nates a very thin 35 mm camera 00·⁹m or less) focussing screen specimen, magnetic glass plate camera lenses can make an fluorescent screen enlarged picture.

This picture is made 1Q-

visible on a fluor- ^...^~

escent screen, which

e

transforms the electron rays to visible

light. In the early ^Figure1: CM 20 STEM

(11)

1930's a transmission electron-microscope (TEM) with 3 lenses could already enlarge a specimen two times more than a light microscope could. Nowadays TEMs with 8 lenses can enlarge up to one million times.

3

When a specimen is exposed to a dense electron beam, several physical phenomena (depending on the kind of specimen) occur. First, elements with a higher atomic number are less penetrable than lighter elements. That is, the brightness of the spot on the

fluorescent screen is determined by the atomic number. Electrons from the beam are deflected, totally reflected or absorbed by the specimen. When electrons from the beam collide unelastically with the atoms in the specimen, the atoms absorb energy. This energy may be radiated as visible light (electro-luminiscence), as X-rays, or as secondary

electrons, emitted by the specimen. An electron microscope which includes different kinds of sensors to monitor these phenomena is called an AEM (Analytical Electron Micros- cope).

§ 2.2 Operation of an electron microscope

§ 2.2.1 TEM Image mode

The specimen to be examined is placed in a specimen holder. The specimen is placed on a disc-shaped copper grid or other supporting material with a diameter of 3 mm. The

specimen has to be very thin (1 nm or less) to let sufficient electrons through to form a picture. The microscopist has to examine very carefully that no dirt or dust can enter the microscope.

The specimen holder is inserted in the microscope.

The microscope is under vacuum. To decrease waiting time when a specimen is changed, valves are placed in the microscope pillar that maintain separate vacuum areas.

The electron source is either a Tungsten filament, a Lanthanium Hexaboride crystal, or a Field Emission Gun. The amount of electrons emitted can be adjusted by means of the intensity knob.

The electrons are accelerated by a high voltage source (200 kV for the CM 20). A higher electron energy produces a shorter wavelength and therefore a better achievable resolution. However, a high electron energy increases the risk of damage by radiation of the specimen.

The condensor lenses distribute the electron beam evenly over the part of the specimen shown. When the image enlargement increases, the electron beam has to be spread over an equally smaller part of the specimen to keep the brightness on the

Figure 2: Philips CMI2 STEM

(12)

Problem solving behaviour of microscopisls: a case study 4 fluorescent screen constant. In high magnification mode, the enlargement is done in two steps. The objective lens forms an intermediate image which is a 20 to 30 times enlargement of the specimen area under examination. The diffraction lens, the intermediate lens, and both projector lenses form the enlarged image on the fluorescent screen. The objective aperture controls the contrast. The smaller the aperture, the higher the contrast and the lower the amount of light on the fluorescent screen. To select a predetermined region of the intermediate image, the selected area apertures can be used. The lighting in the microscope room is dimmed. Light of the environment would decrease the contrast on the fluorescent screen.

§ 2.2.2 Diffraction patterns

Crystals and some other materials have regular atomic structures. When a parallel electron beam is exposed to these materials, electrons are deflected in a specific way. Electron beams are extinguished in some directions and in other directions they are reinforced. A pattern of spots appears on the screen; the diffraction pattern. A diffraction pattern

contains crystallographic information. It can be used to tilt the specimen to find a desired crystal orientation. Another application is to align apertures around the area of interest.

§ 2.2.3 Scanning modes

In transmission mode, a specimen is exposed to a parallel beam. A combination of transmitted and forward scattered electrons form an image on a fluorescent screen.

In scanning modes, a fine probe exposes a small area of the specimen. The probe is deflected to cover an area. The electron beam in a cathode ray tube (CRT) is deflected in the same way. Detectors measure the amount of electrons, X-rays or light emitted from the exposed area of the specimen. The more electrons, X-rays or light measured, the brighter the spot on the CRT.

§ 2.3 Alignment

An electron microscope is a highly sophisticated device. For example, an enlargement of one million times shows an area of one thousandth of a millimeter as one meter. It is evident that the mechanical parts of the microscope have to be made precise. However, even with extremely precise mechanical parts and very stable electrical current sources to feed the electromagnetic lenses, alignments are inevitable.

The only way to judge the proper adjustments is to evaluate the picture that is made by the microscope on the fluorecent screen or the CRT. An unexperienced electron microscopist may confuse different kinds of wrong alignments. Great skills are required to interpret the picture and the causes of the incorrect alignments. When a microscope is not properly aligned before the actual measurement starts, certain aspects on the picture may be ascribed to the specimen, while they are actually caused by the incorrect alignment of the microscope.

(13)

CHAPTER 3 Method

§ 3.1 Objective

The objective of this case study is to investigate the problem-solving behaviour of microscopists in order to design a better user interface. Problem solving is a dynamic process. Much more important than the result is the way that result is reached. To reveal the underlying thought processes, the experimental subject is asked to 'think aloud'. Before the experiment, the subject is asked to tell everything he or she is thinking while performing the task. During the task, the observer reminds the subject to speak. Ericsson and Simon (1984), the authors of 'Verbal protocols as data' consider 'thinking aloud' while performing a task as a very good way to reveal the underlying thought processes.

The audio data obtained is recorded and transcribed later on.

§ 3.2 Design of the experiment

Four subjects perform a representative task with a Philips CM 20 scanning transmission electronmicroscope (STEM) in the actual work environment. The experiments are

performed in a CM 20 room at the application laboratory of Philips Electron Optics. The setting in the CM 20 room doesn't really differ from a typical work environment.

5

During the experiment the observer asks the subjects to think aloud. After the task is done the subjects are asked to give a retrospective report of the session, i.e., the goals they wanted to achieve, difficulties they had to overcome. Finally, the observer interviews the subjects about how they experienced 'thinking aloud' and their opinions on the CM 20.

The analyses of data is very time consuming. Due to the limited available amount of time, we had to limit to only four subjects. In chapter 1 we mentioned the goals for the new user interface. The results of this case study will be supportive to the design of the user interface. For that reason we wanted to know the differences in problem solving behaviour between subjects with different levels of skill in operating an electron microscope.

Furthermore we wanted to know the differences in problem solving behaviour between expert professionals for different scientific fields. Two important scientific fields where electron microscopes are used are material science and biology/pathology. The following subjects where choosen:

Subject 1:

Subject 2:

Subject 3:

Subject 4:

Material science expert, STEM expert with two years of experience, male.

Material science expert, STEM non-expert (worked for one month with an electron microscope during graduation), female.

Material science expert, STEM expert with ten years of experience, male.

Biology/ pathology expert, more than fifteen years of STEM experience, male.

(14)

§ 3.2.1 Task

The task consisted of three parts:

1. alignment of the STEM 2. measurement

3. interpretation.

The non-experienced STEM user (subject 2) skipped the first phase and started with a pre- aligned STEM. She first had a tutorial session where she was helped by an experienced microscopist. Afterwards she had to perform the task on her own.

The tasks lasted about two hours. Preparation of the specimens, a difficult and time- consuming task, was not included. Previously prepared specimens where used.

§ 3.2.2 Equipment

The thinking aloud data are crucial for this type of problem solving studies. Therefore, the audio recording has to be high quality. When difficult situations occur during the task performance, people tend to speak faint. Especially at those moments it's important to hear what is said. To disturb the microscopist as little as possible, we used a directional

microphone. The observer carried a clip-microphone. The audio levels were optimised and both audio signals combined with an audio mixer.

To support the analyses of the audio, video films where taken. The video film helps to place a text in proper context; i.e., which task the microscopist is performing, if he is actually working while he is speaking etc. When an electronmicroscopist is doing his job, the room lightning is dimmed to increase the visibility of the contrast differences on the fluorescent screen. The audiovisual support group of the Philips Natlab advised us to use light sensitive black and white camera's. We used two camera's. One camera showed a global overview from the back of the microscopist, the control panel and most of the handling. The second camera showed the fluorescent screen from a side window of the microscope. The camera's did not disturb the microscopist. Two S-VHS recorders with hifi quality sound recorded the video. To relate both video's, a time code signal was recorded on one audio channel. The other audio channel contained the audio data.

Afterwards a compilation video was made from each experimental session. A small picture of the fluorescent screen was inserted in the overview video. This combination of two views gives a good insight in what happens during the experiment. That is, most decisions of the microscopists are made on their evaluations of the enlargement picture.

The audio on the video cassettes was taped to audio cassettes. Cassette recorders are much quicker in reversing the tape than video recorders. A segment of unclear spoken text can be listened to and quickly repeated with a cassette recorder. One track of the cassette tape contained time code which was read with a separate time code reader. The time code was included in the transciption. The time code made a clear connection between videotape position, audiotape position and transcription text.

(15)

CHAPTER 4 Analysis

§ 4.1 Aim of the analyses

After finishing the experiments, about eight hours of audio and video data is available.

These data are very difficult to manage; it's hard to find and compare wanted passages on a videotape quickly. Video material offers a multititude of information. It's easy to fall into investigation of all kinds of aspects of motion studies, while the actual aim of the experiment, the investigation of problem-solving behaviour, would be missed.

The thinking aloud verbal protocol will be the starting point of the analyses. This protocol approximates the underlying thinking processes of the microscopist as close as possible.

The acquired verbal data of the experiment are transposed in a literal transcription. This transcription is the starting point for a domain independent coding to problem solving strategies and for an analysis of the different tasks and subtasks performed by the microscopist.

§ 4.2 Organization of the analyses

The analysis is divided in two different phases. The first phase contains arrangements and processing of the verbal data. The data will not be classified yet. In the second phase the information will be coded. The benefit is no mixing of data-processing and data-

interpretation, but a separate phase of interpretation.

The analysis of problem-solving behaviour of users is domain independent. During the phase of interpretation the problem-solving approach of the microscopist will be reduced to terms that are independent of the problem-solving domains. The model of

Brouwer-Janse and Reeves (1986) will be used for this analysis. As a result of this domain independence, it is possible to compare the differences in problem-solving behaviour between novices and experts in microscopy and between microscopists and users in other disciplines.

Beside this domain independent analysis, a domain dependant analyses takes place. In a structure diagram the intentions of the microscopist (obtained from thinking aloud) and his actions are reported. These actions are organized in target and sub-targets, or goals and sub-goals. Sub-targets and sub-goals are representated by 'tabs to the right' in the structure diagram. That results in an overview of when the microscopist has finished a target,

whether he executes targets and sub-targets in order, and the number of targets he is working on at the same time.

The results of the domain independent analysis of problem-solving behaviour of microscopist need to be transformed to develop concrete recommendations for the development of a new user-interface. By combining problem-solving behaviour and domain-dependant analyses more tangible recommendations are possible.

(16)

Problem solving behaviour of microscopists: a case study

§ 4.3 Procedure

The most important part is taken by the analysis of the audio data. The video data has been used for support. Video has been taped to add contextual information to the audio.

The video film shows the enlarged images of the specimen the subject was looking at and the mode in which he operated the microscope.

8

By using thinking aloud protocol data it is assumed that the execution of the task does not interfere with the speaking. A closer look at the video and audio data of the experiments shows that this assumption is correct most of the time. But when the task is particulary hard to perform, speaking may fade away. Besides this, an expert microscopist may not see the necessity of telling aloud what he considers a routine task. In other words; the thinking aloud data may deteriorate if the tasks are too difficult or too easy in relation to the expertise of the subject.

§ 4.3.1 Literal transcription

A literal transcription has been made from the audio data. That is, the complete text included also unfinished sentences, half-pronounced words and non-linguistic expressions, like "hm", "uh", etc. This is a time-consuming job. One experiment of two hours takes about one week to transcribe. Still the benefits exceed the costs.

Passages in the written text are quick to be found, by hand but even better using the search function of a data processor. Written text is easy to compare. Softly-spoken text needs only once, at the transcription, to be solved precisely. When listening to audio-data, loudly-spoken text strikes more than softly spoken text. That certainly doesn't mean that sofly-spoken text is less important. It is often very interesting, when objects run up against difficulties. Just when the task becomes difficult, subjects tend to speak softer. In a

transcription, each verbalisation receives the same emphasis.

00:24:42

00:25:34

P: ik probeer nu nog met contrast en brightness te .. 0:

P: ik probeer contrast-brightness setting te vinden 0:

P: maar oh, dat werkt niet 0:

P: goed, ik ga weer aus .. op manual en doe ' t gewoon met .. 0:

P: okee ... nou is het bijna gedaan 0: (hm-mm]

P: nog een keer proberen of hi j 't nog doet ... (hm-mm-mm] 0:

P: ik probeer nog een andere plek te vinden waar hij nog een beetje ...

O: ja

P: ik ga misschien nog .. [eh) 0:

Figure 3: Example of a verbatim transcription (P = subject, 0 = experimenter)

(17)

Problem solving behaviour of microscopisls: a case study 9 To make a link between the written text and the video tape, a time-code was been placed (hours, minutes, seconds) once at every five lines that is shown on the video tape. Besides looking for the matching video fragment of a transcription text, it is also possible to determine the duration of activities by determining the difference between two time-codes.

§ 4.3.2 Propositions

The literal transcription will now be the starting-point for further analysis. The transcription is transformed to an ordered list of propositions that are relevant to the problem. The propositions consist of single sentences, fitting within the context of the text. Spoken language is changed into written language.

§ 4.3.3 Episodes

The propositions can now be grouped into episodes. An episode is a completed unit within the problem solving process. Separations between episodes are defined by semantic and syntactic criteria. They are rarely ambiguous. Often there is a summary at the end of one episode. A transition-sentence like "so, now I can ... " often indicate the start of a new episode. Transitions between episodes are also often marked by exclamations like "well",

"forget it", "ok". In almost every case the transition between episodes is characterized by a short break and a change of inflection in the voice at the start of a new episode. Proposi- tions, being ambiguous in no context, are not ambiguous anymore in the context of an episode. A summary of the performance during the episode is given in keywords.

§ 4.3.4 Subepisodes

The structure of the episodes are domain independent, but looking inside an episode, the propositions are to be divided into different domain dependant groups of actions. In this case, taking the microscope as a domain, it's like: aligning the eucentric height of the specimen, focus, correct astigmatism etc. Figure 4 shows an example of pre-processed data, before the coding takes place. An episode is shown, devided in subepisodes. Each proposition is numbered.

When the pre-processing is done, the aranged data has to be proofread by the subject.

(18)

Episode 1oa

Omschrijving:

Begintijd:

Duur:

contrast-brightness setting

00:24:39 00:01 :48

10.1 lk probeer contrast-brightness setting te vinden.

10.2 Maar oh, dat werkt niet.

10.3 Goed, ik ga weer op manual.

10.4 Okee.

10.5 Nou is het bijna .. bater ja ..

10.6 Nog een keer proberen of hij 't nog doet.

10.7 lk probeer nog een andere plek te vinden waar hij nog een beetje .. (stabiel is)

10.8 Gewoon moet eigenlijk die autocontrast-brightness functie werken maar die doet het niet goed.

10.9 Weer een andere mogelijkheid is met de set-functie, die doet in principe hetzelfde, maar werkt nog ietsjes nauwkeuriger.

10.10 In principe maakt die ook een automatische contrast-brightness setting.

10.11 Maar het is niet goed.

10.12 lk ga een beetje met de vergroting omlaag. 10.13 Nu ben ik weer bij 200000 keer.

10.14 lk probeer nog een mooie plek te vinden om 't nog een keer te checken of het goed is. 10.15 Oat ziet goed uit denk ik.

10.16 Ja ik denk, dat is goed nu.

Figure 4: Pre-processed data, before the coding takes place (subject 1)

§ 4.4 Coding

10

In this phase a domain independent problem solving model is used (Brouwer-Janse 1986).

This model indicates the subdivision of the propositions in twenty-four subroutines, shown in table 1. By using these subroutines the propositions become codeable.

The second level of analysis classifies the subroutines to eight strategies:

- Condition - Act

- General problem solving - Selection

- Hypotheses - Pattern Extraction - Evaluation

- Conclusion

This classification came about empirical. From the results of different experiments,

containing more than two hundred protocols, it appeared that definite subroutines occurred grouped every time (Brouwer-Janse, 1983).

Every proposition in the transcription is labelled with the most appropriate subroutine.

Inside a subepisode, almost every proposition belongs to one strategy. The subepisode is labelled by this dominating strategy.

(19)

Strategy Subroutine Subroutine description Strategy description

Condition SRI List given information Compiling the necessary conditions (cond) SR3 List possible questions for the execution of a program; e.g.

SR14 Select rule/ method/ algorithm directly available information, a question to address and a rule to apply

Act SRI6 Execute program Execution of a program and re-

SR17 Identify feedback cording of the results.

SRI& Tag new information

General SRIO Define initial state Use of means-end algorithm to

Problem SRII Define goal state achieve consecutive subgoals.

( Solver (GPS) SRI2 Identify data needed

Selection SR6 List relevant information Process of selecting the most ap- (sel) SR9 Select relevant questions propriate information, specifying SR13 Identify available algorithms the crucial question to address, SR19 Organize and compile data deliberate choosing of the algorithm, and a selective reorganiza- tion and compiling of the data.

Hypothesizing SR7 Formulate hypotheses The generation of an if-then hy-

(hyp) SR8 Define predictions potheses and its corresponding

SR20 Match data to predictions predictions, and the confirmation SR21 Determine truth value predic- and disconfirmation of predictions.

tions

Pan em SR22 Extract panems from data Identification of relevant panems, Extraction SR23 Summarize relevant panems symmetries, regularities, analogies

(pall extr) in the assembled data.

Evaluation SR2 List assumptions Continuous evaluation process,

(eval) SR4 Select evaluative data monitoring and reappraising the

SR5 Assign priorities total problem-solving process as

SRI5 Edit algorithm well as its constituents.

Conclusion SR24 Output conclusions Outcome or result of the problem-

(cone) solving process reached after de-

liberation and often including a summing-up of the preceding.

Table I: Strategies and consisting subroutines (Brouwer-Janse, M.D. & Reeves, J. I 986)

(20)

Episode 1oa

Omschrijving:

Begintijd:

Duur:

10.1 10.2 Eval 10.3 10.4 10.5 10.6 10.7 Sel 10.8 Eval 10.9

10.10 10.11

10.12 Act 10.13 10.14 10.15 10.16 Sel

contrast-brightness setting

00:24:39 00:01 :48

lk probeer contrast-brightness setting te vinden. (SR 11) Maar oh, dat werkt niet. [SR 4,5)

Goed, ik ga weer op manual. [SR 15) Okee.

Nou is het bijna .. beter ja .. [SR 4,5)

Nog een keer proberen of hij 't nog doet. (SR 14)

lk probeer nog een andere plek te vinden waar hij nog een beetje .. (stabiel is) [SR 13)

Gewoon moet eigenlijk die autocontrast-brightness functie werken maar die doet hat niet goed.

(SR4,5)

Weer een andere mogelijkheid is met de set-functie, die doet in principe hetzelfde, maar werkt nog ietsjes nauwkeuriger. [SR 15)

In principe maakt die ook een automatische contrast-brightness setting.

Maar het is niet goed. [SR 4,5)

lk ga een beetje met de vergroting omlaag. [SR 14) Nu ben ik weer bij 200000 keer. [SR 6)

lk probeer nog een mooie plek te vinden om 'I nog een keer te checken of hat goed is. (SR 16) Dat ziet goed uit denk ik. (SR 17)

Ja ik denk, dat is goed nu. [SR 19)

Figure 5: Coded episode.

Figure 5 shows the coded episode from figure 4. Each proposition is coded with a subroutine (SR) according to table 1. And the subroutines are coded according to the dominating strategy, for example: EVAL, SEL, ACT.

§ 4.5 Different levels in strategies

In this project, we will compare the problem-solving behaviour of novice microscopists and experts. The transcriptions of 'thinking aloud' data are divided in episodes and subepisodes. The subepisodes are labeled by strategies. We measure the relative distribution of the strategies (the number of sub-episodes with a strategy as the dominating strategy divided by the total number of strategies in the transcription).

12

We classify the eight strategies listed in paragraph 4.4 according to the overview the user has over the task to be done. The lowest level is taken by the Condition and Act strategies.

The user looks at the present state, performs an action and appraises the feed-back. The actions take place on a very local level, not considering the wider scope of the problem. The user is confronted with a situation which is not purposefully created by himself. Also in a time-function there is local action, the present only is considered important. There are no references to similar situations in the near past.

(21)

Strategy 5. Conclusion/

(high level) evaluation 4. Hypotheses/

Pattern extraction 3. Selection

2. General Problem Solving.

1 . Condition/ Act

Figure 6: Strategies grouped into five levels

13

Overview level

Evaluation Anticipation Control

Manipulation Confrontation

On a higher level is the General Problem Solving (GPS) strategy. A problem on a local level is approached by comparing the differences between the present situation and the desired one, and further to define how to reach this desired state. As opposed to the lower level, it comes to systematic and efficient working. The user is faced with a problem but manipulates the situation in a way he can solve it.

Again one level higher we find the Selection strategy. Characteristic to this level is the fact that the user is very conscious of his choices. Important information is separated from those less important. Algorithms for problem solving are selected according to their usefulness. The user controls the problem-solving process. After an action is done, recapitulation follows; data is valued to importance and then resumed.

Characteristic to the strategies Hypothesizing and Pattern Extraction, is the clear expec- tation pattern the user has. At Hypothesizing he generates an hypothesis. and concludes the consequences. He anticipates possible problems and he traces the truth value of the

predictions afterwards. At Pattern Extraction the user recognizes relevant patterns and analogies in the data. At both strategies the user is master of the problem solving process because he can anticipate the continuation of the process using the knowledge he already had.

At a high level the Evaluation and Conclusion strategies, the user considers the problem from a total view ("helicopter-view"). He knows then to solving a subproblem and how to place it in the context of the whole problem. He is not losing sight of the final purpose. A continuous evaluation takes place. Sub aspects are valued on importance to achieve this purpose.

(22)

Problem solving behaviour of microscopists: a case study 14 The relative distribution of the strategies in the processed and coded transcription is used to investigate what level is necessary to operate a STEM. The catagorizing of strategies in different levels suggest only novice user will use low level strategies like Condition and Act. This is not the case. Different tasks require different strategies. Sometimes low level strategies are most appropriate. The difference between novice users and expert users is that novice users use very little high level strategies. They have a limited overview over the problem-solving process and fewer expectations about the system's behaviour.

§ 4.6 Classifications in problem-solving behaviour

In our approach, we have taken different levels of overview over the problem-solving process as a measure to value different problem-solving strategies. Compared to the Rasmussen model for human computer interaction, our model makes a finer grained distinction in performance possible.

Rasmussen (1986) distinguishes three typical levels of control of human actions: skill-, rule-, and knowledge-based performance. Skill-based behaviour represents sensorimotor performance during acts or activities that, after an initialization, take place without conscious control. The flexibility of skilled performance is due to the ability to compose from a large repertoire of automated subroutines, these sets that are especially suited for a specific purpose. Rule-based behaviour is the composition of a sequence of subroutines which are consciously controlled by a stored rule or procedure. Rule-based coordination is in general based on explicit know-how. The rules that are used can be reported by the person. Knowledge-based performance is goal-controlled performance. In this situation, the goal is explicitly formulated, based on an analyses of the environment and the overall aims of the person (Rasmussen 1986, pg 99-115).

Skill-based behaviour in electron-microscopy mainly refers to the operation of controls of the electron-microscope. The Condition and Act strategies are rule-based strategies. One recognizes a situation and chooses a corresponding rule. General Problem Solving, Selection, Hypotheses, Pattern Extraction, Evaluation, and Conclusion are all knowledge- based strategies. For these strategies, a goal is explicitly available. Depending on the strategy, however, the goal can be a sub-goal or all-embracing. In other words, the knowledge-based level of the Rasmussen model is structured in layers according to the overview a person has over the problem-solving process.

(23)

CHAPTER 5 Results and discussion

§ 5.1 Experiments

The first subject, a material scientist, with two years of experience on the STEM,

performed a task of one and a half hours. The first thirty minutes were used to align the microscope by using a suitable specimen: PE spheres. This phase is used as a preparation for the research of the actual specimen, investigating the texture of wool. This task was followed till the moment the subject was satisfied with the setting and ready to make an X-ray map to investigate the division of elements in the wool sample.

The second subject, a material scientist, with one month of STEM experience six years ago, worked on a task of titanium/aluminium alloy for two hours. After ten minutes, it appeared that support was necessary in order to operate the microscope. During this tutorial session, a microscopist of the application lab explained the basic functions of the microscope. After this explanation, the subject operated the microscope herself. She could always ask for assistance of the tutor when she got stuck while performing the task. The last thirty minutes the subject worked on the task without any help. The task was not completed.

The third subject, a material science expert with ten years of experience on STEMs worked on two tasks, each lasting one hour. The first task concerned the analyses of the boundaries of Silicon Nitride. Silicon Nitride is an isolating crystal, which is charged under influence of the electron radiation. The alignment of the microscope becomes more difficult. The second task is the research of a semiconductor specimen of Gallium

Arsenide and Silicon. Gallium Arsenide and Silicon have identical atomic structures, but the crystal sizes are different. This causes defects on the boundary that have to be analysed.

The fourth subject, an expert in electron microscopy using biology specimens, analyzed a liver specimen. Analyses of biological specimens does not require to exhaust the microscope at the physical-technical level. This subject only worked at the task for part of the time during the one hour experiment. The remaining time he was explaining the use of STEMs in hospitals. During this part of the experiment, the audio data is not longer a representation of underlying thought processes. So it is important to check this limiting condition.

The audio data of the experiment of subjects 1 and 3 are coded into problem solving strategies as described in section 5.4. Doing this with the data of subject 2 was of little value. She was working task oriented indeed, but reached too often a dead-lock situation so that there was no question of effective thinking aloud data that reflects the problem- solving process. This protocol resembles a tutorial dialogue. The results of this experiment are used to analyse when dead-lock situations occur for a novice electron microscopist.

Subject 4 (biology/ pathology expert with fifteen years of STEM experience) interrupted his task often to give an explanation about the use of electron microscopes in hospitals. A

(24)

summary of these data is shown in § 5.5 It is important to mention that we have two different types of data. Subjects 1, 2 and 3 worked task-oriented during the experiment.

The audio-data contained the verbalisations of what they where thinking. According to the 'thinking aloud' protocol, the verbalisations are as close as we can get to the underlying thought processes. The data applies directly to the task performed. The fourth subject tells about experiences. These data are retrospective. The selection of topics he mentions and the contents are subjective data. The subject is an expert in his domain. The explanation gives a good insight in the issue of the use of STEMs in a hospital environment.

Figure 7: TEM image of a human liver (Application lab., Philips, Eindhoven)

The use of problem-solving strategies by subject 1 and subject 3 are reported in the next section.

(25)

§ 5.2 Problem-solving strategies

The verbal protocols of subject 1 and subject 3 were coded and analysed. The protocol of subject 1 (length: 67 min.) consisted of 22 episodes, 169 subepisodes and 480

propositions. The part of the protocol of subject 3 with a Silicon Nitride specimen (length:

67 min.) consisted of 35 episodes, 222 subepisodes and 633 propositions. The second part, with a Gallium Arsenide specimen (length 34 min.) consisted of 18 episodes, 115

subepisodes and 322 propositions.

§ 5.2. l Differences in experience

The first and third subject have been compared. Both are material scientists. Their skills in operating the STEM differ: subject 1 has two years of experience, subject 3 ten years.

Appendix B and D show part of the transcription to episodes of both subjects. Every subepisode has been categorized to one of the in § 4.4 mentioned strategies. For subject 3, the results of both tasks are combined. Diagram 1 shows for both subjects the relative distribution of each strategy.

Conclusion

Evaluation

Pattern Extraction

Hypotheses

Selection

Gen. Problem Solving

Condition

Act

(0%)

I

^(1%)

:

¹

::::::::::::::::::::::::::::

¹

:::::::

¹

:::::::::::,:~:~:::::r::;.

^::·/¹':.=::::::::::::::::::::::::::::::::1:1:1:::::::1:::::.:1:1:1:1:1:1:::=:::,: ( 2 s % >

( 18%)

(13%)

(12%) ( 14%)

(20%) (23%)

(19%)

!!!:::::::::! Material science expert, 2 years of STEM experience

II

Material science expert, 10 years of STEM experience Diagram 1 : Problem solving strategies; differences in experience

The more experience the user has, the more he has a global overview. This manifests itself in frequently used Pattern Extraction (13 % versus 8% ). A user having less experience, though working systematically, often relies on the General Problem Solving strategy (19%

(26)

versus 10%). Problems are detected and solved locally instead of in wider context. These are in concordance with the results obtained in problem-solving studies in other domains (Brouwer-Janse & Reeves, 1986).

18

That Evaluation appears more often for the less experienced microscopist can be explained by the coding model. Evaluation can take place on different levels. The whole of the problem-solving process might be evaluated, but also a subproblem. The different levels of evaluation should be labelled accordingly. This was not done: all evaluation is coded with one label.

High level strategies like Pattern Extraction and Evaluation are particulary used to judge and interpret microscope images. A microscopist has to specify if artifacts in the image are caused by alignment errors or by the properties of the specimen. This turns out to be a difficult task, that requires a lot of survey of the entire situation. Indeed it has to be

remarked that the less experienced user, certainly isn't a beginner. This appeared from his very systematic way of working.

(27)

§ 5.2.2 Differences in task-difficulty

Subject 3 preformed two different tasks. One of his tasks was a routine task. The other task turned out to be problematic because the Silicon Nitride specimen charged con- tinuously. Diagram 2 gives an overview of the percentile shares of different strategies for these different tasks.

Conclusion

Evaluation

Pattern Ext raction

Hypotheses

Selection (27%)

Gen. Problem Solving

-

~!!!i~!illi!!!!!!!!!!!!!!!!!!!i!!!i)!!!!!!!!!!!!i!ii!ii

⁽¹⁰^%⁾

(9%) Condition

i:::::::~:::::::;:::::::;::::::i:::::::::::!!!!!!!!!!!:!:::::::::::::::::::::

⁽¹³^{% )}

(15%)

Act

~i!!!!!i!!!!:::::::::::::::1:!!!!!!!!!!!!!!!!!))!!!!!!!!!!!!i:::;1:;:::::::::::::::1:;:;::11

⁽^{1 7}^%⁾

( 16%)

ri:::

Routine task, material science expert, 10 years of STEM experience

::::::::::::;

II

Difficult task, material science expert, 10 years of STEM experience Diagram 2 :Problem solving strategies; differences in task difficulty

The Selection strategy is used more for routine task than for non-routine tasks (27% versus 21 %). In contrast, the Evaluation is used more for the non-routine task than for the routine task (20% versus 13% ). During a routine task the microscopist has a better control over the situation and he chooses his methods selectively. It is striking that high level strategies like Pattern Extraction and Hypotheses are used to the same extend. This might be caused by the interaction effects of general domain knowledge and characteristics of the task itself. To clasify this more research is needed.

§ 5.3 Required level for problem solving with a STEM

Subject 2, novice on the area of STEM, got stuck when she tried to solve the given task.

A beginning user tends to start with solving subproblems straight away, without an overview of the total problem. When the problem-solving process is coded to strategies

(28)

Problem solving behaviour of microscopists: a case study 20 this shows as a frequent use of Condition and Act. When the performed action doesn't lead to the proper result, a reappraisal is necessary. The subgoal has to be released temporary and the problem-solving process has to be evaluated from a broader overview.

According to the ranking of strategies (see § 5.5) this means: selection of a higher level of strategy. When the reappraisal doesn't take place, then the possibility that the microscopist deteriorates the situation is great. The wrong settings will be changed. The novice ends up in a vicious circle: the problem turns out worse.

An effective problem-solving strategy for electron microscopy requires the use of high- level strategies. The user has to utilize at least the General Problem Solving strategy. A systematic way of working is necessary. Besides that, an overview over the total problem- solving process is invaluable. Now and then a reappraising of the total process has to take place where strategies like Selection and Evaluation are used.

A user who solves problems mostly on a local level with some high level evaluation now and then will be able to solve a task independently. The more he has a total overview of the problem solving process, the more efficient he can work. A coded transcription of an expert can be recognized by the frequent appearance of high level strategies such as Pattern Extraction, Selection, Hypotheses and Evaluation.

§ 5.4 Analyses of structure diagrams

The goals of the microscopists and the performed actions are shown in structure diagrams.

These goals are obtained from the 'thinking aloud' data. The structure diagrams give an insight in the progress of the problem solving process. Structure diagrams are, as opposed to strategies, domain determined. Structure diagrams show which actions the microscopist performs when he starts with a problem. They also show which actions are successive and which actions are repeated often.

Structure diagrams show the goals and sub-goals in a layered manner using tab-stops and brackets like in computer programming in Pascal or C source code. Part of the structure diagrams of the experiments of subjects 1 and 3 are shown in figure 8 and 9. At this moment only the form of the diagrams is of importance.