Describing and evaluating data collection methods

Overview

Difference between accuracy and reliability and operator error.

What you need to know

Answer each one before you reveal it.

Questions from this subtopic

Each one with the model answer underneath.

1. 1

   What must a description of a data collection method contain?

   Model answer

   Enough for someone else to repeat it exactly: what was measured, with what equipment, in what units, at how many sites and where those sites were, how the sites were selected, how many repeats were taken at each, who carried it out, and when - date, time and conditions. It should also state the rules that were applied, such as how edge cases in a quadrat were treated or how long each pedestrian count ran. A description that omits these is not a method, and its results cannot be checked or reproduced.

2. 2

   What does it mean to evaluate a data collection method?

   Model answer

   To judge how well it produced data capable of answering the question, weighing its strengths against its weaknesses rather than simply listing what went wrong. That means considering whether it measured what it was supposed to measure, how accurate and repeatable the measurements were, whether the sample was representative, what bias the method introduced, and how the conditions on the day affected it - and then saying what difference all that makes to confidence in the results, and what specific change would improve it.

3. 3

   What is the difference between accuracy and reliability?

   Model answer

   Accuracy is how close a measurement is to the true value - a flowmeter that has drifted out of calibration gives inaccurate readings however carefully it is used. Reliability is how consistent the results are when the measurement is repeated, by the same person or by someone else, under the same conditions. The two are independent: a badly calibrated instrument can be highly reliable and consistently wrong, while a well-calibrated one used carelessly gives accurate readings on average but with wide scatter. Good fieldwork needs both, which is why instruments are checked and measurements repeated.

4. 4

   Evaluate the use of a questionnaire as a fieldwork method.

   Model answer

   Its strengths are that it collects data on opinion, behaviour and origin that no measurement can supply, that closed questions produce comparable quantitative data from many respondents quickly, and that it can be delivered at several sites for spatial comparison. Its weaknesses are that respondents are self-selecting, so those in a hurry or unwilling to stop are missed; answers may be inaccurate through faulty memory or a wish to give the expected reply; the sample achieved in a few hours is usually small relative to the population; and wording or the surveyor's manner can lead the response. It is improved by piloting, by sampling at varied times and places, and by increasing the sample size.

5. 5

   Evaluate the measurement of river velocity in the field.

   Model answer

   Velocity is central to any study of channel processes and can be measured quickly with a flowmeter or with a float and stopwatch, and repeating the reading across the channel gives a defensible mean. The weaknesses are real: a float measures only surface velocity, which exceeds the average for the section; a flowmeter must be held at a consistent depth and can be affected by the operator standing in the flow; turbulence and vegetation produce widely varying readings at the same spot; and the values apply only to the flow conditions on the day, which depend on recent rainfall. Repeating the measurement several times at each point and averaging, and recording the antecedent weather, are the main improvements.

6. 6

   Evaluate an environmental quality survey as a method.

   Model answer

   It converts qualities that cannot be measured with equipment into numbers that can be mapped, compared between sites and correlated with other variables, and it is quick, cheap and repeatable across many locations. Its weakness is that every score is a personal judgement, so results differ between surveyors and can be affected by the weather, the time of day and the surveyor's own expectations about the area. Reliability is improved by writing precise descriptors for each point on the scale, by having the same person score every site, by using several independent scorers and averaging, and by photographing each site so that scores can be reviewed afterwards.

Other ways to work this

The same content, four other ways.

What is inside Levo

Six ways to work the same material.