Evaluating methods, evidence and the limitations of conclusions

Overview

Main sources of limitation in fieldwork data, measurement error affect fieldwork and subjectivity in fieldwork.

What you need to know

Answer each one before you reveal it.

Questions from this subtopic

Each one with the model answer underneath.

1. 1

   What are the main sources of limitation in fieldwork data?

   Model answer

   The sample - its size, how it was selected and how representative it is of the wider population. Time - the data usually describe a single day or a short window and may not be typical. Space - the study covers a small area which may not represent the wider region. Measurement - instrument accuracy, operator inconsistency and the precision of the recording. The people - non-response, honesty and self-selection in questionnaires. And secondary data - their date, scale, boundaries and the purpose for which they were originally gathered.

2. 2

   How does sample size limit what you can conclude?

   Model answer

   A small sample is heavily influenced by individual unusual cases, so an apparent pattern may be an accident of which sites or respondents happened to be included, and significance tests deliberately demand a much higher value from a small sample before accepting a result. It also means the conclusion cannot fairly be extended to the whole population. The honest response is to state the sample size, to say how it affects confidence, and to identify the size that would have been needed rather than simply noting that more would have been better.

3. 3

   Why is the timing of collection a limitation, and how should that be expressed?

   Model answer

   Because data collected on one day capture one set of conditions in a system that varies continuously. River discharge reflects the recent rainfall, a beach profile reflects the last storm and the state of the tide, footfall reflects the hour, the day and the weather, and vegetation reflects the season. The limitation should be expressed in terms of its effect on the data - counts were taken on a wet weekday morning, which is likely to have depressed footfall and under-represented weekend shoppers, so the sphere of influence measured is probably smaller than the true one - rather than as a comment on the weather.

4. 4

   Why is a single study area a limitation on the conclusion?

   Model answer

   Because the findings strictly apply only to that place. A relationship demonstrated along one river, one beach or one town may not hold elsewhere, since geology, land use, management, climate and history all differ, and the study area may itself have been unusual in ways that were not obvious. The conclusion should therefore be stated with its geographical scope attached, and any wider claim should be offered as a suggestion for further investigation rather than as a finding.

5. 5

   What kinds of measurement error affect fieldwork, and how do you allow for them?

   Model answer

   Instrument error, where equipment is imprecise, uncalibrated or unsuited to the conditions - a flowmeter reading erratically in turbulence, a clinometer read to the nearest degree on a very gentle slope. Reading error, where values are recorded to a precision the instrument does not support, or read from the wrong scale. And recording or transcription error when figures move from the field sheet to the spreadsheet. They are allowed for by repeating measurements and averaging, by stating the precision of each instrument, by checking a sample of readings independently, and by acknowledging in the evaluation how large the possible error is relative to the differences being claimed.

6. 6

   What is subjectivity in fieldwork, and where does it enter?

   Model answer

   Subjectivity is the influence of personal judgement on what is recorded. It enters wherever no instrument makes the decision: scoring environmental quality or building condition, estimating percentage cover in a quadrat, classifying a land use that could belong to two categories, deciding where a break of slope lies, choosing which photographs to take, and interpreting an open questionnaire response. It also enters through expectation, since a fieldworker who knows the hypothesis may unconsciously score in its favour. Written criteria, consistent scorers, independent repeat scoring and photographic records all reduce it, but it cannot be eliminated.

Other ways to work this

The same content, four other ways.

What is inside Levo

Six ways to work the same material.