Assessment Report

Level 3 Statistics 2016

Standards 91584  91585  91586

 

Part A: Commentary

Candidates who performed well across the standards were able to demonstrate understanding of statistical and probability concepts, integrating these understandings with contextual knowledge when required. Successful candidates demonstrated both calculation-based and analytical-based approaches.

When using contextual knowledge as part of a response, candidates needed to link this knowledge to an aspect of statistical knowledge. Contextual knowledge or speculation alone was not sufficient. 

Part B: Report on Standards

91584:  Evaluate statistically based reports

Candidates who were awarded Achievement commonly:

  • understood the principles of experimental design, e.g. use of random allocation to assign treatments to experimental units
  • constructed confidence intervals from single survey percentages
  • understood margin of error, e.g. explaining the necessity for a margin of error to be included in statistical reports, calculating the margin of error using the ‘rule of thumb’, appreciating the relationship between sample size and the size of the margin of error
  • described features of a statistical study design, e.g. explanatory and response variables, difference between observational study and an experiment.

Candidates who were assessed as Not Achieved commonly:

  • gave vague responses that were not in context or not linked to statistical concepts
  • confused statistical terms, e.g. sample with population, random allocation with random selection, sample error with non-sampling error
  • were unable to construct a confidence interval
  • wrote responses that were not guided by the question asked, e.g. discussing “using the past as a source of data” when asked to discuss “extending the results inappropriately”
  • confused features of experiments and observational studies
  • believed non-sampling errors or sources of bias could be removed completely, e.g. random allocation “removes bias” rather than minimising bias
  • made incorrect inferences about the survey respondents rather than the target population
  • could not explain the need for margin of error in a statistical report and did not understand the relationship between sample size and margin of error.

Candidates who were awarded Achievement with Merit commonly:

  • explained how random allocation reduces bias or creates two fair or balanced groups, using specific contextual examples of bias
  • identified confounding variables for observational studies and illustrated with contextual examples drawn from the text
  • interpreted correctly a confidence interval in context to make an inference about a (target) population
  • recognised the difference between a proportion versus count in a report
  • used the relationship between sample size and the margin of error to reason correctly about two different studies based on their reported margin of errors
  • linked survey table results with report statistics
  • used understanding of study design to support comments on the nature of the claims made.

Candidates who were awarded Achievement with Excellence commonly:

  • responded to a claim concerning the difference between two percentages by constructing and interpreting a confidence interval, and explaining that a claim could not be made because 0 was included in the interval, or because both values of the confidence interval were either both positive or both negative
  • described potential issues with the survey design and described how the issues could affect a specific aspect of the report (e.g. conclusions or claims presented) in context with examples to illustrate the issue
  • understood the ‘rule of thumb’ acceptable percentage range (30% to 70%) and implications for margin of errors of survey percentages outside of this range (when constructing approximate 95% confidence intervals)
  • justified and discussed, in context, potential problems when reporting on a study and provided examples to illustrate these
  • used statistical terms confidently, such as non-sampling error, bias, sampling variability, confounding variables, showing a clear understanding of the meaning of these terms
  • were succinct in their responses and presented clear, well-supported discussion points rather than lengthy generic statements or descriptions
  • integrated statistical and contextual knowledge in their responses when discussing issues with a statistical process or claim presented in the statistical report.

Standard-specific comments

Candidates generally demonstrated sound skills with constructing and interpreting confidence intervals. However, some candidates incorrectly made an inference about the respondents rather than the population, e.g. “I’m pretty sure that somewhere between 55% to 89% of respondents”, while others wrote the confidence intervals incorrectly, e.g. 1.7 ± 59; (60.7%, 57.3%); 57.3% - 60.7%.

Despite some improvement in the construction of confidence intervals, understanding of sampling variability was not well demonstrated by some candidates. At this level, by quantifying the margin of error using the 1/root(n) rule of thumb, candidates should focus on the size of the sample and its effect on the expected variation of sample estimates.

Some candidates struggled to integrate statistical and contextual information when assessing the quality of reports with respect to conclusions made in those reports. Successful candidates looked for the “obvious” issues in the study design (including the survey design) rather than suggesting obscure issues. For example, in question 2(c), the “obvious” issue was the information about sugar provided to participants before they completed the survey. These issues were then linked to the claims or conclusions made in the statistical report.

Candidates were required to assess the quality of reports with respect to conclusions made in those reports, through questions that scaffolded the evaluation of the statistical report. Candidates needed to read the report and the questions carefully, and highlight or underline key words to focus their responses. Often candidates gave responses that were unrelated to the question and appeared to be rote-learned. Good responses communicate key statistical ideas well, focus on a central idea or point, and support the idea or point with relevant features from the report, e.g. relevant contextual knowledge, specific study design features, and specific results from the study.

For example, in question 3(d), candidates were asked to discuss one potential issue with the study design used for the 2008 survey, in respect to the heading for this report “Higher fines discourage disability parking abuse”. Candidates linked statistical and contextual knowledge by discussing issues such as the presence of confounding variables due to the study being observational, or issues with the sampling of disability parks in terms of representativeness (selection bias). However, this discussion did not address the heading for the report, which was the main conclusion made by the report. To demonstrate statistical insight, candidates need to integrate statistical and contextual information to assess the quality of reports with respect to the conclusions made. 



 

91585:  Apply probability concepts in solving problems

Candidates who were awarded Achievement commonly:

  • identified relevant probability concepts and contextual information from the problem
  • used representations such as two-way tables, probability trees and Venn diagrams
  • used methods associated with two-way tables and probability trees
  • used methods related to independence to calculate probabilities
  • used methods related to conditional probabilities
  • demonstrated understanding of true probability versus experimental estimates, e.g. that a sample proportion / relative frequency only gives an estimate for the true probability in the situation
  • demonstrated understanding of experimental estimates versus model estimates, e.g. creating a probability model from the results of only one sample, which highlights consequences of non-random selection. 

Candidates who were assessed as Not Achieved commonly:

  • struggled to identify relevant probability concepts and contextual information from the written problem
  • used probabilities lower than 0 or greater than 1
  • were unable to work with probabilities written in different forms
  • did not understand that data-based estimates of true probability are subject to sampling variability / sample size and issues of generalisability
  • used incorrect methods to calculate probabilities of independent, combined and conditional events
  • compared probabilities using differences rather than quotients.

Candidates who were awarded Achievement with Merit commonly:

  • selected and carried out a logical sequence of steps, e.g. knowing when to add and when to multiply probabilities when combining events
  • demonstrated understanding of the concept of independence, e.g. that for independent events P(B/A) = P(B/A’)
  • connected different concepts and representations, e.g. calculating and comparing conditional probabilities to determine “how many times as likely”, using the relationship between probabilities and expected numbers with probability information from a two-way table
  • communicated thinking using appropriate statements, e.g. correct use of probability notation.

Candidates who were awarded Achievement with Excellence commonly:

  • developed a chain of logical reasoning, e.g. completing a 3-event Venn diagram to represent a situation and interpreting the results to solve a problem
  • devised a strategy to solve a problem, e.g. discussing how a simulation could be used to consider sampling variability when using experimental estimates of probabilities
  • used contextual knowledge to reflect on their answers, e.g. assumptions of independence, assumptions related to sampling without replacement.

Standard-specific comments

When using experimental estimates of probabilities, candidates needed to consider sample size, sampling variability and issues of generalisability. Many candidates incorrectly stated that sample sizes must be equal when comparing groups, e.g. in question 1(a)(iii). Many candidates also lacked familiarity regarding the use of simulations with probability models, e.g. in question 2(b)(ii).

Some candidates used the term “relative risk” when comparing two conditional probabilities using division, e.g. in question 1(a)(ii). The term “relative risk” should be limited to situations where risk is involved. Many candidates also struggled to discuss the concept of statistical independence, e.g. in question 1(b)(i). When events A and B are independent, then knowing that event A has occurred (or not) gives no new information about the chances of event B occurring, i.e. P(B | A) = P(B).



91586:  Apply probability distributions in solving problems

Candidates who were awarded Achievement commonly:

  • sketched probability distribution models
  • described features of probability distribution models
  • selected appropriate probability distribution models
  • calculated simple probabilities using probability distribution models
  • used terms correctly e.g. “more than”, “less than”
  • used discrete random variables represented as probability distribution tables
  • calculated the expected value (mean) of a discrete random variable
  • understood the distribution of true probabilities versus the distribution of model estimates of probabilities, e.g. identifying a factor that should be considered when modelling a situation.

Candidates who were assessed as Not Achieved commonly:

  • used incorrect methods, e.g. the wrong choice of model, incorrect probability calculations
  • did not account for the standard deviation when sketching a normal distribution model
  • misinterpreted instructions such as “fewer than” and “more than”
  • misinterpreted features of a probability distribution graph or table.

Candidates who were awarded Achievement with Merit commonly:

  • linked features of the probability distribution model to the context, e.g. identifying appropriate assumptions in context, giving the conditions for a probability distribution in context
  • combined correctly probabilities and correctly interpreted “OR”
  • considered the nature of the variable involved when calculating the probability
  • used inverse methods to find unknown parameters of probability distribution models, e.g. inverse Poisson to determine lambda, inverse normal to determine standard deviation
  • understood variation from a graphical representation of a probability distribution
  • communicated their thinking using appropriate statements, e.g. stating probability distribution model and parameters, correctly using probability notation.

Candidates who were awarded Achievement with Excellence commonly:

  • discussed the appropriateness of a model by considering features of the probability distribution, statistical evidence and / or the context
  • utilised the relationship between parameters of the probability distribution model and probability (as area under the curve) to compare two different probability distribution models
  • interpreted and attempted to generalise features of an experimental probability distribution when considering the features of a discrete random variable based on this experimental probability distribution
  • clearly explained their reasoning.

Standard-specific comments

Many candidates struggled to connect the concept of variation, as measured by standard deviation, with the visual features or properties of a probability distribution graph or sketch. For example, in question 1, candidates needed to sketch the probability distribution model (a normal distribution with given parameters) but many sketches did not account for standard deviation. Similarly, in question 3, candidates needed to identify which distribution had less variation and explain why using features of the probability distribution.

Candidates needed to be able to discuss situations and data collected from situations where the mathematical probability distributions (e.g. Poisson, normal, binomial, uniform, triangular) were not appropriate. For example, when answering question 3(b)(ii), many candidates struggled to explain why it would not be appropriate to model the ratings for the ‘before’ survey using the conditions for a Poisson distribution. 

Mathematics and Statistics subject page

Find information for...

 
Skip to main page content Accessibility page with list of access keys Home Page Site Map Contact Us newzealand.govt.nz