Sample size for hypothesis testing for the proportion

The BIS.Net Team BIS.Net Team

Hypothesis testing for the proportion has one of the most diverse applications and is relevant particularly in manufacturing, health, political campaigning and market research.

After polling booths close, as results trickle in, political commentators may be interested in determining if results so far indicate the required swing for a candidate to win or lose the election. The answer can be obtained through hypothesis testing. In this instance the analyst cannot dictate the sample size and hypothesis testing can only be performed on the number of results available.

However, there are many applications where samples must be taken to perform a hypothesis test. For example, prior to an election a survey may be carried out to determine if the preference percent for a candidate has increased or decreased. It is then important to know in advance how many samples need to be taken.

To determine the sample size the analyst will need to specify a reference value for the null hypothesis, which can be considered the status quo. The amount of concern, i.e. the minimum change from the reference value that is important to detect must also be specified.

Additionally, the analyst must specify two types of risk. The first is the risk or probability of falsely concluding there is a change at least equal to the specified amount of concern when there is none. This is called the alpha risk. The second is the risk or probability of concluding there has been no change when in fact there was a change. This is called the beta risk. Both risks are due to ‘the way the numbers fall’ when sampling. By chance, sampling may have selected samples with measurements larger or smaller than expected. Finally, the type of change must be specified, which can be > or < or <>

There are two ways to determine the sample size. One is based on the traditional Wald method for determining confidence intervals. Just as the Wald method is known to be unreliable for confidence intervals it is unreliable for hypothesis testing which is related to confidence intervals.

The BIS.Net sample size APP uses the more reliable score interval method as part of its calculations to determine the best sample size to meet the required objective for the hypothesis test. This however not possible through numerical methods.

Instead a machine learning algorithm is used which learns to discard incorrect solution paths, ultimately finding the required sample size.

Once the sample size is determined it is possible to calculate critical values. An amount beyond the critical value implies that there is reasonable evidence that there has been a change beyond the alternative hypothesis. For a > than hypothesis an upper critical value is calculated. For a < than hypothesis a lower critical value is calculated. For a <> equal hypothesis both a lower and upper critical value is calculated.

A preferred and recommended option to using critical values is to perform a hypothesis test after sampling with the calculated sample size and make decisions based on the p value.

The BIS.Net Sample Size app also displays an OC and Power curve such as shown in the image below.

Hypothesis testing on proportions

The analyst can use this chart to answer what if questions, such as what is the power of detecting a change equal to x percent.

Download the Inferences APP, comprised of mainstream and machine-powered analytics for statistical analysis

Analytics as a Service (AaaS) for Quality

Drive quality improvement through actionable insights using analytics you can trust! Use up to 200 analytics tools downloadable through a suite of Apps!

FREE usage of the analytics Apps for quality improvement
  • Augmented with machine-powered smarts
  • Always updated with the latest tools and features
  • No licencing or fixed subscriptions - Pay ONLY for the analysis you run from 20 USD cents per analysis, billed monthly! Set a budget so you don't exceed!