Computing Minimal Pattern Dimension for A/B Checks in Statsmodels: How and Why | by Jason Jia | Might, 2024
A well-liked, high-performing numerical optimization methodology is Brent’s methodology. Brent’s methodology is a root-finding algorithm that mixes numerous strategies such because the bisection methodology, the secant methodology and inverse quadratic interpolation. Additional particulars of its implementation in Statsmodels might be discovered here.
In Python, the implementation appears like this:
def solve_power(self, effect_size=None, nobs1=None, alpha=None, energy=None,
ratio=1., different='two-sided'):
print('--- Arguments: ---')
print('effect_size:', effect_size, 'nobs1:', nobs1, 'alpha:', alpha, 'energy:', energy, 'ratio:', ratio, 'different:', different, 'n')# Test that solely nobs1 is None
kwds = dict(effect_size=effect_size, nobs1=nobs1, alpha=alpha,
energy=energy, ratio=ratio, different=different)
key = [k for k,v in kwds.items() if v is None]
assert(key == ['nobs1'])
# Test that the effect_size is just not 0
if kwds['effect_size'] == 0:
elevate ValueError('Can't detect an effect-size of 0. Attempt altering your effect-size.')
# Initialize the counter
self._counter = 0
# Outline the operate that we need to discover the basis of
# We need to discover nobs1 s.t. present energy = goal energy, i.e. present energy - goal energy = 0
# So func = present energy - goal energy
def func(x):
kwds['nobs1'] = x
target_power = kwds.pop('energy') # all the time the identical goal energy laid out in key phrases, e.g. 0.8
current_power = self.energy(**kwds) # present energy given the present nobs1, word that self.energy doesn't have energy as an argument
kwds['power'] = target_power # add again energy to kwds
fval = current_power - target_power
print(f'Iteration {self._counter}: nobs1 = {x}, present energy - goal energy = {fval}')
self._counter += 1
return fval
# Get the beginning values for nobs1, given the brentq_expanding algorithm
# Within the authentic code, that is the self.start_bqexp dictionary arrange within the __init__ methodology
bqexp_fit_kwds = {'low': 2., 'start_upp': 50.}
# Resolve for nobs1 utilizing brentq_expanding
print('--- Fixing for optimum nobs1: ---')
val, _ = brentq_expanding(func, full_output=True, **bqexp_fit_kwds)
return val
1.2. Writing a stripped-down model of tt_ind_solve_power that’s a precise implementation of the statistical derivation and produces the identical output as the unique operate
The supply file in Statsmodels is offered here. Whereas the unique operate is written to be extra highly effective, its generalizability additionally makes it tougher to realize instinct on how the code works.
I thus regarded via the supply code line-by-line and simplified it down from 1,600 traces of code to 160, and from 10+ features to only 2, whereas making certain the that implementation stays similar.
The stripped-down code comprises simply two features underneath the TTestIndPower class, precisely following the statistical derivation defined in Half 1:
- energy, which computes energy given a pattern dimension
- solve_power, which finds the minimal pattern dimension that achieves a goal energy utilizing Brent’s methodology
That is the complete code for the stripped-down model with a check to verify that it produces the identical output as the unique operate:
