DOE Statistical Sizing Engine | Test & Evaluation Power Design Tool

Model Formula (Sum Coded)

Candidate Space

0 constrained out

Model Rank / Params

Effective α: --

Selected Export Design

-- Runs

Condition: --

📈 Power Analysis Summary Across Conditions

Condition	Threshold N	Factorial N	Min Term Power	MC Wilson LB	D-Criterion	Status	Term-Level Breakdown	Quick Export
Click "Run Sizing & Optimization Engine" to compute results.

🎯 Threshold vs. Multivariate Factorial Power Curves

Red dashed = Target Power | Gray dotted = Recommended N

📝 TEMP-Ready Plain-English Narrative & Term Diagnostics

Run the engine to generate a TEMP-ready narrative.

💻 Engine Search & Monte Carlo Verification Log

Ready.

🛡️ Mode A: Visual Rule Builder (Disallowed Factor Combinations)

Build multi-factor operational/safety exclusion rules (e.g., Altitude == "Low" AND Speed == "Fast"). Any candidate treatment combination matching all conditions in a rule will be excluded before D-optimal sizing.

⌨️ Mode B: Custom Expression Filter (is_disallowed(row))

Supports Python/JS Syntax

Write a boolean expression exactly like the notebook's is_disallowed(row) function. Both Python style (row["F"] == "F1" and row["D"] == "D4") and JavaScript style (row.F === "F1" && row.D === "D4") are automatically supported.

Expression valid (No custom expression active).

☑️ Mode C: Interactive Candidate Space Table (Manual Row Toggle)

Review all 0 full-factorial combinations below. Uncheck any specific row to manually disallow it, or verify which rows are excluded by your Visual Rules / Custom Expression.

📋 Optimized DOE Run Matrix & Treatment Allocation

Inspect Scenario:

Scenario Condition

Recommended DOE Runs

Minimum Omnibus Power

Relative D-Criterion

Run Counts by Factor Combination

Run-by-Run Test Matrix

1. Purpose & Basic Workflow

The DOE Statistical Sizing Engine supports test planning and resource estimation by calculating the number of test runs required to achieve specified statistical power for both:

Threshold-Only Evaluations: One-sample baseline comparison against a requirement (ignores DOE structure).
Multivariate DOE Evaluations: D-optimal factorial sizing ensuring every modeled non-block factor and interaction independently achieves target power.

Execution Sequence: (1) Define response type & effect sizes → (2) Construct full candidate space & apply is_disallowed() constraints → (3) Build Sum-coded model matrix → (4) Optimize multi-start coordinate-exchange D-optimal designs → (5) Perform Coarse/Fine analytical search + Binomial GLM Likelihood-Ratio Monte Carlo validation (for proportional designs).

2. Proportional-Response Path (Binary Outcomes)

Use PROPORTIONAL when each run produces a binary outcome (Pass/Fail, Hit/Miss, Detect/No-Detect). Each run is modeled as Bernoulli(p_i) under a logistic regression framework:

logit(p_i) = log(p_i / (1 - p_i)) = X_i' β

Key Proportional Mechanics

Threshold Sizing: Uses Cohen's arcsine transformation h = 2*asin(sqrt(p1)) - 2*asin(sqrt(p0)) with a one-sample normal power approximation.
Robust Local Logistic D-Optimality: Maximizes det(X'WX) where W = diag(p_i*(1-p_i)) is the Bernoulli information weight averaged across term-specific alternative scenarios.
Multi-Level Factors: Uses a conservative midpoint-centered least-favorable coded-coefficient (LFC) scenario so active contrasts span the intended p0 to p1 range.
Monte Carlo LRT Validation & Wilson Bound: Simulates binary outcomes, fits full and reduced binomial GLMs via IRLS, and computes G2 = 2*(LL_full - LL_reduced). When USE_CONSERVATIVE_MC_BOUND is enabled, the one-sided Wilson lower confidence bound (e.g., 60%) must meet or exceed TARGET_POWER.

3. Continuous-Response Path (Numerical Outcomes)

Use CONTINUOUS for numerical measurements (Range, Miss Distance, Time, Latency, Error) assuming normal residuals y = X*β + ε.

SNR = Δ / σ = Detectable Difference / Residual Std Dev

Threshold Sizing: Exact one-sample noncentral t-test power calculation.
Factorial Sizing: Omnibus noncentral F-test across all non-block model terms (df1 = q, df2 = n - p), maximizing ordinary linear D-optimality det(X'X).

4. Optimization Modes & Full Factorial Option

Mode	How It Works	Best Used When
ITERATIVE	Generates a fresh D-optimal design at every candidate run count N.	Minimizing total run count is the top priority and unequal treatment allocation is acceptable.
REPLICATE	Generates a D-optimal base block and repeats it across cycles.	Phased test blocks, repeatable schedules, and operational simplicity are desired.
FORCE FULL FACTORIAL	Forces the base block to include every feasible candidate combination once.	Complete treatment coverage is required and the candidate space is small.

5. Blocking, Alpha & Multiplicity

Blocking Factor: Accounts for nuisance variation (Day, Crew, Aircraft, Site). Included in the regression model to isolate variance, but excluded from powered test requirements.
Alpha (α): Type I false-positive risk. Screening tests commonly use α = 0.20; confirmatory tests often use α = 0.05 or 0.10.
Multiplicity Method: NONE applies α independently to each term; BONFERRONI divides α across all modeled non-block terms (α_term = α / K).

6. Limitations & Validation Checklist

Assumes independent observations and fixed-effects model terms (random effects, repeated measures, or split-plot structures require mixed-effects modeling).
The exported D-optimal run order is an allocation sequence—randomize physical execution order where operationally appropriate while preserving treatment counts.
This tool supports test planning and resource estimation; it does not replace test authority, safety review, or formal statistical approval.

7. Frequently Asked Questions (FAQ)

Does the Threshold N determine the DOE run count?

No. Threshold N estimates the sample size for a single-sample comparison against baseline. Factorial N sizes the full multivariate experiment so that every modeled factor and interaction achieves target power.

Why is Factorial N often larger than Threshold N?

A factorial DOE must estimate multiple parameters across multiple factor levels and interactions simultaneously, requiring degrees of freedom and leverage across the entire design space.

What does a "Budget Limited" warning mean?

It means MAX_ALLOWABLE_RUNS was reached before every modeled term achieved target power. The tool outputs the best design at the ceiling along with the achieved power so test risk can be briefed transparently.

Point of Contact

Developed and unit-tested with 4 TES analysts. Report questions or edge cases to Lt Max Deihl (maxwell.deihl.1@spaceforce.mil). Current as of 10/09/2026.