Add ./output/ directory for debug verification artifacts and update development workflow
- Create ./output/ with .gitkeep for test run outputs - Add output/* to .gitignore (negated .gitkeep) - Add §4.5 Debug Verification to DEVELOPMENT.md: final step of each dev iteration - Run bs_pnpi -vv with full parameter consistency checks - Required debug logging per component documented - Example commands and success criteria - Update Agent Instructions and Release Checklist - Mark B8.3 debug verification framework as completed in TODO.md
This commit is contained in:
+100
@@ -111,6 +111,103 @@ Notebooks in `notebooks/` are executed as pytest tests using `nbmake`. This ensu
|
||||
pytest notebooks/ --nbmake # or just `pytest` (includes notebooks/)
|
||||
```
|
||||
|
||||
______________________________________________________________________
|
||||
|
||||
### 4.5. Debug Verification (Final Step of Each Dev Iteration)
|
||||
|
||||
After all quality gates pass, run a **debug-level verification** as the final step of every development iteration. This catches configuration mismatches, missing parameter propagation, and silent logic errors before merging.
|
||||
|
||||
**Purpose:** Verify that all parameters (endpoint energy, energy grid, Z, A, physical constants) are consistent throughout the pipeline, and that every spectrum component emits sufficient debug logging for explicit comparison.
|
||||
|
||||
**Procedure:**
|
||||
|
||||
1. **Run `bs_pnpi` with `-vv` (DEBUG logging) and output to `./output/`:**
|
||||
```bash
|
||||
bs_pnpi --nuclide Tc99 -vv --output ./output/verification.csv
|
||||
```
|
||||
|
||||
2. **Capture full debug output to a log file for analysis:**
|
||||
```bash
|
||||
bs_pnpi --nuclide Tc99 -vv --output ./output/verification.csv --log-file ./output/debug.log 2>&1 | tee ./output/debug_stdout.log
|
||||
```
|
||||
|
||||
3. **Verify consistency of key parameters in the debug output:**
|
||||
|
||||
| Parameter | What to check | Where to find it |
|
||||
|---|---|---|
|
||||
| `endpoint_MeV` | Matches expected Q-value | CLI dry-run, config creation, energy grid |
|
||||
| `Z_parent` / `Z_daughter` | Consistent across all components | Component init logs |
|
||||
| `A_number` | Consistent across all components | Component init logs |
|
||||
| `e_step_MeV` | Matches grid size × (endpoint - e_step) | Energy grid creation |
|
||||
| `W0` (endpoint in natural units) | `W0 = 1.0 + endpoint_MeV / m_e` | PhaseSpace init |
|
||||
| Physical constants | `m_e = 0.51099895` MeV, `α = 1/137.036` | Constants module, Fermi function |
|
||||
| Energy grid bounds | `e_step` to `endpoint - e_step` | `get_energy_grid()` output |
|
||||
|
||||
4. **Verify each component's debug logging:**
|
||||
|
||||
Every enabled component must log its initialization parameters and a representative evaluation. Check the debug output for entries like:
|
||||
|
||||
```
|
||||
DEBUG ... Initializing PhaseSpace: W0=1.57664, transition_type=A, m_e=1.0
|
||||
DEBUG ... Evaluating PhaseSpace at 294 energy points
|
||||
DEBUG ... PhaseSpace range: [min=..., max=...]
|
||||
```
|
||||
|
||||
Required log entries per component:
|
||||
|
||||
| Component | Must log |
|
||||
|---|---|
|
||||
| `PhaseSpace` | `W0`, `transition_type`, `m_e`, `m_nu`, energy range |
|
||||
| `FermiFunction` | `Z`, `A`, `α`, Coulomb approximation used |
|
||||
| `FiniteSizeL0` | `Z`, `A`, nuclear radius `R`, L0 approximation |
|
||||
| `ChargeDistributionU` | `Z`, `A`, charge radius, U correction range |
|
||||
| `ScreeningCorrection` | `Z_parent`, screening model, S(Z,W) range |
|
||||
| `ExchangeCorrection` | `Z_parent`, Hayen coefficients used, X(Z,W) range |
|
||||
| `RadiativeCorrection` | `W0`, `δ_r` formula, resummation flag, `delta_cut` value |
|
||||
|
||||
5. **Report any missing logging:**
|
||||
|
||||
If a component's debug output does not allow explicit verification of its parameters or computed range, **add logging to that component** before the iteration is considered complete. The goal is that a developer (or an LLM) reading the debug log can reconstruct every parameter and verify correctness without reading source code.
|
||||
|
||||
6. **Save debug artifacts to `./output/`:**
|
||||
|
||||
After verification, ensure all artifacts are in `./output/`:
|
||||
- `debug_stdout.log` — full stdout+stderr from `bs_pnpi -vv`
|
||||
- `debug.log` — log file output (if `--log-file` was used)
|
||||
- `verification.csv` — the spectrum CSV with metadata header
|
||||
- `verification.png` — analysis plot (if `--plot` was used)
|
||||
|
||||
These files serve as a **verifiable record** of the last working state and can be used for regression comparison.
|
||||
|
||||
**Example verification command:**
|
||||
|
||||
```bash
|
||||
# Full debug run with all outputs
|
||||
bs_pnpi --nuclide Tc99 -vv \
|
||||
--output ./output/verification.csv \
|
||||
--plot ./output/verification.png \
|
||||
--log-file ./output/debug.log 2>&1 | tee ./output/debug_stdout.log
|
||||
|
||||
# Quick parameter extraction from debug log
|
||||
grep -E '(Initializing|Evaluating|W0=|Z=|endpoint|e_step|PhaseSpace|Fermi|FiniteSize|Screening|Exchange|Radiative)' ./output/debug_stdout.log
|
||||
```
|
||||
|
||||
**Success criteria:**
|
||||
|
||||
- [ ] All parameters (Z, A, endpoint, e_step, W0) appear consistently in debug output
|
||||
- [ ] Each enabled component logs its initialization parameters
|
||||
- [ ] Each enabled component logs its evaluation energy range
|
||||
- [ ] Physical constants match expected values
|
||||
- [ ] No silent parameter coercion or unit conversion errors detected
|
||||
- [ ] All artifacts saved to `./output/`
|
||||
|
||||
**If any check fails:**
|
||||
|
||||
1. Add missing debug logging to the component
|
||||
2. Fix any parameter mismatch
|
||||
3. Re-run the verification until all checks pass
|
||||
4. Only then consider the iteration complete
|
||||
|
||||
## 5. Agent Instructions
|
||||
|
||||
When working on this project, agents must:
|
||||
@@ -123,6 +220,7 @@ When working on this project, agents must:
|
||||
6. **Work on a dev branch** — create `dev/<description>` and switch to it
|
||||
7. **Run quality gates** after making changes and report results
|
||||
8. **Update docs** when changing public API or adding features
|
||||
9. **Run debug verification** (`bs_pnpi -vv`) as the final step of each iteration, saving all artifacts to `./output/` and verifying parameter consistency across all components
|
||||
|
||||
## 6. Release Checklist
|
||||
|
||||
@@ -130,6 +228,8 @@ When working on this project, agents must:
|
||||
- [ ] Code is formatted (`black --check`)
|
||||
- [ ] No lint errors (`ruff check`)
|
||||
- [ ] Type checks pass (`mypy`)
|
||||
- [ ] Debug verification passes (`bs_pnpi -vv`) — all parameters consistent, all components logged
|
||||
- [ ] Debug artifacts saved to `./output/` (debug_stdout.log, verification.csv, etc.)
|
||||
- [ ] Update version in `pyproject.toml`
|
||||
- [ ] Update `Development Status` section in `README.md`
|
||||
- [ ] Update `TODO.md` marking corresponding items as complete.
|
||||
|
||||
Reference in New Issue
Block a user