v0.1.3 — Restore Basic Reliability¶
Status: implemented locally; not tagged, published, or pushed. Verification date: October 16, 2026. Proposed release deadline: October 23, 2026 (Asia/Karachi).
Delivered changes¶
| Requirement | Implementation |
|---|---|
| REL-01: Startup | Valid Python source, base-only public imports/help without service calls, lazy optional SDKs, and actionable missing-SDK errors |
| REL-02: Version | pyproject.toml is authoritative; runtime version comes from installed distribution metadata; wheel/source metadata identify 0.1.3 |
| REL-03: Configuration | Strict shapes/types/ranges, contextual active-provider validation, safe YAML errors, no CLI fallback, and preserved deliberate Python defaults |
| REL-04: Empty evaluation | Zero evaluated questions fail across CLI/scorer/facade; empty authoring still works; failed reruns invalidate previous reports |
| REL-05: Files | Candidate validation, preflight before generation, atomic per-file writes, permission preservation, explicit replacement, and unchanged existing data on tested ordinary failures |
| REL-06: Documentation | Corrected setup/configuration/CLI examples, local HTTP fixture, recovery instructions, and known limitations |
init copies hardcoded examples; it does not create questions from documents or embeddings. Use verified manual cases, capture, or document-based generation followed by review. See the quick start.
Compatibility changes¶
-
Requires Python 3.11 or newer. Python 3.10 support was removed after the initial reliability verification.
-
Invalid CLI settings no longer silently become defaults. Correct the reported field or selected path.
- No approved/matching questions means a failed evaluation, even when quality-failure exit behavior is disabled.
- Python users supplying an adapter may still omit the conventional default configuration file; custom missing files and broken links fail.
- Question-list replacement uses
generate --output PATH --overwrite. Output and auto-capture modes cannot be combined. - Invalid existing auto-capture goldens must be repaired or a new destination selected. They are never reset automatically.
- Use an editable installation for source development; runtime version resolution requires installed distribution metadata.
- No baseline database/schema migration is required.
Verification results¶
The following results are local runs, not GitHub-hosted CI results:
| Environment/check | Result |
|---|---|
| Historical Python 3.10.21 run, before the minimum-version change | 270 passed; 15 optional Chroma cases skipped |
| Python 3.11.16, full collected suite with integration enabled | 270 passed; 15 optional Chroma cases skipped |
| Python 3.12.14, full collected suite with integration enabled | 270 passed; 15 optional Chroma cases skipped |
| Source and changed/new test/example/helper lint | Passed |
| Strict MkDocs build | Passed; existing README/index conflict and broken placeholder links corrected |
| Locked Python 3.12 development sync | Passed; Python 3.10-only dependency branches removed |
| Wheel and source archive build | Passed |
| Clean base-only wheel installation outside checkout | Passed |
| Installed CLI local HTTP check → baseline → diff | Passed; intentional empty-tag evaluation failed as expected |
| Startup network/project-data side effects | None observed under the guarded smoke test |
| Runtime/distribution/source-archive version comparison | 0.1.3 throughout |
| Rebuilt package minimum and installer rejection | Requires-Python: >=3.11; Python 3.10 installation rejected |
The Chroma SDK was not installed in these environments. Its 15 optional integration cases were explicitly skipped by their existing fixture. Live cloud backends, paid question generation, Windows/macOS permissions, and the full all-extras CI environment were not exercised. GitHub Actions must still run after the user pushes.
Acceptance traceability¶
Test IDs refer to the v0.1.3 requirements document in the planning workspace.
| Cases | Evidence in the repository |
|---|---|
| T-01/T-02 | tests/unit/test_startup.py: guarded imports/help, source syntax, and missing SDKs for all five optional adapter families |
| T-03 | Installed runtime metadata test plus examples/verify_v013_release.py and clean artifact verification |
| T-04–T-08 | tests/unit/test_config.py: load/default rules, invalid scalar/section values, active/inactive credentials, typed objects, and secret sentinels |
| T-09–T-12 | tests/unit/test_empty_evaluation.py and scorer tests: selection failure, draft inclusion, direct scorer, stale reports, and watch continuation |
| T-13/T-14 | tests/unit/test_cli_file_safety.py: invalid/unreadable preflight, zero paid calls, merge preservation, new drafts |
| T-15/T-16 | tests/unit/test_file_safety.py: serialization, stream write/flush, sync and replacement failure injection; Unicode, nested paths, cleanup, read-only and symbolic-link handling |
| T-17–T-19 | CLI file-safety tests: initialization preservation/force, export overwrite rules, and zero-capture byte preservation |
| T-20 | tests/integration/test_reliability_workflow.py: local HTTP capture → approval → check → baseline → diff, plus regression and preservation paths |
Atomic saves are scoped to a single regular-file writer. Tests cover POSIX mode preservation and injected permission failures; no claim is made that every ACL/filesystem/power-loss condition has been exercised.
Reproducing the checks¶
From the project root:
uv sync --locked --dev --python 3.12
uv run pytest tests/ --run-integration -q
uv run ruff check src/
uv build
The local HTTP integration requires permission to bind a localhost socket. Base installation and artifact verification should run separately from a development environment:
uv venv /tmp/longprobe-wheel-check --python 3.12
uv pip install --python /tmp/longprobe-wheel-check/bin/python dist/longprobe-0.1.3-py3-none-any.whl
/tmp/longprobe-wheel-check/bin/python -I examples/verify_v013_release.py
The verification script uses the installed wheel rather than the editable checkout, confirms optional SDKs/test dependencies are absent, starts the synthetic HTTP example, and prints artifact SHA-256 hashes. Keep the wheel environment base-only. Supported-version tests run on Python 3.11 and 3.12. The earlier Python 3.10 result is historical and does not indicate current support.
Release identity and disposition¶
Read-only checks on October 16, 2026 returned PyPI versions 0.1.0 and 0.1.1, and public repository tags v0.1.0 and v0.1.1. Sources: PyPI metadata, GitHub tags API. Version availability must be rechecked before tagging/publishing because public state can change.
The syntax-breaking URL was a pre-existing uncommitted addition in this local checkout. It was preserved in a temporary backup and removed, restoring the tracked valid adapter source. No unrelated initial working-tree changes were present.
Local disposition: implementation checks pass for the specified reliability scope. Publication remains a maintainer action after remote CI and release review. No push, tag creation, or package publication was performed.
Remaining limitations¶
- Normal evaluation does not wire
embeddersettings into vector-only backends. Pinecone/Qdrant text-question integration remains deferred. - Semantic matching is word-frequency similarity, not neural embedding matching.
- The default pytest adapter fixture still needs an override.
- Baseline-save quality policies differ between CLI/Python; baseline compatibility and equal-recall evidence-change detection are unchanged.
- JSON stdout may still include notices/trailing command output; general cleanup is deferred.
- Explain does not calculate rank/score deltas. Watch still reacts only to golden-file changes.
- No automatically verified ground truth is promised for LLM-generated/captured evidence; review drafts before approval.
Verified artifact hashes¶
These identify the locally built artifacts used by the final clean-install verifier.
| Artifact | SHA-256 |
|---|---|
longprobe-0.1.3-py3-none-any.whl |
5788213aab9c4a6064ceaf1d14cdb47732c7293faeb395e7db68955cbcebae08 |
longprobe-0.1.3.tar.gz |
ea89a8cf9aa266963981e5b1d1f544510e7f73303c357a59f897656f5a03a40c |