Skip to content

v0.1.3 — Restore Basic Reliability

Status: implemented locally; not tagged, published, or pushed. Verification date: October 16, 2026. Proposed release deadline: October 23, 2026 (Asia/Karachi).

Delivered changes

Requirement Implementation
REL-01: Startup Valid Python source, base-only public imports/help without service calls, lazy optional SDKs, and actionable missing-SDK errors
REL-02: Version pyproject.toml is authoritative; runtime version comes from installed distribution metadata; wheel/source metadata identify 0.1.3
REL-03: Configuration Strict shapes/types/ranges, contextual active-provider validation, safe YAML errors, no CLI fallback, and preserved deliberate Python defaults
REL-04: Empty evaluation Zero evaluated questions fail across CLI/scorer/facade; empty authoring still works; failed reruns invalidate previous reports
REL-05: Files Candidate validation, preflight before generation, atomic per-file writes, permission preservation, explicit replacement, and unchanged existing data on tested ordinary failures
REL-06: Documentation Corrected setup/configuration/CLI examples, local HTTP fixture, recovery instructions, and known limitations

init copies hardcoded examples; it does not create questions from documents or embeddings. Use verified manual cases, capture, or document-based generation followed by review. See the quick start.

Compatibility changes

  • Requires Python 3.11 or newer. Python 3.10 support was removed after the initial reliability verification.

  • Invalid CLI settings no longer silently become defaults. Correct the reported field or selected path.

  • No approved/matching questions means a failed evaluation, even when quality-failure exit behavior is disabled.
  • Python users supplying an adapter may still omit the conventional default configuration file; custom missing files and broken links fail.
  • Question-list replacement uses generate --output PATH --overwrite. Output and auto-capture modes cannot be combined.
  • Invalid existing auto-capture goldens must be repaired or a new destination selected. They are never reset automatically.
  • Use an editable installation for source development; runtime version resolution requires installed distribution metadata.
  • No baseline database/schema migration is required.

Verification results

The following results are local runs, not GitHub-hosted CI results:

Environment/check Result
Historical Python 3.10.21 run, before the minimum-version change 270 passed; 15 optional Chroma cases skipped
Python 3.11.16, full collected suite with integration enabled 270 passed; 15 optional Chroma cases skipped
Python 3.12.14, full collected suite with integration enabled 270 passed; 15 optional Chroma cases skipped
Source and changed/new test/example/helper lint Passed
Strict MkDocs build Passed; existing README/index conflict and broken placeholder links corrected
Locked Python 3.12 development sync Passed; Python 3.10-only dependency branches removed
Wheel and source archive build Passed
Clean base-only wheel installation outside checkout Passed
Installed CLI local HTTP check → baseline → diff Passed; intentional empty-tag evaluation failed as expected
Startup network/project-data side effects None observed under the guarded smoke test
Runtime/distribution/source-archive version comparison 0.1.3 throughout
Rebuilt package minimum and installer rejection Requires-Python: >=3.11; Python 3.10 installation rejected

The Chroma SDK was not installed in these environments. Its 15 optional integration cases were explicitly skipped by their existing fixture. Live cloud backends, paid question generation, Windows/macOS permissions, and the full all-extras CI environment were not exercised. GitHub Actions must still run after the user pushes.

Acceptance traceability

Test IDs refer to the v0.1.3 requirements document in the planning workspace.

Cases Evidence in the repository
T-01/T-02 tests/unit/test_startup.py: guarded imports/help, source syntax, and missing SDKs for all five optional adapter families
T-03 Installed runtime metadata test plus examples/verify_v013_release.py and clean artifact verification
T-04–T-08 tests/unit/test_config.py: load/default rules, invalid scalar/section values, active/inactive credentials, typed objects, and secret sentinels
T-09–T-12 tests/unit/test_empty_evaluation.py and scorer tests: selection failure, draft inclusion, direct scorer, stale reports, and watch continuation
T-13/T-14 tests/unit/test_cli_file_safety.py: invalid/unreadable preflight, zero paid calls, merge preservation, new drafts
T-15/T-16 tests/unit/test_file_safety.py: serialization, stream write/flush, sync and replacement failure injection; Unicode, nested paths, cleanup, read-only and symbolic-link handling
T-17–T-19 CLI file-safety tests: initialization preservation/force, export overwrite rules, and zero-capture byte preservation
T-20 tests/integration/test_reliability_workflow.py: local HTTP capture → approval → check → baseline → diff, plus regression and preservation paths

Atomic saves are scoped to a single regular-file writer. Tests cover POSIX mode preservation and injected permission failures; no claim is made that every ACL/filesystem/power-loss condition has been exercised.

Reproducing the checks

From the project root:

uv sync --locked --dev --python 3.12
uv run pytest tests/ --run-integration -q
uv run ruff check src/
uv build

The local HTTP integration requires permission to bind a localhost socket. Base installation and artifact verification should run separately from a development environment:

uv venv /tmp/longprobe-wheel-check --python 3.12
uv pip install --python /tmp/longprobe-wheel-check/bin/python dist/longprobe-0.1.3-py3-none-any.whl
/tmp/longprobe-wheel-check/bin/python -I examples/verify_v013_release.py

The verification script uses the installed wheel rather than the editable checkout, confirms optional SDKs/test dependencies are absent, starts the synthetic HTTP example, and prints artifact SHA-256 hashes. Keep the wheel environment base-only. Supported-version tests run on Python 3.11 and 3.12. The earlier Python 3.10 result is historical and does not indicate current support.

Release identity and disposition

Read-only checks on October 16, 2026 returned PyPI versions 0.1.0 and 0.1.1, and public repository tags v0.1.0 and v0.1.1. Sources: PyPI metadata, GitHub tags API. Version availability must be rechecked before tagging/publishing because public state can change.

The syntax-breaking URL was a pre-existing uncommitted addition in this local checkout. It was preserved in a temporary backup and removed, restoring the tracked valid adapter source. No unrelated initial working-tree changes were present.

Local disposition: implementation checks pass for the specified reliability scope. Publication remains a maintainer action after remote CI and release review. No push, tag creation, or package publication was performed.

Remaining limitations

  • Normal evaluation does not wire embedder settings into vector-only backends. Pinecone/Qdrant text-question integration remains deferred.
  • Semantic matching is word-frequency similarity, not neural embedding matching.
  • The default pytest adapter fixture still needs an override.
  • Baseline-save quality policies differ between CLI/Python; baseline compatibility and equal-recall evidence-change detection are unchanged.
  • JSON stdout may still include notices/trailing command output; general cleanup is deferred.
  • Explain does not calculate rank/score deltas. Watch still reacts only to golden-file changes.
  • No automatically verified ground truth is promised for LLM-generated/captured evidence; review drafts before approval.

Verified artifact hashes

These identify the locally built artifacts used by the final clean-install verifier.

Artifact SHA-256
longprobe-0.1.3-py3-none-any.whl 5788213aab9c4a6064ceaf1d14cdb47732c7293faeb395e7db68955cbcebae08
longprobe-0.1.3.tar.gz ea89a8cf9aa266963981e5b1d1f544510e7f73303c357a59f897656f5a03a40c