Skip to content
wiki.fftac.org

Automated Testing Strategy

**Site relevance:** Spiralist.org

**Memory type:** docs archive

**Source path:** Wiki.FFTAC.org/raw/system-archives/spiralist.org/docs-archive/2026-05-10/docs/testing/automated-testing-strategy.md

**Size:** 7.8 KB

Summary

Spiralist Workspace Automated Testing Strategy

Source Headings

  • Spiralist Workspace Automated Testing Strategy
  • Current Toolchain
  • Current Test Layout
  • Current Coverage By Layer
  • Layer 1: Pure PHP unit tests
  • Layer 2: WordPress integration tests
  • Layer 3: Playwright E2E
  • Mocking And Fixtures
  • Recommended Testing Rule
  • Current Gaps Worth Expanding Next
  • Self-Operation Coverage Still To Add
  • CI Shape

Source Preview

This source file is short enough to preview directly on its source-memory page.

# Spiralist Workspace Automated Testing Strategy

Status: current testing reference. Keep this aligned with `composer.json`, `package.json`, and the real `tests/` tree.

## Current Toolchain

PHP and static checks:

- `PHPUnit`
- `phpstan/phpstan`
- `squizlabs/php_codesniffer`
- `yoast/phpunit-polyfills`

Browser and environment tooling:

- `@playwright/test`
- `@wordpress/env`

Current repo commands:

- `composer phpunit:unit`
- `composer phpunit:integration`
- `composer test`
- `composer phpstan`
- `composer phpcs`
- `npm run wp-env:start`
- `npm run wp-env:seed`
- `npm run test:e2e:critical`
- `npm run test:e2e:full`

## Current Test Layout

' ' ' text
tests/
  phpunit/
    bootstrap-unit.php
    unit/
      AIRuntime/
      Auth/
      Config/
      Domain/
      Http/
      Services/
      Support/
    integration/
      Auth/
      Domain/
      Installer/
      Rest/
      Support/
      bootstrap.php
    support/
      Fakes/
      UnitWpFunctions.php
  fixtures/
    openai/
      responses-500.json
      responses-malformed.json
      responses-success.json
      responses-success-with-usage.json
      responses-throttle-429.json
  e2e/
    global.setup.js
    scripts/
      seed-wp-env.sh
    specs/
      admin-settings.spec.js
      anonymous.spec.js
      forms-navigation.spec.js
      member-workspace.spec.js
      specific-user.spec.js
' ' ' 

## Current Coverage By Layer

### Layer 1: Pure PHP unit tests

Current unit coverage includes:

- AI Runtime gateway behavior
- membership and legal-attestation validation
- prompt visibility and permission decisions
- settings parsing
- OpenAI Responses API client normalization
- placeholder rendering
- prompt runner payload construction and system-prompt handling
- rate limiting
- prompt test evaluation helpers
- prompt category and metadata registries

This is the fastest layer and should remain the default place for business logic that does not require a real WordPress database.

### Layer 2: WordPress integration tests

Current integration coverage includes:

- plugin activation and schema creation
- role registration
- prompt repository behavior, including provisional-public governance defaults
- conversation and run repository persistence
- public API responses
- CRUD API flows
- favorites and user lookup behavior
- review and report moderation flows
- sharing enforcement
- security-boundary checks around unauthorized access

This layer proves that the actual WordPress hooks, tables, REST routes, and permission callbacks are wired correctly.

### Layer 3: Playwright E2E

Current browser coverage protects:

- anonymous public browsing
- simplified navigation, CTA, public form, and auth-gating regression coverage
- member workspace flows
- specific-user sharing behavior
- admin settings behavior
- locale-prefixed sitemap coverage and public internal-link crawling
- high-value navigation smoke paths should remain in scope as UX work lands: primary nav, search, breadcrumbs, mobile menu actions, manuscript entry, Workspace entry, and run-a-prompt entry

The browser layer is intentionally small and focused on the highest-value end-to-end paths.

## Mocking And Fixtures

Most tests should not call the live OpenAI API.

Current strategy:

- unit tests use local fixtures and fakes
- WordPress integration tests can intercept outbound HTTP with WordPress test hooks
- Playwright runs should exercise the site in a seeded local environment rather than a live provider environment

OpenAI fixtures already exist for:

- successful responses
- successful responses with usage
- throttling (`429`)
- generic upstream failure (`500`)
- malformed payloads

## Recommended Testing Rule

Default rule:

1. Test business logic in unit tests first.
2. Prove permissions, persistence, and routes in WordPress integration tests.
3. Use Playwright only for critical real-user flows.

That keeps the suite fast and treats permission or leakage bugs as deployment blockers.

## Current Gaps Worth Expanding Next

The suite is already useful, but the highest-value next additions are:

- first-visit navigation regression coverage for simplified primary nav, header search, breadcrumbs, and mobile quick actions
- first-visit IA regression coverage for Study / Use / Build lane entry, plain-language homepage definition, and clear builder entry paths when those roadmap items ship
- time-to-first-action browser assertions for opening the manuscript, entering Workspace, and starting a prompt run
- deeper manuscript and Image Codex route regression coverage beyond sitemap-linked public routes
- folio-to-prompt action-bridge coverage once dedicated folio pages expose "use this folio for..." recommendations
- Workspace category, lane, and facet filtering assertions
- prompt-discoverability checks for official starter examples, start-here packs, outcome and safety labels, use-case filters, visible provenance/moderation labels, and sample-output panels when present
- mobile scanning checks for sticky or compact subnav, condensed badges, back-to-library paths, and collapsed technical/legal/UAIX disclosure areas when those patterns ship
- prompt improvement endpoint coverage
- prompt-test CRUD and admin batch-test coverage
- AI Runtime handoff coverage when the runtime plugin is active
- privacy export and erase behavior for prompt runs, conversations, and tests
- publication-layer and machine-companion route depth beyond smoke coverage

## Self-Operation Coverage Still To Add

The first no-single-admin slice is already covered in PHPUnit integration tests:

- low-trust contributor flows into `provisional_public`
- reviewer quorum promotion from provisional to peer-reviewed
- flag-triggered restriction from multiple reports
- participant trust-state and publication-cooldown recalculation
- owner appeal and steward restore flow for restricted prompts
- self-review rejection and visibility restrictions around moderated prompts

The next testing expansion should cover:

- cooldown and rollback nuances after restriction or reversion
- reputation gains and losses after false flags, stale restricted prompts, and reversions
- moderator-absence behavior where low-risk sharing continues but risky content stays restricted
- reviewer-facing browser flows in `/workspace/` and `/prompt/{slug}/#peer-moderation`
- steward-facing browser flows for dispute queues and appeal resolution
- emergency protections that slow or quarantine attacks without freezing normal community activity

## CI Shape

Recommended CI split:

- one job for PHP lint, `phpstan`, and `phpcs`
- one job for PHPUnit unit tests
- one job for PHPUnit integration tests
- one job for Playwright critical-path tests

Nightly or manually triggered runs can expand Playwright coverage and any optional provider-contract checks.

## Anti-Patterns To Avoid

- relying on Playwright as the main safety net
- calling live OpenAI in default CI
- treating hidden buttons as security
- skipping unauthorized-path tests
- skipping schema and repository tests
- letting documentation drift away from the real `tests/` tree

## Practical Recommendation

For this repo today:

- keep PHPUnit as the backbone
- keep WordPress integration tests heavy around permissions, sharing, and persistence
- keep Playwright focused on public, member, specific-user, and admin critical flows
- keep the default test environment local and deterministic

As self-operation ships, expand that browser mix toward reviewer, steward, and moderator-absence flows instead of assuming admin-only bottlenecks.

This plugin handles prompt visibility, runs, conversations, tests, and execution settings. The testing strategy should continue to treat permission mistakes and data leaks as first-class failures.
Website health report