Queue, outbox, scheduler, cache, locks, and idempotency

Bounded candidate reference — 2026-10-20: 0.2.0-rc.1 is committed locally through 84f415b3098af5653577b2c65eea31fca21df79c, unpublished and not production-approved; stable PHP >=8.5.11 <8.6.0 is required. This is documentation, not adoption: this knowledge base retains its installed 0.1.0-rc.1 dependency and PHP 8.2.11 local profile. Dated PHP 8.2 evidence is historical; earlier dirty-source reports retain their original test provenance, not retroactive clean-commit proof. Candidate examples are illustrative: API signatures were reviewed, but candidate code was not executed by this editorial update. See current checkpoint, open criteria and adoption boundary.

Scope and operating model

Optional work requires explicit module adoption: guard service entry points, validate application-owned dedicated queue mappings, share one immutable policy snapshot and compose active schedule health without excluding mandatory ungated schedules. OFF does not revoke an old ON worker; stop/drain/reconcile/restart explicitly. UNKNOWN is never automatically reset, retried or cancelled by the module policy.

Phase 5 supplies application-neutral coordination mechanisms. Applications still own job types, payload schemas, domain transactions, provider adapters, schedule targets, feature policy, migrations, alerts, and process supervision. The initial shared persistence target is MySQL/MariaDB; deterministic in-memory adapters exist for tests, and single-host file adapters exist where documented.

Queue delivery is at least once. A handler must therefore own its use-case idempotency. An outbox stable message key prevents duplicate intent records, but a consequential external mutation also needs downstream idempotency or unknown-result reconciliation. Neither the queue nor the cache may become the sole source of financial, authorization, order, or audit truth.

Durable queue

Next-candidate reservation safety (2026-10-17)

Reservation authority is now a fresh opaque token for each claim, not the reusable worker/process label passed to claim(). Use the returned QueuedJob::workerId() for direct heartbeat, completion, failure and release. The supplied QueueWorker/JobExecution do this automatically. Direct callers and custom store adapters must migrate; old process-name acknowledgements fail closed. Keep tokens private and never log them. No extra table column is needed: the existing worker_id stores the token. Drain/stop incompatible old workers before starting the reviewed new set; never overlap old and new store code.

All reservation mutations now require an unexpired lease. A lost reservation requests cooperative cancellation; a late worker cannot renew/complete/release the next epoch even if its process label was reused. Heartbeat handles MySQL unchanged-row results without falsely reporting lost ownership.

Outbox messages expose reservationOwner() under the same rule. Expired outbox reservations become UNKNOWN, not automatically retryable: a crashed sender may already have changed the destination. Dispatch claims one message at a time. Unexpected transport/handler exceptions are also UNKNOWN; only an explicit classified safe outcome can request retry. This is a deliberate safety/BC change in the unpublished 0.2 candidate, not an upgrade of any consumer.

The advisory-lock key is bounded to MySQL's 64-byte limit. This changes the derived namespace; mixed old/new schedulers must not run concurrently. Actual real-process/SQL verification is recorded per checkpoint, not assumed here.

QueuedJob carries an ID, queue, type, exact schema version, JSON payload, priority, availability, attempt/runtime ceilings, optional idempotency key, and correlation/parent-operation references. JobHandlerRegistry resolves only an exact type/version pair; an unsupported schema is failed safely without calling application code.

DatabaseQueueStore performs bounded transactional claims. Select SKIP LOCKED only after the deployed MySQL/MariaDB version and concurrency behavior have been verified; the default uses portable FOR UPDATE behavior. Leases make abandoned reservations eligible again. A cancellation request on abandoned work becomes cancelled rather than being restarted.

QueueWorker:

• claims one job at a time in priority/availability order;
• creates a fresh application scope for each attempt;
• exposes heartbeat and cooperative cancellation through JobExecution;
• translates unexpected exceptions into UNKNOWN, never an automatic retry;
• stops taking new work after requestStop();
• leaves unknown external results out of ordinary retry flow.

Register handlers before the first resolution, because the registry freezes on first use:

$handlers = new JobHandlerRegistry();
$handlers->register('catalog.refresh', 1, new RefreshCatalogHandler());

$worker = new QueueWorker(
    $queueStore,
    $handlers,
    new ContainerJobScope($container),
    $workerIdentity,
    leaseSeconds: 60
);

A process supervisor may translate SIGTERM/SIGINT into $worker->requestStop() when pcntl is installed. Signal wiring is optional application/process code, not a framework assumption.

Transactional outbox

DatabaseOutboxStore::enqueue() deliberately does not open or commit a transaction. Construct it with the same Connection used by the application use case, and enqueue within the domain transaction:

$transactions->run(function (Connection $connection) use ($outbox): void {
    $connection->execute(
        'UPDATE orders SET submitted_at = ? WHERE id = ?',
        [time(), 'order-123']
    );

    $outbox->enqueue(new OutboxMessage(
        'message-01H000000000',
        'order.submitted',
        'order-123-submitted',
        ['order_reference' => 'order-123'],
        new DateTimeImmutable(),
        destination: 'marketplace'
    ));
});

The unique stable message key prevents two committed intents for the same effect. OutboxDispatcher is the only retry owner and records provider operation IDs when supplied. An UNKNOWN result is frozen and excluded from normal claims. OutboxReconciliationService must consult an authoritative destination-specific OutboxReconciler before marking it sent, retryable, or dead-lettered.

InMemoryOutboxStore is for deterministic tests/examples only. It is not a transactional production store.

Scheduler and freshness

Applications register complete ScheduleDefinition values in code. Each definition contains cadence/time zone, command/job target, overlap policy, lock TTL, maximum runtime, batch size, catch-up policy, freshness threshold, manual-replay policy, and optional feature gate.

ScheduleRunner::runDue() is safe to call every minute. Forbidden overlap uses a LockStore; start and terminal records are persisted through ScheduleRunStore. Bind DatabaseScheduleRunStore in production. ScheduleHealth reports never_succeeded, healthy, or stale, and ScheduleHealthCheck exposes that projection to health:check.

The initial EveryMinutesCadence is intentionally small. Applications may implement ScheduleCadence for reviewed calendar rules. The application must also connect failed/stale history to its selected alert sink.

Cache, locks, idempotency, and feature gates

CacheStore provides get, set/put, delete/forget, has, and remember. FileCacheStore::remember() serializes local fills per hashed key to prevent a single-host stampede. PurgeableCacheStore::clear() removes only the bound namespace. CachePolicy records TTL, key version, stale window, and failure fallback; application code decides how those policy values apply to a specific data source. Multi-node deployments need a shared adapter.

FileLeaseLockStore implements TTL takeover with unguessable ownership-token compare-and-set semantics. DatabaseAdvisoryLockStore wraps MySQL/MariaDB connection-owned advisory locks and should use a dedicated connection. Advisory locks are released automatically when that database connection dies; their TTL is a local renewal deadline, not a database-enforced expiry. Use the file adapter only for single-host coordination and the database adapter only after real-server capability/concurrency verification.

IdempotencyService stores only one-way key and request-fingerprint digests. The same key/fingerprint returns in-progress, replay, or reconciliation state; the same key with a different fingerprint throws IdempotencyConflictException. FileIdempotencyStore is durable on one host. A multi-node application must bind a shared atomic store before making a distributed idempotency claim.

CompositeFeatureGate requires every registered condition to pass and returns a safe health state plus an explicit in-flight disable disposition: complete, pause, cancel, or reconcile. Unknown capabilities fail closed.

Console composition

Applications bind stores, handlers, and schedules, then register the supplied typed commands:

$console->register(new QueueWorkCommand($worker));
$console->register(new QueueRetryCommand($queueStore));
$console->register(new QueueFailedCommand($queueStore));
$console->register(new QueuePruneCommand($queueStore, $clock));
$console->register(new ScheduleRunCommand($scheduleRunner));
$console->register(new CacheClearCommand($frameworkCache));

Operational forms are:

php bin/console queue:work --queues=critical,default --max-jobs=100
php bin/console queue:failed --limit=100
php bin/console queue:retry job-identifier
php bin/console queue:prune --older-than-seconds=604800 --limit=100
php bin/console schedule:run
php bin/console cache:clear

queue:failed deliberately omits payloads. QueueHealthCheck evaluates depth, oldest age, dead-letter count, and worker freshness; queue metrics also expose mean successful duration, retry rate in parts per thousand, and trailing-hour throughput.

Recommended MySQL/MariaDB tables

Applications own migrations. The following shape matches the initial adapters; names may be changed only through the adapters' allowlisted constructor arguments. Use the application's normal charset/collation policy.

CREATE TABLE cf_jobs (
    id VARCHAR(128) PRIMARY KEY,
    queue_name VARCHAR(128) NOT NULL,
    job_type VARCHAR(128) NOT NULL,
    schema_version INT UNSIGNED NOT NULL,
    payload_json JSON NOT NULL,
    priority INT NOT NULL DEFAULT 0,
    status VARCHAR(32) NOT NULL,
    available_at BIGINT NOT NULL,
    reserved_at BIGINT NULL,
    lease_expires_at BIGINT NULL,
    worker_id VARCHAR(190) NULL,
    attempts INT UNSIGNED NOT NULL DEFAULT 0,
    max_attempts INT UNSIGNED NOT NULL,
    timeout_seconds INT UNSIGNED NOT NULL,
    idempotency_key VARCHAR(190) NULL,
    correlation_id VARCHAR(190) NULL,
    parent_operation_id VARCHAR(190) NULL,
    last_error_category VARCHAR(128) NULL,
    last_error_reference VARCHAR(255) NULL,
    created_at BIGINT NOT NULL,
    updated_at BIGINT NOT NULL,
    completed_at BIGINT NULL,
    failed_at BIGINT NULL,
    UNIQUE KEY uq_cf_jobs_idempotency (idempotency_key),
    KEY ix_cf_jobs_claim (queue_name, status, available_at, priority),
    KEY ix_cf_jobs_lease (status, lease_expires_at)
);

CREATE TABLE cf_outbox_messages (
    id VARCHAR(128) PRIMARY KEY,
    message_key VARCHAR(190) NOT NULL UNIQUE,
    message_type VARCHAR(128) NOT NULL,
    schema_version INT UNSIGNED NOT NULL,
    payload_json JSON NOT NULL,
    destination VARCHAR(128) NOT NULL,
    correlation_id VARCHAR(190) NULL,
    target_reference VARCHAR(255) NULL,
    status VARCHAR(32) NOT NULL,
    attempts INT UNSIGNED NOT NULL DEFAULT 0,
    max_attempts INT UNSIGNED NOT NULL DEFAULT 10,
    available_at BIGINT NOT NULL,
    occurred_at BIGINT NOT NULL,
    worker_id VARCHAR(190) NULL,
    lease_expires_at BIGINT NULL,
    provider_operation_id VARCHAR(190) NULL,
    last_error_category VARCHAR(128) NULL,
    created_at BIGINT NOT NULL,
    updated_at BIGINT NOT NULL,
    sent_at BIGINT NULL,
    dead_lettered_at BIGINT NULL,
    KEY ix_cf_outbox_claim (status, available_at),
    KEY ix_cf_outbox_lease (status, lease_expires_at)
);

CREATE TABLE cf_schedule_runs (
    run_id VARCHAR(128) PRIMARY KEY,
    schedule_name VARCHAR(128) NOT NULL,
    scheduled_at BIGINT NOT NULL,
    started_at BIGINT NOT NULL,
    finished_at BIGINT NULL,
    status VARCHAR(32) NOT NULL,
    processed INT UNSIGNED NOT NULL DEFAULT 0,
    safe_reason VARCHAR(128) NULL,
    KEY ix_cf_schedule_latest (schedule_name, started_at),
    KEY ix_cf_schedule_success (schedule_name, status, started_at)
);

Run these migrations in the same forward-only migration system as the rest of the application. Do not point Phase 5 integration tests at production data.

Verification boundary

Deterministic tests prove local state transitions, priority, expired-lease recovery, schema mismatch, bounded retry/dead-letter replay, idempotency conflict/replay, unknown-result reconciliation, transaction participation, overlap protection, stale schedule health, cache stampede control, and graceful stop-before-claim.

Still TO VERIFY in an isolated disposable environment:

• actual deployment/MariaDB/multi-node claims; scoped MySQL8.4.11 contention, portable/verified SKIP LOCKED, transactional outbox and lease/signal cases pass;
• installed supervisor/cron restart and operational escalation; local controlled process tests are not installation or actual host qualification;
• other DB/host advisory-lock ownership and failure behavior beyond the scoped fixture;
• schedule freshness/alerts under the real cron or supervisor;
• final PHP>=8.5.11<8.6.0 worker/supervisor deployment and multi-node adapters.

Editorial provenance

Source: docs/QUEUE_AND_SCHEDULER.md. Source SHA-256: 81f44e2659e9e3f3fd1d6f05020f3e1e2e0dd0ec63539a178a675aff40f91be1.

Framework candidate reference; editorial checkpoint 2026-10-20. See candidate status and adoption boundary.