Opal Art Pipeline

This document defines the technical direction for Opal's artist-facing source pipeline. The goal is dependable iteration for production art, not guided content creation.

Product principles

• Source files remain editable in artists' existing tools.
• Asset IDs are stable references, independent from paths and display names.
• Reimport is inspectable: Opal can explain what changed before it mutates the project.
• Missing or broken sources degrade visibly, never destructively.
• Derived outputs are reproducible and retain the source and import settings that created them.
• Batch operations and automation use the same pipeline as the editor UI.

Milestone 1: source snapshot and diff

Implemented for local project asset folders:

1. Scan supported source files and record SHA-256 content hashes, size, modification time, and MIME type.
2. Compare the snapshot with imported asset records.
3. Classify each source as added, modified, moved, missing, restored, or unchanged.
4. Infer moves only for a unique source/record content-hash match.
5. Apply the diff while preserving asset IDs across modifications and moves.
6. Retain the last imported payload for missing sources and surface a non-blocking Project Health warning.

GET /api/projects/:id/assets/diff previews the next sync without changing records. POST /api/projects/:id/assets/sync applies that same model and returns applied counts plus the scanned summary.

Asset state model

Each folder-sourced asset stores:

• sourceFolderFile: project-relative source path
• sourceHash and sourceHashAlgorithm: identity of the imported bytes
• sourceSize and sourceMtimeMs: scan metadata
• sourceMissing and sourceMissingSince: retained-import state
• stable asset id: the reference used by scenes, prefabs, and graphs

Paths organize sources; they are not identity. Content hashes support conservative move detection; they are not a long-term substitute for explicit source identity.

Next milestones

2. Asset Changes workspace

• Review grouped additions, reimports, moves, ambiguous matches, and missing sources.
• Apply or ignore individual changes, or sync a folder as a batch.
• Show thumbnail, dimensions, alpha bounds, color-space changes, and dependent scenes/prefabs.
• Preserve a short local import history and allow rollback to the previous payload.

3. Import recipes and derived data

• Store per-asset import settings separately from source metadata.
• Add deterministic processors for trimming, padding, slicing, pivots, atlases, compression, and platform variants.
• Separate source assets from generated artifacts and rebuild only invalidated outputs.
• Run processors through a versioned job graph with cancellation and atomic commits.

4. Explicit source identity

• Introduce sidecar metadata with a generated source UUID for moves that cannot be inferred from content.
• Detect copies, source replacement, and conflicts without path heuristics.
• Support relinking a missing asset to a new source while retaining its asset ID and import recipe.

5. Toolchain integration

• Watch folders with debounced, race-safe scans instead of relying only on polling.
• Expose headless scan, diff, validate, and reimport commands for CI and build farms.
• Add processor/plugin APIs and structured diagnostics.
• Track dependencies so a source change invalidates only affected atlases, animations, prefabs, and builds.

Safety requirements before broad format support

• Stage new payloads before swapping records so failed imports leave the previous version intact.
• Validate content independently from filename extensions.
• Bound concurrency, file sizes, decode work, and generated output sizes.
• Reject path escapes and ignore symlinks in project source folders.
• Make sync serialization explicit so overlapping scans cannot race record writes.
• Include processor version and settings in derived-artifact cache keys.