RawCull Tech Documentation

• 1: Thumbnails and Scan Pipeline
• 2: Memory Cache
• 3: Memory Pressure
• 4: Concurrency
• 5: Focus Mask and Sharpness
• 6: Detailed Sharpness Scoring
• 7: Detailed Focus Mask Computation
• 8: Burst Groups
• 9: RawCull Packages
• 9.1: How PhotoAIKit Is Constructed
• 9.2: How PhotoAnalysisKit Is Constructed
• 9.3: How RawCullCore Is Constructed
• 9.4: How RawParserKit Is Constructed
• 10: Artificial Intelligence
• 10.1: How RawCull Loads and Uses CLIP
• 10.2: AI Model Download Service
• 10.3: AI Model Licence
• 10.4: Evaluating CLIP Models
• 10.5: New RawCull AI models
• 10.6: CLIP Model Evaluation Results
• 11: File Read and Write Reference
• 12: Synchronous Code
• 13: Repository Git Workflow
• 14: Sony/Nikon MakerNote Parser
• 15: Security-Scoped URLs

flowchart LR
    UI["SwiftUI views"] --> VM["RawCullViewModel and feature view models"]
    VM --> Actors["Actors for scan, cache, thumbnail, export, JSON writes"]
    Actors --> Parser["RawParserKit: ARW/NEF dispatch and extraction"]
    VM --> Core["RawCullCore: value models and pure algorithms"]
    Actors --> Disk["Caches, savedfiles.json, security-scoped folders"]
    VM --> Core

1 - Thumbnails and Scan Pipeline

Thumbnails and Scan Pipeline

RawCull separates catalog scanning, thumbnail preloading, UI-driven thumbnail requests, and full-size preview extraction. They share RawParserKit and disk-cache infrastructure, but they have deliberately different memory-admission rules.

The key current invariant is: catalog preloading warms the 200 px grid cache and disk cache, but only UI demand may admit an image to the preview-size RAM cache. This keeps scan order from displacing the images the user is actively viewing.

Source Map

Catalog Load Flow

flowchart TD
    A["User selects catalog folder"] --> B["startCatalogLoad"]
    B --> C["Acquire security-scoped access"]
    C --> D["ScanFiles.scanFiles"]
    D --> E["RawImageLoading.fileMetadata per file"]
    E --> F["Sort and filter on MainActor"]
    F --> G["Load ratings and valid saved scores"]
    G --> H["ScanAndCreateThumbnails.preloadCatalog"]
    H --> I["200 px grid RAM cache"]
    H --> J["JPEG thumbnail disk cache"]
    F --> K["SwiftUI grid and detail views"]
    K --> L["ThumbnailLoader / RequestThumbnail"]
    L --> M["Preview RAM cache"]
    L --> J

Catalog Load Ownership

RawCullViewModel.startCatalogLoad(for:) cancels the older load, starts security-scoped access, and creates a background-priority catalogLoadTask that runs handleSourceChange(url:).

handleSourceChange(url:) is main-actor orchestration. File I/O, metadata work, decoding, and cache I/O are awaited through actors or detached work. After every significant suspension, isActiveCatalogLoad(_:) and cancellation checks prevent an old catalog from publishing into the current UI.

Changing catalogs also cancels thumbnail/full-preview work, clears burst analysis state, stops the old security scope, and releases the current scanning actors.

File Discovery

DiscoverFiles enumerates a directory off the main actor and filters extensions through RawFormatRegistry.allExtensions. Its recursive parameter controls whether subdirectories are traversed.

The current registry contains:

ScanFiles performs its own non-recursive directory listing and asks RawFormatRegistry.format(for:) whether each item is supported. Discovery and scan therefore share registry policy rather than maintaining separate hard-coded extension lists.

Metadata Scan

ScanFiles.scanFiles(url:onProgress:) starts its own security-scoped access and creates one child task per supported file. Each child:

1. reads URL resource values for name, byte size, content type, and modification date;
2. calls the injected RawImageLoading.fileMetadata(for:) abstraction;
3. builds a FileItem with EXIF metadata and the normalized AF point;
4. records the package focus-location string when available.

The production adapter, RawParserKitImageLoader, maps RawParserKit.RawImageLoader.metadata(for:) into the app’s ExifMetadata. RawParserKit reads ImageIO EXIF/TIFF data, dispatches body-specific details through RawFormatRegistry, and resolves MakerNote or EXIF subject-area focus evidence.

This is a single metadata pass per file. The app no longer runs separate EXIF and MakerNote extraction passes.

If no native focus-location strings are found for the catalog, ScanFiles falls back to focuspoints.json. This is catalog-wide fallback behavior; it does not merge JSON entries into a partly populated native result.

FileItem is an app typealias for RawCullCore.RawCullFileItem, keeping scan output usable by the pure package engines and tests.

Thumbnail Preload

After scan, sort, ratings, and valid saved scores are applied, ScanAndCreateThumbnails.preloadCatalog(at:targetSize:) walks the catalog again to warm browsing caches. A catalog URL already recorded in processedURLs is not preloaded again during the same app session.

The outer task group admits at most:

RawImageLoadingConcurrency.batchExtractionLimit
= max(1, activeProcessorCount * 2)

RawParserKit adds an internal six-slot thumbnail decode limiter and coalesces identical URL/size requests. The outer limit bounds catalog work; the package limit prevents simultaneous RAW decodes from growing without bound.

For each file, preload uses this path:

flowchart LR
    A["RAW URL"] --> B{"Preview RAM hit?"}
    B -->|"yes"| C["Touch preview LRU"]
    B -->|"no"| D{"Disk JPEG hit?"}
    D -->|"yes"| E["Decode oriented JPEG"]
    D -->|"no"| F["RawImageLoading.thumbnailImage"]
    C --> G["Populate 200 px grid cache"]
    E --> G
    F --> G
    F --> H["Encode JPEG data"]
    H --> I["Background atomic disk save"]

All three branches populate the dedicated grid cache when needed. Disk and source branches do not insert into the preview-size RAM cache. RequestThumbnail is its only admitter, so preview LRU ordering reflects user demand instead of scan order.

On a cold extraction, ScanAndCreateThumbnails converts the actor-owned image to JPEG Data before launching the background disk save. This avoids sending CGImage or NSImage across the detached-task boundary.

The Two Memory Caches

SharedMemoryCache owns two NSCache instances:

Both caches are cost-limited. The preview item count limit is intentionally high so byte cost is the normal binding constraint. Under warning memory pressure both limits shrink to 60%; under critical pressure both caches are cleared and the preview cache is temporarily capped at 50 MiB.

UI-Driven Thumbnail Loading

ThumbnailImageView has two routes:

• .grid calls the shared ThumbnailLoader actor;
• .list calls RequestThumbnail directly.

For grid targets of 200 px or less, ThumbnailLoader first checks the grid cache without acquiring a concurrency slot. A miss joins its FIFO-like slot queue, which allows at most six active requests and removes cancelled waiters safely.

After a slot is acquired, the loader reads saved settings and asks RequestThumbnail for settings.thumbnailSizePreview. The passed grid target controls the grid-cache fast path; the slower preview request uses the configured preview size.

RequestThumbnail resolves a request in this order:

1. preview RAM cache;
2. oriented JPEG disk cache;
3. RawImageLoading.thumbnailCGImage source decode.

A disk hit is promoted into preview RAM. A cold source decode is inserted into preview RAM and asynchronously encoded to the thumbnail disk cache. The cache also records UI demand, cold extraction, eviction, and “boomerang” diagnostics for measuring whether a recently evicted image had to be reloaded from disk.

Disk Thumbnail Cache

DiskCacheManager stores quality-0.7 JPEGs. Its key is an MD5 filename hash of a version string plus the standardized source path. MD5 is used only as a compact filesystem key, not for security.

The current key version includes oriented-thumbnail semantics, so thumbnails from older orientation behavior are automatically invalidated. Loads use OrientationNormalizedImageLoader, and writes are atomic.

Full-Size And Zoom Preview Paths

Full-size inspection does not use the normal thumbnail disk cache.

flowchart TD
    A["Zoom / comparison source"] --> B{"Source choice"}
    B -->|"Thumbnail"| C["RequestThumbnail"]
    B -->|"Embedded JPEG"| D["FullSizePreviewLoader"]
    B -->|"Developed RAW"| E["ZoomPreviewHandler"]
    D --> F{"Sidecar JPEG exists?"}
    F -->|"yes"| G["Load oriented sidecar"]
    F -->|"no"| H{"Full-size disk cache hit?"}
    H -->|"yes"| I["Load cached embedded preview"]
    H -->|"no"| J["RawImageLoading.previewCGImage"]
    J --> K["Save embeddedJPG variant"]
    E --> L{"developedRAW cache hit?"}
    L -->|"no"| M["SonyRawFormat.createFullSizeJPEG"]

FullSizePreviewLoader prefers a same-basename .jpg sidecar, then the full-size disk cache, then RawParserKit extraction. RawParserKit itself coalesces duplicate preview requests and limits expensive full-size decode work to two concurrent operations.

The developed-RAW route is currently Sony-specific and uses its own cache variant. The full-size cache stores quality-0.85 JPEG data and has a versioned orientation-aware key. Keeping these larger images on disk prevents a few pixel-peeping previews from evicting many browsing thumbnails from RAM.

App Loading Abstraction

The app depends on the RawImageLoading protocol rather than calling vendor conformers directly:

RawParserKitImageLoader is the production adapter. Tests can inject alternate loaders without performing real RAW decoding.

Within RawParserKit, RawFormat provides vendor policy:

extractFullJPEG remains only as a deprecated compatibility requirement; new code uses extractEmbeddedPreview.

What To Check When Changing This Area

• Preserve the rule that scan/preload does not admit disk or source results to preview RAM.
• Check both the grid-cache fast path and configured preview-size path when changing thumbnail settings.
• Keep RawFormatRegistry.allExtensions and RawFormatRegistry.all aligned when adding a format.
• Keep expensive decode limits and cancellation behavior covered by concurrency tests.
• Convert actor-owned images to Data before detached cache writes.
• Version cache keys when orientation or encoded-image semantics change.
• Inspect isActiveCatalogLoad(_:) guards if stale results appear after switching folders.

2 - Memory Cache

Cache System

RawCull uses several caches because RAW decoding is expensive and because different UI surfaces need different image sizes. The normal thumbnail path is layered RAM -> disk -> RAW extraction. The zoom path has its own disk cache for larger embedded JPEGs. Burst analysis has a separate JSON cache for derived analysis data.

Source Map

Thumbnail Lookup

flowchart TD
    A["Request thumbnail"] --> B{"Preview memory cache?"}
    B -->|"hit"| C["Return NSImage"]
    B -->|"miss"| D{"Disk thumbnail cache?"}
    D -->|"hit"| E["Decode JPEG and refill memory"]
    D -->|"miss"| F["RawParserKit extractThumbnail"]
    F --> G["Create NSImage"]
    G --> H["Store preview memory"]
    G --> I["Store grid memory copy"]
    G --> J["Save disk JPEG"]

The same lookup idea is used by preload and on-demand requests. ScanAndCreateThumbnails warms the cache after a catalog opens. ThumbnailLoader and RequestThumbnail resolve individual thumbnails while the user scrolls.

CachedThumbnail

CachedThumbnail wraps an immutable NSImage and stores its cache cost.

The cost is calculated from image representations:

cost = sum(rep.pixelsWide * rep.pixelsHigh * 4) * 1.1

The 4 is SharedMemoryCache.costPerPixel, fixed for RGBA. The 1.1 multiplier gives a small overhead buffer for the wrapper and image metadata.

CachedThumbnail is @unchecked Sendable. The project invariant is: construct the NSImage fully before caching it, then treat it as immutable.

The class intentionally does not conform to NSDiscardableContent. Earlier versions did, but diagnostics showed NSCache discarded entries too aggressively and destroyed the RAM hit rate. Eviction is now controlled by explicit cache limits and memory-pressure handling.

SharedMemoryCache

SharedMemoryCache is an actor singleton:

actor SharedMemoryCache {
    nonisolated static let shared = SharedMemoryCache()
}

It is still an actor because it owns configuration, pressure monitoring, disk-cache references, and statistics. Its two NSCache objects are marked nonisolated(unsafe) because NSCache is already thread-safe and the app needs synchronous lookup APIs.

That gives RawCull both:

• actor isolation for mutable app-owned state,
• fast synchronous cache reads for hot thumbnail paths.

Adaptive Limits

CacheRecommendationPolicy chooses cache caps from physical memory, current used memory, user maximums, and memory pressure.

Baseline limits:

At normal pressure, RawCull calculates available headroom:

freeMB = physicalMB - usedMB
expandableMB = max(0, freeMB - 3072)
extraBudgetMB = expandableMB * 0.5
previewMB = baselinePreview + extraBudgetMB * 0.65
gridMB = baselineGrid + extraBudgetMB * 0.35

Values are rounded up to 256 MB steps and capped by both the machine tier and user settings.

At warning or critical pressure, the policy falls back toward 60 percent of the baseline, again respecting user limits and minimum settings.

Eviction Tracking

CacheDelegate is shared by both NSCache instances. It records three counters:

The delegate also calls back into SharedMemoryCache to decrement manual cost/count mirrors. This is necessary because NSCache does not expose current cost or item count.

For preview-cache evictions, the delegate stores the evicted URL in a recent-eviction ring. If the same URL is requested shortly after and falls back to disk, diagnostics can label that as a boomerang miss: the item was useful, but the cache evicted it too soon.

Memory Pressure

SharedMemoryCache owns a DispatchSourceMemoryPressure.

The current pressure level is stored behind an OSAllocatedUnfairLock and exposed as a synchronous nonisolated property. This lets MemoryViewModel and diagnostics read pressure state without an actor hop.

Disk Caches

DiskCacheManager stores generated thumbnail JPEGs. It is used by both preload and on-demand thumbnail loading.

FullSizeJPGDiskCache stores larger embedded previews for the zoom overlay. It is deliberately separate from the normal thumbnail cache because zoom images are larger and should not compete with grid scrolling for memory.

Burst Analysis Cache

BurstAnalysisCache is not an image cache. It stores analysis artifacts under Application Support:

• Vision feature-print embeddings,
• sharpness scores and saliency info,
• burst groups and boundary evidence,
• ranking results,
• review states.

A snapshot is accepted only when the catalog, file count, file sizes/modification dates, thumbnail size, sharpness signature, similarity signature, schema version, and algorithm version still match.

What To Check When Changing This Area

• Keep CachedThumbnail immutable after insertion.
• Update manual cost/count mirrors whenever adding or removing cache entries.
• Check diagnostics if cache hit rate drops; boomerang misses often point to a limit or pollution problem.
• Do not feed zoom-preview images into the normal thumbnail RAM cache unless you intentionally want them competing with grid thumbnails.
• When changing scoring or burst algorithms, bump signatures/versions so stale burst-analysis cache data is rejected.

3 - Memory Pressure

Memory Pressure

RawCull can hold many images in memory while scanning and culling. The memory-pressure code exists to keep the app responsive when macOS reports pressure and to make cache behavior visible during development.

Source Map

Runtime Flow

flowchart TD
    A["SharedMemoryCache.ensureReady"] --> B["startMemoryPressureMonitoring"]
    B --> C["DispatchSourceMemoryPressure"]
    C --> D{"event"}
    D -->|"normal"| E["refreshConfig from settings/adaptive policy"]
    D -->|"warning"| F["shrink cache limits to 60 percent"]
    D -->|"critical"| G["clear memory caches and set 50 MB floor"]
    F --> H["Notify file handler warning"]
    G --> H
    E --> I["Clear warning"]

SharedMemoryCache starts monitoring lazily during ensureReady(). That is called before thumbnail cache use, so the memory-pressure handler is active before large thumbnail work begins.

Measuring System Memory

MemoryViewModel.getUsedSystemMemory() uses Mach VM statistics:

usedMemory = min((wired + active + compressed) * pageSize, physicalMemory)

The model intentionally uses a simple definition of “used”:

The result is clamped to physical memory so the UI cannot show impossible totals.

Measuring App Memory

RawCull measures its own process footprint with:

task_info(... TASK_VM_INFO ...).phys_footprint

That is the value displayed as app memory usage. It is a better signal than simply adding image sizes because it reflects what the kernel says the process currently costs.

Percentages In The UI

MemoryViewModel exposes:

memoryPressurePercentage = memoryPressureThreshold / totalMemory * 100
usedMemoryPercentage = usedMemory / totalMemory * 100
appMemoryPercentage = appMemory / usedMemory * 100

The default pressure threshold is 85 percent of physical memory:

memoryPressureThreshold = totalMemory * 0.85

This UI threshold is informational. The actual pressure response is driven by macOS DispatchSourceMemoryPressure events.

Cache Limit Policy

CacheRecommendationPolicy.adaptiveLimits(...) is the first place to inspect when tuning memory behavior.

At normal pressure:

1. Choose a baseline from physical RAM.
2. Estimate free memory from Mach stats.
3. Keep a 3 GB reserve.
4. Use half of the remaining expandable memory as extra cache budget.
5. Split that extra budget 65 percent preview cache and 35 percent grid cache.
6. Round to 256 MB steps.
7. Clamp by machine tier and user maximums.

At non-normal pressure, RawCull returns reduced baseline limits and then the live pressure handler may shrink or clear active caches immediately.

Pressure Responses

The grid cache is cleared on critical pressure too. The disk caches are not deleted; only RAM is released.

Why Pressure State Is Lock-Backed

currentPressureLevel is read by UI and diagnostics without await. The value is stored in an OSAllocatedUnfairLock, which makes the read/write contract explicit while avoiding a main-actor or cache-actor hop during frequent sampling.

The same pattern is used for pressure counters and cache cost/count mirrors.

What To Check When Changing This Area

• Memory sampling should stay off the main actor; Mach calls run in Task.detached.
• Do not rely only on the 85 percent UI threshold; the real emergency signal is the kernel pressure event.
• If cache limits look strange, inspect both user settings and the adaptive tier caps.
• If diagnostics miss short pressure spikes, use cumulative pressure counters rather than only sampled pressure labels.
• Critical pressure should free RAM quickly and should not delete disk caches.

4 - Concurrency

Concurrency

This page documents concurrency in the RawCullAIModels branch of RawCull and in the first-party Swift packages checked out beside it. The audit covers production source in the RawCull app, DecodeEncodeGeneric, ParseRsyncOutput, PhotoAIKit, PhotoAnalysisKit, RawCullCore, RawParserKit, RsyncArguments, RsyncProcessStreaming, and RsyncAnalyse.

RawCull uses concurrency to keep SwiftUI responsive while it scans folders, reads RAW metadata, extracts and caches previews, calculates sharpness, creates similarity artifacts, groups bursts, runs AI models, downloads model assets, and streams rsync output.

The rule: isolate work, do not guess about threads

A Swift Task is not a thread. Swift schedules tasks on executors, and executors use threads. At an await, a task may suspend and later resume on a different worker thread even when it resumes on the same actor. RawCull must never depend on the worker thread being unchanged.

In this document, hop means a change of actor, executor, or dispatch queue that is required or explicitly requested by the source code:

• Calling an actor-isolated method from outside that actor hops to the actor’s executor.
• @MainActor, MainActor.run, and Task { @MainActor in ... } hop to the main actor when the caller is elsewhere.
• @concurrent and Task { @concurrent in ... } explicitly move non-UI work off inherited actor isolation and onto the concurrent executor.
• A task-group child is independently scheduled so that sibling work can run in parallel.
• Task.detached creates an unstructured task that does not inherit actor isolation.
• DispatchQueue.async submits work to the named GCD queue.

None of await, Task.sleep, task priority, or cancellation alone requests an executor hop. They may cause suspension and a later change of worker thread, but that is scheduler behavior and must not be observed as application logic.

Every executor hop in RawCull must be intentional. Add an explicit isolation annotation or concurrency primitive and document the reason. A change of worker thread after suspension is neither an error nor a behavior RawCull may rely on.

Compiler concurrency model

swift-tools-version selects the PackageDescription API and minimum toolchain. It is not, by itself, proof of the package’s Swift language mode; packages that declare swiftLanguageModes: [.v6] do so explicitly.

Concurrency architecture

flowchart TD
    UI["SwiftUI and @MainActor state"]
    APP["RawCull background actors"]
    GROUPS["Bounded task groups and async let"]
    PARSER["RawParserKit actors and ImageIO bridge"]
    ANALYSIS["PhotoAnalysisKit workers"]
    AI["PhotoAIKit model and storage actors"]
    CORE["RawCullCore nonisolated Sendable values"]
    RSYNC["Rsync callbacks, StreamAccumulator actor, GCD drain queue"]
    GCD["GCD worker threads for blocking framework and pipe work"]

    UI -->|"await: intentional actor hop"| APP
    UI -->|"@concurrent: intentional exit from MainActor"| GROUPS
    APP --> GROUPS
    APP -->|"await"| PARSER
    GROUPS --> ANALYSIS
    GROUPS --> AI
    APP --> CORE
    UI --> RSYNC
    PARSER --> GCD
    RSYNC --> GCD
    GCD -->|"Task @MainActor"| UI

1. Main-actor concurrency

The main actor owns SwiftUI and user-visible mutable state. This is correctness isolation, not a way to make work parallel.

SwiftUI .task modifiers and button-created Task values inherit main-actor isolation when their synchronous prefix reads or mutates view state. That entry is intentional. They then hop only when awaiting a background actor or an explicitly concurrent function, and resume on the main actor before changing UI state.

DispatchQueue.main.async is used in the comparison filmstrip and vertical image table to defer scrollTo by one run-loop turn so lazy view IDs have entered scroll geometry. These calls do not exist merely to reach the main thread; they intentionally delay already-main-actor UI work.

2. Actors that serialize mutable state

An actor hop is wanted because it establishes exclusive access to the actor’s state. It may suspend the caller, but it does not promise a particular worker thread.

RawCull app actors

SharedMemoryCache deliberately exposes its NSCache instances through nonisolated(unsafe): NSCache supplies its own thread safety. Synchronous counters, eviction history, and memory-pressure samples use OSAllocatedUnfairLock. This avoids an actor hop in synchronous cache delegate callbacks while retaining an explicit synchronization invariant.

Package actors

PhotoAIKit’s Core AI model providers are actors because model loading and inference resources are shared and not safe to use concurrently without ownership. An await from a RawCull workflow to a provider is therefore an intentional provider-actor hop. The pinned coreai-models implementation may use its own async execution internally; RawCull does not assume which thread it uses.

3. Structured parallelism

Structured child tasks inherit cancellation and cannot outlive their scope. RawCull uses them when operations are independent and results must be joined.

Task-group children are scheduled independently of the actor running the group. That is the explicit parallelism boundary. Each child captures Sendable values and calls the actor or provider that owns mutable state.

4. Unstructured tasks and lifetime

RawCull creates unstructured tasks to bridge synchronous UI/callback entry points, coalesce duplicate requests, and retain cancellable long-running work.

Important stored tasks include catalog load, thumbnail preload, full-size cache warming, zoom extraction, burst analysis, similarity and semantic hydration, scoring, semantic search, model download, deep review, memory sampling, mask loading, and image-source loading. Their exact properties live with the owning view model or actor; ownership, not thread identity, defines their lifetime.

A bare Task { ... } inherits the surrounding actor. It does not hop away merely because it is asynchronous. This is wanted when its synchronous prefix accesses actor-owned state. If a new task has no actor work before its first await, use Task { @concurrent in ... } or call an explicitly @concurrent function so that leaving the main actor is visible in source.

5. Explicit concurrent-executor work

@concurrent is the Swift 6.2 spelling RawCull uses for work that must not inherit the caller’s actor. It is an executor hop, not a guarantee that a new OS thread will be created.

The PhotoAIKit disk-store tasks intentionally leave actor isolation, but @concurrent still uses Swift’s cooperative executor. It is appropriate for short bounded operations; any operation shown by profiling to block for a long time should use a dedicated blocking-I/O bridge like RawParserKit’s GCD continuation bridge.

6. Detached tasks and blocking work

Task.detached is used only where losing actor inheritance is deliberate. It does not inherit actor isolation or task-local values, and cancellation is not automatically structured unless the parent explicitly awaits and/or cancels it.

RawParserKit’s CancellableImageIOWork is the stronger escape hatch for blocking framework calls. It uses withCheckedThrowingContinuation and submits the operation to DispatchQueue.global(qos:). This is an intentional queue hop: blocking ImageIO/CoreImage work runs on a GCD worker rather than occupying a Swift cooperative-pool thread. ImageIOCancellationToken and WorkState use OSAllocatedUnfairLock so cancellation can race safely with exactly-once continuation resumption.

The queue submission may run on a different worker thread, but GCD does not promise a fresh thread or a stable thread identity. The guarantee RawCull relies on is queue/executor separation, not a thread number.

7. Async streams, callbacks, and continuations

8. GCD, locks, and framework callbacks

GCD remains where the source API is callback-based or blocking:

• RsyncProcessStreaming uses a named serial termination queue to wait briefly, drain stdout/stderr, clear handlers, and then creates Task { @MainActor in ... } to finalize process state. The first hop avoids blocking the main actor; the return hop protects main-actor process/UI state.
• PTY and pipe callbacks create main-actor tasks before accessing RsyncProcess.
• A short DispatchQueue.main.asyncAfter in PTY cleanup deliberately defers final drain/close; it is a timing boundary, not an accidental thread correction.
• RawParserKit’s ImageIO bridge uses a global GCD queue specifically for blocking decode work.
• The two SwiftUI DispatchQueue.main.async calls deliberately defer scrolling by one run loop.

Locks are limited to synchronous callback-compatible state:

• SharedMemoryCache and CacheDelegate use OSAllocatedUnfairLock for counters, pressure state, and eviction history.
• ImageIOCancellationToken uses it for the cancellation bit.
• WorkState uses it to enforce exactly-once continuation completion.

Code must never hold one of these locks across await, call arbitrary client code while holding it, or use it as a substitute for actor ownership of asynchronous mutable state.

9. Sendable and nonisolated data flow

Values crossing task and actor boundaries are immutable or explicitly Sendable:

• RawCullCore declares its catalog, EXIF, saliency, burst grouping, boundary, ranking, and review values nonisolated and Sendable. Its algorithms are synchronous and stateless; concurrency is supplied by callers.
• RawParserKit exposes Sendable metadata/value types and actor-owned loaders. Images are consumed inside their owning isolation domain or encoded to Data before crossing a detached-task boundary.
• PhotoAnalysisKit batch requests contain @Sendable async input providers. FocusMaskEngine is @unchecked Sendable under the documented invariant that its CIContext has no mutable model/UI state and every operation receives immutable configuration.
• PhotoAIKit contracts are Sendable values and @Sendable provider/progress closures. CoreAISAM3Provider temporarily marks upstream runtime types @unchecked Sendable; the documented invariant is that they never leave that actor, and the conformances should be removed when upstream supplies correct annotations.
• CachedThumbnail, rsync ProcessHandlers, and lock/continuation bridge state use @unchecked Sendable only with a local immutability or synchronization invariant.

nonisolated does not mean “background thread.” It means the declaration is not owned by an actor. A synchronous nonisolated function runs on the calling thread; an async one follows its declared/caller isolation rules. Use @concurrent when execution must explicitly leave inherited actor isolation.

Package coverage

The app also pins external packages through PhotoAIKit, including Apple’s coreai-models. Their source is not part of the first-party checkouts under /Users/thomas/GitHub/RawCull; RawCull treats their async APIs as suspension points and does not make thread-affinity claims about their internals.

Cancellation, stale results, and backpressure

Cancellation is cooperative. Long-running loops check Task.isCancelled or Task.checkCancellation() before expensive steps and after suspension points. Task groups call cancelAll() when abandoning partial work. Continuation-based limiters remove and resume cancelled waiters. Framework work receives an explicit cancellation token where the underlying synchronous API cannot observe Swift task cancellation directly.

Cancellation alone does not prevent an old operation from publishing late. Main-actor workflows also use task identity, generation counters, selected catalog IDs, source IDs, or current-query checks before applying results. RawCullViewModel.cancelCatalogLoad() cancels catalog-related tasks, tells actors to cancel their inner work, resets dependent analysis state, and releases the active security-scoped resource.

Backpressure is intentional:

• RAW thumbnail decoding is capped at 6 and full-size decoding at 2 in RawParserKit.
• App batch extraction uses max(1, activeProcessorCount * 2).
• PhotoAnalysisKit and PhotoAIKit batch APIs maintain a sliding window rather than enqueueing every file at once.
• ThumbnailLoader parks excess callers with continuations instead of blocking threads.

Hop audit checklist

Before adding or changing concurrent work, answer these questions in code review:

1. Who owns the mutable state? Use @MainActor for UI state and an actor for shared background state.
2. Does the synchronous prefix need that actor? Keep a bare inherited Task only when work before the first await uses actor-owned state. Otherwise make the exit explicit with @concurrent.
3. Is parallelism structured? Prefer async let or a bounded task group. Store an unstructured task only when a synchronous entry point or independently cancellable lifetime requires it.
4. Is the operation blocking? An actor or @concurrent prevents main-actor occupation but does not make blocking safe for Swift’s cooperative pool. Use the GCD continuation bridge for long ImageIO/CoreImage or pipe-drain work.
5. What causes each hop? Name the actor call, @concurrent, detached task, group child, MainActor.run, or dispatch queue. Do not write “await moves to a background thread.”
6. How does work return? UI publication must occur through main-actor isolation; continuation waiters resume on their required executor automatically.
7. What crosses the boundary? Transfer Sendable values. Encode non-Sendable images to Data, or consume them inside the actor that owns them.
8. How does it stop? Define task ownership, cancellation checks, continuation cleanup, and stale-result rejection.
9. Is a thread change being observed? Remove correctness logic based on Thread.current or thread IDs. Thread logging is diagnostic only.
10. Is every hop wanted? If no correctness, responsiveness, or parallelism reason can be written beside the boundary, remove the boundary rather than relying on an accidental scheduling effect.

5 - Focus Mask and Sharpness

Focus Mask and Sharpness

RawCull’s focus system does two related jobs:

1. produce a visible focus mask for image inspection,
2. compute a scalar sharpness score for sorting, burst ranking, and saved scoring.

Both jobs use the same core engine, but the UI overlay and the numeric score are not identical. The overlay tries to show useful focus evidence. The score tries to rank files fairly across a catalog or burst.

Source Map

Scoring Flow

flowchart TD
    A["SharpnessScoringModel.scoreFiles"] --> B["Bounded task group"]
    B --> C["Per-file FocusDetectorConfig"]
    C --> D["FocusMaskEngine.computeSharpnessScore"]
    D --> E{"Scoring source"}
    E -->|"embeddedPreview"| F["Embedded JPEG or ImageIO thumbnail"]
    E -->|"rawDemosaic"| G["CIRAWFilter demosaiced thumbnail"]
    F --> H["Normalize to sRGB RGBA"]
    G --> H
    H --> I["Vision saliency and optional classification"]
    I --> J["Laplacian/detail scoring"]
    J --> K["SharpnessBreakdown"]
    K --> L["scores, saliencyInfo, breakdowns"]

SharpnessScoringModel is @Observable @MainActor. It owns progress, current options, scores, saliency info, and per-file breakdowns. It uses a bounded task group so multiple files can score in parallel without launching the whole catalog at once.

Scoring Options

The main knobs are:

effectiveFocusConfig combines photo type and quality into the config used for scoring. Per file, the model also injects ISO and an aperture hint derived from EXIF.

Image Source Choices

For Sony ARW files, FocusMaskEngine has a fallback that reads an embedded JPEG directly through SonyMakerNoteParser when ImageIO cannot produce the preview.

Saliency And AF Evidence

The engine uses Vision saliency to find likely subject regions. When enabled, it also runs image classification and stores the best subject label in SaliencyInfo.

The camera AF point comes from FileItem.afFocusNormalized, which is parsed during catalog scan. When present, it helps choose the relevant subject/patch and improves burst-ranking confidence.

Region selection favors:

1. saliency regions that contain or align with the AF point,
2. regions closer to the AF point,
3. higher Vision confidence,
4. stronger local detail,
5. larger region area as a final tie-breaker.

Sharpness Breakdown

SharpnessBreakdown is the main debugging record for a scored file.

The breakdown is useful when a score looks wrong because it tells you whether the engine trusted subject detail, AF-local detail, global detail, or a fallback.

Normalization For UI

Raw scores can vary by catalog and subject. SharpnessScoringModel keeps maxScore as an O(1) normalization denominator for UI badges:

if score count < 2: use the only score or 1.0
if score count < 10: use raw max score
if score count >= 10: use 90th percentile

The 90th percentile avoids one extreme outlier compressing every other image badge.

Calibration

calibrateFromBurst(_:) samples a burst/catalog and calls FocusMaskModel.calibrateAndApplyFromBurstParallel(...). The calibration updates the active focus config from measured detail distribution. It needs enough scoreable images; otherwise it logs a warning and leaves the current config in place.

This is used before burst scoring when sharpness data is missing.

Persistence

Sharpness data is persisted through savedfiles.json:

• raw sharpness score,
• saliency subject label,
• scoring signature,
• source file size,
• source modification date.

On catalog load, RawCull restores persisted scores only when the file identity and scoring signature still match. This prevents old scores from being reused after changing quality/source/config or after a RAW file changes.

Focus Mask Overlay

The overlay path uses the same engine family but is tuned for visual inspection. ZoomOverlayView can regenerate masks for the current zoom image and stores the resulting breakdown/saliency back into SharpnessScoringModel.

The visual mask can isolate to subject or AF evidence, and it has a “guarantee visible focus evidence” mode for making weak evidence inspectable. That visual relaxation should not be confused with the scalar score.

Tests

The package tests cover supporting pure/package behavior:

App-target focus scoring itself is still mainly verified through source review/manual behavior in this snapshot.

What To Check When Changing This Area

• If a scoring option changes score meaning, update SharpnessScoringSignature.
• If you tune scoring weights, inspect both single-image badges and burst ranking.
• Keep RAW demosaic concurrency low; it is much heavier than embedded-preview scoring.
• Treat overlay visibility tweaks separately from scalar scoring tweaks.
• When a score looks wrong, inspect SharpnessBreakdown before changing thresholds.

6 - Detailed Sharpness Scoring

Detailed Sharpness Scoring

This page explains, step by step, where RawCull computes sharpness scores, what code is involved, what the score means, and which factors can move the result up or down.

The short version: RawCull does not ask “is the whole image contrasty?”. It asks “how much reliable edge detail exists in the useful part of the image, especially around the subject or the camera AF point?”. The score is then used for sharpness sorting, burst ranking, saved scoring results, and inspection badges.

Source Map

End-To-End Flow

flowchart TD
    A["User clicks Score Sharpness"] --> B["RawCullViewModel.calibrateAndScoreCurrentCatalog"]
    B --> C["sharpnessScoringTargetFiles"]
    C --> D["SharpnessScoringModel.calibrateFromBurst"]
    D --> E["SharpnessScoringModel.scoreFiles"]
    E --> F["Bounded task group"]
    F --> G["Per-file FocusDetectorConfig"]
    G --> H["FocusMaskEngine.computeSharpnessScore"]
    H --> I["decodeScoringImage"]
    I --> J["Vision saliency and optional classification"]
    J --> K["buildScoringLaplacian"]
    K --> L["Crop Laplacian to the source image extent"]
    L --> M["full, saliency, AF, and patch scores"]
    M --> N["Blend + penalties + blur gate"]
    N --> O["SharpnessBreakdown"]
    O --> P["scores, saliencyInfo, breakdowns"]
    P --> Q["persistScoringResultsInMemory"]
    P --> R["sortBySharpness"]

The main call chain is:

SharpnessControlsView
  -> RawCullViewModel.calibrateAndScoreCurrentCatalog()
  -> RawCullViewModel.calibrateAndScoreFiles(_:)
  -> SharpnessScoringModel.calibrateFromBurst(_:)
  -> SharpnessScoringModel.scoreFiles(_:)
  -> FocusMaskEngine.computeSharpnessScore(...)
  -> FocusMaskEngine.computeSharpnessBreakdown(...)
  -> FocusMaskEngine.computeSharpnessAnalysis(...)

Step 1: The User Starts Scoring

The visible button lives in SharpnessControlsView.swift.

When the user presses Score Sharpness or Re-score, the button starts:

Task { await viewModel.calibrateAndScoreCurrentCatalog() }

The target set is computed by RawCullViewModel.sharpnessScoringTargetFiles:

sharpnessScoringTargetFiles = burstAnalysisOrderedFiles()

That means scoring follows the currently relevant catalog scope. Depending on selection and filtering, the UI description can be:

Example:

If a catalog contains 2,000 RAW files but the user has selected 45 files, RawCull scores those 45 selected files. If no files are selected but a 3-star filter is active, it scores the visible 3-star set. If neither is true, it scores the catalog files.

Step 2: RawCull Calibrates Before Scoring

RawCullViewModel+Sharpness.swift runs calibration before scoring:

await sharpnessModel.calibrateFromBurst(files)
await sharpnessModel.scoreFiles(files)

Calibration calls:

FocusMaskModel.calibrateAndApplyFromBurstParallel(...)

The purpose is to tune the active focus-mask threshold from the current file set. Calibration is mostly about the visual focus mask threshold, but it uses the same image/detail family as scoring. If there are too few scoreable files, RawCull logs a warning and keeps the existing config.

Important detail: calibration does not itself store final per-file sharpness scores. The score table is filled by scoreFiles(_:).

Step 3: SharpnessScoringModel Owns The Score Run

SharpnessScoringModel is @Observable @MainActor. It owns the UI-visible state:

At the beginning of a run, scoreFiles(_:) resets:

scores = [:]
saliencyInfo = [:]
breakdowns = [:]
scoringProgress = 0
scoringTotal = files.count

The model stores results locally while the task group is running. It only assigns the dictionaries back to observable state at the end of a clean, non-cancelled run. This avoids partially completed score tables being treated as a successful catalog score.

Step 4: Work Is Parallel, But Bounded

scoreFiles(_:) creates a bounded task group. It does not launch one task for every file in a large catalog. Instead it keeps a fixed number of active tasks and adds another file each time one completes.

The maximum concurrency depends on the selected source and quality:

This cap matters because RAW demosaic uses CIRAWFilter and is much heavier than scoring camera previews.

Example:

If 300 files are scored in Fast Embedded Preview mode, RawCull starts 6 tasks. When one finishes, task 7 starts. The UI progress updates after every completed file. If the same 300 files are scored from RAW Demosaic, only 2 are active at a time to reduce memory and CPU pressure.

Step 5: Per-File Config Is Built From Global Options And EXIF

Before each file is sent to the engine, SharpnessScoringModel creates a per-file FocusDetectorConfig:

fileConfig = effectiveFocusConfig
fileConfig.iso = file.exifData?.isoValue ?? 400
fileConfig.apertureHint = FocusDetectorConfig.ApertureHint.from(aperture: file.exifData?.apertureValue)
afPoint = file.afFocusNormalized

The base config comes from:

effectiveFocusConfig =
    scoringQuality.applying(
        to: photoType.applying(
            to: focusMaskModel.config
        )
    )

So the final config is a blend of:

1. the current focus-mask model config,
2. photo-type preset,
3. scoring quality preset,
4. per-file ISO,
5. per-file aperture hint,
6. per-file AF point, when parsed during scan.

Photo Type Impact

SharpnessPhotoType changes how strongly RawCull trusts subject detail versus full-frame detail.

Example:

For a bird against a detailed forest, Birds/Wildlife uses a high subject weight so the bird can outrank a frame where the background is crisp but the bird is soft. Landscape does almost the opposite: a single Vision salient object should not dominate a mountain scene where detail across the frame is meaningful.

Quality Impact

SharpnessScoringQuality changes both speed and detail sensitivity.

Balanced and High Precision blend a second, finer Laplacian pass into the score. This helps small details survive the noise-reduction pre-blur.

Example:

A small bird eye may occupy only a few pixels in a 512 px preview. Fast mode may mostly score wing/body edges. High Precision at 2048 px with a fine-detail blend is more likely to preserve eye and feather detail, but it takes longer.

Source Impact

SharpnessScoringSource chooses which image is measured.

Embedded previews are usually what the photographer sees quickly. RAW demosaic is closer to measuring the RAW file itself, but it costs more and may produce slightly different contrast/detail characteristics.

Step 6: The Scoring Image Is Decoded

The engine entry point is:

FocusMaskEngine.computeSharpnessScore(
    fromRawURL: url,
    config: fileConfig,
    thumbnailMaxPixelSize: thumbSize,
    afPoint: afPoint,
    scoringSource: scoringSource
)

The first engine step is decodeScoringImage(...).

For Embedded Preview, RawCull tries:

1. extractSonyEmbeddedPreview(...)
2. decodeThumbnail(...)

The Sony-specific fallback reads embedded JPEG byte ranges through SonyMakerNoteParser for newer ARW files where ImageIO cannot create a thumbnail.

For RAW Demosaic, RawCull uses:

CIRAWFilter(imageURL: url)

with scoring-oriented settings:

After decode, every image is normalized through an 8-bit sRGB RGBA CGContext. This keeps the Metal pipeline predictable regardless of the source image color space or bit depth.

Step 7: Vision Finds The Subject

detectSaliencyAndClassify(for:classify:) runs Vision:

VNGenerateAttentionBasedSaliencyImageRequest
VNClassifyImageRequest, when enabled

The saliency pass returns candidate bounding boxes for visually important regions. RawCull filters candidates:

area > 0.03 OR confidence >= 0.9

This means a saliency region is kept if it covers at least 3 percent of the image, or if Vision is very confident even though the region is small.

When classification is enabled, RawCull also stores a useful subject label, such as a wildlife or portrait-related label, in SaliencyInfo. The label can later be persisted with the score.

How The Winning Saliency Region Is Chosen

If Vision finds multiple candidates, selectSaliencyCandidate(...) ranks them by:

1. whether the candidate contains the AF point,
2. distance to the AF point,
3. Vision confidence,
4. measured interior detail,
5. candidate area,
6. deterministic coordinate tie-breaker.

Example:

Vision finds two subjects:

Even though the branch has higher Vision confidence and more detail, the camera AF point tells RawCull the intended subject was probably the bird.

Step 8: Metal Converts The Image To Edge Energy

The sharpness score is based on Laplacian edge energy. The Metal kernel is focusLaplacian in Kernels.ci.metal.

For every pixel, the kernel samples a 3 by 3 neighborhood:

laplace = 8 * center - sum(8 neighbors)
energy = luminance(abs(laplace.rgb))

Flat areas produce values near zero. Strong second-derivative edges produce larger values. The RGB edge response is collapsed into a single scalar using Rec. 601 luminance weights:

0.299 red + 0.587 green + 0.114 blue

Before the Metal kernel runs, buildAmplifiedLaplacian(...) applies Gaussian pre-blur:

effectiveRadius =
    preBlurRadius
    * isoScalingFactor(iso)
    * resolutionScalingFactor(imageSize)
    * apertureHint.blurDamp

The radius is capped at 100 px.

Extent Normalization After Blur

Core Image’s Gaussian blur expands the filter output beyond the input bounds. After the primary and optional fine-detail Laplacian passes are built, computeSharpnessAnalysis(...) now crops the result back to the decoded image extent:

rawBoosted = buildScoringLaplacian(...)
boosted = rawBoosted.cropped(to: inputImage.extent)

This is required before calculating bitmap width and height, border insets, and normalized AF/saliency rectangles. Without the crop, those coordinates would be interpreted against blur padding rather than the real photo and could sample shifted or incorrectly scaled regions.

Why Pre-Blur Exists

Without pre-blur, the Laplacian would fire on noise, hot pixels, sensor pattern noise, and JPEG artifacts. Pre-blur makes the score care more about meaningful edges than random high-frequency noise.

ISO Impact

isoScalingFactor(iso:) increases pre-blur as ISO rises:

Example:

With preBlurRadius = 2.2 and a 1024 px image:

resolution factor = sqrt(1024 / 512) = 1.414

ISO 400:
effective radius = 2.2 * 1.0 * 1.414 = 3.11

ISO 3200:
effective radius = 2.2 * 1.6 * 1.414 = 4.98

The ISO 3200 image is blurred more before edge detection, so random noise is less likely to inflate the score.

Resolution Impact

resolutionScalingFactor(for:) is:

sqrt(max(longestSide, 512) / 512)

clamped from 1.0 to 3.0.

Example:

Larger scoring images need a larger pre-blur radius so “detail scale” remains comparable across sizes.

Aperture Impact

FocusDetectorConfig.ApertureHint is derived from EXIF aperture:

The aperture hint changes:

Landscape aperture reduces the blur radius and reduces subject dominance. This protects deep-depth-of-field scenes where sharpness across the frame matters.

Step 9: RawCull Collects Sample Sets

computeSharpnessAnalysis(...) renders the source-extent-cropped Laplacian output to an RGBAf bitmap and reads the red channel as a Float edge-energy sample. The bitmap dimensions therefore match the decoded scoring image.

It collects these sample sets:

The full-frame sample excludes the border:

borderCols = width * borderInsetFraction
borderRows = height * borderInsetFraction

This avoids Gaussian edge artifacts near image boundaries.

Example:

For a 1024 by 683 scoring image with borderInsetFraction = 0.05:

borderCols = 1024 * 0.05 = 51 px
borderRows = 683 * 0.05 = 34 px

The full-frame score ignores the outer 51 columns on the left/right and 34 rows on the top/bottom.

Step 10: Each Sample Set Becomes A Robust Tail Score

RawCull does not average all edge pixels. A normal average would be dominated by large smooth areas, and a simple maximum would be dominated by one artifact.

Instead, robustTailScore(_:):

1. sorts the sample values,
2. finds p20, p90, and p97,
3. takes values between p90 and p97,
4. subtracts the p20 noise floor,
5. averages that upper-tail band,
6. applies a density factor if too few pixels are in the band.

In formula form:

tail samples = values where p90 <= value <= p97
band mean = average(max(0, value - p20))
density factor = min(1, (tailCount / totalCount) / 0.06)
robust tail score = band mean * density factor

The 6 percent density target penalizes sparse edge hits. A frame with only a few isolated hard edges can look deceptively sharp if measured by maximum value alone.

Example: Real Detail Versus Sparse Edge

Image A has many fine feathers:

p20 = 0.03
p90 = 0.28
p97 = 0.42
tailCount / totalCount = 0.06
bandMean after p20 subtraction = 0.32
densityFactor = 1.0
score = 0.32

Image B has one high-contrast branch but little subject texture:

p20 = 0.03
p90 = 0.26
p97 = 0.40
tailCount / totalCount = 0.015
bandMean after p20 subtraction = 0.31
densityFactor = 0.015 / 0.06 = 0.25
score = 0.31 * 0.25 = 0.0775

Both images have strong edge values, but Image B has too few useful edge pixels, so the robust score is much lower.

Step 11: AF, Saliency, And Local Patches Form A Subject Score

RawCull computes:

fullScore
salientScore
afScore
afCenterScore
afNeighborhoodScore
afLocalPatchScore
subjectInteriorPatchScore

First it builds a broad subject score:

if AF and saliency exist:
    broadSubjectScore = AF * 0.6 + saliency * 0.4
else if AF exists:
    broadSubjectScore = AF
else if saliency exists:
    broadSubjectScore = saliency

Then it blends in local patch detail:

if AF local patch and subject interior patch exist:
    localDetailScore = AFLocalPatch * 0.6 + subjectInteriorPatch * 0.4
else if only one exists:
    localDetailScore = that one

if broad subject and local detail exist:
    effectiveSubjectScore = broadSubjectScore * 0.75 + localDetailScore * 0.25
else:
    effectiveSubjectScore = whichever exists

This is intentionally conservative. A broad region can contain too much background. A tiny patch can overreact to one crisp edge. Blending them gives local detail a say without letting it fully override the broad subject region.

Example: AF And Saliency Agree

afScore = 0.36
salientScore = 0.32
afLocalPatchScore = 0.42
subjectInteriorPatchScore = 0.34

broadSubjectScore = 0.36 * 0.6 + 0.32 * 0.4
                  = 0.344

localDetailScore = 0.42 * 0.6 + 0.34 * 0.4
                 = 0.388

effectiveSubjectScore = 0.344 * 0.75 + 0.388 * 0.25
                      = 0.355

The local patch improves the subject score slightly because it found crisp detail near the intended focus area.

Example: AF Point On The Wrong Object

afScore = 0.12
salientScore = 0.34

broadSubjectScore = 0.12 * 0.6 + 0.34 * 0.4
                  = 0.208

The score does not blindly use saliency and ignore AF, or blindly use AF and ignore saliency. It records weaker subject evidence because the two signals disagree.

Step 12: Full-Frame And Subject Scores Are Blended

The main blend is:

base = fullScore * (1 - salientWeight)
     + effectiveSubjectScore * salientWeight

salientWeight comes from:

config.explicitSalientWeightOverride
    ?? apertureHint.salientWeightOverride
    ?? config.salientWeight

Photo type presets often set explicitSalientWeightOverride, so they usually win over the aperture hint. Landscape photo type uses a much lower subject weight than Birds/Wildlife.

Example: Birds/Wildlife

fullScore = 0.22
effectiveSubjectScore = 0.36
salientWeight = 0.85

base = 0.22 * 0.15 + 0.36 * 0.85
     = 0.033 + 0.306
     = 0.339

The final score is close to the subject score because the bird/wildlife preset cares strongly about the subject.

Example: Landscape

fullScore = 0.30
effectiveSubjectScore = 0.22
salientWeight = 0.35

base = 0.30 * 0.65 + 0.22 * 0.35
     = 0.195 + 0.077
     = 0.272

The full frame carries most of the result because the scene is expected to have broad depth of field.

If There Is No Subject

If RawCull has a full-frame score but no saliency or AF subject score, it applies a strong penalty:

p = 1 - salientWeight
base = fullScore * p * p * p

Example in Birds/Wildlife mode:

fullScore = 0.30
salientWeight = 0.85
p = 0.15

base = 0.30 * 0.15 * 0.15 * 0.15
     = 0.0010125

That looks harsh, but it is intentional for wildlife-first culling: a sharp background without a detected subject is usually not a strong keeper candidate.

In Landscape mode:

fullScore = 0.30
salientWeight = 0.35
p = 0.65

base = 0.30 * 0.65 * 0.65 * 0.65
     = 0.082

The penalty is much softer because whole-frame detail is meaningful for landscapes.

Step 13: Silhouette Penalty Reduces Rim-Only Sharpness

For each subject region, RawCull tracks how much edge energy lives in the outer 12 percent rim of the region versus the interior.

This matters for backlit birds, aircraft, or portraits where only the outline is sharp. A silhouette rim can produce a high edge score even if the face, eye, feathers, or body detail is soft.

When more than 62 percent of subject edge energy is in the rim:

over = (borderFraction - 0.62) / (1.0 - 0.62)
base *= 1.0 - silhouettePenaltyStrength * over

Example:

base = 0.34
borderFraction = 0.80
silhouettePenaltyStrength = 0.55

over = (0.80 - 0.62) / 0.38 = 0.474
multiplier = 1.0 - 0.55 * 0.474 = 0.739
base after penalty = 0.34 * 0.739 = 0.251

The photo still receives credit for real outline detail, but it is no longer ranked as if the subject interior were crisp.

Step 14: Subject Size Can Add A Bonus

If there is no AF region and the score depends on Vision saliency, RawCull can apply a subject-size bonus:

base *= 1.0 + area * subjectSizeFactor

This does not apply when AF is available, because AF already gives a high-confidence subject location and the bonus would double-count subject evidence.

Example:

base = 0.28
saliency area = 0.20
subjectSizeFactor = 0.10

bonus multiplier = 1.0 + 0.20 * 0.10 = 1.02
base after bonus = 0.2856

The default wildlife-style bonus is small. It nudges closer/larger subjects upward without overpowering real sharpness.

Step 15: Blur Gate Applies The Final Attenuation

The blur gate uses subject micro-contrast:

subjectMicro = standard deviation of subject-region Laplacian samples

If subject micro-contrast is below the aperture-specific low threshold, the final score is multiplied by 0.20. If it is above the high threshold, no attenuation is applied. Between low and high, it ramps linearly:

t = clamp((subjectMicro - blurGateLow) / (blurGateHigh - blurGateLow), 0, 1)
blurAttenuation = 0.20 + t * 0.80
finalScore = base * blurAttenuation

Example for .mid aperture:

blurGateLow = 0.008
blurGateHigh = 0.022
base = 0.34
subjectMicro = 0.015

t = (0.015 - 0.008) / (0.022 - 0.008)
  = 0.5

blurAttenuation = 0.20 + 0.5 * 0.80
                = 0.60

finalScore = 0.34 * 0.60
           = 0.204

This prevents low-texture or genuinely soft subjects from ranking too high, while avoiding an all-or-nothing cliff.

Step 16: Focus Failure Is Classified

classifyFocusFailure(...) stores a diagnostic label in SharpnessBreakdown.

It can return:

Example: motion blur

globalScore = 0.05
subjectScore = 0.04
afPointScore = 0.03
blurGateSigma = 0.009

result = motionBlur

Everything is weak, including micro-contrast.

Example: missed focus

globalScore = 0.26
subjectScore = 0.10
subject / global = 0.385

result = missedFocus

The image has sharp detail somewhere, but not on the subject.

Step 17: SharpnessBreakdown Explains The Result

The final result is not only a number. computeSharpnessBreakdown(...) returns:

When a score looks surprising, inspect the breakdown before changing thresholds. It usually tells whether the problem is subject detection, AF disagreement, silhouette dominance, blur attenuation, or source image quality.

Step 18: Scores Are Stored And Sorting Turns On

At the end of a successful score run, SharpnessScoringModel assigns:

self.scores = localScores
self.saliencyInfo = localSaliency
self.breakdowns = localBreakdowns
self.sortBySharpness = true

Then RawCullViewModel.calibrateAndScoreFiles(_:) persists scoring results:

persistScoringResultsInMemory(files: files)
await handleSortOrderChange()

Sorting happens in RawCullViewModel+Catalog.swift:

if sharpnessModel.sortBySharpness, !sharpnessModel.scores.isEmpty {
    result.sort { (scores[$0.id] ?? -1) > (scores[$1.id] ?? -1) }
}

Files with higher scores appear first. Files without a score sort below scored files because they use -1 as the fallback.

Step 19: Scores Are Normalized For Badges

The raw score is a Float, often in a small range such as 0.03 to 0.50. UI badges need a relative label, so SharpnessScoringModel maintains maxScore.

recomputeMaxScore() uses:

The 90th percentile prevents one extreme outlier from making every other file look soft.

SharpnessLabel then uses:

normalized = score / maxScore

0.85...1.00 = sharp
0.65...0.85 = good
0.35...0.65 = check
0.00...0.35 = soft

Example:

maxScore = 0.40
score = 0.34
normalized = 0.85
label = sharp

Another file:

maxScore = 0.40
score = 0.20
normalized = 0.50
label = check

Important: badge labels are relative to the current scored set. The raw finalScore is what is stored and sorted.

Step 20: Scores Are Persisted With A Signature

After scoring, RawCull merges results into savedfiles.json through CullingModel.mergeScoringResults(...).

The persisted fields are:

On catalog load, loadPersistedScoringandSaliency() only restores a score when:

1. the file name matches,
2. the file size matches,
3. the file modification date matches,
4. the saved SharpnessScoringSignature equals the current signature.

The signature includes:

algorithmVersion
isoScalingPolicyVersion
apertureHintPolicyVersion
scoringPhotoType
scoringQuality
scoringSource
thumbnailMaxPixelSize
preBlurRadius
borderInsetFraction
salientWeight
explicitSalientWeightOverride
subjectSizeFactor
silhouettePenaltyStrength
afRegionRadius
fineDetailBlendWeight
stableScoringEnergyMultiplier

This protects against stale scores after changing scoring logic, quality/source, image size, or important config values.

Current Signature Compatibility Finding

SharpnessScoringSignature.currentAlgorithmVersion is currently 3. The July 2026 source update corrected score geometry by cropping the blur-expanded Laplacian back to the source extent, but it did not change that version. A score or burst cache written by an earlier version-3 build can therefore still pass signature validation even though it was computed with the old padded geometry.

Before distributing the corrected scoring behavior, increment currentAlgorithmVersion so persisted file scores and burst-analysis snapshots are recomputed.

Worked Examples

These examples use simplified numbers, but they match the structure of the code.

Example 1: Sharp Bird, Soft Background

Settings:

photoType = Birds/Wildlife
salientWeight = 0.85
subjectSizeFactor = 0.05
silhouettePenaltyStrength = 0.55

Measurements:

fullScore = 0.18
afScore = 0.38
salientScore = 0.34
afLocalPatchScore = 0.44
subjectInteriorPatchScore = 0.36
subjectMicro = 0.026
blur gate = 1.0
borderFraction = 0.45

Calculation:

broadSubject = 0.38 * 0.6 + 0.34 * 0.4 = 0.364
localDetail = 0.44 * 0.6 + 0.36 * 0.4 = 0.408
effectiveSubject = 0.364 * 0.75 + 0.408 * 0.25 = 0.375

base = 0.18 * 0.15 + 0.375 * 0.85 = 0.346
silhouette penalty = none
finalScore = 0.346

Result:

The file sorts high because the subject and AF region are sharp, even though the whole frame is not.

Example 2: Sharp Background, Soft Bird

Settings:

photoType = Birds/Wildlife
salientWeight = 0.85

Measurements:

fullScore = 0.32
afScore = 0.09
salientScore = 0.11
subjectMicro = 0.011
blur gate for mid aperture = about 0.37

Calculation:

broadSubject = 0.09 * 0.6 + 0.11 * 0.4 = 0.098
base = 0.32 * 0.15 + 0.098 * 0.85 = 0.131
finalScore = 0.131 * 0.37 = 0.048

Failure classification:

globalScore = 0.32
subjectScore = 0.11
subject/global = 0.34
result = missedFocus

Result:

This file should not rank high for wildlife culling. The image has detail, but the intended subject is soft.

Example 3: Backlit Bird Silhouette

Measurements:

fullScore = 0.22
effectiveSubjectScore = 0.36
salientWeight = 0.85
borderFraction = 0.82
silhouettePenaltyStrength = 0.55
blur gate = 1.0

Calculation:

base before penalty = 0.22 * 0.15 + 0.36 * 0.85 = 0.339

over = (0.82 - 0.62) / 0.38 = 0.526
multiplier = 1.0 - 0.55 * 0.526 = 0.711

finalScore = 0.339 * 0.711 = 0.241

Result:

The strong outline still counts, but RawCull avoids ranking the file as if internal subject detail were sharp.

Example 4: Landscape With Broad Detail

Settings:

photoType = Landscape
salientWeight = 0.35
apertureHint = landscape
blurDamp = 0.8

Measurements:

fullScore = 0.31
salientScore = 0.24
no AF point
subjectMicro = 0.020
blur gate = 1.0

Calculation:

base = 0.31 * 0.65 + 0.24 * 0.35 = 0.286
finalScore = 0.286

Result:

The file can score well because landscape scoring trusts broad full-frame detail.

Example 5: High ISO Noise Does Not Automatically Win

Assume two files have similar raw edge peaks, but one is ISO 6400.

At ISO 6400:

isoScalingFactor = about 1.9

The pre-blur radius becomes larger before Laplacian detection. Random noise is smoothed more aggressively. If true subject detail remains, it will still produce robust edge energy. If the apparent sharpness was mostly noise, the robust tail score drops.

Result:

High ISO can still score well, but only when there is stable edge structure after noise-aware smoothing.

Factors That Increase The Score

Factors That Decrease The Score

Common Debugging Questions

Why did a visually crisp photo get a low score?

Check SharpnessBreakdown:

Why did changing quality change scores?

Balanced and High Precision can use larger images and a fine-detail Laplacian blend. They may rank small eyes, feather detail, eyelashes, or texture differently than Fast mode. This is expected, and the scoring signature prevents old Fast scores from being reused as High Precision scores.

Why are badge labels relative?

The raw score range depends on source image size, source type, ISO, subject matter, and config. Badge labels use score / maxScore, where maxScore is usually the 90th percentile for scored sets of 10 or more images. This gives useful in-catalog labels without letting one outlier flatten the scale.

Why can RAW Demosaic differ from Embedded Preview?

Embedded Preview scores the camera-generated JPEG preview or ImageIO thumbnail. RAW Demosaic scores a CIRAWFilter rendering with RawCull’s scoring settings. They can differ because camera JPEG processing, sharpening, noise reduction, tone curves, and embedded preview size are not identical to RawCull’s demosaic path.

Changing This Code Safely

When changing sharpness scoring:

1. Increment SharpnessScoringSignature.currentAlgorithmVersion when score meaning or sampling geometry changes; the current blur-extent crop still needs that invalidation bump.
2. Check SharpnessBreakdown fields before tuning thresholds.
3. Test wildlife, portrait, landscape, and high-ISO examples separately.
4. Keep RAW demosaic concurrency low.
5. Treat overlay visibility changes separately from numeric scoring changes.
6. Re-check burst ranking, because burst winner selection uses sharpness as one of its strongest signals.

The most important rule: do not tune from the final number alone. The breakdown tells which part of the pipeline produced the result.

7 - Detailed Focus Mask Computation

Detailed Focus Mask Computation

This page explains, step by step, how RawCull computes the visible focus mask: where the work starts in the UI, which model and engine methods run, how the bitmap overlay is produced, what the intermediate values mean, and which factors can change the final result.

The short version: the focus mask is not a camera focus-point display. It is a computed overlay built from local edge energy. RawCull first finds useful regions to inspect, then computes a Laplacian edge-energy image, ranks local patches, thresholds the strongest evidence, colorizes it, clips it to the selected patch areas, and finally draws the result over the photo.

The red focus-point marker and the focus mask are related but separate:

• the focus-point marker shows where the camera says autofocus was attempted,
• the focus mask shows where RawCull found likely in-focus edge detail,
• when an autofocus point exists, the mask can use it to choose and rank evidence.

Source Map

End-To-End Flow

flowchart TD
    A["User opens image or toggles focus mask"] --> B{"View context"}
    B -->|"Zoom"| C["ZoomOverlayView.regenerateMaskFromCG"]
    B -->|"Main thumbnail"| D["MainThumbnailImageView.regenerateMask"]
    B -->|"Comparison grid"| E["ComparisonGridImageCoordinator.focusResult"]
    C --> F["Build per-file FocusDetectorConfig"]
    D --> F
    E --> F
    F --> G["FocusMaskModel.generateFocusMask or generateFocusMaskWithBreakdown"]
    G --> H["FocusMaskEngine.generateFocusMask..."]
    H --> I["Detached cancellable worker"]
    I --> J["Vision saliency and optional sharpness breakdown"]
    J --> K["buildFocusMask"]
    K --> L["Scale image and build amplified Laplacian"]
    L --> M["Select saliency, AF, mixed, or global region"]
    M --> N["Rank local patches"]
    N --> O["Choose up to 3 evidence patches"]
    O --> P["Adaptive visual threshold"]
    P --> Q["Color threshold + morphology + feather"]
    Q --> R["Clip mask to selected patches"]
    R --> S["Return CGImage overlay and diagnostics"]

Step 1: The User Asks For A Mask

The focus mask can be generated from three user-facing places.

In ZoomOverlayView, mask generation is kicked off by .task(id: viewModel.zoomOverlayCGImage?.hashValue) and by changes to viewModel.sharpnessModel.effectiveFocusConfig. The concrete method is:

ZoomOverlayView.regenerateMaskFromCG()
  -> FocusMaskModel.generateFocusMaskWithBreakdown(...)

In MainThumbnailImageView, the flow is:

FocusMaskControlsView toggles showFocusMask
  -> MainThumbnailImageView.generateFocusMaskIfNeeded()
  -> MainThumbnailImageView.regenerateMask()
  -> FocusMaskModel.generateFocusMask(...)

In the comparison grid, the flow is:

ComparisonGridImageCoordinator.loadState(...)
  -> populateFocusMask(...)
  -> focusResult(...)
  -> FocusMaskModel.generateFocusMaskWithBreakdown(...)

Example:

If the user opens a 6000 x 4000 RAW preview in the zoom overlay, ZoomOverlayView downsizes the CGImage to 1024 pixels wide before computing the mask. The displayed image can still be larger in the UI, but the mask computation runs on the smaller inspection image so the overlay is fast and memory-friendly.

Step 2: RawCull Builds A Per-File Config

Before calling the engine, each view builds a FocusDetectorConfig.

The base config comes from:

viewModel.sharpnessModel.effectiveFocusConfig

Then the view injects per-file metadata:

config.iso = file.exifData?.isoValue ?? 400
config.apertureHint = FocusDetectorConfig.ApertureHint.from(aperture: file.exifData?.apertureValue)
afPoint = file.afFocusNormalized

If the file already has a sharpness score and the UI label says it is sharp, the view can also enable:

config.guaranteeVisibleFocusEvidence = true

That flag is visual-only. It lets the mask relax the threshold if the selected evidence would otherwise be nearly invisible.

Example:

Suppose two files have the same subject and same edge detail:

The ISO 6400 image may show fewer tiny red speckles, because buildAmplifiedLaplacian increases the blur radius before the Laplacian kernel runs.

Step 3: FocusMaskModel Bridges UI And Engine

FocusMaskModel is @Observable @MainActor, because it belongs to UI state. It owns:

var config = FocusDetectorConfig()
private nonisolated let engine = FocusMaskEngine()

It exposes two public generation methods:

The distinction matters. The simple method can reuse existing FocusEvidence from previous scoring. The breakdown method computes or refreshes diagnostic information while generating the overlay.

Example:

If a file was already sharpness-scored, MainThumbnailImageView passes:

viewModel.sharpnessModel.breakdowns[file.id]?.focusEvidence

That lets the overlay reuse the same winning region or patch evidence that explained the score, so the visual mask and the inspector diagnostics stay aligned.

Step 4: FocusMaskEngine Runs Off The Main Actor

FocusMaskEngine is immutable and @unchecked Sendable. It owns a shared CIContext configured with:

.workingColorSpace: NSNull()
.workingFormat: CIFormat.RGBAf

The engine methods call runCancellableWorker, which creates a detached task. That keeps Core Image, Vision, patch sampling, and Metal-backed filtering away from the main actor.

Cancellation checks appear throughout the pipeline:

• before saliency detection,
• before and after Laplacian creation,
• before patch ranking returns,
• before the UI stores the final mask.

Example:

If the user holds the right-arrow key in the zoom overlay, the selected image changes repeatedly. Each old maskTask is cancelled. RawCull avoids spending time finishing masks for images that are no longer visible.

Step 5: The Engine Chooses Saliency And Breakdown Data

There are two engine entry points:

FocusMaskEngine.generateFocusMask(...)
FocusMaskEngine.generateFocusMaskWithBreakdown(...)

The simpler path decides a saliency region only when needed:

if evidence has winningSaliencyRect:
    use it
else if config.isolateMaskToSubject:
    run Vision saliency and select a candidate
else:
    use no salient region

The breakdown path always does more diagnostic work:

detectSaliencyAndClassify(...)
computeSharpnessBreakdown(...)
buildFocusMask(...)
store mask diagnostics back into breakdown

This is why zoom and comparison views can later show fields like:

• mask region,
• mask threshold,
• winning region,
• rendered region,
• location confidence,
• selected patch summaries.

Step 6: buildFocusMask Starts With A Scaled CIImage

The private method that actually builds the bitmap is:

FocusMaskEngine.buildFocusMask(...)

Its first real image operation is:

scaledImage = inputImage.transformed(by: CGAffineTransform(scaleX: scale, y: scale))
rawLaplacian = buildAmplifiedLaplacian(from: scaledImage, config: config)

Most callers pass scale: 1.0, because they already choose an appropriate source image size before calling the engine.

The Laplacian image is the raw focus-evidence map. Bright pixels mean strong local second-derivative edge energy. Flat pixels are dark.

Step 7: The Shared Laplacian Pipeline Computes Edge Energy

buildAmplifiedLaplacian lives in FocusMaskEngine+Scoring.swift, because both sharpness scoring and focus-mask rendering use it.

It does three things:

1. apply Gaussian pre-blur,
2. run the Metal focusLaplacian kernel,
3. multiply the edge energy by config.energyMultiplier.

The effective blur radius is:

effectiveRadius = preBlurRadius * isoFactor * resFactor * apertureBlurDamp

Where:

The Metal kernel in Kernels.ci.metal samples a 3 x 3 neighborhood:

laplace = 8 * center - sum(8 neighbors)
energy = luminance_weighted_absolute_laplace

It writes the same scalar energy to RGB with alpha 1. Downstream Core Image filters can then threshold or colorize the result without converting formats.

Example:

Imagine a black bird eye surrounded by bright feathers:

• at the dark pupil edge, neighboring pixels change abruptly, so Laplacian energy is high,
• on a smooth out-of-focus background, neighboring pixels are similar, so energy is near zero,
• on high-ISO grain, raw energy could be high, but pre-blur reduces isolated noise before the kernel sees it.

Step 8: Border Artifacts Are Removed

If config.borderInsetFraction > 0, buildFocusMask masks out the outer border of the Laplacian:

innerRect = scaledImage.extent.insetBy(
    dx: width * borderInsetFraction,
    dy: height * borderInsetFraction
)
boostedLaplacian = rawLaplacian cropped to innerRect over black background

This prevents Gaussian blur and kernel edge behavior near the image border from creating false highlighted edges.

Example:

A bright sky frame with a dark border or vignette can produce strong edge energy at the frame boundary. The border inset stops that artifact from becoming the “best focus” evidence.

Step 9: Raw Laplacian Debug Mode Can Return Early

When:

config.showRawLaplacian == true

buildFocusMask returns the boosted Laplacian directly instead of a colorized, thresholded, clipped focus mask.

This mode is useful for debugging. It shows the whole edge-energy field rather than the final user-facing mask. It is not the normal overlay users see during culling.

Step 10: RawCull Selects The Search Region

The normal path decides where to look for focus evidence.

If:

config.isolateMaskToSubject == true

RawCull calls:

focusMaskRegionSelection(
    extent: scaledImage.extent,
    salientRegion: salientRegion,
    afPoint: afPoint,
    afRegionRadius: config.afRegionRadius
)

That can produce:

If subject isolation is disabled, the search region becomes the full image extent.

Coordinate note: AF points are stored in normalized image coordinates, but Core Image and Vision-style rectangles use different vertical orientation conventions in this code path. The engine flips the Y coordinate with:

visionY = 1.0 - afPoint.y

Example:

If the camera AF point is at (0.50, 0.35) in normalized display coordinates, the Core Image search center becomes approximately (0.50, 0.65) in the unit rectangle used by the mask selection helper.

Step 11: The Visual Evidence Region Is Chosen

The mask can be asked to visualize a specific kind of evidence. FocusEvidenceRegion can be:

If no explicit evidence region is available, the fallback order is:

AF point if available
else saliency if available
else global

Example:

For a bird photo with a parsed AF point on the eye, the mask usually prefers AF-anchored evidence. For a portrait without an AF point but with a detected salient face/body region, it uses saliency. For a landscape with no subject and no AF point, it falls back to global edges.

Step 12: AF-Anchored Masks Get A Finer Local Laplacian

When AF evidence exists, the mask builds a second local Laplacian with less blur:

fineConfig.preBlurRadius = max(0.35, config.preBlurRadius * 0.52)
fineLaplacian = buildAmplifiedLaplacian(from: scaledImage, config: fineConfig)

This is visual-overlay behavior. It helps small AF-local details survive the normal noise-reduction blur.

Example:

A small bird eye may be only a few pixels wide in the downscaled preview. The primary Laplacian is good at avoiding noisy false positives, but it can soften the tiny eye highlight. The finer AF pass helps the local patch ranking see that compact detail.

Step 13: AF-Weighted Source Can Favor The Center

For AF-local search rectangles, the engine can apply:

centerWeightedLaplacian(...)

This creates a radial gradient centered on the AF point and multiplies it with the Laplacian. Details closer to the AF point keep more weight; details near the edge of the AF rectangle lose weight.

Example:

If the AF rectangle covers both the bird’s eye and a high-contrast branch just beside the head, center weighting helps the eye region win when it is close to the camera’s AF point.

Step 14: Search Regions Become Patch Rankings

For each selected search region, the engine calls:

patchRankings(in:sourceImage:extent:afPoint:visualRegion:context:)

That method tiles the region with overlapping patches:

patchWidth  = clamp(region.width  * 0.34, image.width  * 0.035, image.width  * 0.14)
patchHeight = clamp(region.height * 0.34, image.height * 0.035, image.height * 0.14)
stepX = patchWidth  * 0.50
stepY = patchHeight * 0.50

It also adds a patch centered on the AF point when possible.

Each patch is scored by:

patchRanking(for:searchRegion:sourceImage:extent:afPoint:visualRegion:context:)

The ranking records:

• robust high-end edge energy,
• micro-contrast,
• threshold coverage,
• distance to AF point,
• silhouette fraction,
• ring/compact detail shape,
• linear-edge penalty,
• whether the patch contains the AF point,
• final composite score.

Step 15: Patch Composite Score Decides Which Area Is Worth Showing

The patch composite is built roughly like this:

composite =
    robustTailScore
  + microContrast * 0.35
  + coverage * 0.08
  + afProximity * 0.12
  + interiorBonus
  - silhouetteFraction * silhouettePenalty
  + eyeHeadHeuristicAdjustment

The exact code clamps the result to zero or above.

The intent is to prefer patches with real interior detail, not just any contrast line.

Example:

Consider a bird against the sky:

• Patch A covers the bird’s outline only. It has strong edges but little interior texture, so silhouette penalty applies.
• Patch B covers the eye and feather detail. It has compact/ring detail and real micro-contrast, so it gets a stronger composite score.
• Patch C covers a branch near the AF point. It has high linear edge energy, but the linear-edge penalty can reduce it.

The final mask should prefer Patch B.

Step 16: Up To Three Evidence Patches Are Selected

After ranking all patches, RawCull calls:

selectEvidencePatches(from:rankings, visualRegion:visualRegion)

The selection rules are:

1. discard patches with non-finite or zero composite score,
2. sort strongest first,
3. in AF-anchored mode, prefer the nearest AF patch if it is within 15% of the strongest patch’s score,
4. keep patches that do not overlap too much,
5. stop after three patches.

Overlap is rejected when:

overlapRatio(candidate, selected) >= 0.55

Example:

If the top three ranked patches all sit on the same bird eye, RawCull keeps the best one and skips heavily overlapping duplicates. It may then include a nearby feather patch or head patch so the overlay shows a compact area of focus evidence rather than three identical rectangles.

Step 17: The Visual Threshold Is Adaptive

RawCull samples the selected patch rectangles from the boosted Laplacian:

visualSamples = redSamples(in: selectedPatchRects, from: boostedLaplacian)

Then it computes:

adaptiveVisualThreshold(
    samples,
    fallback: config.threshold,
    percentile: AF anchored ? 0.82 : 0.90,
    floorMultiplier: AF anchored ? 0.32 : 0.55,
    capAtFallback: AF anchored
)

This means the threshold is derived from the actual local evidence distribution, not only from the settings slider.

The result is clamped to:

max(fallback * floorMultiplier, 0.01) ... 0.95

Example:

If a subject has weak but consistent feather detail, the local percentile threshold may settle below the global config.threshold, allowing visible edges. If a subject has very strong contrast, the threshold rises so the mask does not flood the entire region.

Step 18: The Mask Can Relax Threshold For Visibility

If:

config.guaranteeVisibleFocusEvidence == true

RawCull checks coverage:

coverage = pixelsAboveThreshold / sampledPixels

If coverage is below:

config.minimumEvidenceCoverage

it tries a relaxed threshold using the 70th percentile and a lower floor.

Example:

A file can score as sharp because a small AF-local detail patch is genuinely strong, but the visual threshold might produce only a few red pixels. With visibility guarantee enabled, the overlay can relax enough to make that focus evidence inspectable. This changes the overlay, not the saved score.

Step 19: Edges Are Thresholded, Cleaned, And Colorized

The method:

buildColorizedThresholdedEdges(from:threshold:config:)

turns the boosted Laplacian into the red/orange overlay.

It performs:

1. CIColorMatrix to extract edge energy into grayscale,
2. CIColorThreshold using the adaptive visual threshold,
3. optional CIMorphologyMinimum for erosion,
4. optional CIMorphologyMaximum for dilation,
5. a small CIMorphologyMinimum to thin the dilated result,
6. CIColorMatrix to colorize the binary mask.

The colorization uses strong red, a little green, very little blue, and high alpha:

red   = 1.0
green = 0.22
blue  = 0.02
alpha = 0.92

Example:

If erosionRadius is raised, isolated noise pixels disappear more aggressively. If dilationRadius is raised, nearby edge fragments connect and look more continuous. Too much dilation can make the mask look thick; too much erosion can remove legitimate fine detail.

Step 20: The Mask Is Clipped To Selected Evidence Patches

The red edge image still covers the whole Laplacian extent. RawCull clips it to selected patch rectangles:

clip(edgeMask, to: patchRects, extent: scaledImage.extent)

This creates a white mask from the selected rectangles and blends the colorized edges through it.

If config.featherRadius > 0, the clipped mask is then Gaussian-blurred slightly:

feather.radius = config.featherRadius

Finally the result is cropped back to the image extent and rendered:

context.createCGImage(croppedMask, from: croppedMask.extent)

Example:

A photo may contain sharp grass in the background and a slightly soft bird in the foreground. If the selected evidence region is saliency or AF, clipping prevents background grass from covering the whole overlay. The mask can still show the best subject-local evidence, even if other parts of the frame have stronger raw texture.

Step 21: Diagnostics Are Returned With The Mask

The engine returns:

FocusMaskRenderResult(
    image: CGImage?,
    diagnostics: FocusMaskDiagnostics,
    evidence: FocusEvidence?
)

For the breakdown path, generateFocusMaskWithBreakdown stores:

breakdown.focusMaskRegionSource = maskResult.diagnostics.regionSource
breakdown.focusMaskVisualThreshold = maskResult.diagnostics.visualThreshold
breakdown.focusEvidence = maskResult.evidence

The UI can then show those values in CandidateInspectorView.

Important diagnostic fields include:

Example:

If the inspector says:

Mask Region: AF + saliency
Rendered Region: AF point
Evidence Threshold: 0.31
Location Confidence: High
Reason: AF-local patch is spatially aligned

that means RawCull had both subject and AF data, chose an AF-local visualization, used an adaptive threshold of 0.31, and found the selected patch close enough to the AF point to trust its location.

Step 22: The UI Draws The Mask Over The Image

In ZoomOverlayView, the final drawing is:

if showFocusMask, let mask = focusMask {
    Image(decorative: mask, scale: 1.0, orientation: .up)
        .resizable()
        .scaledToFit()
        .blendMode(.screen)
        .opacity(0.95)
}

The same visual idea is used by thumbnail and comparison image views: the generated bitmap is drawn above the source image, aligned to the same aspect-fit frame.

The separate focus-point overlay uses FocusPointMarker and red corner brackets. It is controlled by showFocusPoints, not by the focus-mask image.

What Most Affects The Result

Practical Examples

Example 1: Sharp Bird Eye, Busy Background

Inputs:

• AF point exists on or near the bird’s eye,
• Vision saliency finds the bird,
• background branches contain many high-contrast lines.

Likely flow:

1. region source becomes saliencyAndAF,
2. visual region falls back to AF-local evidence,
3. fine Laplacian and center weighting favor the AF area,
4. patches around the eye/head outrank branch patches,
5. the mask is clipped to one to three subject-local patches.

Result:

The overlay should highlight small red/orange edges around the eye, head, or feather detail instead of painting every sharp branch in the frame.

Example 2: Bird Outline Sharp, Interior Soft

Inputs:

• subject silhouette has strong edge contrast,
• eye and feather interior are soft,
• saliency finds the bird.

Likely flow:

1. outline patches produce high raw Laplacian energy,
2. silhouetteFraction rises because edge energy is concentrated near the outside of the patch,
3. the silhouette penalty reduces composite score,
4. if no stronger interior patch exists, confidence may be medium or low.

Result:

The mask may show less than expected, or it may highlight only the best available edge. This is intentional: a sharp outline alone does not prove the subject detail is in focus.

Example 3: High-ISO Indoor Portrait

Inputs:

• ISO is high,
• face/subject is visible,
• image contains sensor noise in flat areas.

Likely flow:

1. isoScalingFactor increases the pre-blur radius,
2. isolated grain is smoothed before the Metal Laplacian,
3. patches with real facial detail keep stronger micro-contrast,
4. erosion removes remaining isolated noise pixels.

Result:

The overlay should avoid sparkling across flat walls or skin noise. It should prefer consistent detail such as eyes, eyelashes, hair, or clothing texture.

Example 4: Landscape With No Clear Subject

Inputs:

• no useful AF point,
• Vision saliency may be weak or irrelevant,
• aperture is f/8 or smaller.

Likely flow:

1. if subject isolation cannot find a useful region, mask falls back toward global evidence,
2. landscape aperture hint damps blur so whole-frame detail is not smoothed away,
3. global patches are ranked by robust edge energy and micro-contrast.

Result:

The mask can show distributed detail in trees, rocks, buildings, or horizon texture. This is different from a wildlife frame, where background detail is intentionally less important.

Example 5: Mask Looks Sparse On A Sharp Image

Inputs:

• image has a high sharpness score,
• selected patch is small,
• adaptive threshold is strict.

Likely flow:

1. the score can be high from compact, reliable evidence,
2. the visual mask may still have low coverage,
3. if the image is labeled sharp, guaranteeVisibleFocusEvidence may relax the threshold,
4. diagnostics set relaxedForVisibility = true.

Result:

The overlay becomes easier to inspect, but this does not change the computed sharpness score.

Common Misreadings

How To Debug A Surprising Mask

Start with the inspector fields, because they tell you why a mask was drawn where it was.

1. Check Mask Region: did the engine have saliency, AF, both, or neither?
2. Check Winning Region and Rendered Region: did it choose AF, saliency, mixed, or global?
3. Check Evidence Threshold: a high value means only very strong edges survived.
4. Check Evidence Coverage: very low coverage means the mask is intentionally sparse.
5. Check Visibility Relaxed: if yes, the overlay was made more visible for inspection.
6. Check the top patch summaries: look for AF distance, coverage, silhouette fraction, and composite score.
7. Toggle focus points: compare the red AF marker with the red/orange computed mask.
8. Adjust settings one at a time: threshold, pre-blur, amplify, erosion, dilation.

Change Checklist

When changing focus-mask behavior, check these code paths together:

• FocusMaskEngine+MaskGeneration.swift for visual overlay logic,
• FocusMaskEngine+Scoring.swift for shared Laplacian behavior,
• FocusDetectorConfig.swift for defaults and settings impact,
• ZoomOverlayView.swift for zoom mask generation and diagnostics persistence,
• MainThumbnailImageView.swift for thumbnail/detail mask generation,
• ComparisonGridImageCoordinator.swift for comparison-grid masks,
• CandidateInspectorView.swift for diagnostics display.

Be especially careful with buildAmplifiedLaplacian. It is shared by visual focus masks and sharpness scoring. A change there can alter both the overlay and the numeric score behavior.

8 - Burst Groups

Burst Groups

Burst analysis turns a flat catalog into groups of adjacent, visually similar frames. It ranks each multi-frame group, presents review queues, and supports culling decisions such as keeping the best frame, keeping the top two, deferring a group, or setting a manual pick.

The implementation separates orchestration, Vision feature-print indexing, pure grouping and ranking engines, cache persistence, and the review UI.

Source Map

End-to-End Flow

flowchart TD
    A["Analyze Bursts"] --> B["RawCullViewModel.analyzeBursts"]
    B --> C{"Valid BurstAnalysisCache?"}
    C -->|"yes"| D["Remap cached IDs and apply snapshot"]
    C -->|"no"| E{"Sharpness scores missing?"}
    E -->|"yes"| F["Calibrate and score target files"]
    E -->|"no"| G["Reuse scores"]
    F --> H{"Similarity embeddings missing?"}
    G --> H
    H -->|"yes"| I["Index Vision feature prints"]
    H -->|"no"| J["Reuse embeddings"]
    I --> K["Group adjacent frames"]
    J --> K
    K --> L["Rank multi-frame groups"]
    L --> M["Apply manual winner overrides"]
    M --> N["Save cache and review-state snapshots"]
    N --> O["Show home dashboard and review queues"]

Every awaited phase is protected by the analysis generation and selected catalog. A cancelled or superseded run cannot publish late results into a newer catalog.

The target is normally every catalog file sorted by localized filename, which acts as shot order. If files are selected, visible selected files are followed by hidden selected files. If no selection exists and a star filter is active, only files with that rating are analyzed.

Similarity Embeddings

SimilarityScoringModel creates Vision feature-print embeddings from thumbnail-resolution images. It archives each VNFeaturePrintObservation as Data keyed by FileItem.id, avoiding a large collection of live Vision objects.

Embedding decode first uses the RawParserKit thumbnail path and then falls back to ImageIO. The feature-print request uses .scaleFill.

The same model can rank a catalog by distance from an anchor image. Burst grouping instead calculates feature-print distances only between adjacent files. Those adjacent distances are cached in memory and reused when possible during regrouping.

Grouping Rules

BurstGroupingEngine.group(...) makes one sequential pass. It starts a new group when any boundary rule fires.

Lens changes are recorded as evidence but do not independently split a group. They do make group metadata unstable during ranking.

Default configuration:

visualDistanceThreshold = 0.25
maxTimeGapSeconds = 2.0
requireSameCamera = true
requireSimilarFocalLength = true
maxFocalLengthDeltaMM = 3.0
algorithmVersion = 2

The burst sensitivity control changes only the visual threshold. reGroupBursts() cancels older grouping work, reuses embeddings and adjacent-distance data, rebuilds rankings, and saves a new cache. The current home/category presentation is preserved instead of being forced into the grouped grid.

Ranking Formula

BurstRankingEngine computes:

overall =
    rankingSharpness * 0.62
  + focusPoint      * 0.12
  + saliency        * 0.10
  + metadata        * 0.16

Sharpness is normalized by SharpnessScoringModel.maxScore. When at least two group members have scores and their normalized spread is at least 0.03, global and burst-relative sharpness are blended:

rankingSharpness = normalizedSharpness * 0.65
                 + burstRelativeSharpness * 0.35

The other components are heuristic evidence:

• Focus is 0.70 when camera AF data exists and 0.45 otherwise.
• Saliency is 0.75 when the subject label matches the group’s dominant label, 0.25 on a mismatch, and 0.45–0.60 when evidence is incomplete.
• Metadata starts from group stability, gains 0.15 for tight similarity, loses 0.10 at ISO 6400 or above, and gains 0.05 at f/5.6 or wider.

Ties in overall score retain original shot order.

Confidence And One-Click Safety

isSafeForOneClickCulling is true only for high-confidence results. keepBestInGroup and keepTopTwoInGroup also require the current sharpness score table to be non-empty; otherwise they return without changing ratings.

Home Dashboard And Review Queues

After analysis, BurstGroupsHomeView shows catalog coverage, group counts, the active similarity threshold, up to three suggested picks, and these queue categories:

The grouped culling grid can collapse a burst to its top three ranked frames. A group header opens the dedicated workspace and toggles Reviewed or Deferred state.

BurstCullingWorkspaceView displays ranked frames as a filmstrip with rating, source, zoom, focus overlay, manual-pick, defer, and review controls. P/N move between frames and G advances to the next eligible multi-frame group in the current queue. The detailed comparison grid remains available from the workspace.

Review States

RawCullCore.BurstReviewState currently defines:

• .none
• .needsReview
• .reviewed
• .deferred
• .algorithmReviewed
• .manualWinnerOverride
• .decisionApplied

The app and package enum are now aligned. algorithmReviewed remains for cache compatibility.

BurstReviewQueuePolicy.effectiveState(for:) preserves explicit modern states. For .none or legacy .algorithmReviewed, it derives Needs Review when confidence is not high, cautions exist, no recommendation exists, or one-click culling is unsafe; otherwise it treats the group as reviewed.

Toggling Reviewed or Deferred a second time resets the result to .none and removes that group from the explicit state dictionary. Review states are persisted using a catalog-and-membership BurstGroupSignature, so they can be restored after a threshold change even if numeric group IDs change.

Manual Winner Overrides

A manual winner is stored in savedfiles.json as BurstWinnerOverride:

Overrides use filenames because saved-file persistence is filename-based. CullingModel canonicalizes member names so lookup is order-independent and prunes overrides whose files no longer exist. Applying an override promotes the selected file, recalculates second place, and sets .manualWinnerOverride.

User Actions

One-click rating actions capture a BurstUndoEntry before writing ratings. Only the most recent burst action is retained for undo.

Cache Validity

BurstAnalysisCache stores embeddings, scores, saliency, groups, boundary evidence, ranked results, and review-state snapshots. A snapshot is accepted only when all of these still match:

• cache schema version (currently 4),
• grouping algorithm version,
• catalog path,
• effective sharpness thumbnail size and complete sharpness signature,
• grouping, embedding, and Vision similarity signature,
• file count,
• every file path, size, and modification date.

On load, cached UUIDs are remapped to the current scan’s UUIDs by file path because FileItem.id values are recreated. Cache saves are also guarded by the completed analysis context, generation, catalog, and similarity signature so stale asynchronous work cannot overwrite a newer result.

What To Check When Changing This Area

• Bump BurstGroupingConfig.algorithmVersion when grouping semantics change.
• Update the embedding pipeline version or similarity signature when embedding inputs change.
• Update the sharpness scoring signature when score meaning changes.
• Keep review-state decoding backward compatible and preserve signature-based restoration across regrouping.
• Keep manual winner overrides durable across cache invalidation and UUID remapping.
• Treat high confidence plus available sharpness scores as the one-click culling gate.

9 - RawCull Packages

RawCull Packages

RawCull is split across four reusable Swift packages. Each package owns one kind of knowledge and deliberately avoids importing the others. RawCull is the composition root: it decodes a file and capture metadata with RawParserKit, asks PhotoAnalysisKit or PhotoAIKit to measure or index it, translates the evidence into RawCullCore values and decisions, and presents or persists the outcome.

flowchart LR
    Files["ARW, NEF, and rendered files"] --> Parser["RawParserKit\ndecode + capture metadata"]
    Parser --> Host["RawCull adapters\nsource and cache policy"]
    Host --> Analysis["PhotoAnalysisKit\nsharpness + focus evidence"]
    Host --> AI["PhotoAIKit\nimage/text embeddings + segmentation"]
    Analysis --> Host
    AI --> Host
    Host --> Core["RawCullCore\nburst grouping + ranking"]
    Core --> App["RawCull UI, persistence, and culling actions"]

The arrows are runtime data flow, not package dependencies. The four sibling packages do not depend on one another; the RawCull application imports and adapts them.

Recommended Reading Order

1. Start with RawParserKit to see how a file becomes a normalized image and metadata.
2. Continue with PhotoAnalysisKit to see how a decoded image becomes sharpness and focus evidence.
3. Read RawCullCore to see how measurements become burst boundaries and recommendations.
4. Read PhotoAIKit for optional CLIP image similarity, text-query semantic search, and SAM 3 segmentation workflows.

The source for this section is the current PhotoAIKit, PhotoAnalysisKit, RawCullCore, RawParserKit, and RawCull repositories under /Users/thomas/GitHub/RawCull. In each package guide, paths beginning with Sources/ and Tests/ are relative to that package. Paths beginning with RawCull/ are relative to the RawCull application repository.

9.1 - How PhotoAIKit Is Constructed

How PhotoAIKit Is Constructed

PhotoAIKit is a reusable Swift package extracted from application code. Its most important achievement is not merely that CLIP and SAM 3 run. It is that reusable AI behavior has been separated from RawCull’s UI, RAW-file handling, paths, sandbox rules, and culling policy.

This document explains the construction from the bottom up and gives the reason for each boundary.

1. Begin With The Package Boundary

The package owns:

• typed, Sendable contracts;
• model-bundle validation and fingerprinted model identity;
• Core AI CLIP image and text inference plus SAM 3 inference;
• validated image/text semantic comparison;
• Apple Vision feature-print generation and comparison;
• bounded similarity indexing and explicit fallback;
• segmentation, mask selection, geometry, and catalog workflows;
• optional artifact codecs and mask stores.

The host application owns:

• model download, installation, candidate URLs, and sandbox bookmarks;
• RAW decoding and application image models;
• SwiftUI, Observation state, and display wording;
• app-specific task staleness such as “latest selection wins”;
• query admission, result ordering, filtering, and semantic-search presentation;
• culling, burst ranking, sharpness, saliency, and rating policy;
• helper-process launch and application restart behavior.

This boundary makes PhotoAIKit reusable. A package that imports FileItem or searches Bundle.main for a RawCull resource might be convenient for one app, but it would silently encode application policy into the AI layer.

2. Read Package.swift As An Architecture Diagram

PhotoAIKit/Package.swift declares Swift tools 6.4, macOS 27, Swift 6 language mode, six library products, and one test target. The only external package dependency is a pinned revision of apple/coreai-models.

flowchart TD
    Contracts["PhotoAIContracts\nvalues + protocols"]
    CLIP["CoreAICLIPBackend"] --> Contracts
    SAM3["CoreAISAM3Backend"] --> Contracts
    Vision["VisionFeaturePrintBackend"] --> Contracts
    Workflows["PhotoAIWorkflows"] --> Contracts
    Storage["PhotoAIStorage"] --> Contracts
    CoreAI["apple/coreai-models\nCoreAISegmentation product"] --> CLIP
    CoreAI --> SAM3
    Tests["PhotoAIKitTests"] --> Contracts
    Tests --> CLIP
    Tests --> SAM3
    Tests --> Vision
    Tests --> Workflows
    Tests --> Storage

There is intentionally no large umbrella target in which every feature can reach every implementation. Each higher-level product depends on PhotoAIContracts, but the backends, workflows, and storage products do not depend on one another.

Why this shape helps:

• A host can import only the products it uses.
• Workflow code is testable with fake providers and decoders.
• Storage does not need to know which inference backend produced a value.
• A backend cannot accidentally reach into RawCull or into another backend.
• Framework-heavy dependencies remain concentrated in concrete backend targets.

3. PhotoAIContracts: The Stable Center

Contracts are the innermost layer. They contain data and protocol definitions, not application decisions.

3.1 Translate Host Photos Into AIImageSource

Sources/PhotoAIContracts/AIImageSource.swift defines the package-owned source value:

public struct AIImageSource: Codable, Hashable, Identifiable, Sendable {
    public let id: UUID
    public let url: URL
    public let displayName: String
}

RawCull maps FileItem into this type at its integration boundary. PhotoAIKit therefore receives only what a reusable indexing or segmentation workflow needs. It never learns ratings, focus data, camera metadata, or view state.

SourceFileIdentity reads size and modification date. SourceFingerprint adds the standardized path. These values let persisted artifacts answer a crucial cache question: “Was this result produced for the current contents of this source file?”

3.2 Invert Image Decoding

PhotoAIKit declares:

public protocol ImageDecoding: Sendable {
    func image(for source: AIImageSource) async throws -> CGImage
}

The package needs a CGImage, but it should not prescribe how one is obtained. A JPEG host can use ImageIO; RawCull can try RawParserKit first; tests can return a generated image. The workflow depends on the capability, not on a camera-format implementation.

This is the dependency-inversion principle in a small, concrete form:

flowchart LR
    Workflow["PhotoAIKit indexer"] --> Protocol["ImageDecoding protocol"]
    Raw["RawCull RAW decoder"] -. conforms .-> Protocol
    Test["Test decoder"] -. conforms .-> Protocol

3.3 Separate Generation From Comparison

Sources/PhotoAIContracts/SimilarityArtifact.swift defines two protocols:

• ImageSimilarityArtifactProviding creates an artifact from a CGImage and source.
• ImageSimilarityArtifactComparing computes the distance between two compatible artifacts.

The combined ImageSimilarityBackend type alias requires both.

The split matters because generation can be actor-isolated and expensive, while comparison may be synchronous and nonisolated. It also keeps distance semantics with the backend that understands its payload. RawCull does not decode a VNFeaturePrintObservation, nor does it implement CLIP cosine distance from unverified arbitrary data.

3.4 Keep Text Queries Query-Scoped

Sources/PhotoAIContracts/TextEmbedding.swift defines a second, deliberately different similarity value. TextEmbedding contains a normalized text vector and a TextEmbeddingDescriptor with the complete backend identity, dimensions, tokenizer version, and schema version.

Text embeddings are not file-backed SimilarityArtifact values. They have no source fingerprint and are intended to live for one query. The split public protocols mirror the image API:

• TextEmbeddingProviding tokenizes and encodes a query.
• ImageTextSimilarityComparing compares a compatible image artifact with the text embedding.
• ImageTextSimilarityBackend combines both capabilities.

The comparison returns cosine similarity in -1...1, where a larger value is a closer semantic match. It is a relative retrieval score, not a confidence or a keep/reject decision.

3.5 Make Persisted Artifacts Self-Describing

A SimilarityArtifact has two fields:

SimilarityArtifact
├── descriptor
│   ├── backend
│   ├── model fingerprint
│   ├── dimensions
│   ├── representation
│   ├── preprocessing version
│   ├── normalization version
│   ├── configuration version
│   ├── source fingerprint
│   └── schema version
└── payload (backend-owned Data)

This looks more elaborate than storing [Float], but it prevents subtle cache bugs. A vector is not reusable merely because its dimension matches. Reuse is valid only if the source, model asset, preprocessing, normalization, configuration, representation, and schema are still the same.

SimilarityArtifactDescriptor.isCompatibleForDistance(with:) intentionally compares every backend/configuration field but excludes the source fingerprint. Two different photos must have different source fingerprints, yet their artifacts can still be compared when they were produced by the same backend definition.

3.6 Treat Model Identity As Data

The model types are spread across:

• ModelIdentity.swift;
• ModelAssetFingerprint.swift;
• ModelBundleResolver.swift;
• ModelResource.swift.

ModelIdentity.cacheIdentifier preserves compatibility with older cache naming. artifactIdentifier adds the selected asset fingerprint for new artifacts. This distinction allows migration without pretending that two different model binaries are the same model.

ModelResourceDescriptor.clip and .sam3 describe package-neutral requirements and version strings. ModelCapabilityStatus reports available, missing, or invalid without user-facing wording. The host translates that status into its own settings UI.

4. Model Bundles Are Supplied, Not Discovered

PhotoAIKit never chooses an application directory. The host supplies one URL or an ordered list of candidates.

A valid bundle has this conceptual layout:

ModelBundle/
├── metadata.json
├── tokenizer/
│   └── tokenizer.json
└── selected-model.aimodel  (or .aimodelc)

metadata.json names the selected model in assets.main. New exports can also provide asset_fingerprints.main.

ModelBundleResolver validates, in order:

1. the URL exists and is a directory;
2. metadata.json decodes;
3. assets.main is present and non-empty;
4. the asset extension is accepted;
5. the selected asset exists;
6. required resources such as tokenizer/tokenizer.json exist;
7. the model asset can be fingerprinted;
8. a manifest checksum, when present, matches the actual asset.

For a file asset, the cryptographic algorithm is SHA-256. For a compiled model directory, PhotoAIKit hashes a stable sorted tree representation. Older manifests without a checksum receive a size/modification-time fallback fingerprint. That fallback can detect replacement in place but is marked as not cryptographically verified.

ModelProviderFactory<Provider> connects validation to a backend constructor. A backend supplies “how to make me from a validated URL”; the host supplies “which URLs should be considered, and in what order.”

Candidate order is policy, not just presentation. ModelResourceResolver skips a candidate that is missing, but returns immediately when it finds an invalid candidate. This prevents a damaged higher-priority installation from being silently hidden by a lower-priority fallback.

CLIP has a second validation layer after generic bundle resolution. CLIPRuntimeConfiguration reads the model contract from metadata: source model and revision, architecture, pretrained checkpoint, embedding dimensions, preprocessing dimensions and normalization, tokenizer context and padding, named image/text functions, normalization version, and configuration version. Current bundles use shortest-side resize, a centered square crop, and bicubic interpolation. Older PhotoAIKit bundles remain compatible through the original 224-pixel stretch and bilinear defaults.

This separation prevents a library update from unexpectedly changing an application’s installation or sandbox policy.

5. Concrete Backend Products

5.1 CoreAICLIPBackend

CoreAICLIPProvider is an actor conforming to image embedding, artifact generation/comparison, text embedding, and image/text comparison protocols.

Its responsibilities are deliberately backend-specific:

• validate the supplied CLIP bundle;
• lazily load and cache the named image and text Core AI functions;
• validate tensor names, shapes, scalar types, dimensions, and metadata;
• create the image, token, and attention-mask NDArrays;
• apply the model-declared resize/crop policy in sRGB and CLIP channel normalization;
• L2-normalize image vectors through ImageEmbedding;
• validate that the exported graph’s text vector is finite, non-empty, correctly shaped, and already L2-normalized;
• encode the vector as an artifact payload;
• compute cosine distance only after descriptor and payload validation.

The actor owns loadedModel, so lazy initialization is isolated from concurrent callers.

For a semantic query, the same provider creates a TextEmbedding and similarity(image:text:) rejects every incompatible model, representation, dimension, preprocessing, normalization, configuration, tokenizer, schema, or payload before computing a dot product. Vision artifacts are rejected because they are not in CLIP’s image/text embedding space.

RawCull’s RawCullCLIPSemanticSearchService keeps the product policy outside the backend: it trims and admits a literal query, selects already-persisted compatible image artifacts, isolates per-file failures, reports progress, and orders equal scores deterministically. It never decodes an image or persists a query embedding.

5.2 CoreAISAM3Backend

CoreAISAM3Provider conforms to SubjectSegmenting. It owns tokenizer setup, lazy model loading, request inference, query selection, confidence conversion, mask decoding, resizing, thresholding, feathering, timing, and diagnostics.

The public contract speaks in SubjectSegmentationRequest and SubjectSegmentationResult, not Core AI tensors. This lets workflows and hosts work at the domain level while the backend handles framework details.

5.3 VisionFeaturePrintBackend

This actor uses VNGenerateImageFeaturePrintRequest. The produced VNFeaturePrintObservation is securely archived into an opaque artifact payload.

Comparison returns to this backend, which unarchives the observations and calls Vision’s native computeDistance. The observation never becomes part of RawCull’s persistence API. This is an example of information hiding: callers can store and route the artifact without learning its private representation.

6. PhotoAIWorkflows: Reusable Orchestration

Backends operate on one image or request. Workflows coordinate many sources and reusable policies.

6.1 Similarity Indexing

EmbeddingIndexer is the vector-specific API. SimilarityArtifactIndexer is the more general API used when the fallback might have an opaque representation such as a Vision feature print.

Both indexers accept:

• package-owned sources;
• an injected decoder;
• a primary provider;
• an optional fallback provider;
• an explicit fallback policy;
• a concurrency limit;
• an async progress callback.

They use a throwing task group but enqueue only concurrencyLimit children. Each time one finishes, one new source is added. This is bounded concurrency: a catalog with 20,000 files does not create 20,000 live tasks and decoded images.

The fallback policies are:

The indexer does not decide which policy is correct for RawCull. The host selects .wholeBatch for CLIP-to-Vision fallback.

6.2 Segmentation Workflows

The SAM 3 side is split into focused units:

The package can decide which mask best satisfies a general selection strategy. RawCull still decides which photo is a culling candidate or winner. Reusable mask quality is not the same concern as product ranking policy.

7. PhotoAIStorage: Optional Persistence Mechanics

Storage depends only on contracts.

For similarity data:

• SimilarityArtifactCodec wraps an artifact in a versioned envelope.
• EmbeddingCodec writes descriptor-complete CLIP artifacts.
• decodeMigrating reads current data and two version-1 formats, but legacy values are accepted only against the real current model identity and are marked for immediate rewrite.
• LegacyCLIPEmbeddingCodec remains a staged migration reader; its identity-incomplete writer is deprecated.

For segmentation:

• SubjectMaskMemoryStore is an injected actor-backed dictionary.
• SubjectMaskDiskStore stores PNG mask data plus JSON metadata, validates the storage key, reports disk usage, supports pruning, and rejects stale or corrupt entries.

The host injects the disk directory. PhotoAIKit owns how the record is encoded and validated, while the application owns where records live and when they are removed.

8. Concurrency And Isolation

PhotoAIKit targets Swift 6 and makes boundary values Sendable.

The main patterns are:

• Actors for mutable runtime state: providers lazily cache loaded models; services and stores protect mutable state.
• Value types for transport: sources, identities, descriptors, progress, failures, and configuration values cross tasks safely.
• Bounded task groups: indexers and prefetch workflows limit simultaneous work.
• Cooperative cancellation: workflow boundaries and expensive stages call Task.checkCancellation().
• Async progress callbacks: the package reports neutral progress values; a host decides how and where to publish UI state.

Actor isolation does not remove the need for a host policy. RawCull still owns generation tokens and catalog checks that prevent an older task from replacing results for a newer selection.

9. Testing The Architecture, Not Only The Math

Most of Tests/PhotoAIKitTests/ exercises the products through public APIs. The CLIP text suite also uses @testable import for deterministic tokenizer, batch, tensor-shape, and preprocessing checks that sit below the provider boundary. Together the tests cover architectural promises such as:

• model bundles are accepted through supplied URLs;
• manifest fingerprints become artifact identity;
• provider factories and resource resolvers share validation;
• CLIP normalization and cosine distance remain stable;
• model-declared CLIP preprocessing and legacy preprocessing remain compatible;
• token batches, attention masks, text-output validation, and image/text compatibility checks reject malformed or mismatched data;
• whole-batch fallback produces a homogeneous result set;
• Vision artifacts stay opaque and use the native metric;
• segmentation caches by package-owned source values;
• disk stores reject stale or corrupt entries;
• legacy data is a rewrite candidate, not silently current data;
• batch transport has an explicit versioned schema;
• cancellation crosses the service boundary.

Fake decoders, providers, and stores make these tests possible. That testability is a direct consequence of putting protocols in the innermost target.

10. Developer Tools And Model Assets

The package includes tools under Tools/ for exporting CLIP and SAM 3 assets, selecting a SAM 3 asset, generating fingerprints, and producing reference CLIP image/text similarities. Output paths are explicit.

export_clip.py supports the existing OpenAI clip-vit-base-patch32 model and OpenCLIP ViT-B-32-256 with datacomp_s34b_b86k weights. It writes the image and text encoders as named functions in one Core AI asset, verifies tokenizer parity for the OpenCLIP export, and records the runtime metadata consumed by CLIPRuntimeConfiguration.

The Swift package itself declares no model resources. It does not embed .aimodel, .aimodelc, tokenizer, or metadata assets. Model binaries are large deployment inputs with their own lifecycle; keeping them out of the library avoids coupling package source, application installation, and model distribution.

11. How To Add Another Similarity Backend

Use the existing layering as a checklist:

1. Add or reuse package-neutral contract values in PhotoAIContracts; do not add host types.
2. Create a separate backend target that depends on contracts.
3. Conform to artifact providing and comparing protocols.
4. Give the backend a complete SimilarityBackendDescriptor.
5. Put framework-specific payload encoding and distance semantics inside that backend.
6. If the backend supports text, add a query-scoped descriptor and keep image/text compatibility checks with that backend.
7. Reuse SimilarityArtifactIndexer and inject the host’s decoder.
8. Decide explicitly whether fallback is none, per-item, or whole-batch.
9. Add public-API tests with fake sources and model-free inputs where possible.
10. Let the host choose model URLs, cache locations, settings, and product policy.

If a proposed type needs SwiftUI, FileItem, a RawCull path, or a burst rating, it probably belongs in RawCull’s adapter layer instead.

Source Map

Next, follow these abstractions into the host application in How RawCull Enables and Uses CLIP.

9.2 - How PhotoAnalysisKit Is Constructed

How PhotoAnalysisKit Is Constructed

PhotoAnalysisKit is the reusable measurement layer extracted from RawCull. It turns a decoded CGImage plus neutral capture metadata into sharpness, saliency, focus evidence, and an optional focus-mask image. It can also create opaque Apple Vision feature prints.

The package measures a photo; it does not decide whether to keep it. That distinction prevents image-processing code from becoming coupled to RawCull’s files, settings, views, caches, or culling policy.

1. Begin With The Package Boundary

PhotoAnalysisKit owns:

• sRGB normalization for predictable analysis input;
• Core Image and Metal Laplacian processing;
• Vision saliency and optional subject classification;
• scalar sharpness metrics and failure classification;
• AF-point, subject, local-patch, and global focus evidence;
• focus-mask selection and rendering;
• numeric configuration, presets, quality levels, and analysis identity;
• bounded batch analysis and calibration;
• Vision feature-print generation and native comparison.

The host application owns:

• RAW, JPEG, or other source decoding;
• file URLs, security-scoped access, and sandbox bookmarks;
• source identity, cache keys, cache directories, and persistence;
• observable task state, progress presentation, and cancellation policy;
• settings labels and saved-settings migration;
• sorting, ratings, burst decisions, and culling actions.

The input boundary is intentionally small:

flowchart LR
    Source["RAW or rendered source"] --> Decoder["Host decoder"]
    Decoder --> Input["PhotoAnalysisInput\nCGImage + ISO + aperture + AF point"]
    Input --> Analyzer["PhotoAnalyzer"]
    Analyzer --> Result["PhotoAnalysisResult\nsaliency + breakdown + optional mask"]
    Result --> Host["Host cache, UI, and culling policy"]

PhotoAnalysisKit does not import RawParserKit or RawCullCore. RawCull provides the adapters between them.

2. Read Package.swift As The First Design Document

The manifest declares Swift tools 6.2, Swift 6 language mode, macOS 26, one library product, and one test target.

PhotoAnalysisKit package
├── PhotoAnalysisKit library
│   ├── public contracts and facade
│   ├── Core Image, Vision, and Accelerate implementation
│   └── packaged default.metallib
└── PhotoAnalysisKitTests

There are no third-party package dependencies. The implementation uses Apple frameworks including Core Graphics, Core Image, Vision, Accelerate, and Foundation.

The target enables InferIsolatedConformances and NonisolatedNonsendingByDefault. The package is therefore built under the same strict Swift 6 concurrency assumptions as the host without adopting UI isolation.

3. PhotoAnalysisInput Is The Decoding Boundary

Sources/PhotoAnalysisKit/PhotoAnalysisInput.swift defines the package’s source value:

public struct PhotoAnalysisInput: Sendable {
    public let image: CGImage
    public let iso: Int
    public let aperture: Double?
    public let normalizedAFPoint: CGPoint?
}

The AF point uses normalized 0...1 coordinates with the origin at the visual top-left. This matches the camera metadata shape used by RawCull. Vision uses a bottom-left coordinate system, so the package performs the vertical conversion internally.

ISO is clamped to at least 1. Aperture and AF point are optional because not every image or camera provides them. The package can still compute a global result when either is absent.

This value has no URL or file identifier. Two consequences follow:

1. The package cannot reopen a source behind the host’s back.
2. The host must associate returned results with the correct file and invalidate them when that file or decode policy changes.

RawCull makes that ownership concrete in RawCullPhotoAnalysisAdapter. The adapter chooses an embedded-preview decode through RawParserKit or a host-owned RAW demosaic, bounds the requested pixel size, and then constructs PhotoAnalysisInput. It also translates the returned neutral saliency value into RawCullCore’s SaliencyInfo. Neither package needs to import the other to participate in that flow.

PhotoAnalysisResult returns:

An absent breakdown means analysis could not produce a valid result or was cancelled. It is different from a valid score of zero.

4. PhotoAnalyzer Is The Public Facade

Sources/PhotoAnalysisKit/PhotoAnalyzer.swift keeps the public entry points small:

The facade contains an immutable FocusMaskEngine. Both types are @unchecked Sendable because CIContext does not declare sendability even though this package holds no mutable model or UI state and reuses the context concurrently.

For each input, PhotoAnalyzer copies the supplied configuration, replaces its ISO with the input ISO, and derives an aperture hint from the input aperture. Callers can safely reuse one base configuration across many files.

When a host already stores SharpnessBreakdown.focusEvidence, passing it to focusMask avoids repeating the saliency selection step. This makes the measurement result useful as an input to later presentation work.

5. Follow The Scalar Sharpness Pipeline

The core implementation is in FocusMaskEngine+Scoring.swift.

flowchart TD
    Image["CGImage"] --> SRGB["8-bit sRGB normalization"]
    SRGB --> Vision["Vision saliency + optional classification"]
    SRGB --> Preblur["ISO- and aperture-aware Gaussian pre-blur"]
    Preblur --> Laplacian["Metal focusLaplacian kernel"]
    Laplacian --> Samples["Global, saliency, AF, and local-patch samples"]
    Vision --> Samples
    Samples --> Tail["Robust p90-p97 tail scores"]
    Tail --> Blend["Subject/global blend"]
    Blend --> Adjust["Silhouette, subject-size, and blur-gate adjustments"]
    Adjust --> Breakdown["SharpnessBreakdown"]

5.1 Normalize Before Measuring

normalizeToSRGB redraws the input into an 8-bit sRGB RGBA bitmap. The Metal pipeline therefore receives a predictable pixel format regardless of the source image’s bit depth or color space.

5.2 Find Candidate Subject Regions

VNGenerateAttentionBasedSaliencyImageRequest supplies salient-object rectangles. Small weak objects are removed, and candidates are ordered using AF overlap or distance, saliency confidence, interior detail, area, and deterministic coordinates.

When classification is enabled, VNClassifyImageRequest supplies a neutral subject label. The package filters out broad environment descriptions and prefers likely subjects such as animals or people. This label is evidence, not a keep/reject decision.

5.3 Build Edge Energy

The packaged focusLaplacian Metal kernel runs after Gaussian pre-blur. The pre-blur grows with ISO so high-frequency sensor noise is less likely to masquerade as detail. Aperture hints damp or strengthen parts of this behavior for wide, middle, and landscape apertures.

The result is rendered as floating-point RGBA data. The red channel carries edge energy.

5.4 Score Several Regions

The engine samples:

• the full image after excluding a configurable border;
• the selected salient region;
• the camera AF region;
• smaller AF-center and AF-neighborhood regions;
• ranked local patches within subject or AF regions.

robustTailScore sorts the sample values, measures the p90-p97 energy band relative to a p20 noise floor, and penalizes a band that is too sparse. microContrast computes the standard deviation of finite edge samples.

The normal blend combines full-frame and subject evidence. When both AF and saliency scores exist, AF has the larger share of the subject score. A conservative local-detail component can then refine the broad subject measurement.

The final value may be adjusted for:

• a silhouette-dominated subject whose strongest energy is mostly at the outer rim;
• the area of a saliency-only subject;
• an aperture-aware soft blur gate driven by subject micro-contrast.

SharpnessBreakdown preserves the component scores and selected evidence so a host can explain the result instead of presenting a single unexplained number.

5.5 Distinguish Failure Shapes

FocusFailureKind classifies the evidence as:

• .motionBlur when global, subject, and AF detail are all weak and micro-contrast is low;
• .missedFocus when the frame has usable global detail but the subject is much weaker;
• .none when neither pattern is established.

These are algorithmic classifications, not final user-facing copy or culling actions.

6. Focus Evidence And Mask Rendering Are Related But Separate

Scalar scoring answers “how much reliable detail is present?” Mask rendering answers “where should the UI draw visible focused edges?” They share evidence but have different configuration needs.

FocusEvidence records the winning region, AF scores, selected patch rankings, spatial alignment, dominance, silhouette handling, visual threshold, coverage, and confidence diagnostics. FocusPatchRanking exposes the detail, coverage, shape, position, and penalty components behind each candidate patch.

The overlay pipeline:

1. chooses AF-center, AF-neighborhood, AF, saliency, mixed, or global evidence;
2. builds primary and, for AF regions, fine-detail Laplacians;
3. ranks local patches and selects the best evidence patches;
4. chooses an adaptive percentile threshold;
5. optionally relaxes the threshold to guarantee minimum visible coverage;
6. applies erosion and dilation to the binary edge mask;
7. colorizes, clips, feathers, and crops the mask;
8. returns updated evidence and render diagnostics with the image.

FocusMaskRegionSource describes whether saliency, AF, both, or neither provided the overlay region. FocusEvidenceOverlayStyle distinguishes subject edges from global edges. RawCull decides how these neutral values appear in its interface.

7. Configuration, Presets, And Cache Identity

SharpnessConfiguration is a value snapshot containing numeric algorithm settings. It separates host-editable behavior from observable settings models.

Notable groups are:

SharpnessPreset applies high-level subject tuning for automatic, birds and wildlife, portrait, landscape, or general action use. SharpnessQuality selects fast, balanced, or high-precision fine-detail work. The host may persist its own UI enums, but should map them into these package values instead of duplicating constants.

Persisted results need more than a filename. SharpnessAnalysisDescriptor records:

• descriptor schema version;
• scalar algorithm version;
• ISO-scaling policy version;
• aperture-hint policy version;
• every host-configurable value that affects non-mask scoring;
• the stable scoring energy multiplier.

The descriptor deliberately excludes per-image ISO and aperture, mask-only presentation settings, decoded dimensions, source choice, and source-file identity. A host must add those values to its cache identity.

Host sharpness cache identity
├── PhotoAnalysisKit SharpnessAnalysisDescriptor
├── per-image ISO and aperture
├── source file identity
├── selected preview/decode policy
└── decoded pixel dimensions

8. Bounded Batch Analysis

PhotoAnalysisBatchRequest<Identifier> contains an identifier and an async input provider. The provider inversion is important: PhotoAnalysisKit coordinates work, while the host retains file access and decoding policy.

analyzeBatch starts only up to maximumConcurrentTasks child tasks. When one completes, it enqueues one more request. This prevents a large catalog from creating an unbounded number of decoded bitmaps.

The API has three ordering and failure guarantees:

• progress is emitted in completion order;
• the returned results preserve request order;
• a decode failure is represented by a result whose analysis is nil.

If the parent task is cancelled, the method cancels the group and returns nil, telling the host to discard partial results rather than mistake them for a complete batch.

9. Calibration Changes The Overlay, Not The Score

Calibration samples Laplacian energies from host-provided decoded inputs with bounded concurrency. It downsamples the collected sample set, sorts it, and returns p50, p90, p95, and p99 statistics plus a clamped threshold at the requested percentile.

At least minimumSuccessfulImages inputs must succeed. Cancellation or too few samples returns nil.

Calibration changes only the visual edge threshold. Core sharpness scores keep a stable gain and do not depend on the current catalog. This prevents the same photo from receiving a different scalar score merely because unrelated photos were added or removed.

10. Vision Feature Prints Stay Opaque

VisionFeaturePrintBackend is an actor that creates a VNFeaturePrintObservation using Vision revision 2 by default. The observation is securely archived inside VisionFeaturePrint.payload.

The value also stores its Vision revision and representation version. Before comparison, the backend verifies both values, securely unarchives each observation, and calls Vision’s native computeDistance.

The host can persist the opaque payload, but it must associate it with source-file identity. Incompatible prints return nil; corrupt archives or failed Vision operations throw a typed VisionFeaturePrintError.

PhotoAnalysisKit intentionally does not copy PhotoAIKit’s general similarity artifact descriptors, CLIP fallback, or batch indexing. It supplies only the focused Vision measurement primitive.

11. Metal Resources Are Part Of The Algorithm

Sources/PhotoAnalysisKit/Resources/Kernels.ci.metal is the source for the Core Image kernel. SwiftPM copies Metal source resources but does not compile them for command-line builds, so the package also checks in default.metallib.

Tools/build_metallib.sh regenerates the binary. A kernel change is incomplete until the checked-in library has been rebuilt and tests have been run. The binary resource affects algorithm output and must be treated like source, not an optional deployment file.

12. Concurrency And Cancellation

The package uses four complementary techniques:

• Immutable facade and engine: callers pass input and configuration snapshots; no app state is retained.
• Explicit concurrent workers: synchronous Core Image and Vision work runs outside caller UI isolation while retaining task priority and task-local context.
• Bounded task groups: batch analysis and calibration cap simultaneous inputs.
• Cooperative checks: expensive stages test cancellation before Vision work, rendering, large loops, and result publication.

@unchecked Sendable is confined to the immutable Core Image facade and engine. Public transport values are Sendable value types.

13. Testing The Public Contract

Tests/PhotoAnalysisKitTests/ imports the library through import PhotoAnalysisKit, not @testable import. Tests therefore exercise the public surface used by a host.

The suite covers:

• end-to-end sharpness analysis with the packaged Metal kernel;
• focus-mask images, evidence, and diagnostics;
• calibration from decoded images and neutral metadata;
• bounded concurrency, order preservation, progress, decode failure, and cancellation;
• descriptor encoding and changes in identity-affecting settings;
• robust-tail, micro-contrast, ISO scaling, and focus-failure metrics;
• configuration presets and aperture behavior;
• Vision feature-print compatibility, round trips, and malformed payloads.

Synthetic images keep the suite deterministic and independent of RAW files, model downloads, cache directories, and application state.

14. How To Add Another Analysis

Use the existing boundary as a checklist:

1. Accept CGImage and only the neutral metadata the algorithm needs.
2. Keep URLs, camera decoding, settings state, and source selection in the host.
3. Return a Sendable package-owned result with enough evidence to explain it.
4. Put numeric tuning in a value configuration, not UI preferences.
5. Define a versioned descriptor when hosts may persist the output.
6. Make synchronous framework work cancellation-aware and independent of UI isolation.
7. Bound multi-image work instead of spawning one live task per catalog item.
8. Test through the public API with synthetic images and values.

If the proposed API needs FileItem, SwiftUI, a cache directory, or a rating, that concern belongs in RawCull’s adapter or policy layer.

Source Map

Next, see how package-neutral measurements become culling-domain decisions in How RawCullCore Is Constructed.

9.3 - How RawCullCore Is Constructed

How RawCullCore Is Constructed

RawCullCore is the small domain layer at the center of RawCull. It does not decode or analyze photos. Instead, it receives file metadata and measurements already produced elsewhere, groups sequential files into bursts, and ranks the candidates using deterministic culling rules.

Its main architectural value is that a recommendation can be tested without opening a RAW file, running Vision, creating a Metal context, loading application settings, or constructing a view model.

1. Begin With The Package Boundary

RawCullCore owns:

• package-safe file, EXIF, source-catalog, and saliency values;
• normalized focus-point parsing;
• burst group, boundary-evidence, ranking, confidence, and review-state models;
• pure burst grouping rules;
• pure burst ranking and one-click-safety rules;
• a lightweight 256-bin luminance histogram.

RawCullCore deliberately does not own:

• TIFF, MakerNote, ARW, or NEF parsing;
• thumbnail, preview, or full RAW decoding;
• Vision, Core Image, Metal, CLIP, or SAM 3 analysis;
• caches, settings, security-scoped URLs, and saved-file coordination;
• observable view models, SwiftUI, selection, sorting, and culling actions.

This produces a clear flow:

flowchart LR
    Parser["RawParserKit\nmetadata + AF location"] --> Adapter["RawCull adapter"]
    Analysis["PhotoAnalysisKit\nsharpness + saliency"] --> Adapter
    AI["PhotoAIKit\nvisual distance"] --> Adapter
    Adapter --> Values["RawCullCore values"]
    Values --> Group["BurstGroupingEngine"]
    Group --> Rank["BurstRankingEngine"]
    Rank --> Decision["RawCull presentation and action"]

The arrows describe how RawCull composes runtime values. RawCullCore imports none of the other packages.

2. The Manifest Optimizes For A Small Domain Library

Package.swift declares Swift tools 6.2, Swift 6 language mode, macOS 26, one RawCullCore library target, and one test target. It has no external package dependencies and no bundled resources.

The target uses:

.defaultIsolation(MainActor.self)
.enableUpcomingFeature("InferIsolatedConformances")
.enableUpcomingFeature("NonisolatedNonsendingByDefault")

Default main-actor isolation matches the surrounding application environment, but the package’s public value types and pure engines are explicitly nonisolated. Background grouping and ranking therefore do not acquire unnecessary main-actor hops.

This is a useful distinction: the build default is conservative, while each proven pure API opts out explicitly.

3. Package-Safe Models Replace Application State

The package uses transport values instead of importing RawCull’s FileItem or view models.

3.1 ExifMetadata

ExifMetadata is a Codable, Hashable, and Sendable snapshot containing display strings and numeric values for shutter speed, focal length, aperture, ISO, exposure compensation, camera, lens, RAW type, size class, and pixel dimensions.

It contains both strings and numbers because they serve different purposes:

• strings preserve display-ready metadata such as shutter notation and lens names;
• numeric exposure time, focal length, aperture, ISO, and exposure compensation support deterministic stop-based rules without reparsing display text.

RawCull or an adapter constructs this value from RawParserKit’s RawImageMetadata. RawCullCore does not know where the values came from.

3.2 RawCullFileItem

RawCullFileItem contains an ID, URL, name, byte size, modification date, optional capture date and capture-time-zone offset, optional EXIF snapshot, and optional normalized AF point.

Equality and hashing use only id. Two snapshots with the same ID are the same logical file item even if another stored field changed. This matches the identity behavior expected by RawCull collections.

effectiveCaptureDate prefers the EXIF capture instant and falls back to the file modification date. usesFileModificationDateForCaptureTime exposes which path was used so grouping and confidence can treat the fallback conservatively. formattedSize is a lightweight Foundation convenience. File access and mutable scan state remain outside the value.

3.3 Catalog And Saliency Summaries

RawCullSourceCatalog identifies a named source URL without adding bookmark or access-lifecycle behavior.

SaliencyInfo stores only an optional subject label and confidence. It deliberately does not import Vision or expose a Vision observation. A PhotoAnalysisKit result can be translated into this small culling-domain value at the host boundary.

4. Focus-Point Parsing Connects Vendor Metadata To Analysis

RawParserKit’s vendor parsers return a focus string in this shape:

imageWidth imageHeight focusX focusY

FocusPointParser.normalizedPoint(from:) splits on arbitrary whitespace, requires exactly four numeric values and positive image dimensions, and returns:

x = focusX / imageWidth
y = focusY / imageHeight

The origin remains at the visual top-left. PhotoAnalysisKit accepts that convention and performs its own Vision coordinate conversion.

The parser does not clamp the returned point. Vendor parsers and host validation are expected to supply meaningful sensor coordinates. Malformed input or non-positive dimensions return nil.

5. Burst Models Preserve Evidence, Not Just A Winner

BurstAnalysisModels.swift defines the values exchanged by the two engines.

BurstGroupingOutput
├── groups: [BurstGroup]
└── boundaryEvidence: [BurstBoundaryEvidence]

BurstAnalysisResult
├── ordered candidate scores
├── recommended and second-best IDs
├── confidence and review state
├── one-click safety flag
├── reasons
└── cautions

This design avoids throwing away intermediate decisions. RawCull can show why a boundary was created, why a file won, and why automation is or is not considered safe.

BurstGroupingConfig.algorithmVersion is currently 4 and gives hosts a version marker for persisted grouping output. Its configurable thresholds cover visual distance, EXIF and fallback time gaps, camera equality, focal-length similarity, and maximum shutter, aperture, ISO, and exposure-compensation changes in EV. Custom decoding supplies defaults for fields that are absent from older saved configurations.

BurstReviewState keeps historical states for cache compatibility. Unknown decoded raw values fall back to .none. BurstWinnerOverride also migrates older data by generating a missing ID and defaulting missing member filenames to an empty array.

6. BurstGroupingEngine: Decide Where A Burst Splits

The grouping engine expects files in shot order. It starts with the first file and evaluates each adjacent pair.

flowchart TD
    Pair["Previous + current file"] --> Visual{"Visual distance present\nand below threshold?"}
    Visual -- No --> Split["Start new group"]
    Visual -- Yes --> Time{"Time gap allowed?"}
    Time -- No --> Split
    Time -- Yes --> Camera{"Required camera same?"}
    Camera -- No --> Split
    Camera -- Yes --> Focal{"Required focal delta allowed?"}
    Focal -- No --> Split
    Focal -- Yes --> Exposure{"Exposure stable?"}
    Exposure -- No --> Split
    Exposure -- Yes --> Continue["Append to current group"]

A new group begins when any enabled boundary rule fires:

• similarity evidence is missing;
• visual distance is greater than or equal to visualDistanceThreshold;
• the absolute capture gap exceeds maxTimeGapSeconds, or maxFallbackTimeGapSeconds when either file lacks a parsed EXIF capture instant;
• camera identity changes when requireSameCamera is enabled;
• numeric or parsed focal length changes by more than maxFocalLengthDeltaMM when similarity is required;
• shutter speed, aperture, ISO, or exposure compensation changes by more than its configured EV threshold.

The default EXIF capture gap is 2 seconds; the default file-date fallback gap is 10 seconds. Each boundary records captureTimeUsedFallback, allowing later ranking to distinguish precise camera time from filesystem time.

Exposure comparison works in photographic stops:

shutter delta  = |log2(current seconds / previous seconds)|
aperture delta = 2 × |log2(current f-number / previous f-number)|
ISO delta      = |log2(current ISO / previous ISO)|
compensation   = |current EV - previous EV|

The defaults are 0.5 EV for shutter, aperture, and ISO and 0.34 EV for exposure compensation. The engine prefers numeric values. When a numeric shutter, aperture, or ISO comparison is unavailable but both normalized display strings are present and differ, it still treats the exposure as changed. The largest available adjustment is retained as exposureAdjustmentEV.

Focal length similarly prefers focalLengthMM and falls back to the first number in the display string. If either side has no usable value, that rule has no delta to evaluate.

Lens changes are recorded in BurstBoundaryEvidence, but do not by themselves split a group. They later make the group’s ranking metadata unstable. Keeping evidence separate from the grouping decision makes this behavior visible rather than implicit.

Missing similarity is conservative: it creates a boundary instead of assuming two files belong together. BurstPairKey.cacheKey standardizes the ordered adjacent-pair key used to supply those distances.

Each boundary record stores the measured values, boolean changes, final decision, and human-readable reasons. The output assigns stable sequential group IDs beginning at zero for that run.

7. BurstRankingEngine: Turn Evidence Into A Recommendation

Ranking consumes groups, files keyed by ID, sharpness scores, a score normalization maximum, saliency summaries, boundary evidence, and optional review states.

7.1 Determine Group Conditions

For each group, the engine derives:

• metadata stability: none of its internal boundaries report exposure, camera, or lens changes;
• capture-time reliability: every member has a parsed EXIF capture date rather than the modification-date fallback;
• tight similarity: every internal boundary has a visual distance below 0.22;
• dominant subject: the most frequent non-nil saliency label;
• burst-relative sharpness: a within-group 0...1 normalization when at least two valid scores exist and their normalized spread is at least 0.03.

The configured grouping threshold decides membership; the fixed tighter 0.22 check contributes to ranking confidence. They answer different questions.

7.2 Score Each Candidate

If burst-relative sharpness is available, the ranking sharpness component is:

ranking sharpness = 0.65 × catalog-normalized sharpness
                  + 0.35 × burst-relative sharpness

The final candidate score is:

overall = 0.62 × ranking sharpness
        + 0.12 × focus-point evidence
        + 0.10 × saliency consistency
        + 0.16 × metadata evidence

The supporting components are deliberately simple and inspectable:

Every component is clamped or normalized into a predictable range before use. Candidates are sorted by descending overall score; equal scores preserve the group’s original shot order.

Motion risk uses exposureTimeSeconds with focalLengthMM when both are available. A shutter at least twice as fast as the reciprocal focal-length rule gets the positive adjustment; a shutter slower than the reciprocal rule gets a stop-based penalty. Without focal length, 1/500 second or faster is treated as lower risk and 1/60 second or slower as elevated risk.

Each candidate also carries reasons and cautions such as measured sharpness, available AF evidence, classified subject, missing sharpness, changed metadata, fast or slow shutter behavior, and high ISO.

7.3 Assign Confidence Separately

Winning a group does not automatically mean a decision is safe to automate.

High confidence requires:

• at least one sharpness score;
• at least three files in the group;
• a best-versus-second score gap of at least 0.12;
• best absolute sharpness of at least 0.65 after normalization;
• stable metadata;
• tight visual similarity;
• parsed capture times for every member.

Medium confidence requires a gap of at least 0.05 and stable metadata. Other results are low confidence.

isSafeForOneClickCulling is true only for high confidence. canApplyOneClickCulling(hasSharpnessScores:) adds an explicit host-supplied confirmation that sharpness data is available before an action is enabled. RawCull still owns the action itself.

Reasons and cautions on BurstAnalysisResult are capped to three each so the result remains concise enough for inspection UI and persistence.

A modification-date fallback therefore does not prevent grouping, but it does prevent high-confidence automation and adds a capture-time caution.

8. Histogram Calculation Is Intentionally Lightweight

HistogramCalculator.normalizedLuminanceHistogram(from:) directly reads an 8-bit RGB or RGBA-style CGImage buffer. Each pixel is placed in one of 256 bins using Rec. 601 luminance:

Y = 0.299R + 0.587G + 0.114B

The bins are normalized by the largest bin count, so the peak has value 1. This is shape normalization, not probability normalization; the bins do not necessarily sum to 1.

The implementation requires positive dimensions, provider data, 8 bits per component, and at least three bytes per pixel. Unsupported input returns a zero-filled 256-bin array, preserving a stable return shape for views and callers.

The helper is appropriate for display and lightweight comparisons. Color conversion, RAW development, high-bit-depth analysis, and channel-layout generalization are outside this package.

9. Concurrency Is Achieved Through Purity

RawCullCore contains no actor because it owns no shared mutable runtime state. Models are Sendable values and engines are namespaces of nonisolated static functions.

The package performs no asynchronous I/O. Callers do expensive decoding, Vision work, embeddings, and sharpness analysis on the appropriate tasks, then pass snapshots into RawCullCore.

This design has several practical benefits:

• grouping and ranking are deterministic for the same inputs;
• no application singleton can change a result midway through a call;
• tests do not require async setup or framework assets;
• background work does not cross the main actor merely because the host uses main-actor default isolation.

10. Testing Rules And Migrations

Tests/RawCullCoreTests/ uses Swift Testing and synthetic values. Coverage includes:

• model coding, hashing, identity, and source-catalog values;
• valid, decimal, whitespace-varied, and malformed focus strings;
• empty and stable groups;
• boundaries caused by missing distance, time, camera, focal length, and exposure;
• EXIF capture ordering, time-zone offsets, modification-date fallback, and the different fallback gap;
• numeric and display-string exposure comparisons in photographic stops;
• boundary evidence content;
• absolute and burst-relative sharpness ranking, motion risk, and progressive ISO penalties;
• missing scores, stable tie order, confidence, review-state propagation, and one-click eligibility;
• RGB and RGBA histogram bins, peak normalization, and unsupported input.

No test needs an actual catalog, RAW file, Vision model, Metal device, cache directory, or settings store. That is the strongest evidence that the package boundary is doing useful work.

11. How To Extend Culling Logic

Use these constraints when adding a rule:

1. Pass the required fact into a Sendable package value; do not reach into a view model.
2. Preserve raw evidence separately from the final boolean or winner.
3. Keep scoring weights and confidence thresholds deterministic and testable.
4. Decide whether a rule affects group membership, candidate ranking, confidence, or only a caution.
5. Version persisted behavior when a changed rule can invalidate cached results.
6. Maintain decoding fallbacks for existing review and override data.
7. Leave framework observations and heavy image processing in the producing package.
8. Leave user actions and presentation state in RawCull.

Source Map

Return to the package overview in RawCull Packages, or continue with the file-decoding layer in How RawParserKit Is Constructed.

9.4 - How RawParserKit Is Constructed

How RawParserKit Is Constructed

RawParserKit is RawCull’s camera-file boundary. It knows how Sony ARW and Nikon NEF files are structured, how to locate their embedded JPEGs and AF metadata, and how to turn those sources into orientation-normalized images and display-ready metadata.

The package stops at decoding. It does not score sharpness, generate embeddings, group bursts, cache application results, or decide which photo should be kept.

1. Begin With The Package Boundary

RawParserKit owns:

• vendor-neutral RAW format dispatch;
• Sony ARW and Nikon NEF format knowledge;
• TIFF IFD and vendor MakerNote traversal;
• focus-location and embedded-JPEG offset parsing;
• RAW and rendered-image thumbnails and previews;
• source-orientation normalization;
• display-ready and numeric EXIF/RAW metadata extraction, including a structured capture instant and time-zone offset;
• Sony full sensor development to JPEG;
• decode task deduplication, concurrency limiting, and cooperative cancellation;
• diagnostics and staged compatibility APIs.

The host application owns:

• security-scoped URL access and sandbox bookmarks;
• directory discovery, catalogs, and source selection;
• memory and disk cache locations and eviction policy;
• image-analysis inputs and result identity;
• observable loading state, placeholders, retries, and presentation;
• ratings, burst grouping, ranking, and saved-file behavior.

RawParserKit supplies inputs to higher layers without importing them:

flowchart LR
    File["ARW, NEF, JPEG, PNG, or TIFF"] --> Parser["RawParserKit"]
    Parser --> Image["CGImage or NSImage"]
    Parser --> Metadata["RawImageMetadata + RawFocusPoint"]
    Image --> Host["RawCull adapters"]
    Metadata --> Host
    Host --> Analysis["PhotoAnalysisKit / PhotoAIKit"]
    Host --> Core["RawCullCore"]

2. Package Shape And Framework Boundary

Package.swift declares Swift tools 6.2, Swift 6 language mode, macOS 26, one library product, and one test target. There are no third-party package dependencies.

The implementation uses Apple frameworks appropriate to file decoding: Foundation, AppKit, Core Graphics, ImageIO, Core Image, OSLog, and synchronization primitives from os.

Like RawCullCore, the target enables main-actor default isolation plus InferIsolatedConformances and NonisolatedNonsendingByDefault. Pure parsing and static format APIs opt out with nonisolated; the stateful loader and limiter use actors.

The source is arranged in layers:

RawImageLoader                         high-level deduplicated facade
├── RawFormatRegistry + RawFormat      vendor-neutral dispatch
│   ├── SonyRawFormat
│   └── NikonRawFormat
├── thumbnail and preview extractors   ImageIO + binary fallback
├── MakerNote parsers                  TIFF byte traversal
├── OrientationNormalizedImageLoader   rendered/embedded image helpers
└── cancellation + decode limiter      concurrency control

Callers can use the facade for normal browser behavior or a lower layer when they need explicit control.

3. RawFormat Makes Vendor Dispatch Explicit

Sources/RawParserKit/RawFormat.swift describes the static capabilities every camera format supplies:

• supported filename extensions and a display name;
• thumbnail extraction;
• embedded-preview extraction;
• AF focus-location parsing;
• a readable label for compression codes;
• camera-specific megapixel thresholds for S, M, and L size classes.

Conformers are stateless enums. RawFormatRegistry.all currently registers:

format(for:) lowercases a URL’s extension and returns a format metatype. Callers invoke static protocol requirements on that value without switching on brands.

The default rawSizeClass implementation converts dimensions to megapixels, obtains body-specific L and M thresholds, and returns L, M, or S. Unknown bodies use a generic fallback supplied by the conformer.

This is a simple plugin architecture inside the package. Adding a camera brand means adding a conformer and registering it, not adding vendor switches throughout the loader.

4. RawImageLoader Is The High-Level Facade

RawImageLoader.shared is an actor. It offers four current operations:

4.1 Deduplicate Equivalent Work

The actor stores in-flight tasks:

• thumbnails keyed by URL and requested pixel size;
• previews keyed by URL;
• metadata keyed by URL.

If another caller requests the same work while it is running, both await the existing task. The entry is removed after completion. This prevents rapid view updates from decoding the same large image repeatedly.

4.2 Bound Expensive Decodes

The facade uses two DecodeConcurrencyLimiter actors:

• up to six concurrent thumbnail decodes;
• up to two concurrent full-size preview decodes.

These limits control memory as much as CPU. A few full-resolution bitmaps can consume substantially more memory than the compressed RAW files that produced them.

4.3 Follow The Thumbnail Strategy

For a rendered JPEG, PNG, or TIFF, the loader asks ImageIO for an orientation-aware thumbnail. For RAW input, it tries an embedded ImageIO thumbnail first and then dispatches to the registered vendor extractor. The vendor result is orientation-normalized before it becomes an NSImage.

4.4 Follow The Preview Strategy

The preview path checks for a same-basename .jpg sidecar first. If absent, it tries an orientation-aware embedded preview and then the registered format’s embedded-preview extractor. Vendor output is normalized using the source orientation.

The sidecar-first decision is host-facing convenience, not RAW parsing. A caller that requires only bytes physically embedded in the RAW can call the format or extractor API directly.

5. Metadata Is Normalized Into A Neutral Snapshot

RawImageMetadata contains optional display and numeric values for camera, lens, shutter speed, exposure time in seconds, aperture, focal length in millimeters, ISO, exposure compensation in EV, capture time, dimensions, focus point, RAW compression label, size class, and pixel dimensions.

RawImageLoader.metadata(for:) combines several sources:

1. ImageIO properties from the source or a sidecar fallback;
2. TIFF make, model, compression, and display-date fields;
3. EXIF exposure time, aperture, focal length, ISO, exposure bias, lens, and dimensions;
4. EXIF DateTimeOriginal, SubsecTimeOriginal, and OffsetTimeOriginal;
5. the registered vendor MakerNote parser for an AF point;
6. EXIF subject-area coordinates when vendor focus data is unavailable;
7. vendor-specific compression labels and size-class thresholds.

captureDate parses the original capture timestamp as a real Date, preserving variable-length fractional seconds. OffsetTimeOriginal accepts Z, +HH:MM, -HH:MM, and compact +HHMM/-HHMM forms; the parsed offset is also retained as captureTimeZoneOffsetSeconds. When the EXIF offset is missing, parsing uses the current time zone. A missing or invalid DateTimeOriginal leaves captureDate nil rather than substituting the TIFF display date.

capturedAt remains a display string and can fall back to TIFF DateTime. Keeping it separate from captureDate prevents formatted UI text from becoming burst-ordering evidence.

rows returns non-empty display label/value pairs, while isEmpty lets the facade omit an empty metadata object. Numeric exposure time, aperture, focal length, ISO, exposure compensation, width, and height remain available so hosts do not have to parse formatted strings for analysis or domain rules.

RawFocusPoint stores normalized X and Y coordinates. Its failable focus-string initializer accepts the common "width height x y" shape, checks positive dimensions, and rejects coordinates outside 0...1.

RawCull’s RawParserKitImageLoader maps this parser-owned snapshot into its own ExifMetadata and RawCullFileItem values. ScanFiles carries the parsed capture instant and offset into the catalog; RawCullCore can then prefer camera time and explicitly detect a modification-date fallback. The similar types exist on purpose: each package owns the vocabulary at its boundary and neither must depend on the other.

6. Thumbnail Extraction Prefers Embedded Work

Thumbnails should not require full RAW development when a camera already stored a usable JPEG.

6.1 Sony

SonyThumbnailExtractor first asks SonyMakerNoteParser for embedded JPEG locations and decodes a selected JPEG directly. This bypasses macOS RAW decoder failures seen with newer ARW layouts. If the binary path cannot locate a JPEG, it falls back to ImageIO’s embedded-thumbnail behavior.

6.2 Nikon

NikonThumbnailExtractor asks ImageIO for a transformed embedded thumbnail. Both vendor extractors then redraw into an 8-bit premultiplied sRGB bitmap using interpolation quality derived from qualityCost.

Both APIs run their synchronous ImageIO work through CancellableImageIOWork and throw ThumbnailError for an invalid source, failed generation, or failed bitmap context.

ThumbnailSharpener is an optional lower-level Core Image helper for producing a sharpened preview at a requested maximum dimension. The high-level boundary does not force sharpening on every caller.

7. Embedded Preview Extraction Has Two Paths

The two vendors use the same building blocks in a different order. SonyEmbeddedJPEGExtractor prefers the binary TIFF locator, avoiding RAW decoder initialization on affected ARW files, then falls back to ImageIO. NikonEmbeddedJPEGExtractor inspects ImageIO sub-images first and uses the binary locator when the preview is not exposed there.

flowchart TD
    Source["RAW URL"] --> ImageIO["Inspect ImageIO image indexes"]
    Source --> Parser["Vendor TIFF parser"]
    ImageIO -->|usable JPEG| Decode["Decode or downsample"]
    Parser --> Offset["Absolute JPEG offset + length"]
    Offset --> Bytes["Read JPEG bytes"]
    Bytes --> Decode
    Decode --> Result["CGImage"]

fullSize: true permits a longest edge up to 8640 pixels. The normal preview path limits large images to 4320 pixels.

The Nikon fallback prefers the full-resolution SubIFD preview for full-size requests and IFD1 for smaller requests. Sony chooses the largest available JPEG first, with preview and thumbnail fallbacks.

Extractor-level limiters default to two concurrent operations, and a caller can inject a shared limiter. This allows a host facade to enforce one budget across several decode paths instead of accidentally stacking independent limits.

8. Sony Full-Size JPEG Creation Is Different From Extraction

SonyRawFormat.createFullSizeJPEG(from:quality:) does not return a camera-embedded preview. SonyRAWJPEGCreator develops the ARW sensor data through macOS CIRAWFilter, renders it into sRGB, and encodes JPEG data.

Quality must be in 0...1. The operation can fail with:

• invalidQuality;
• unsupportedOrInvalidRAW when the installed macOS RAW decoder cannot develop that file;
• encodingFailed.

This distinction matters in UI and caching. An embedded preview and a developed sensor image have different cost, pixels, appearance, and invalidation semantics even if both are JPEG-encoded at the end.

9. Sony MakerNote And TIFF Parsing

Sony ARW is TIFF-based. Focus parsing follows this structure:

TIFF IFD0
└── ExifIFD tag 0x8769
    └── MakerNote tag 0x927C
        └── Sony MakerNote IFD
            └── FocusLocation tag 0x2027
                (fallback: 0x204A)

The focus tag contains four unsigned 16-bit values: image width, image height, X, and Y. Sony MakerNote IFD offsets are interpreted as absolute file offsets.

The production parser uses a fast path and a fallback:

• focus location reads the first 4 MB, then retries with the full file when necessary;
• embedded-JPEG discovery reads the first 512 KB, then retries the full file when no locations were found.

The embedded locator walks TIFF IFDs and returns optional absolute locations for a small thumbnail, preview, and full JPEG. readEmbeddedJPEGData seeks directly to a validated location and reads its byte range.

The fast path keeps normal scans inexpensive; the full-file fallback supports bodies that place relevant TIFF structures near the end of the file.

10. Nikon MakerNote And TIFF Parsing

Nikon NEF is also TIFF-based, but modern Nikon Type-3 MakerNotes contain their own TIFF header:

TIFF IFD0
└── ExifIFD tag 0x8769
    └── MakerNote tag 0x927C
        ├── "Nikon\0" signature + version
        └── inner TIFF header
            └── Nikon IFD
                └── AFInfo2 tag 0x00B7

Offsets in the inner TIFF are relative to the MakerNote TIFF-header base, not the start of the NEF file. Keeping this offset rule inside the Nikon parser prevents a generic loader from acquiring vendor-specific exceptions.

For supported modern AFInfo2 layouts, the parser reads AF image dimensions, area position, and area size, and returns the same "width height x y" shape as Sony. The public shape lets all downstream code use one focus-point adapter.

Nikon embedded preview discovery examines Compression=6 SubIFDs referenced from IFD0 and the IFD1 JPEG interchange fields. It returns optional locations for the largest preview and IFD1 JPEG.

As on Sony, focus parsing starts with 4 MB and falls back to the full file. Embedded-location parsing uses a 1 MB fast path followed by a full-file retry when required.

11. Diagnostics Report The Failed Stage

The ordinary parser APIs return optionals because missing or unsupported MakerNote data is expected during normal browsing.

For troubleshooting, each vendor also exposes diagnostic forms for focus and embedded-JPEG lookup. RawParserDiagnostics<Value> contains:

• the optional parsed value;
• an ordered trace of stages and offsets checked;
• an optional final failure explanation.

This keeps logging policy outside the binary parser while allowing RawCull’s diagnostics UI to show whether failure occurred at file access, TIFF validation, IFD lookup, MakerNote traversal, tag interpretation, or fallback.

12. Orientation Helpers Normalize Visual Coordinates

OrientationNormalizedImageLoader provides lower-level operations for rendered URLs, encoded JPEG data, source thumbnails, embedded thumbnails, and embedded previews.

ImageIO’s transform option is used when available. The loader also implements the eight EXIF orientation transforms, including mirrored and transposed cases, and can read orientation from the RAW source when decoding separately extracted JPEG data.

SupportedFileType enumerates .arw, .nef, .jpeg, .jpg, .png, .tif, and .tiff. Its rendered-image set distinguishes sources that can be loaded directly from formats that need RAW dispatch.

Orientation is part of the parser boundary because AF coordinates and subject analysis must refer to the same visual image the user sees.

13. Cancellation And Decode Limiting Solve Different Problems

CancellableImageIOWork bridges a synchronous ImageIO closure to async code on a global dispatch queue. It creates an ImageIOCancellationToken and uses a locked state machine to ensure the checked continuation is resumed exactly once, even when cancellation races completion.

Cancellation is cooperative. The token is checked before and after synchronous framework calls and between multi-stage loops. A framework function already executing may not stop internally, but its result is discarded when cancellation is observed.

DecodeConcurrencyLimiter is an actor that solves admission control. It:

1. grants work immediately while slots are available;
2. queues additional continuations;
3. removes and resumes a cancelled waiter;
4. transfers a released slot directly to the next waiter;
5. releases the slot with defer when work completes.

Cancellation prevents obsolete work; limiting prevents too much valid work from running simultaneously. The loader needs both.

14. Compatibility APIs Support Staged Migration

The package retains deprecated names such as:

• BrowserExifInfo and BrowserFocusPoint;
• thumbnail200px, extractembeddedJPG, and exifInfo;
• JPGSonyARWExtractor and JPGNikonNEFExtractor;
• extractFullJPEG on RawFormat.

Each shim forwards to a current neutral name. This lets RawCull migrate call sites without forcing an all-at-once source break, while deprecation warnings make the intended direction visible.

New code should use RawImageMetadata, RawFocusPoint, thumbnail, previewImage, metadata, the embedded-JPEG extractors, and extractEmbeddedPreview.

15. Test Binary Rules Without Shipping Camera Files

Tests/RawParserKitTests/ uses Swift Testing and mostly synthetic TIFF-like byte buffers. The suite covers:

• registry extension matching and dispatch;
• Sony focus tags, offset rules, invalid byte-order markers, fallback tags, embedded JPEG locations, and diagnostics;
• Nikon Type-3 MakerNote and AFInfo2 layouts, SubIFDs, IFD1 JPEGs, offset rules, and diagnostics;
• direct reading of embedded JPEG bytes;
• cancellation before decode and cancellation behavior in vendor extractors;
• decode-limiter capacity;
• Sony full-size JPEG quality validation and generated JPEG properties;
• numeric exposure-time, focal-length, aperture, ISO, and exposure-compensation metadata;
• capture-date offsets, subsecond precision, invalid-date behavior, and offset parsing;
• current public naming and format-helper behavior.

Synthetic binary fixtures make edge cases reproducible and avoid committing large proprietary ARW and NEF samples. Framework integration is tested with small generated images where needed.

16. How To Add Another Camera Vendor

Use the existing extension points:

1. Add a stateless RawFormat conformer with its extension and display name.
2. Implement vendor-specific thumbnail, preview, focus, compression, and size-class behavior.
3. Put TIFF or MakerNote byte rules in a dedicated parser, including explicit offset bases and bounds checks.
4. Return the common normalized focus-location string at the public boundary.
5. Add ImageIO behavior first and a binary embedded-JPEG fallback when the framework does not expose the preview reliably.
6. Make blocking decode stages cooperative with CancellableImageIOWork.
7. Register the conformer in RawFormatRegistry.all.
8. Add synthetic binary tests for endian, offsets, missing tags, corrupt lengths, and diagnostics.
9. Leave analysis, cache placement, and UI policy in their owning layers.

Source Map

Continue from decoded images into How PhotoAnalysisKit Is Constructed, or return to the package overview.

10 - Artificial Intelligence

Artificial Intelligence in RawCull

This section explains how RawCull uses reusable AI components without allowing model-runtime details to spread through the application. It is written as a learning path: first understand the boundary between RawCull and PhotoAIKit, then study the package itself, and finally follow CLIP from application startup to a persisted similarity artifact.

The source for this section comes from the sibling PhotoAIKit and RawCull projects. Paths beginning with Sources/ refer to PhotoAIKit. Paths beginning with Model/, Main/, Views/, or Actors/ refer to the RawCull app target.

What This Section Documents

PhotoAIKit currently contains two main AI families:

• CLIP image embeddings for visual similarity.
• SAM 3 subject segmentation for subject masks.

It also contains an Apple Vision feature-print backend. In RawCull, Vision is both the always-available similarity implementation and the whole-batch fallback when CLIP indexing fails.

The detailed CLIP document follows code that is connected to RawCull’s similarity and burst-analysis features. The package architecture document also covers the SAM 3 contracts, workflows, and storage so that the complete package design is understandable. Not every reusable package capability is necessarily exposed as a finished RawCull user workflow.

The Central Design Idea

RawCull and PhotoAIKit answer different kinds of questions.

PhotoAIKit asks:

• What does an image source look like at a package boundary?
• How is a model bundle validated and identified?
• How does a backend produce and compare a similarity artifact?
• How is bounded indexing or segmentation orchestrated?
• How can reusable artifacts and masks be encoded or cached?

RawCull asks:

• Where are models installed for this application?
• How is a Sony or Nikon RAW file decoded for AI input?
• Which backend did the user request?
• When should a catalog be indexed or reindexed?
• How do similarity distances affect burst grouping and culling?
• What state and wording should SwiftUI present?

This is dependency inversion in practical form. PhotoAIKit defines small protocols such as ImageDecoding, ImageSimilarityArtifactProviding, and ImageSimilarityArtifactComparing. RawCull injects app-specific implementations and keeps its file model, UI, sandbox policy, and culling decisions outside the package.

flowchart LR
    UI["RawCull SwiftUI and settings"] --> Integration["RawCullAIIntegration composition root"]
    Integration --> AppAdapter["RawCull adapters: paths, RAW decoding, policy"]
    AppAdapter --> Contracts["PhotoAIContracts"]
    Integration --> CLIP["CoreAICLIPBackend"]
    Integration --> SAM3["CoreAISAM3Backend"]
    Integration --> Vision["VisionFeaturePrintBackend"]
    AppAdapter --> Workflows["PhotoAIWorkflows"]
    Workflows --> Contracts
    Storage["PhotoAIStorage"] --> Contracts
    CLIP --> Contracts
    SAM3 --> Contracts
    Vision --> Contracts

The arrow direction matters: PhotoAIKit does not import RawCull. A reusable package should not need to know what FileItem, RawCullViewModel, an app-specific model folder, or a burst winner means.

Responsibility Boundary

Current Runtime Shape

RawCull deliberately has a safe startup path:

1. RawCullApp creates one RawCullAIIntegration composition root.
2. The main view model initially receives the Vision similarity service.
3. RawCullAISettingsModel.refresh() asks the integration to validate the CLIP and SAM 3 model candidates.
4. PhotoAIKit validates the selected model bundle and fingerprints its asset.
5. If CLIP validation and provider construction succeed, the saved CLIP preference can select RawCullCLIPSimilarityService.
6. If CLIP is disabled, missing, invalid, or cannot be constructed, RawCull continues with Vision.
7. If CLIP starts a batch but any item fails, the complete requested batch is retried with Vision.

stateDiagram-v2
    [*] --> VisionStartup
    VisionStartup --> CheckingModels: refresh capabilities
    CheckingModels --> VisionSelected: CLIP disabled, missing, or invalid
    CheckingModels --> CLIPSelected: preference enabled and provider ready
    CLIPSelected --> CLIPArtifacts: all requested items succeed
    CLIPSelected --> VisionFallbackArtifacts: any requested item fails
    VisionSelected --> VisionArtifacts: index catalog

The whole-batch fallback is an integrity rule, not only an error-recovery convenience. CLIP vectors and Vision feature prints have different representations and distance semantics. Recomputing the entire batch prevents a catalog from containing artifacts that cannot be compared with one another.

Vocabulary

Suggested Learning Order

Read How PhotoAIKit Is Constructed first if dependency boundaries, protocols, or package products are unfamiliar. It builds the mental model from the bottom up.

Then read How RawCull Enables and Uses CLIP. That document follows the runtime path and shows where the clean package abstractions meet app-specific concerns such as model locations, RAW files, settings, burst grouping, and cache validation.

Finally, keep AI Runtime Step by Step beside the source code. It expands the runtime path into exact function hops and branches, including what does and does not invoke CLIP when a developer indexes similarity, analyzes bursts, opens a burst, or enters the detailed comparison view.

10.1 - How RawCull Loads and Uses CLIP

How RawCull Loads and Uses CLIP

RawCull supports two CLIP models:

• OpenAI CLIP ViT-B/32, which processes a 224 × 224 image;
• OpenCLIP DataComp ViT-B-32-256, which processes a 256 × 256 image.

The user selects one model in Settings > AI. That one selection defines the vector space used for both image-to-image similarity and text-to-image semantic search. RawCull never mixes vectors from the two models.

This page starts with the basic idea behind CLIP and then follows the current source code from application startup, through model validation and Core AI inference, to search results, burst groups, and persisted artifacts.

1. CLIP In Plain Language

CLIP stands for Contrastive Language-Image Pre-training. It was trained to place matching images and text descriptions near each other in a shared mathematical space.

For example, CLIP can turn these two inputs into vectors:

image: a photograph of a red fox at dusk  ──► [0.012, -0.084, ...]
text:  "red fox at dusk"                  ──► [0.018, -0.079, ...]

A vector is just an ordered list of numbers. RawCull does not display those numbers to the user. It compares their directions:

• two image vectors pointing in similar directions represent visually or semantically related images;
• an image vector pointing in a similar direction to a text vector is a possible match for that text;
• vectors produced by different CLIP models are not comparable, even when they contain the same number of values.

CLIP does not write a caption, locate the exact pixels of an object, rate a photo, or decide which burst frame is sharpest. It provides similarity evidence. RawCull decides how to use that evidence.

The original OpenAI CLIP introduction and CLIP paper explain the contrastive training method in more depth.

2. Why There Are Two CLIP Models

Both choices use the same general CLIP idea and the same size of output vector, but they use different learned weights and different image resolutions.

The model details do not come from hard-coded branches inside CoreAICLIPProvider. The exporter writes them into each bundle’s metadata.json, and the provider builds a CLIPRuntimeConfiguration from that metadata.

2.1 OpenAI CLIP ViT-B/32

The OpenAI choice is the original clip-vit-base-patch32 checkpoint used by PhotoAIKit’s exporter.

ViT-B/32 can be read as:

• ViT: the image encoder is a Vision Transformer;
• B: the base-size transformer family;
• 32: the image is divided into 32 × 32 pixel patches before transformer processing.

The exported image encoder receives a 224 × 224 image and returns a 512-dimensional, L2-normalized image vector. The exported text encoder receives up to 77 CLIP BPE tokens plus an attention mask and returns a normalized 512-dimensional text vector.

The original CLIP family was trained on 400 million image-text pairs collected from the internet. The aim was not to learn a fixed list of RawCull categories. It learned a broad relationship between visual content and natural-language descriptions.

2.2 OpenCLIP DataComp ViT-B-32-256

The DataComp choice uses the open-source OpenCLIP implementation and the datacomp_s34b_b86k checkpoint. Its architecture is ViT-B-32-256: a base Vision Transformer with 32 × 32 patches and a 256 × 256 image input.

This checkpoint was trained on the DataComp-1B dataset and its published name records that the training run saw approximately 34 billion samples. The DataComp model card describes the released checkpoint.

The DataComp model also produces 512-dimensional normalized vectors and uses a 77-token CLIP BPE context. Its exported text graph follows OpenCLIP’s input contract: token IDs are padded with zero and no separate attention-mask input is required.

2.3 What The Model Choice Means In RawCull

The two models can rank the same catalog differently because their weights, training data, and input resolution differ. RawCull does not declare one model universally better. It treats them as two valid alternatives.

The important rule is:

An OpenAI image vector can be compared only with OpenAI image or text vectors from the same model fingerprint and processing configuration. The equivalent rule applies to DataComp.

Both output 512 values, but matching dimensions alone do not make two vector spaces compatible.

3. Packages And Frameworks In The CLIP Path

RawCull is the host application. Most reusable CLIP mechanics live in PhotoAIKit, while RAW decoding and culling policy remain in RawCull.

3.1 Swift Packages Used At Runtime

RawCull also links CoreAISAM3Backend, but SAM 3 is a separate segmentation model. Its resource check happens alongside the two CLIP checks; it is not used to calculate CLIP vectors.

PhotoAIKit has one external Swift package dependency: apple/coreai-models, pinned to an exact revision. Its CoreAISegmentation product makes the Core AI runtime APIs available to the CLIP and SAM 3 backend targets.

3.2 Apple Frameworks Used Around The Packages

3.3 Python Packages Used Only To Export Models

The model bundle is created before RawCull runs. PhotoAIKit’s Tools/export_clip.py declares these build-time tools:

These Python packages are not shipped as part of RawCull’s Swift runtime.

4. The Complete Runtime Shape

The main path is:

flowchart TD
    Settings["Settings: choose OpenAI or DataComp"] --> Integration["RawCullAIIntegration"]
    Integration --> Managers["Two CLIP resource managers"]
    Managers --> Validate["PhotoAIKit validates both bundles"]
    Validate --> Providers["One provider per valid bundle"]
    Providers --> Selected{"Which model is selected?"}
    Selected -->|OpenAI| OpenAI["OpenAI CoreAICLIPProvider"]
    Selected -->|DataComp| DataComp["DataComp CoreAICLIPProvider"]
    OpenAI --> Similarity["Image similarity and burst grouping"]
    OpenAI --> Search["Text-to-image semantic search"]
    DataComp --> Similarity
    DataComp --> Search

There may be two validated providers in memory, but RawCull exposes only the selected provider to the active similarity and semantic-search features.

5. Startup Uses Vision First

Main/RawCullApp.swift creates the object graph synchronously:

let integration = RawCullAIIntegration()
let viewModel = RawCullViewModel(
    similarityService: integration.visionSimilarityService,
    semanticSearchCapability: integration.capabilities()
        .semanticSearchStatus(for: .defaultSelection),
    deepAIReviewFeature: integration.deepAIReviewFeature
)

Vision is installed first because it needs no downloaded model bundle. The app can open even when both CLIP folders are absent.

RawCullAISettingsModel then receives two callbacks:

• one installs a new RawCullSimilarityServicing value in the main view model;
• one installs the selected model’s semantic-search capability and service.

The asynchronous .task attached to the main window calls aiSettingsModel.refresh(). Model directory inspection, fingerprinting, and provider construction therefore do not block RawCullApp.init().

6. Preferences And The Selected Model

Model/ViewModels/RawCullAISettingsModel.swift stores two values:

RawCullAI.useCLIPForSimilarity
RawCullAI.selectedCLIPModel

When no values have been saved:

• CLIP similarity defaults to enabled;
• the OpenAI model is the default selection.

The model picker is mutually exclusive. Choosing DataComp replaces OpenAI as the selected model; it does not combine their results.

The CLIP toggle is a request, not proof that a model is usable:

Semantic-search readiness is tracked separately because Vision can compare two images but cannot compare an image with text. A valid selected CLIP provider is required for semantic search.

RawCull can read already-cached compatible CLIP artifacts for semantic search. To create missing semantic-search artifacts through the normal indexing button, the selected CLIP model must also be active as the similarity backend.

Changing the selected model immediately reapplies both services. It is not necessary to restart RawCull.

7. Model Locations

RawCullAIPaths.live() creates two separate installed-model locations:

~/Library/Application Support/RawCull/Models/
├── CLIP-DataComp/
└── CLIP-OpenAI/

A sandboxed build resolves the same relative folders inside its container:

~/Library/Containers/no.blogspot.RawCull/Data/Library/Application Support/
└── RawCull/Models/
    ├── CLIP-DataComp/
    └── CLIP-OpenAI/

Settings > AI shows the exact paths for the running build.

RawCullAIModelCandidates.urls(...) puts the installed directory first. Debug builds may add app-bundle resource candidates such as Models/CLIP-OpenAI; release builds do not use bundled fallback unless it is explicitly enabled during construction.

PhotoAIKit does not search these paths. RawCull owns path policy and passes an ordered URL list into the package.

8. What A CLIP Bundle Contains

Each model has its own self-contained directory:

CLIP-OpenAI/ or CLIP-DataComp/
├── metadata.json
├── tokenizer/
│   └── tokenizer.json
└── selected-model.aimodel

The selected model may instead be a compiled .aimodelc directory.

A current metadata version 0.4 export describes:

• model family, source, revision, architecture, and pretrained checkpoint;
• 512 embedding dimensions;
• input width and height;
• shortest-side resize, center crop, and interpolation policy;
• RGB mean and standard deviation;
• tokenizer type, version, context length, and padding token;
• the image_encoder and text_encoder function names;
• output normalization and configuration versions;
• assets.main, which names the selected Core AI asset;
• asset_fingerprints.main, which records the expected asset fingerprint.

This matters because a model is more than its weight file. The tokenizer and preprocessing rules must match the weights used during training.

9. Validation And Provider Construction

RawCullAIIntegration owns one resource manager for each model:

clipDataCompModelResourceManager
clipOpenAIModelResourceManager

During refreshCapabilities(), RawCull loads SAM 3, DataComp CLIP, and OpenAI CLIP concurrently:

async let sam3Load = sam3ModelResourceManager.load()
async let clipDataCompLoad = clipDataCompModelResourceManager.load()
async let clipOpenAILoad = clipOpenAIModelResourceManager.load()

Each RawCullAIModelResourceManager is an actor. It first builds a lightweight snapshot from paths, file kinds, sizes, modification dates, and resolved symbolic-link paths. If nothing changed, it can reuse the previous capability and provider instead of hashing a large model again.

When the snapshot changes, PhotoAIKit remains the authority:

flowchart TD
    Candidate["RawCull candidate URL"] --> Exists{"Directory exists?"}
    Exists -->|No| Next["Try the next candidate"]
    Exists -->|Yes| Metadata["Decode metadata.json"]
    Metadata --> Asset["Resolve assets.main"]
    Asset --> Resources["Check tokenizer and selected asset"]
    Resources --> Fingerprint["Fingerprint asset and verify manifest checksum"]
    Fingerprint --> Valid{"Valid?"}
    Valid -->|No| Invalid["Report invalid and stop"]
    Valid -->|Yes| Provider["Construct CoreAICLIPProvider"]

A missing candidate is skipped. An invalid higher-priority candidate stops resolution instead of being hidden by a lower-priority debug bundle.

The provider constructor validates the bundle again, stores its fingerprinted ModelIdentity, decodes metadata.json, and validates the model-specific CLIPRuntimeConfiguration. It does not load the large Core AI model yet.

Capability and provider-construction failure are recorded separately. Settings can therefore distinguish a missing or damaged bundle from a bundle that validated but could not initialize its runtime configuration.

10. Lazy Core AI Loading

The first image or text embedding request calls CoreAICLIPProvider.loadModel().

The provider then:

1. reads assets.main and creates an AIModel;
2. asks Core AI to prefer GPU specialization;
3. creates CLIPTokenizer from the bundle’s tokenizer folder;
4. finds and loads the metadata-selected image and text functions;
5. requires pixel_values and image_embeds on the image side;
6. requires input_ids and text_embeds on the text side;
7. accepts attention_mask when the text function declares it;
8. validates tensor ranks, dimensions, scalar types, resolution, and token context length;
9. caches the functions, descriptors, tokenizer, and compatibility inputs inside the provider actor.

New bundles use two named functions in one asset:

image_encoder(pixel_values)                 -> image_embeds
text_encoder(input_ids[, attention_mask])  -> text_embeds

This avoids running the text tower while indexing an image and avoids running the image tower for every search query.

PhotoAIKit still supports older single-function CLIP exports. If an older image function also requires text inputs, the provider supplies tokens for "a photo". If an older text function requires an image, it supplies a zero image. These are compatibility inputs, not RawCull’s semantic-search query.

The loaded model is cached in the actor, so later requests reuse it. The first request can take longer because macOS may specialize a portable model for the current Mac.

11. From A RAW File To Model Pixels

PhotoAIKit deliberately does not know how to decode camera RAW formats. RawCull’s RawCullSimilarityImageDecoder supplies that application-specific step.

For each AIImageSource, the decoder:

1. checks task cancellation;
2. asks RawParserKitImageLoader for a thumbnail no larger than 512 pixels;
3. if that fails, asks ImageIO for an existing embedded thumbnail;
4. reports imageDecodeFailed if neither path returns a CGImage.

The 512-pixel value belongs to RawCull’s embedding pipeline. It gives the model provider a bounded image instead of decoding an unnecessarily large RAW preview.

The selected provider then applies the exact preprocessing declared by that model:

1. convert to an 8-bit sRGB RGBA drawing buffer;
2. resize the shortest side while preserving aspect ratio;
3. take a centered 224 × 224 OpenAI crop or 256 × 256 DataComp crop;
4. convert red, green, and blue from bytes to values in 0 ... 1;
5. subtract CLIP means 0.48145466, 0.4578275, and 0.40821073;
6. divide by CLIP standard deviations 0.26862954, 0.26130258, and 0.27577711;
7. change interleaved RGB into channel-first [1, 3, height, width] order;
8. write a Float16 or Float32 Core AI NDArray, as required by the model.

The center crop can remove pixels near the long edges of a photograph. That is intentional: preprocessing must match the model’s training and export contract.

12. Creating A CLIP Image Artifact

After preprocessing, the provider runs image_encoder and reads image_embeds.

Both current graphs export an L2-normalized vector. PhotoAIKit wraps the result in ImageEmbedding, which also applies safe L2 normalization:

magnitude = sqrt(sum(value²))
normalizedValue = value / magnitude

The provider then creates a SimilarityArtifact:

SimilarityArtifact
├── descriptor
│   ├── backend = "clip"
│   ├── model fingerprint
│   ├── dimensions = 512
│   ├── representation version
│   ├── preprocessing version
│   ├── normalization version
│   ├── configuration version
│   ├── source path, size, and modification date
│   └── artifact schema version
└── payload
    └── JSON-encoded ImageEmbedding vector

The descriptor prevents RawCull from treating a vector as reusable merely because it can be decoded.

13. Indexing, Concurrency, And Failure Recovery

RawCullCLIPSimilarityService creates a PhotoAIKit SimilarityArtifactIndexer with:

• the selected CLIP provider;
• RawCull’s diagnosing RAW decoder;
• no per-image or whole-batch fallback provider;
• a current concurrency limit of 1.

The current CLIP behavior is:

flowchart TD
    Source["One catalog image"] --> Decode["Decode bounded CGImage"]
    Decode --> Inference["Run selected CLIP image encoder"]
    Inference --> Finite{"Vector finite and non-empty?"}
    Finite -->|Yes| Keep["Validate, persist, and keep artifact"]
    Finite -->|No| Retry["Retry once with the loaded provider"]
    Retry --> RetryOK{"Valid now?"}
    RetryOK -->|Yes| Keep
    RetryOK -->|No| Replace["Construct a fresh provider and retry"]
    Replace --> ReplaceOK{"Valid now?"}
    ReplaceOK -->|Yes| Keep
    ReplaceOK -->|No| Exclude["Record failure; exclude this image"]

Only non-finite embedding output receives the two recovery attempts. A decode error or another model error is recorded for that source without discarding valid CLIP artifacts already created for other images.

This is different from the earlier whole-batch fallback design:

• RawCull no longer throws away a successful CLIP pass because one image failed;
• it does not mix Vision artifacts into a selected CLIP indexing pass;
• a failed image has no current artifact and is excluded from similarity comparisons and burst grouping until it can be indexed successfully.

When CLIP is disabled or its selected bundle is unavailable at service selection time, RawCull uses RawCullVisionSimilarityService instead. Vision currently indexes with a concurrency limit of 4.

14. How RawCull Uses Image-To-Image Distance

For two compatible CLIP artifacts, PhotoAIKit computes cosine distance:

cosineDistance(a, b) =
    1 - dot(a, b) / (magnitude(a) × magnitude(b))

For normalized vectors:

• 0 means the directions are effectively identical;
• 1 means the directions are orthogonal;
• 2 means the directions are opposite.

Before computing the value, the provider verifies that both artifacts have the same model fingerprint, dimensions, representation, preprocessing, normalization, configuration, and schema. It also checks that each JSON payload matches its descriptor.

RawCull uses this primitive distance in three ways:

1. Find similar images. It compares a selected anchor image with every compatible catalog artifact and sorts smaller distances first.
2. Apply a small product-policy adjustment. If available saliency labels identify different subjects, SimilarityScoringModel adds 0.10 to the distance.
3. Group bursts. It compares adjacent images in capture order and passes the distances to RawCullCore.BurstGroupingEngine. The current default visual threshold is 0.25.

CLIP supplies the distance. RawCull owns the saliency adjustment, threshold, burst rules, sharpness scoring, recommendation, and rating behavior.

If an image has no valid artifact, burst grouping splits the ordered catalog into eligible runs around that gap. It does not invent a distance across the missing image.

15. How Semantic Search Works

Semantic search uses the text half of the same selected CLIP provider and the already-persisted image artifacts.

For a query such as red fox at dusk, RawCull:

1. trims surrounding whitespace but otherwise sends the literal query;
2. tokenizes it with the bundle’s CLIP BPE tokenizer;
3. truncates or pads it to the model’s 77-token context;
4. runs text_encoder;
5. validates that the returned 512-value vector is finite, non-empty, and L2-normalized;
6. compares it only with cached image artifacts that match the selected model and every backend descriptor field;
7. sorts larger cosine-similarity scores first.

Image-to-text uses cosine similarity, not distance:

cosineSimilarity(image, text) = dot(imageVector, textVector)

The provider clamps the result to -1 ... 1. It is a relative ranking score, not a calibrated probability or confidence percentage.

The text embedding is query-scoped and is not written to disk. The expensive image encoder does not run during a search. Only the text encoder runs once, followed by ordinary vector dot products over the compatible cached images.

RawCull’s deterministic tie order is:

1. higher score;
2. earlier catalog order;
3. localized filename order;
4. UUID string.

The UI initially presents the highest-ranked 20 images. The user can expand the selection without recomputing embeddings or scores. Rating and filename filters are applied before semantic ranking, and the selected semantic results become the active catalog working set until the search is cleared.

16. Persisting And Reusing Image Artifacts

RawCull persists successful similarity artifacts per source file under:

Application Support/RawCull/
└── AnalysisArtifacts/
    └── Similarity/

PerFileAnalysisArtifactStore writes one atomic JSON record per source/backend/pipeline identity. This lets an artifact survive application restart and be reused by both image similarity and semantic search.

The store validates:

• standardized source path, file size, and modification date;
• selected model fingerprint;
• artifact schema and vector representation;
• preprocessing, normalization, and configuration versions;
• RawCull’s 512-pixel input limit;
• RawCull embedding pipeline version 3.

Moving or renaming a source file changes its standardized path and is therefore a cache miss. The default pruning policy removes records unused for 90 days and caps the store at 50,000 entries.

RawCull also saves catalog-wide burst analysis under:

Application Support/RawCull/BurstAnalysis/

The current burst cache schema is 9. Its similarity signature includes grouping configuration, the selected backend descriptor, accepted artifact descriptors, artifact schema, 512-pixel input limit, and embedding pipeline version. It also verifies the catalog, file identities, and a digest of the artifact set.

These identities create independent invalidation paths:

model asset changed       -> model fingerprint mismatch
selected model changed    -> backend descriptor mismatch
preprocessing changed     -> preprocessing or pipeline mismatch
source file changed       -> source fingerprint mismatch
artifact format changed   -> schema mismatch
grouping policy changed   -> burst signature mismatch

17. Switching Between OpenAI And DataComp

When the selected model changes, RawCullAISettingsModel asks the composition root for new similarity and semantic-search services.

RawCullViewModel.setSimilarityService(_:) then:

1. cancels and resets active burst analysis;
2. resets image indexing, distances, grouping, ranking, progress, and diagnostics;
3. installs the new backend;
4. tries to hydrate only artifacts compatible with the new descriptor.

The semantic-search service performs a similar reset and compatible-artifact hydration.

This reset is required even though both models return 512 values. Keeping an OpenAI vector while labeling the active service DataComp would make every subsequent comparison mathematically meaningless.

18. Cancellation And Stale-Result Protection

The code uses two complementary mechanisms:

• Task cancellation asks ongoing decode, inference, persistence, or ranking work to stop.
• Generation counters prevent a result from being published after the user has started a newer operation or switched models.

The decoder checks cancellation before and after image loading. The CLIP recovery path never converts CancellationError into a model failure. Indexing, image ranking, semantic search, semantic artifact hydration, and burst grouping each keep their own generation.

Cancellation answers “should this work continue?” A generation check answers “even if it finished, does it still belong to the current user action?”

19. Troubleshooting

Use this order because it follows the runtime path:

1. Check the selected model. OpenAI and DataComp use different folders.
2. Check the CLIP toggle. It must be on to index missing CLIP artifacts through the normal similarity workflow.
3. Check Settings capability text. A saved preference does not mean that the selected provider exists.
4. Check the displayed folder. Use the exact path shown by the running sandboxed or non-sandboxed build.
5. Check the bundle structure. Confirm metadata.json, tokenizer/tokenizer.json, and the .aimodel or .aimodelc selected by assets.main.
6. Check the fingerprint. A declared checksum must match the selected asset.
7. Check metadata compatibility. Resolution, crop policy, token context, function names, tensor names, shapes, and scalar types must match the Core AI asset.
8. Remove or repair an invalid higher-priority install. Resolution stops at an invalid installed directory instead of silently using a debug-bundled copy.
9. Expect the first inference to be slower. Core AI may specialize the model for the Mac.
10. Inspect similarity diagnostics. A partial CLIP result identifies decode or inference failures by image. It does not mean that the complete catalog switched to Vision.
11. Reindex missing images. Images that still produced invalid output after recovery remain excluded until a later successful indexing pass.
12. Check semantic-search coverage. Search ranks compatible cached CLIP artifacts only; it never decodes and indexes missing images as a side effect of submitting a query.