RawCull Packages

• 1: How PhotoAIKit Is Constructed
• 2: How PhotoAnalysisKit Is Constructed
• 3: How RawCullCore Is Constructed
• 4: How RawParserKit Is Constructed

flowchart LR
    Files["ARW, NEF, and rendered files"] --> Parser["RawParserKit\ndecode + capture metadata"]
    Parser --> Host["RawCull adapters\nsource and cache policy"]
    Host --> Analysis["PhotoAnalysisKit\nsharpness + focus evidence"]
    Host --> AI["PhotoAIKit\nimage/text embeddings + segmentation"]
    Analysis --> Host
    AI --> Host
    Host --> Core["RawCullCore\nburst grouping + ranking"]
    Core --> App["RawCull UI, persistence, and culling actions"]

1 - How PhotoAIKit Is Constructed

How PhotoAIKit Is Constructed

PhotoAIKit is a reusable Swift package extracted from application code. Its most important achievement is not merely that CLIP and SAM 3 run. It is that reusable AI behavior has been separated from RawCull’s UI, RAW-file handling, paths, sandbox rules, and culling policy.

This document explains the construction from the bottom up and gives the reason for each boundary.

1. Begin With The Package Boundary

The package owns:

• typed, Sendable contracts;
• model-bundle validation and fingerprinted model identity;
• Core AI CLIP image and text inference plus SAM 3 inference;
• validated image/text semantic comparison;
• Apple Vision feature-print generation and comparison;
• bounded similarity indexing and explicit fallback;
• segmentation, mask selection, geometry, and catalog workflows;
• optional artifact codecs and mask stores.

The host application owns:

• model download, installation, candidate URLs, and sandbox bookmarks;
• RAW decoding and application image models;
• SwiftUI, Observation state, and display wording;
• app-specific task staleness such as “latest selection wins”;
• query admission, result ordering, filtering, and semantic-search presentation;
• culling, burst ranking, sharpness, saliency, and rating policy;
• helper-process launch and application restart behavior.

This boundary makes PhotoAIKit reusable. A package that imports FileItem or searches Bundle.main for a RawCull resource might be convenient for one app, but it would silently encode application policy into the AI layer.

2. Read Package.swift As An Architecture Diagram

PhotoAIKit/Package.swift declares Swift tools 6.4, macOS 27, Swift 6 language mode, six library products, and one test target. The only external package dependency is a pinned revision of apple/coreai-models.

flowchart TD
    Contracts["PhotoAIContracts\nvalues + protocols"]
    CLIP["CoreAICLIPBackend"] --> Contracts
    SAM3["CoreAISAM3Backend"] --> Contracts
    Vision["VisionFeaturePrintBackend"] --> Contracts
    Workflows["PhotoAIWorkflows"] --> Contracts
    Storage["PhotoAIStorage"] --> Contracts
    CoreAI["apple/coreai-models\nCoreAISegmentation product"] --> CLIP
    CoreAI --> SAM3
    Tests["PhotoAIKitTests"] --> Contracts
    Tests --> CLIP
    Tests --> SAM3
    Tests --> Vision
    Tests --> Workflows
    Tests --> Storage

There is intentionally no large umbrella target in which every feature can reach every implementation. Each higher-level product depends on PhotoAIContracts, but the backends, workflows, and storage products do not depend on one another.

Why this shape helps:

• A host can import only the products it uses.
• Workflow code is testable with fake providers and decoders.
• Storage does not need to know which inference backend produced a value.
• A backend cannot accidentally reach into RawCull or into another backend.
• Framework-heavy dependencies remain concentrated in concrete backend targets.

3. PhotoAIContracts: The Stable Center

Contracts are the innermost layer. They contain data and protocol definitions, not application decisions.

3.1 Translate Host Photos Into AIImageSource

Sources/PhotoAIContracts/AIImageSource.swift defines the package-owned source value:

public struct AIImageSource: Codable, Hashable, Identifiable, Sendable {
    public let id: UUID
    public let url: URL
    public let displayName: String
}

RawCull maps FileItem into this type at its integration boundary. PhotoAIKit therefore receives only what a reusable indexing or segmentation workflow needs. It never learns ratings, focus data, camera metadata, or view state.

SourceFileIdentity reads size and modification date. SourceFingerprint adds the standardized path. These values let persisted artifacts answer a crucial cache question: “Was this result produced for the current contents of this source file?”

3.2 Invert Image Decoding

PhotoAIKit declares:

public protocol ImageDecoding: Sendable {
    func image(for source: AIImageSource) async throws -> CGImage
}

The package needs a CGImage, but it should not prescribe how one is obtained. A JPEG host can use ImageIO; RawCull can try RawParserKit first; tests can return a generated image. The workflow depends on the capability, not on a camera-format implementation.

This is the dependency-inversion principle in a small, concrete form:

flowchart LR
    Workflow["PhotoAIKit indexer"] --> Protocol["ImageDecoding protocol"]
    Raw["RawCull RAW decoder"] -. conforms .-> Protocol
    Test["Test decoder"] -. conforms .-> Protocol

3.3 Separate Generation From Comparison

Sources/PhotoAIContracts/SimilarityArtifact.swift defines two protocols:

• ImageSimilarityArtifactProviding creates an artifact from a CGImage and source.
• ImageSimilarityArtifactComparing computes the distance between two compatible artifacts.

The combined ImageSimilarityBackend type alias requires both.

The split matters because generation can be actor-isolated and expensive, while comparison may be synchronous and nonisolated. It also keeps distance semantics with the backend that understands its payload. RawCull does not decode a VNFeaturePrintObservation, nor does it implement CLIP cosine distance from unverified arbitrary data.

3.4 Keep Text Queries Query-Scoped

Sources/PhotoAIContracts/TextEmbedding.swift defines a second, deliberately different similarity value. TextEmbedding contains a normalized text vector and a TextEmbeddingDescriptor with the complete backend identity, dimensions, tokenizer version, and schema version.

Text embeddings are not file-backed SimilarityArtifact values. They have no source fingerprint and are intended to live for one query. The split public protocols mirror the image API:

• TextEmbeddingProviding tokenizes and encodes a query.
• ImageTextSimilarityComparing compares a compatible image artifact with the text embedding.
• ImageTextSimilarityBackend combines both capabilities.

The comparison returns cosine similarity in -1...1, where a larger value is a closer semantic match. It is a relative retrieval score, not a confidence or a keep/reject decision.

3.5 Make Persisted Artifacts Self-Describing

A SimilarityArtifact has two fields:

SimilarityArtifact
├── descriptor
│   ├── backend
│   ├── model fingerprint
│   ├── dimensions
│   ├── representation
│   ├── preprocessing version
│   ├── normalization version
│   ├── configuration version
│   ├── source fingerprint
│   └── schema version
└── payload (backend-owned Data)

This looks more elaborate than storing [Float], but it prevents subtle cache bugs. A vector is not reusable merely because its dimension matches. Reuse is valid only if the source, model asset, preprocessing, normalization, configuration, representation, and schema are still the same.

SimilarityArtifactDescriptor.isCompatibleForDistance(with:) intentionally compares every backend/configuration field but excludes the source fingerprint. Two different photos must have different source fingerprints, yet their artifacts can still be compared when they were produced by the same backend definition.

3.6 Treat Model Identity As Data

The model types are spread across:

• ModelIdentity.swift;
• ModelAssetFingerprint.swift;
• ModelBundleResolver.swift;
• ModelResource.swift.

ModelIdentity.cacheIdentifier preserves compatibility with older cache naming. artifactIdentifier adds the selected asset fingerprint for new artifacts. This distinction allows migration without pretending that two different model binaries are the same model.

ModelResourceDescriptor.clip and .sam3 describe package-neutral requirements and version strings. ModelCapabilityStatus reports available, missing, or invalid without user-facing wording. The host translates that status into its own settings UI.

4. Model Bundles Are Supplied, Not Discovered

PhotoAIKit never chooses an application directory. The host supplies one URL or an ordered list of candidates.

A valid bundle has this conceptual layout:

ModelBundle/
├── metadata.json
├── tokenizer/
│   └── tokenizer.json
└── selected-model.aimodel  (or .aimodelc)

metadata.json names the selected model in assets.main. New exports can also provide asset_fingerprints.main.

ModelBundleResolver validates, in order:

1. the URL exists and is a directory;
2. metadata.json decodes;
3. assets.main is present and non-empty;
4. the asset extension is accepted;
5. the selected asset exists;
6. required resources such as tokenizer/tokenizer.json exist;
7. the model asset can be fingerprinted;
8. a manifest checksum, when present, matches the actual asset.

For a file asset, the cryptographic algorithm is SHA-256. For a compiled model directory, PhotoAIKit hashes a stable sorted tree representation. Older manifests without a checksum receive a size/modification-time fallback fingerprint. That fallback can detect replacement in place but is marked as not cryptographically verified.

ModelProviderFactory<Provider> connects validation to a backend constructor. A backend supplies “how to make me from a validated URL”; the host supplies “which URLs should be considered, and in what order.”

Candidate order is policy, not just presentation. ModelResourceResolver skips a candidate that is missing, but returns immediately when it finds an invalid candidate. This prevents a damaged higher-priority installation from being silently hidden by a lower-priority fallback.

CLIP has a second validation layer after generic bundle resolution. CLIPRuntimeConfiguration reads the model contract from metadata: source model and revision, architecture, pretrained checkpoint, embedding dimensions, preprocessing dimensions and normalization, tokenizer context and padding, named image/text functions, normalization version, and configuration version. Current bundles use shortest-side resize, a centered square crop, and bicubic interpolation. Older PhotoAIKit bundles remain compatible through the original 224-pixel stretch and bilinear defaults.

This separation prevents a library update from unexpectedly changing an application’s installation or sandbox policy.

5. Concrete Backend Products

5.1 CoreAICLIPBackend

CoreAICLIPProvider is an actor conforming to image embedding, artifact generation/comparison, text embedding, and image/text comparison protocols.

Its responsibilities are deliberately backend-specific:

• validate the supplied CLIP bundle;
• lazily load and cache the named image and text Core AI functions;
• validate tensor names, shapes, scalar types, dimensions, and metadata;
• create the image, token, and attention-mask NDArrays;
• apply the model-declared resize/crop policy in sRGB and CLIP channel normalization;
• L2-normalize image vectors through ImageEmbedding;
• validate that the exported graph’s text vector is finite, non-empty, correctly shaped, and already L2-normalized;
• encode the vector as an artifact payload;
• compute cosine distance only after descriptor and payload validation.

The actor owns loadedModel, so lazy initialization is isolated from concurrent callers.

For a semantic query, the same provider creates a TextEmbedding and similarity(image:text:) rejects every incompatible model, representation, dimension, preprocessing, normalization, configuration, tokenizer, schema, or payload before computing a dot product. Vision artifacts are rejected because they are not in CLIP’s image/text embedding space.

RawCull’s RawCullCLIPSemanticSearchService keeps the product policy outside the backend: it trims and admits a literal query, selects already-persisted compatible image artifacts, isolates per-file failures, reports progress, and orders equal scores deterministically. It never decodes an image or persists a query embedding.

5.2 CoreAISAM3Backend

CoreAISAM3Provider conforms to SubjectSegmenting. It owns tokenizer setup, lazy model loading, request inference, query selection, confidence conversion, mask decoding, resizing, thresholding, feathering, timing, and diagnostics.

The public contract speaks in SubjectSegmentationRequest and SubjectSegmentationResult, not Core AI tensors. This lets workflows and hosts work at the domain level while the backend handles framework details.

5.3 VisionFeaturePrintBackend

This actor uses VNGenerateImageFeaturePrintRequest. The produced VNFeaturePrintObservation is securely archived into an opaque artifact payload.

Comparison returns to this backend, which unarchives the observations and calls Vision’s native computeDistance. The observation never becomes part of RawCull’s persistence API. This is an example of information hiding: callers can store and route the artifact without learning its private representation.

6. PhotoAIWorkflows: Reusable Orchestration

Backends operate on one image or request. Workflows coordinate many sources and reusable policies.

6.1 Similarity Indexing

EmbeddingIndexer is the vector-specific API. SimilarityArtifactIndexer is the more general API used when the fallback might have an opaque representation such as a Vision feature print.

Both indexers accept:

• package-owned sources;
• an injected decoder;
• a primary provider;
• an optional fallback provider;
• an explicit fallback policy;
• a concurrency limit;
• an async progress callback.

They use a throwing task group but enqueue only concurrencyLimit children. Each time one finishes, one new source is added. This is bounded concurrency: a catalog with 20,000 files does not create 20,000 live tasks and decoded images.

The fallback policies are:

The indexer does not decide which policy is correct for RawCull. The host selects .wholeBatch for CLIP-to-Vision fallback.

6.2 Segmentation Workflows

The SAM 3 side is split into focused units:

The package can decide which mask best satisfies a general selection strategy. RawCull still decides which photo is a culling candidate or winner. Reusable mask quality is not the same concern as product ranking policy.

7. PhotoAIStorage: Optional Persistence Mechanics

Storage depends only on contracts.

For similarity data:

• SimilarityArtifactCodec wraps an artifact in a versioned envelope.
• EmbeddingCodec writes descriptor-complete CLIP artifacts.
• decodeMigrating reads current data and two version-1 formats, but legacy values are accepted only against the real current model identity and are marked for immediate rewrite.
• LegacyCLIPEmbeddingCodec remains a staged migration reader; its identity-incomplete writer is deprecated.

For segmentation:

• SubjectMaskMemoryStore is an injected actor-backed dictionary.
• SubjectMaskDiskStore stores PNG mask data plus JSON metadata, validates the storage key, reports disk usage, supports pruning, and rejects stale or corrupt entries.

The host injects the disk directory. PhotoAIKit owns how the record is encoded and validated, while the application owns where records live and when they are removed.

8. Concurrency And Isolation

PhotoAIKit targets Swift 6 and makes boundary values Sendable.

The main patterns are:

• Actors for mutable runtime state: providers lazily cache loaded models; services and stores protect mutable state.
• Value types for transport: sources, identities, descriptors, progress, failures, and configuration values cross tasks safely.
• Bounded task groups: indexers and prefetch workflows limit simultaneous work.
• Cooperative cancellation: workflow boundaries and expensive stages call Task.checkCancellation().
• Async progress callbacks: the package reports neutral progress values; a host decides how and where to publish UI state.

Actor isolation does not remove the need for a host policy. RawCull still owns generation tokens and catalog checks that prevent an older task from replacing results for a newer selection.

9. Testing The Architecture, Not Only The Math

Most of Tests/PhotoAIKitTests/ exercises the products through public APIs. The CLIP text suite also uses @testable import for deterministic tokenizer, batch, tensor-shape, and preprocessing checks that sit below the provider boundary. Together the tests cover architectural promises such as:

• model bundles are accepted through supplied URLs;
• manifest fingerprints become artifact identity;
• provider factories and resource resolvers share validation;
• CLIP normalization and cosine distance remain stable;
• model-declared CLIP preprocessing and legacy preprocessing remain compatible;
• token batches, attention masks, text-output validation, and image/text compatibility checks reject malformed or mismatched data;
• whole-batch fallback produces a homogeneous result set;
• Vision artifacts stay opaque and use the native metric;
• segmentation caches by package-owned source values;
• disk stores reject stale or corrupt entries;
• legacy data is a rewrite candidate, not silently current data;
• batch transport has an explicit versioned schema;
• cancellation crosses the service boundary.

Fake decoders, providers, and stores make these tests possible. That testability is a direct consequence of putting protocols in the innermost target.

10. Developer Tools And Model Assets

The package includes tools under Tools/ for exporting CLIP and SAM 3 assets, selecting a SAM 3 asset, generating fingerprints, and producing reference CLIP image/text similarities. Output paths are explicit.

export_clip.py supports the existing OpenAI clip-vit-base-patch32 model and OpenCLIP ViT-B-32-256 with datacomp_s34b_b86k weights. It writes the image and text encoders as named functions in one Core AI asset, verifies tokenizer parity for the OpenCLIP export, and records the runtime metadata consumed by CLIPRuntimeConfiguration.

The Swift package itself declares no model resources. It does not embed .aimodel, .aimodelc, tokenizer, or metadata assets. Model binaries are large deployment inputs with their own lifecycle; keeping them out of the library avoids coupling package source, application installation, and model distribution.

11. How To Add Another Similarity Backend

Use the existing layering as a checklist:

1. Add or reuse package-neutral contract values in PhotoAIContracts; do not add host types.
2. Create a separate backend target that depends on contracts.
3. Conform to artifact providing and comparing protocols.
4. Give the backend a complete SimilarityBackendDescriptor.
5. Put framework-specific payload encoding and distance semantics inside that backend.
6. If the backend supports text, add a query-scoped descriptor and keep image/text compatibility checks with that backend.
7. Reuse SimilarityArtifactIndexer and inject the host’s decoder.
8. Decide explicitly whether fallback is none, per-item, or whole-batch.
9. Add public-API tests with fake sources and model-free inputs where possible.
10. Let the host choose model URLs, cache locations, settings, and product policy.

If a proposed type needs SwiftUI, FileItem, a RawCull path, or a burst rating, it probably belongs in RawCull’s adapter layer instead.

Source Map

Next, follow these abstractions into the host application in How RawCull Enables and Uses CLIP.

2 - How PhotoAnalysisKit Is Constructed

How PhotoAnalysisKit Is Constructed

PhotoAnalysisKit is the reusable measurement layer extracted from RawCull. It turns a decoded CGImage plus neutral capture metadata into sharpness, saliency, focus evidence, and an optional focus-mask image. It can also create opaque Apple Vision feature prints.

The package measures a photo; it does not decide whether to keep it. That distinction prevents image-processing code from becoming coupled to RawCull’s files, settings, views, caches, or culling policy.

1. Begin With The Package Boundary

PhotoAnalysisKit owns:

• sRGB normalization for predictable analysis input;
• Core Image and Metal Laplacian processing;
• Vision saliency and optional subject classification;
• scalar sharpness metrics and failure classification;
• AF-point, subject, local-patch, and global focus evidence;
• focus-mask selection and rendering;
• numeric configuration, presets, quality levels, and analysis identity;
• bounded batch analysis and calibration;
• Vision feature-print generation and native comparison.

The host application owns:

• RAW, JPEG, or other source decoding;
• file URLs, security-scoped access, and sandbox bookmarks;
• source identity, cache keys, cache directories, and persistence;
• observable task state, progress presentation, and cancellation policy;
• settings labels and saved-settings migration;
• sorting, ratings, burst decisions, and culling actions.

The input boundary is intentionally small:

flowchart LR
    Source["RAW or rendered source"] --> Decoder["Host decoder"]
    Decoder --> Input["PhotoAnalysisInput\nCGImage + ISO + aperture + AF point"]
    Input --> Analyzer["PhotoAnalyzer"]
    Analyzer --> Result["PhotoAnalysisResult\nsaliency + breakdown + optional mask"]
    Result --> Host["Host cache, UI, and culling policy"]

PhotoAnalysisKit does not import RawParserKit or RawCullCore. RawCull provides the adapters between them.

2. Read Package.swift As The First Design Document

The manifest declares Swift tools 6.2, Swift 6 language mode, macOS 26, one library product, and one test target.

PhotoAnalysisKit package
├── PhotoAnalysisKit library
│   ├── public contracts and facade
│   ├── Core Image, Vision, and Accelerate implementation
│   └── packaged default.metallib
└── PhotoAnalysisKitTests

There are no third-party package dependencies. The implementation uses Apple frameworks including Core Graphics, Core Image, Vision, Accelerate, and Foundation.

The target enables InferIsolatedConformances and NonisolatedNonsendingByDefault. The package is therefore built under the same strict Swift 6 concurrency assumptions as the host without adopting UI isolation.

3. PhotoAnalysisInput Is The Decoding Boundary

Sources/PhotoAnalysisKit/PhotoAnalysisInput.swift defines the package’s source value:

public struct PhotoAnalysisInput: Sendable {
    public let image: CGImage
    public let iso: Int
    public let aperture: Double?
    public let normalizedAFPoint: CGPoint?
}

The AF point uses normalized 0...1 coordinates with the origin at the visual top-left. This matches the camera metadata shape used by RawCull. Vision uses a bottom-left coordinate system, so the package performs the vertical conversion internally.

ISO is clamped to at least 1. Aperture and AF point are optional because not every image or camera provides them. The package can still compute a global result when either is absent.

This value has no URL or file identifier. Two consequences follow:

1. The package cannot reopen a source behind the host’s back.
2. The host must associate returned results with the correct file and invalidate them when that file or decode policy changes.

RawCull makes that ownership concrete in RawCullPhotoAnalysisAdapter. The adapter chooses an embedded-preview decode through RawParserKit or a host-owned RAW demosaic, bounds the requested pixel size, and then constructs PhotoAnalysisInput. It also translates the returned neutral saliency value into RawCullCore’s SaliencyInfo. Neither package needs to import the other to participate in that flow.

PhotoAnalysisResult returns:

An absent breakdown means analysis could not produce a valid result or was cancelled. It is different from a valid score of zero.

4. PhotoAnalyzer Is The Public Facade

Sources/PhotoAnalysisKit/PhotoAnalyzer.swift keeps the public entry points small:

The facade contains an immutable FocusMaskEngine. Both types are @unchecked Sendable because CIContext does not declare sendability even though this package holds no mutable model or UI state and reuses the context concurrently.

For each input, PhotoAnalyzer copies the supplied configuration, replaces its ISO with the input ISO, and derives an aperture hint from the input aperture. Callers can safely reuse one base configuration across many files.

When a host already stores SharpnessBreakdown.focusEvidence, passing it to focusMask avoids repeating the saliency selection step. This makes the measurement result useful as an input to later presentation work.

5. Follow The Scalar Sharpness Pipeline

The core implementation is in FocusMaskEngine+Scoring.swift.

flowchart TD
    Image["CGImage"] --> SRGB["8-bit sRGB normalization"]
    SRGB --> Vision["Vision saliency + optional classification"]
    SRGB --> Preblur["ISO- and aperture-aware Gaussian pre-blur"]
    Preblur --> Laplacian["Metal focusLaplacian kernel"]
    Laplacian --> Samples["Global, saliency, AF, and local-patch samples"]
    Vision --> Samples
    Samples --> Tail["Robust p90-p97 tail scores"]
    Tail --> Blend["Subject/global blend"]
    Blend --> Adjust["Silhouette, subject-size, and blur-gate adjustments"]
    Adjust --> Breakdown["SharpnessBreakdown"]

5.1 Normalize Before Measuring

normalizeToSRGB redraws the input into an 8-bit sRGB RGBA bitmap. The Metal pipeline therefore receives a predictable pixel format regardless of the source image’s bit depth or color space.

5.2 Find Candidate Subject Regions

VNGenerateAttentionBasedSaliencyImageRequest supplies salient-object rectangles. Small weak objects are removed, and candidates are ordered using AF overlap or distance, saliency confidence, interior detail, area, and deterministic coordinates.

When classification is enabled, VNClassifyImageRequest supplies a neutral subject label. The package filters out broad environment descriptions and prefers likely subjects such as animals or people. This label is evidence, not a keep/reject decision.

5.3 Build Edge Energy

The packaged focusLaplacian Metal kernel runs after Gaussian pre-blur. The pre-blur grows with ISO so high-frequency sensor noise is less likely to masquerade as detail. Aperture hints damp or strengthen parts of this behavior for wide, middle, and landscape apertures.

The result is rendered as floating-point RGBA data. The red channel carries edge energy.

5.4 Score Several Regions

The engine samples:

• the full image after excluding a configurable border;
• the selected salient region;
• the camera AF region;
• smaller AF-center and AF-neighborhood regions;
• ranked local patches within subject or AF regions.

robustTailScore sorts the sample values, measures the p90-p97 energy band relative to a p20 noise floor, and penalizes a band that is too sparse. microContrast computes the standard deviation of finite edge samples.

The normal blend combines full-frame and subject evidence. When both AF and saliency scores exist, AF has the larger share of the subject score. A conservative local-detail component can then refine the broad subject measurement.

The final value may be adjusted for:

• a silhouette-dominated subject whose strongest energy is mostly at the outer rim;
• the area of a saliency-only subject;
• an aperture-aware soft blur gate driven by subject micro-contrast.

SharpnessBreakdown preserves the component scores and selected evidence so a host can explain the result instead of presenting a single unexplained number.

5.5 Distinguish Failure Shapes

FocusFailureKind classifies the evidence as:

• .motionBlur when global, subject, and AF detail are all weak and micro-contrast is low;
• .missedFocus when the frame has usable global detail but the subject is much weaker;
• .none when neither pattern is established.

These are algorithmic classifications, not final user-facing copy or culling actions.

6. Focus Evidence And Mask Rendering Are Related But Separate

Scalar scoring answers “how much reliable detail is present?” Mask rendering answers “where should the UI draw visible focused edges?” They share evidence but have different configuration needs.

FocusEvidence records the winning region, AF scores, selected patch rankings, spatial alignment, dominance, silhouette handling, visual threshold, coverage, and confidence diagnostics. FocusPatchRanking exposes the detail, coverage, shape, position, and penalty components behind each candidate patch.

The overlay pipeline:

1. chooses AF-center, AF-neighborhood, AF, saliency, mixed, or global evidence;
2. builds primary and, for AF regions, fine-detail Laplacians;
3. ranks local patches and selects the best evidence patches;
4. chooses an adaptive percentile threshold;
5. optionally relaxes the threshold to guarantee minimum visible coverage;
6. applies erosion and dilation to the binary edge mask;
7. colorizes, clips, feathers, and crops the mask;
8. returns updated evidence and render diagnostics with the image.

FocusMaskRegionSource describes whether saliency, AF, both, or neither provided the overlay region. FocusEvidenceOverlayStyle distinguishes subject edges from global edges. RawCull decides how these neutral values appear in its interface.

7. Configuration, Presets, And Cache Identity

SharpnessConfiguration is a value snapshot containing numeric algorithm settings. It separates host-editable behavior from observable settings models.

Notable groups are:

SharpnessPreset applies high-level subject tuning for automatic, birds and wildlife, portrait, landscape, or general action use. SharpnessQuality selects fast, balanced, or high-precision fine-detail work. The host may persist its own UI enums, but should map them into these package values instead of duplicating constants.

Persisted results need more than a filename. SharpnessAnalysisDescriptor records:

• descriptor schema version;
• scalar algorithm version;
• ISO-scaling policy version;
• aperture-hint policy version;
• every host-configurable value that affects non-mask scoring;
• the stable scoring energy multiplier.

The descriptor deliberately excludes per-image ISO and aperture, mask-only presentation settings, decoded dimensions, source choice, and source-file identity. A host must add those values to its cache identity.

Host sharpness cache identity
├── PhotoAnalysisKit SharpnessAnalysisDescriptor
├── per-image ISO and aperture
├── source file identity
├── selected preview/decode policy
└── decoded pixel dimensions

8. Bounded Batch Analysis

PhotoAnalysisBatchRequest<Identifier> contains an identifier and an async input provider. The provider inversion is important: PhotoAnalysisKit coordinates work, while the host retains file access and decoding policy.

analyzeBatch starts only up to maximumConcurrentTasks child tasks. When one completes, it enqueues one more request. This prevents a large catalog from creating an unbounded number of decoded bitmaps.

The API has three ordering and failure guarantees:

• progress is emitted in completion order;
• the returned results preserve request order;
• a decode failure is represented by a result whose analysis is nil.

If the parent task is cancelled, the method cancels the group and returns nil, telling the host to discard partial results rather than mistake them for a complete batch.

9. Calibration Changes The Overlay, Not The Score

Calibration samples Laplacian energies from host-provided decoded inputs with bounded concurrency. It downsamples the collected sample set, sorts it, and returns p50, p90, p95, and p99 statistics plus a clamped threshold at the requested percentile.

At least minimumSuccessfulImages inputs must succeed. Cancellation or too few samples returns nil.

Calibration changes only the visual edge threshold. Core sharpness scores keep a stable gain and do not depend on the current catalog. This prevents the same photo from receiving a different scalar score merely because unrelated photos were added or removed.

10. Vision Feature Prints Stay Opaque

VisionFeaturePrintBackend is an actor that creates a VNFeaturePrintObservation using Vision revision 2 by default. The observation is securely archived inside VisionFeaturePrint.payload.

The value also stores its Vision revision and representation version. Before comparison, the backend verifies both values, securely unarchives each observation, and calls Vision’s native computeDistance.

The host can persist the opaque payload, but it must associate it with source-file identity. Incompatible prints return nil; corrupt archives or failed Vision operations throw a typed VisionFeaturePrintError.

PhotoAnalysisKit intentionally does not copy PhotoAIKit’s general similarity artifact descriptors, CLIP fallback, or batch indexing. It supplies only the focused Vision measurement primitive.

11. Metal Resources Are Part Of The Algorithm

Sources/PhotoAnalysisKit/Resources/Kernels.ci.metal is the source for the Core Image kernel. SwiftPM copies Metal source resources but does not compile them for command-line builds, so the package also checks in default.metallib.

Tools/build_metallib.sh regenerates the binary. A kernel change is incomplete until the checked-in library has been rebuilt and tests have been run. The binary resource affects algorithm output and must be treated like source, not an optional deployment file.

12. Concurrency And Cancellation

The package uses four complementary techniques:

• Immutable facade and engine: callers pass input and configuration snapshots; no app state is retained.
• Explicit concurrent workers: synchronous Core Image and Vision work runs outside caller UI isolation while retaining task priority and task-local context.
• Bounded task groups: batch analysis and calibration cap simultaneous inputs.
• Cooperative checks: expensive stages test cancellation before Vision work, rendering, large loops, and result publication.

@unchecked Sendable is confined to the immutable Core Image facade and engine. Public transport values are Sendable value types.

13. Testing The Public Contract

Tests/PhotoAnalysisKitTests/ imports the library through import PhotoAnalysisKit, not @testable import. Tests therefore exercise the public surface used by a host.

The suite covers:

• end-to-end sharpness analysis with the packaged Metal kernel;
• focus-mask images, evidence, and diagnostics;
• calibration from decoded images and neutral metadata;
• bounded concurrency, order preservation, progress, decode failure, and cancellation;
• descriptor encoding and changes in identity-affecting settings;
• robust-tail, micro-contrast, ISO scaling, and focus-failure metrics;
• configuration presets and aperture behavior;
• Vision feature-print compatibility, round trips, and malformed payloads.

Synthetic images keep the suite deterministic and independent of RAW files, model downloads, cache directories, and application state.

14. How To Add Another Analysis

Use the existing boundary as a checklist:

1. Accept CGImage and only the neutral metadata the algorithm needs.
2. Keep URLs, camera decoding, settings state, and source selection in the host.
3. Return a Sendable package-owned result with enough evidence to explain it.
4. Put numeric tuning in a value configuration, not UI preferences.
5. Define a versioned descriptor when hosts may persist the output.
6. Make synchronous framework work cancellation-aware and independent of UI isolation.
7. Bound multi-image work instead of spawning one live task per catalog item.
8. Test through the public API with synthetic images and values.

If the proposed API needs FileItem, SwiftUI, a cache directory, or a rating, that concern belongs in RawCull’s adapter or policy layer.

Source Map

Next, see how package-neutral measurements become culling-domain decisions in How RawCullCore Is Constructed.

3 - How RawCullCore Is Constructed

How RawCullCore Is Constructed

RawCullCore is the small domain layer at the center of RawCull. It does not decode or analyze photos. Instead, it receives file metadata and measurements already produced elsewhere, groups sequential files into bursts, and ranks the candidates using deterministic culling rules.

Its main architectural value is that a recommendation can be tested without opening a RAW file, running Vision, creating a Metal context, loading application settings, or constructing a view model.

1. Begin With The Package Boundary

RawCullCore owns:

• package-safe file, EXIF, source-catalog, and saliency values;
• normalized focus-point parsing;
• burst group, boundary-evidence, ranking, confidence, and review-state models;
• pure burst grouping rules;
• pure burst ranking and one-click-safety rules;
• a lightweight 256-bin luminance histogram.

RawCullCore deliberately does not own:

• TIFF, MakerNote, ARW, or NEF parsing;
• thumbnail, preview, or full RAW decoding;
• Vision, Core Image, Metal, CLIP, or SAM 3 analysis;
• caches, settings, security-scoped URLs, and saved-file coordination;
• observable view models, SwiftUI, selection, sorting, and culling actions.

This produces a clear flow:

flowchart LR
    Parser["RawParserKit\nmetadata + AF location"] --> Adapter["RawCull adapter"]
    Analysis["PhotoAnalysisKit\nsharpness + saliency"] --> Adapter
    AI["PhotoAIKit\nvisual distance"] --> Adapter
    Adapter --> Values["RawCullCore values"]
    Values --> Group["BurstGroupingEngine"]
    Group --> Rank["BurstRankingEngine"]
    Rank --> Decision["RawCull presentation and action"]

The arrows describe how RawCull composes runtime values. RawCullCore imports none of the other packages.

2. The Manifest Optimizes For A Small Domain Library

Package.swift declares Swift tools 6.2, Swift 6 language mode, macOS 26, one RawCullCore library target, and one test target. It has no external package dependencies and no bundled resources.

The target uses:

.defaultIsolation(MainActor.self)
.enableUpcomingFeature("InferIsolatedConformances")
.enableUpcomingFeature("NonisolatedNonsendingByDefault")

Default main-actor isolation matches the surrounding application environment, but the package’s public value types and pure engines are explicitly nonisolated. Background grouping and ranking therefore do not acquire unnecessary main-actor hops.

This is a useful distinction: the build default is conservative, while each proven pure API opts out explicitly.

3. Package-Safe Models Replace Application State

The package uses transport values instead of importing RawCull’s FileItem or view models.

3.1 ExifMetadata

ExifMetadata is a Codable, Hashable, and Sendable snapshot containing display strings and numeric values for shutter speed, focal length, aperture, ISO, exposure compensation, camera, lens, RAW type, size class, and pixel dimensions.

It contains both strings and numbers because they serve different purposes:

• strings preserve display-ready metadata such as shutter notation and lens names;
• numeric exposure time, focal length, aperture, ISO, and exposure compensation support deterministic stop-based rules without reparsing display text.

RawCull or an adapter constructs this value from RawParserKit’s RawImageMetadata. RawCullCore does not know where the values came from.

3.2 RawCullFileItem

RawCullFileItem contains an ID, URL, name, byte size, modification date, optional capture date and capture-time-zone offset, optional EXIF snapshot, and optional normalized AF point.

Equality and hashing use only id. Two snapshots with the same ID are the same logical file item even if another stored field changed. This matches the identity behavior expected by RawCull collections.

effectiveCaptureDate prefers the EXIF capture instant and falls back to the file modification date. usesFileModificationDateForCaptureTime exposes which path was used so grouping and confidence can treat the fallback conservatively. formattedSize is a lightweight Foundation convenience. File access and mutable scan state remain outside the value.

3.3 Catalog And Saliency Summaries

RawCullSourceCatalog identifies a named source URL without adding bookmark or access-lifecycle behavior.

SaliencyInfo stores only an optional subject label and confidence. It deliberately does not import Vision or expose a Vision observation. A PhotoAnalysisKit result can be translated into this small culling-domain value at the host boundary.

4. Focus-Point Parsing Connects Vendor Metadata To Analysis

RawParserKit’s vendor parsers return a focus string in this shape:

imageWidth imageHeight focusX focusY

FocusPointParser.normalizedPoint(from:) splits on arbitrary whitespace, requires exactly four numeric values and positive image dimensions, and returns:

x = focusX / imageWidth
y = focusY / imageHeight

The origin remains at the visual top-left. PhotoAnalysisKit accepts that convention and performs its own Vision coordinate conversion.

The parser does not clamp the returned point. Vendor parsers and host validation are expected to supply meaningful sensor coordinates. Malformed input or non-positive dimensions return nil.

5. Burst Models Preserve Evidence, Not Just A Winner

BurstAnalysisModels.swift defines the values exchanged by the two engines.

BurstGroupingOutput
├── groups: [BurstGroup]
└── boundaryEvidence: [BurstBoundaryEvidence]

BurstAnalysisResult
├── ordered candidate scores
├── recommended and second-best IDs
├── confidence and review state
├── one-click safety flag
├── reasons
└── cautions

This design avoids throwing away intermediate decisions. RawCull can show why a boundary was created, why a file won, and why automation is or is not considered safe.

BurstGroupingConfig.algorithmVersion is currently 4 and gives hosts a version marker for persisted grouping output. Its configurable thresholds cover visual distance, EXIF and fallback time gaps, camera equality, focal-length similarity, and maximum shutter, aperture, ISO, and exposure-compensation changes in EV. Custom decoding supplies defaults for fields that are absent from older saved configurations.

BurstReviewState keeps historical states for cache compatibility. Unknown decoded raw values fall back to .none. BurstWinnerOverride also migrates older data by generating a missing ID and defaulting missing member filenames to an empty array.

6. BurstGroupingEngine: Decide Where A Burst Splits

The grouping engine expects files in shot order. It starts with the first file and evaluates each adjacent pair.

flowchart TD
    Pair["Previous + current file"] --> Visual{"Visual distance present\nand below threshold?"}
    Visual -- No --> Split["Start new group"]
    Visual -- Yes --> Time{"Time gap allowed?"}
    Time -- No --> Split
    Time -- Yes --> Camera{"Required camera same?"}
    Camera -- No --> Split
    Camera -- Yes --> Focal{"Required focal delta allowed?"}
    Focal -- No --> Split
    Focal -- Yes --> Exposure{"Exposure stable?"}
    Exposure -- No --> Split
    Exposure -- Yes --> Continue["Append to current group"]

A new group begins when any enabled boundary rule fires:

• similarity evidence is missing;
• visual distance is greater than or equal to visualDistanceThreshold;
• the absolute capture gap exceeds maxTimeGapSeconds, or maxFallbackTimeGapSeconds when either file lacks a parsed EXIF capture instant;
• camera identity changes when requireSameCamera is enabled;
• numeric or parsed focal length changes by more than maxFocalLengthDeltaMM when similarity is required;
• shutter speed, aperture, ISO, or exposure compensation changes by more than its configured EV threshold.

The default EXIF capture gap is 2 seconds; the default file-date fallback gap is 10 seconds. Each boundary records captureTimeUsedFallback, allowing later ranking to distinguish precise camera time from filesystem time.

Exposure comparison works in photographic stops:

shutter delta  = |log2(current seconds / previous seconds)|
aperture delta = 2 × |log2(current f-number / previous f-number)|
ISO delta      = |log2(current ISO / previous ISO)|
compensation   = |current EV - previous EV|

The defaults are 0.5 EV for shutter, aperture, and ISO and 0.34 EV for exposure compensation. The engine prefers numeric values. When a numeric shutter, aperture, or ISO comparison is unavailable but both normalized display strings are present and differ, it still treats the exposure as changed. The largest available adjustment is retained as exposureAdjustmentEV.

Focal length similarly prefers focalLengthMM and falls back to the first number in the display string. If either side has no usable value, that rule has no delta to evaluate.

Lens changes are recorded in BurstBoundaryEvidence, but do not by themselves split a group. They later make the group’s ranking metadata unstable. Keeping evidence separate from the grouping decision makes this behavior visible rather than implicit.

Missing similarity is conservative: it creates a boundary instead of assuming two files belong together. BurstPairKey.cacheKey standardizes the ordered adjacent-pair key used to supply those distances.

Each boundary record stores the measured values, boolean changes, final decision, and human-readable reasons. The output assigns stable sequential group IDs beginning at zero for that run.

7. BurstRankingEngine: Turn Evidence Into A Recommendation

Ranking consumes groups, files keyed by ID, sharpness scores, a score normalization maximum, saliency summaries, boundary evidence, and optional review states.

7.1 Determine Group Conditions

For each group, the engine derives:

• metadata stability: none of its internal boundaries report exposure, camera, or lens changes;
• capture-time reliability: every member has a parsed EXIF capture date rather than the modification-date fallback;
• tight similarity: every internal boundary has a visual distance below 0.22;
• dominant subject: the most frequent non-nil saliency label;
• burst-relative sharpness: a within-group 0...1 normalization when at least two valid scores exist and their normalized spread is at least 0.03.

The configured grouping threshold decides membership; the fixed tighter 0.22 check contributes to ranking confidence. They answer different questions.

7.2 Score Each Candidate

If burst-relative sharpness is available, the ranking sharpness component is:

ranking sharpness = 0.65 × catalog-normalized sharpness
                  + 0.35 × burst-relative sharpness

The final candidate score is:

overall = 0.62 × ranking sharpness
        + 0.12 × focus-point evidence
        + 0.10 × saliency consistency
        + 0.16 × metadata evidence

The supporting components are deliberately simple and inspectable:

Every component is clamped or normalized into a predictable range before use. Candidates are sorted by descending overall score; equal scores preserve the group’s original shot order.

Motion risk uses exposureTimeSeconds with focalLengthMM when both are available. A shutter at least twice as fast as the reciprocal focal-length rule gets the positive adjustment; a shutter slower than the reciprocal rule gets a stop-based penalty. Without focal length, 1/500 second or faster is treated as lower risk and 1/60 second or slower as elevated risk.

Each candidate also carries reasons and cautions such as measured sharpness, available AF evidence, classified subject, missing sharpness, changed metadata, fast or slow shutter behavior, and high ISO.

7.3 Assign Confidence Separately

Winning a group does not automatically mean a decision is safe to automate.

High confidence requires:

• at least one sharpness score;
• at least three files in the group;
• a best-versus-second score gap of at least 0.12;
• best absolute sharpness of at least 0.65 after normalization;
• stable metadata;
• tight visual similarity;
• parsed capture times for every member.

Medium confidence requires a gap of at least 0.05 and stable metadata. Other results are low confidence.

isSafeForOneClickCulling is true only for high confidence. canApplyOneClickCulling(hasSharpnessScores:) adds an explicit host-supplied confirmation that sharpness data is available before an action is enabled. RawCull still owns the action itself.

Reasons and cautions on BurstAnalysisResult are capped to three each so the result remains concise enough for inspection UI and persistence.

A modification-date fallback therefore does not prevent grouping, but it does prevent high-confidence automation and adds a capture-time caution.

8. Histogram Calculation Is Intentionally Lightweight

HistogramCalculator.normalizedLuminanceHistogram(from:) directly reads an 8-bit RGB or RGBA-style CGImage buffer. Each pixel is placed in one of 256 bins using Rec. 601 luminance:

Y = 0.299R + 0.587G + 0.114B

The bins are normalized by the largest bin count, so the peak has value 1. This is shape normalization, not probability normalization; the bins do not necessarily sum to 1.

The implementation requires positive dimensions, provider data, 8 bits per component, and at least three bytes per pixel. Unsupported input returns a zero-filled 256-bin array, preserving a stable return shape for views and callers.

The helper is appropriate for display and lightweight comparisons. Color conversion, RAW development, high-bit-depth analysis, and channel-layout generalization are outside this package.

9. Concurrency Is Achieved Through Purity

RawCullCore contains no actor because it owns no shared mutable runtime state. Models are Sendable values and engines are namespaces of nonisolated static functions.

The package performs no asynchronous I/O. Callers do expensive decoding, Vision work, embeddings, and sharpness analysis on the appropriate tasks, then pass snapshots into RawCullCore.

This design has several practical benefits:

• grouping and ranking are deterministic for the same inputs;
• no application singleton can change a result midway through a call;
• tests do not require async setup or framework assets;
• background work does not cross the main actor merely because the host uses main-actor default isolation.

10. Testing Rules And Migrations

Tests/RawCullCoreTests/ uses Swift Testing and synthetic values. Coverage includes:

• model coding, hashing, identity, and source-catalog values;
• valid, decimal, whitespace-varied, and malformed focus strings;
• empty and stable groups;
• boundaries caused by missing distance, time, camera, focal length, and exposure;
• EXIF capture ordering, time-zone offsets, modification-date fallback, and the different fallback gap;
• numeric and display-string exposure comparisons in photographic stops;
• boundary evidence content;
• absolute and burst-relative sharpness ranking, motion risk, and progressive ISO penalties;
• missing scores, stable tie order, confidence, review-state propagation, and one-click eligibility;
• RGB and RGBA histogram bins, peak normalization, and unsupported input.

No test needs an actual catalog, RAW file, Vision model, Metal device, cache directory, or settings store. That is the strongest evidence that the package boundary is doing useful work.

11. How To Extend Culling Logic

Use these constraints when adding a rule:

1. Pass the required fact into a Sendable package value; do not reach into a view model.
2. Preserve raw evidence separately from the final boolean or winner.
3. Keep scoring weights and confidence thresholds deterministic and testable.
4. Decide whether a rule affects group membership, candidate ranking, confidence, or only a caution.
5. Version persisted behavior when a changed rule can invalidate cached results.
6. Maintain decoding fallbacks for existing review and override data.
7. Leave framework observations and heavy image processing in the producing package.
8. Leave user actions and presentation state in RawCull.

Source Map

Return to the package overview in RawCull Packages, or continue with the file-decoding layer in How RawParserKit Is Constructed.

4 - How RawParserKit Is Constructed

How RawParserKit Is Constructed

RawParserKit is RawCull’s camera-file boundary. It knows how Sony ARW and Nikon NEF files are structured, how to locate their embedded JPEGs and AF metadata, and how to turn those sources into orientation-normalized images and display-ready metadata.

The package stops at decoding. It does not score sharpness, generate embeddings, group bursts, cache application results, or decide which photo should be kept.

1. Begin With The Package Boundary

RawParserKit owns:

• vendor-neutral RAW format dispatch;
• Sony ARW and Nikon NEF format knowledge;
• TIFF IFD and vendor MakerNote traversal;
• focus-location and embedded-JPEG offset parsing;
• RAW and rendered-image thumbnails and previews;
• source-orientation normalization;
• display-ready and numeric EXIF/RAW metadata extraction, including a structured capture instant and time-zone offset;
• Sony full sensor development to JPEG;
• decode task deduplication, concurrency limiting, and cooperative cancellation;
• diagnostics and staged compatibility APIs.

The host application owns:

• security-scoped URL access and sandbox bookmarks;
• directory discovery, catalogs, and source selection;
• memory and disk cache locations and eviction policy;
• image-analysis inputs and result identity;
• observable loading state, placeholders, retries, and presentation;
• ratings, burst grouping, ranking, and saved-file behavior.

RawParserKit supplies inputs to higher layers without importing them:

flowchart LR
    File["ARW, NEF, JPEG, PNG, or TIFF"] --> Parser["RawParserKit"]
    Parser --> Image["CGImage or NSImage"]
    Parser --> Metadata["RawImageMetadata + RawFocusPoint"]
    Image --> Host["RawCull adapters"]
    Metadata --> Host
    Host --> Analysis["PhotoAnalysisKit / PhotoAIKit"]
    Host --> Core["RawCullCore"]

2. Package Shape And Framework Boundary

Package.swift declares Swift tools 6.2, Swift 6 language mode, macOS 26, one library product, and one test target. There are no third-party package dependencies.

The implementation uses Apple frameworks appropriate to file decoding: Foundation, AppKit, Core Graphics, ImageIO, Core Image, OSLog, and synchronization primitives from os.

Like RawCullCore, the target enables main-actor default isolation plus InferIsolatedConformances and NonisolatedNonsendingByDefault. Pure parsing and static format APIs opt out with nonisolated; the stateful loader and limiter use actors.

The source is arranged in layers:

RawImageLoader                         high-level deduplicated facade
├── RawFormatRegistry + RawFormat      vendor-neutral dispatch
│   ├── SonyRawFormat
│   └── NikonRawFormat
├── thumbnail and preview extractors   ImageIO + binary fallback
├── MakerNote parsers                  TIFF byte traversal
├── OrientationNormalizedImageLoader   rendered/embedded image helpers
└── cancellation + decode limiter      concurrency control

Callers can use the facade for normal browser behavior or a lower layer when they need explicit control.

3. RawFormat Makes Vendor Dispatch Explicit

Sources/RawParserKit/RawFormat.swift describes the static capabilities every camera format supplies:

• supported filename extensions and a display name;
• thumbnail extraction;
• embedded-preview extraction;
• AF focus-location parsing;
• a readable label for compression codes;
• camera-specific megapixel thresholds for S, M, and L size classes.

Conformers are stateless enums. RawFormatRegistry.all currently registers:

format(for:) lowercases a URL’s extension and returns a format metatype. Callers invoke static protocol requirements on that value without switching on brands.

The default rawSizeClass implementation converts dimensions to megapixels, obtains body-specific L and M thresholds, and returns L, M, or S. Unknown bodies use a generic fallback supplied by the conformer.

This is a simple plugin architecture inside the package. Adding a camera brand means adding a conformer and registering it, not adding vendor switches throughout the loader.

4. RawImageLoader Is The High-Level Facade

RawImageLoader.shared is an actor. It offers four current operations:

4.1 Deduplicate Equivalent Work

The actor stores in-flight tasks:

• thumbnails keyed by URL and requested pixel size;
• previews keyed by URL;
• metadata keyed by URL.

If another caller requests the same work while it is running, both await the existing task. The entry is removed after completion. This prevents rapid view updates from decoding the same large image repeatedly.

4.2 Bound Expensive Decodes

The facade uses two DecodeConcurrencyLimiter actors:

• up to six concurrent thumbnail decodes;
• up to two concurrent full-size preview decodes.

These limits control memory as much as CPU. A few full-resolution bitmaps can consume substantially more memory than the compressed RAW files that produced them.

4.3 Follow The Thumbnail Strategy

For a rendered JPEG, PNG, or TIFF, the loader asks ImageIO for an orientation-aware thumbnail. For RAW input, it tries an embedded ImageIO thumbnail first and then dispatches to the registered vendor extractor. The vendor result is orientation-normalized before it becomes an NSImage.

4.4 Follow The Preview Strategy

The preview path checks for a same-basename .jpg sidecar first. If absent, it tries an orientation-aware embedded preview and then the registered format’s embedded-preview extractor. Vendor output is normalized using the source orientation.

The sidecar-first decision is host-facing convenience, not RAW parsing. A caller that requires only bytes physically embedded in the RAW can call the format or extractor API directly.

5. Metadata Is Normalized Into A Neutral Snapshot

RawImageMetadata contains optional display and numeric values for camera, lens, shutter speed, exposure time in seconds, aperture, focal length in millimeters, ISO, exposure compensation in EV, capture time, dimensions, focus point, RAW compression label, size class, and pixel dimensions.

RawImageLoader.metadata(for:) combines several sources:

1. ImageIO properties from the source or a sidecar fallback;
2. TIFF make, model, compression, and display-date fields;
3. EXIF exposure time, aperture, focal length, ISO, exposure bias, lens, and dimensions;
4. EXIF DateTimeOriginal, SubsecTimeOriginal, and OffsetTimeOriginal;
5. the registered vendor MakerNote parser for an AF point;
6. EXIF subject-area coordinates when vendor focus data is unavailable;
7. vendor-specific compression labels and size-class thresholds.

captureDate parses the original capture timestamp as a real Date, preserving variable-length fractional seconds. OffsetTimeOriginal accepts Z, +HH:MM, -HH:MM, and compact +HHMM/-HHMM forms; the parsed offset is also retained as captureTimeZoneOffsetSeconds. When the EXIF offset is missing, parsing uses the current time zone. A missing or invalid DateTimeOriginal leaves captureDate nil rather than substituting the TIFF display date.

capturedAt remains a display string and can fall back to TIFF DateTime. Keeping it separate from captureDate prevents formatted UI text from becoming burst-ordering evidence.

rows returns non-empty display label/value pairs, while isEmpty lets the facade omit an empty metadata object. Numeric exposure time, aperture, focal length, ISO, exposure compensation, width, and height remain available so hosts do not have to parse formatted strings for analysis or domain rules.

RawFocusPoint stores normalized X and Y coordinates. Its failable focus-string initializer accepts the common "width height x y" shape, checks positive dimensions, and rejects coordinates outside 0...1.

RawCull’s RawParserKitImageLoader maps this parser-owned snapshot into its own ExifMetadata and RawCullFileItem values. ScanFiles carries the parsed capture instant and offset into the catalog; RawCullCore can then prefer camera time and explicitly detect a modification-date fallback. The similar types exist on purpose: each package owns the vocabulary at its boundary and neither must depend on the other.

6. Thumbnail Extraction Prefers Embedded Work

Thumbnails should not require full RAW development when a camera already stored a usable JPEG.

6.1 Sony

SonyThumbnailExtractor first asks SonyMakerNoteParser for embedded JPEG locations and decodes a selected JPEG directly. This bypasses macOS RAW decoder failures seen with newer ARW layouts. If the binary path cannot locate a JPEG, it falls back to ImageIO’s embedded-thumbnail behavior.

6.2 Nikon

NikonThumbnailExtractor asks ImageIO for a transformed embedded thumbnail. Both vendor extractors then redraw into an 8-bit premultiplied sRGB bitmap using interpolation quality derived from qualityCost.

Both APIs run their synchronous ImageIO work through CancellableImageIOWork and throw ThumbnailError for an invalid source, failed generation, or failed bitmap context.

ThumbnailSharpener is an optional lower-level Core Image helper for producing a sharpened preview at a requested maximum dimension. The high-level boundary does not force sharpening on every caller.

7. Embedded Preview Extraction Has Two Paths

The two vendors use the same building blocks in a different order. SonyEmbeddedJPEGExtractor prefers the binary TIFF locator, avoiding RAW decoder initialization on affected ARW files, then falls back to ImageIO. NikonEmbeddedJPEGExtractor inspects ImageIO sub-images first and uses the binary locator when the preview is not exposed there.

flowchart TD
    Source["RAW URL"] --> ImageIO["Inspect ImageIO image indexes"]
    Source --> Parser["Vendor TIFF parser"]
    ImageIO -->|usable JPEG| Decode["Decode or downsample"]
    Parser --> Offset["Absolute JPEG offset + length"]
    Offset --> Bytes["Read JPEG bytes"]
    Bytes --> Decode
    Decode --> Result["CGImage"]

fullSize: true permits a longest edge up to 8640 pixels. The normal preview path limits large images to 4320 pixels.

The Nikon fallback prefers the full-resolution SubIFD preview for full-size requests and IFD1 for smaller requests. Sony chooses the largest available JPEG first, with preview and thumbnail fallbacks.

Extractor-level limiters default to two concurrent operations, and a caller can inject a shared limiter. This allows a host facade to enforce one budget across several decode paths instead of accidentally stacking independent limits.

8. Sony Full-Size JPEG Creation Is Different From Extraction

SonyRawFormat.createFullSizeJPEG(from:quality:) does not return a camera-embedded preview. SonyRAWJPEGCreator develops the ARW sensor data through macOS CIRAWFilter, renders it into sRGB, and encodes JPEG data.

Quality must be in 0...1. The operation can fail with:

• invalidQuality;
• unsupportedOrInvalidRAW when the installed macOS RAW decoder cannot develop that file;
• encodingFailed.

This distinction matters in UI and caching. An embedded preview and a developed sensor image have different cost, pixels, appearance, and invalidation semantics even if both are JPEG-encoded at the end.

9. Sony MakerNote And TIFF Parsing

Sony ARW is TIFF-based. Focus parsing follows this structure:

TIFF IFD0
└── ExifIFD tag 0x8769
    └── MakerNote tag 0x927C
        └── Sony MakerNote IFD
            └── FocusLocation tag 0x2027
                (fallback: 0x204A)

The focus tag contains four unsigned 16-bit values: image width, image height, X, and Y. Sony MakerNote IFD offsets are interpreted as absolute file offsets.

The production parser uses a fast path and a fallback:

• focus location reads the first 4 MB, then retries with the full file when necessary;
• embedded-JPEG discovery reads the first 512 KB, then retries the full file when no locations were found.

The embedded locator walks TIFF IFDs and returns optional absolute locations for a small thumbnail, preview, and full JPEG. readEmbeddedJPEGData seeks directly to a validated location and reads its byte range.

The fast path keeps normal scans inexpensive; the full-file fallback supports bodies that place relevant TIFF structures near the end of the file.

10. Nikon MakerNote And TIFF Parsing

Nikon NEF is also TIFF-based, but modern Nikon Type-3 MakerNotes contain their own TIFF header:

TIFF IFD0
└── ExifIFD tag 0x8769
    └── MakerNote tag 0x927C
        ├── "Nikon\0" signature + version
        └── inner TIFF header
            └── Nikon IFD
                └── AFInfo2 tag 0x00B7

Offsets in the inner TIFF are relative to the MakerNote TIFF-header base, not the start of the NEF file. Keeping this offset rule inside the Nikon parser prevents a generic loader from acquiring vendor-specific exceptions.

For supported modern AFInfo2 layouts, the parser reads AF image dimensions, area position, and area size, and returns the same "width height x y" shape as Sony. The public shape lets all downstream code use one focus-point adapter.

Nikon embedded preview discovery examines Compression=6 SubIFDs referenced from IFD0 and the IFD1 JPEG interchange fields. It returns optional locations for the largest preview and IFD1 JPEG.

As on Sony, focus parsing starts with 4 MB and falls back to the full file. Embedded-location parsing uses a 1 MB fast path followed by a full-file retry when required.

11. Diagnostics Report The Failed Stage

The ordinary parser APIs return optionals because missing or unsupported MakerNote data is expected during normal browsing.

For troubleshooting, each vendor also exposes diagnostic forms for focus and embedded-JPEG lookup. RawParserDiagnostics<Value> contains:

• the optional parsed value;
• an ordered trace of stages and offsets checked;
• an optional final failure explanation.

This keeps logging policy outside the binary parser while allowing RawCull’s diagnostics UI to show whether failure occurred at file access, TIFF validation, IFD lookup, MakerNote traversal, tag interpretation, or fallback.

12. Orientation Helpers Normalize Visual Coordinates

OrientationNormalizedImageLoader provides lower-level operations for rendered URLs, encoded JPEG data, source thumbnails, embedded thumbnails, and embedded previews.

ImageIO’s transform option is used when available. The loader also implements the eight EXIF orientation transforms, including mirrored and transposed cases, and can read orientation from the RAW source when decoding separately extracted JPEG data.

SupportedFileType enumerates .arw, .nef, .jpeg, .jpg, .png, .tif, and .tiff. Its rendered-image set distinguishes sources that can be loaded directly from formats that need RAW dispatch.

Orientation is part of the parser boundary because AF coordinates and subject analysis must refer to the same visual image the user sees.

13. Cancellation And Decode Limiting Solve Different Problems

CancellableImageIOWork bridges a synchronous ImageIO closure to async code on a global dispatch queue. It creates an ImageIOCancellationToken and uses a locked state machine to ensure the checked continuation is resumed exactly once, even when cancellation races completion.

Cancellation is cooperative. The token is checked before and after synchronous framework calls and between multi-stage loops. A framework function already executing may not stop internally, but its result is discarded when cancellation is observed.

DecodeConcurrencyLimiter is an actor that solves admission control. It:

1. grants work immediately while slots are available;
2. queues additional continuations;
3. removes and resumes a cancelled waiter;
4. transfers a released slot directly to the next waiter;
5. releases the slot with defer when work completes.

Cancellation prevents obsolete work; limiting prevents too much valid work from running simultaneously. The loader needs both.

14. Compatibility APIs Support Staged Migration

The package retains deprecated names such as:

• BrowserExifInfo and BrowserFocusPoint;
• thumbnail200px, extractembeddedJPG, and exifInfo;
• JPGSonyARWExtractor and JPGNikonNEFExtractor;
• extractFullJPEG on RawFormat.

Each shim forwards to a current neutral name. This lets RawCull migrate call sites without forcing an all-at-once source break, while deprecation warnings make the intended direction visible.

New code should use RawImageMetadata, RawFocusPoint, thumbnail, previewImage, metadata, the embedded-JPEG extractors, and extractEmbeddedPreview.

15. Test Binary Rules Without Shipping Camera Files

Tests/RawParserKitTests/ uses Swift Testing and mostly synthetic TIFF-like byte buffers. The suite covers:

• registry extension matching and dispatch;
• Sony focus tags, offset rules, invalid byte-order markers, fallback tags, embedded JPEG locations, and diagnostics;
• Nikon Type-3 MakerNote and AFInfo2 layouts, SubIFDs, IFD1 JPEGs, offset rules, and diagnostics;
• direct reading of embedded JPEG bytes;
• cancellation before decode and cancellation behavior in vendor extractors;
• decode-limiter capacity;
• Sony full-size JPEG quality validation and generated JPEG properties;
• numeric exposure-time, focal-length, aperture, ISO, and exposure-compensation metadata;
• capture-date offsets, subsecond precision, invalid-date behavior, and offset parsing;
• current public naming and format-helper behavior.

Synthetic binary fixtures make edge cases reproducible and avoid committing large proprietary ARW and NEF samples. Framework integration is tested with small generated images where needed.

16. How To Add Another Camera Vendor

Use the existing extension points:

1. Add a stateless RawFormat conformer with its extension and display name.
2. Implement vendor-specific thumbnail, preview, focus, compression, and size-class behavior.
3. Put TIFF or MakerNote byte rules in a dedicated parser, including explicit offset bases and bounds checks.
4. Return the common normalized focus-location string at the public boundary.
5. Add ImageIO behavior first and a binary embedded-JPEG fallback when the framework does not expose the preview reliably.
6. Make blocking decode stages cooperative with CancellableImageIOWork.
7. Register the conformer in RawFormatRegistry.all.
8. Add synthetic binary tests for endian, offsets, missing tags, corrupt lengths, and diagnostics.
9. Leave analysis, cache placement, and UI policy in their owning layers.

Source Map

Continue from decoded images into How PhotoAnalysisKit Is Constructed, or return to the package overview.