RawCull

How PhotoAIKit Is Constructed

Thu, 30 Jul 2026 00:00:00 +0000

How PhotoAIKit Is Constructed

PhotoAIKit is a reusable Swift package extracted from application code. Its most important achievement is not merely that CLIP and SAM 3 run. It is that reusable AI behavior has been separated from RawCull’s UI, RAW-file handling, paths, sandbox rules, and culling policy.

This document explains the construction from the bottom up and gives the reason for each boundary.

1. Begin With The Package Boundary

The package owns:

Thumbnails and Scan Pipeline

Wed, 15 Jul 2026 00:00:00 +0000

Thumbnails and Scan Pipeline

RawCull separates catalog scanning, thumbnail preloading, UI-driven thumbnail requests, and full-size preview extraction. They share RawParserKit and disk-cache infrastructure, but they have deliberately different memory-admission rules.

The key current invariant is: catalog preloading warms the 200 px grid cache and disk cache, but only UI demand may admit an image to the preview-size RAM cache. This keeps scan order from displacing the images the user is actively viewing.

How PhotoAnalysisKit Is Constructed

Thu, 30 Jul 2026 00:00:00 +0000

How PhotoAnalysisKit Is Constructed

PhotoAnalysisKit is the reusable measurement layer extracted from RawCull. It turns a decoded CGImage plus neutral capture metadata into sharpness, saliency, focus evidence, and an optional focus-mask image. It can also create opaque Apple Vision feature prints.

The package measures a photo; it does not decide whether to keep it. That distinction prevents image-processing code from becoming coupled to RawCull’s files, settings, views, caches, or culling policy.

How RawCull Loads and Uses CLIP

Wed, 29 Jul 2026 00:00:00 +0000

How RawCull Loads and Uses CLIP

RawCull supports two CLIP models:

OpenAI CLIP ViT-B/32, which processes a 224 × 224 image;
OpenCLIP DataComp ViT-B-32-256, which processes a 256 × 256 image.

The user selects one model in Settings > AI. That one selection defines the vector space used for both image-to-image similarity and text-to-image semantic search. RawCull never mixes vectors from the two models.

This page starts with the basic idea behind CLIP and then follows the current source code from application startup, through model validation and Core AI inference, to search results, burst groups, and persisted artifacts.

Memory Cache

Tue, 19 May 2026 00:00:00 +0000

Cache System

RawCull uses several caches because RAW decoding is expensive and because different UI surfaces need different image sizes. The normal thumbnail path is layered RAM -> disk -> RAW extraction. The zoom path has its own disk cache for larger embedded JPEGs. Burst analysis has a separate JSON cache for derived analysis data.

Source Map

Cache	Main files	Stores
Preview memory cache	`Actors/SharedMemoryCache.swift`, `Model/Cache/CachedThumbnail.swift`	Larger `NSImage` thumbnails for detail/loupe use
Grid memory cache	`Actors/SharedMemoryCache.swift`	Small grid thumbnails, downscaled before insertion
Thumbnail disk cache	`Actors/DiskCacheManager.swift`	JPEG thumbnail files under the app cache directory
Full-size JPEG disk cache	`Actors/FullSizeJPGDiskCache.swift`, `Model/Handlers/ZoomPreviewHandler.swift`	Larger embedded JPEG previews for zoom
Burst analysis cache	`Actors/BurstAnalysisCache.swift`	JSON snapshots of embeddings, scores, groups, rankings, and review state
Diagnostics	`Model/Cache/CacheDelegate.swift`, `Model/Cache/CacheStatistics.swift`, `Views/Diagnostics/MemoryDiagnosticsView.swift`	Hit/miss/eviction/pressure counters

Thumbnail Lookup

flowchart TD
 A["Request thumbnail"] --> B{"Preview memory cache?"}
 B -->|"hit"| C["Return NSImage"]
 B -->|"miss"| D{"Disk thumbnail cache?"}
 D -->|"hit"| E["Decode JPEG and refill memory"]
 D -->|"miss"| F["RawParserKit extractThumbnail"]
 F --> G["Create NSImage"]
 G --> H["Store preview memory"]
 G --> I["Store grid memory copy"]
 G --> J["Save disk JPEG"]

The same lookup idea is used by preload and on-demand requests. ScanAndCreateThumbnails warms the cache after a catalog opens. ThumbnailLoader and RequestThumbnail resolve individual thumbnails while the user scrolls.

AI Model Download Service

Mon, 10 Aug 2026 00:00:00 +0000

AI model download service

RawCull supports exactly three optional model bundles:

Model	PhotoAIKit bundle	Runtime asset	Use
DataComp CLIP	`CLIP-DataComp`	`ViT-B-32-256-datacomp_s34b_b86k_float16_static.aimodel`	Similarity and semantic search
OpenAI CLIP	`CLIP-OpenAI`	`clip-vit-base-patch32_float16_static.aimodel`	Similarity and semantic search
Meta SAM 3	`SAM3`	`sam3_float16.aimodel`	Promptable segmentation

SigLIP2 and EfficientSAM are not part of the RawCull download catalogue or release manifest. Do not include their bundles, notices, archives, or asset-pack IDs in a RawCull model release.

AI Model Licence

Sun, 02 Aug 2026 00:00:00 +0000

AI model licence and provenance clearance procedure

Status: publication hold
Last reviewed: 2026-08-03

Purpose and current decision

This document defines how RawCull clears the DataComp CLIP, OpenAI CLIP, and Meta SAM 3 model packs for public download. It covers technical provenance, licence evidence, upstream contacts, questions to ask, acceptable answers, and the final release gate.

No .aar model archive has been uploaded. Do not create a public GitHub Release, publish a download manifest, or change a production descriptor to ready until every model is either:

How RawCullCore Is Constructed

Thu, 30 Jul 2026 00:00:00 +0000

How RawCullCore Is Constructed

RawCullCore is the small domain layer at the center of RawCull. It does not decode or analyze photos. Instead, it receives file metadata and measurements already produced elsewhere, groups sequential files into bursts, and ranks the candidates using deterministic culling rules.

Its main architectural value is that a recommendation can be tested without opening a RAW file, running Vision, creating a Metal context, loading application settings, or constructing a view model.

Memory Pressure

Mon, 01 Jan 0001 00:00:00 +0000

Memory Pressure

RawCull can hold many images in memory while scanning and culling. The memory-pressure code exists to keep the app responsive when macOS reports pressure and to make cache behavior visible during development.

Source Map

File	Role
`Actors/SharedMemoryCache.swift`	Owns cache limits, memory-pressure dispatch source, pressure counters, and cache clearing
`Model/ViewModels/MemoryViewModel.swift`	Samples total system memory, used system memory, app memory, and pressure threshold for the UI
`Model/Cache/CacheConfig.swift`	Contains `CacheRecommendationPolicy` and cache-limit calculation
`Model/ViewModels/MemoryDiagnosticsViewModel.swift`	Writes diagnostics samples for cache/memory review
`Views/Settings/MemoryTab.swift`	Displays memory stats, cache limits, and pressure status
`Views/RawCullSidebarMainView/MemoryWarningLabelView.swift`	Shows the sidebar warning during pressure

Runtime Flow

flowchart TD
 A["SharedMemoryCache.ensureReady"] --> B["startMemoryPressureMonitoring"]
 B --> C["DispatchSourceMemoryPressure"]
 C --> D{"event"}
 D -->|"normal"| E["refreshConfig from settings/adaptive policy"]
 D -->|"warning"| F["shrink cache limits to 60 percent"]
 D -->|"critical"| G["clear memory caches and set 50 MB floor"]
 F --> H["Notify file handler warning"]
 G --> H
 E --> I["Clear warning"]

SharedMemoryCache starts monitoring lazily during ensureReady(). That is called before thumbnail cache use, so the memory-pressure handler is active before large thumbnail work begins.

Evaluating CLIP Models

Wed, 12 Aug 2026 00:00:00 +0000

Evaluating CLIP Models Against Source-Framework Cosine Values and RawCullFB

This document is the complete repeatable procedure for validating CLIP model bundles used by RawCullFB. It covers both supported candidates:

OpenAI CLIP ViT-B/32 at 224 × 224.
OpenCLIP DataComp ViT-B/32-256 with datacomp_s34b_b86k weights.

SigLIP2 is outside the current evaluation scope.

The procedure has two independent validation layers:

Numerical parity: compare embeddings produced by the Core AI bundle with reference embeddings produced by the original Hugging Face Transformers or OpenCLIP implementation.
Product behavior: use RawCullFB to build a real catalog index, execute the same 77 semantic queries, inspect image-similarity neighborhoods, and measure labeled retrieval quality.

A model must pass both layers. Numerical parity proves that the conversion and integration reproduce the source model; it does not prove that the model retrieves useful photographs. Conversely, plausible search results do not prove that the converted model is correct.

New RawCull AI models

Mon, 10 Aug 2026 00:00:00 +0000

Publishing new RawCull AI models

This runbook publishes exactly three optional models:

Model	Asset-pack ID	Installed destination
DataComp CLIP	`no.blogspot.RawCull.models.clip-datacomp`	`Models/CLIP-DataComp`
OpenAI CLIP	`no.blogspot.RawCull.models.clip-openai`	`Models/CLIP-OpenAI`
Meta SAM 3	`no.blogspot.RawCull.models.sam3`	`Models/SAM3`

SigLIP2 and EfficientSAM are deliberately excluded. Do not add them to the release staging tree, packaging manifests, download manifest, production catalogue, settings UI, notices, or tests.

The examples use the v2 release in RawCull-AI-Models. Tag names and GitHub release URLs are case-sensitive. If another tag is chosen, replace v2 everywhere and keep both RawCull manifest URLs identical.

CLIP Model Evaluation Results

Wed, 12 Aug 2026 00:00:00 +0000

CLIP Model Evaluation Results

Report date: 2026-08-12

This report compares the product-behavior results produced by RawCullFB for:

OpenAI CLIP ViT-B/32 at 224 × 224; and
OpenCLIP DataComp ViT-B/32-256 using the datacomp_s34b_b86k checkpoint.

The runs follow the product-behavior portion of Evaluating CLIP Models. They cover the canonical 77 semantic queries and a complete all-pairs image-similarity pass over 453 indexed images.

The reports show that both integrations completed cleanly, neither model shows semantic collapse, and both produce coherent image-neighbourhood structures. DataComp is modestly faster and more consistent across paraphrases. Those are promising sanity-check results, but they are not labeled accuracy metrics.

Concurrency

Sat, 01 Aug 2026 00:00:00 +0000

Concurrency

This page documents concurrency in the RawCullAIModels branch of RawCull and in the first-party Swift packages checked out beside it. The audit covers production source in the RawCull app, DecodeEncodeGeneric, ParseRsyncOutput, PhotoAIKit, PhotoAnalysisKit, RawCullCore, RawParserKit, RsyncArguments, RsyncProcessStreaming, and RsyncAnalyse.

RawCull uses concurrency to keep SwiftUI responsive while it scans folders, reads RAW metadata, extracts and caches previews, calculates sharpness, creates similarity artifacts, groups bursts, runs AI models, downloads model assets, and streams rsync output.

How RawParserKit Is Constructed

Thu, 30 Jul 2026 00:00:00 +0000

How RawParserKit Is Constructed

RawParserKit is RawCull’s camera-file boundary. It knows how Sony ARW and Nikon NEF files are structured, how to locate their embedded JPEGs and AF metadata, and how to turn those sources into orientation-normalized images and display-ready metadata.

The package stops at decoding. It does not score sharpness, generate embeddings, group bursts, cache application results, or decide which photo should be kept.

1. Begin With The Package Boundary

RawParserKit owns:

Focus Mask and Sharpness

Tue, 19 May 2026 00:00:00 +0000

Focus Mask and Sharpness

RawCull’s focus system does two related jobs:

produce a visible focus mask for image inspection,
compute a scalar sharpness score for sorting, burst ranking, and saved scoring.

Both jobs use the same core engine, but the UI overlay and the numeric score are not identical. The overlay tries to show useful focus evidence. The score tries to rank files fairly across a catalog or burst.

Detailed Sharpness Scoring

Wed, 15 Jul 2026 00:00:00 +0000

Detailed Sharpness Scoring

This page explains, step by step, where RawCull computes sharpness scores, what code is involved, what the score means, and which factors can move the result up or down.

The short version: RawCull does not ask “is the whole image contrasty?”. It asks “how much reliable edge detail exists in the useful part of the image, especially around the subject or the camera AF point?”. The score is then used for sharpness sorting, burst ranking, saved scoring results, and inspection badges.

Detailed Focus Mask Computation

Mon, 29 Jun 2026 00:00:00 +0000

Detailed Focus Mask Computation

This page explains, step by step, how RawCull computes the visible focus mask: where the work starts in the UI, which model and engine methods run, how the bitmap overlay is produced, what the intermediate values mean, and which factors can change the final result.

The short version: the focus mask is not a camera focus-point display. It is a computed overlay built from local edge energy. RawCull first finds useful regions to inspect, then computes a Laplacian edge-energy image, ranks local patches, thresholds the strongest evidence, colorizes it, clips it to the selected patch areas, and finally draws the result over the photo.

Burst Groups

Wed, 15 Jul 2026 00:00:00 +0000

Burst Groups

Burst analysis turns a flat catalog into groups of adjacent, visually similar frames. It ranks each multi-frame group, presents review queues, and supports culling decisions such as keeping the best frame, keeping the top two, deferring a group, or setting a manual pick.

The implementation separates orchestration, Vision feature-print indexing, pure grouping and ranking engines, cache persistence, and the review UI.

Source Map

Area	Main files
Analysis orchestration and user actions	`sourcecode/RawCull/RawCull/Model/ViewModels/RawCullViewModel+BurstGrouping.swift`
Similarity indexing and grouping state	`sourcecode/RawCull/RawCull/Model/ViewModels/SimilarityScoringModel.swift`
Pure grouping and ranking	`sourcecode/RawCullCore/Sources/RawCullCore/BurstGroupingEngine.swift`, `BurstRankingEngine.swift`
Shared models	`sourcecode/RawCullCore/Sources/RawCullCore/BurstAnalysisModels.swift`, app `BurstAnalysisModels.swift`, `BurstReviewQueueModels.swift`
Cache	`sourcecode/RawCull/RawCull/Actors/BurstAnalysisCache.swift`
Ratings and manual overrides	`CullingModel.swift`, `SavedFiles.swift`
Burst home and review list	`BurstGroupsHomeView.swift`, `SimilarityGridSelectionView.swift`, `CullingGridView.swift`
Single-burst workspace and comparison	`BurstCullingWorkspaceView.swift`, `ComparisonGridView.swift`
Tests	`RawCullCore/Tests/RawCullCoreTests/BurstGroupingEngineTests.swift`, `BurstRankingEngineTests.swift`, app burst/culling tests

End-to-End Flow

flowchart TD
 A["Analyze Bursts"] --> B["RawCullViewModel.analyzeBursts"]
 B --> C{"Valid BurstAnalysisCache?"}
 C -->|"yes"| D["Remap cached IDs and apply snapshot"]
 C -->|"no"| E{"Sharpness scores missing?"}
 E -->|"yes"| F["Calibrate and score target files"]
 E -->|"no"| G["Reuse scores"]
 F --> H{"Similarity embeddings missing?"}
 G --> H
 H -->|"yes"| I["Index Vision feature prints"]
 H -->|"no"| J["Reuse embeddings"]
 I --> K["Group adjacent frames"]
 J --> K
 K --> L["Rank multi-frame groups"]
 L --> M["Apply manual winner overrides"]
 M --> N["Save cache and review-state snapshots"]
 N --> O["Show home dashboard and review queues"]

Every awaited phase is protected by the analysis generation and selected catalog. A cancelled or superseded run cannot publish late results into a newer catalog.

File Read and Write Reference

Mon, 01 Jan 0001 00:00:00 +0000

File Read and Write Reference

This page lists the main places RawCull reads and writes files. Use it before changing sandbox access, cache locations, persistence, or export behavior.

File Map

File/folder	Access	Owner
User-selected catalog folder	Read	`RawCullViewModel`, `ScanFiles`, `DiscoverFiles`, parser package
RAW files (`.arw`, `.nef`)	Read	scan, thumbnails, focus parsing, zoom, export, diagnostics
`focuspoints.json` beside catalog	Read optional	`ScanFiles` fallback
App Support `savedfiles.json`	Read/write	`CullingModel`, `ReadSavedFilesJSON`, `WriteSavedFilesJSON`
App Support burst cache	Read/write/delete	`BurstAnalysisCache`
Thumbnail cache directory	Read/write/delete	`DiskCacheManager`
Full-size JPEG preview cache	Read/write/prune	`FullSizeJPGDiskCache`, `ZoomPreviewHandler`
Extracted `.jpg` sidecars	Write	`ExtractAndSaveJPGs`, `SaveJPGImage`
rsync include files / process output	Write/read process stream	`ExecuteCopyFiles`, `ArgumentsSynchronize`, `PrepareOutputFromRsync`
Security-scoped bookmarks	Read/write UserDefaults	`OpencatalogView`, copy workflow
Settings	Read/write UserDefaults	`SettingsViewModel`

Catalog Reads

The active catalog comes from the sidebar folder selection. RawCullViewModel.startCatalogLoad(for:) starts security-scoped access and then runs the scan.

Synchronous Code

Tue, 19 May 2026 00:00:00 +0000

Synchronous Code

Most RawCull code uses async/await, actors, and task groups. Some Apple framework work is still fundamentally synchronous: ImageIO thumbnail decode, CoreImage RAW rendering, JPEG encoding, and direct binary MakerNote parsing. This page explains where that work lives and how the app keeps it from freezing the UI or starving Swift’s cooperative executor.

Source Map

Area	Files
Cancellation-aware blocking bridge	`RawParserKit/Sources/RawParserKit/CancellableImageIOWork.swift`
Thumbnail extraction	`SonyThumbnailExtractor.swift`, `NikonThumbnailExtractor.swift`
Embedded JPEG extraction	`JPGSonyARWExtractor.swift`, `JPGNikonNEFExtractor.swift`, `SonyRAWJPEGCreator.swift`
MakerNote parsing	`SonyMakerNoteParser.swift`, `NikonMakerNoteParser.swift`
App callers	`RequestThumbnail.swift`, `ScanAndCreateThumbnails.swift`, `ExtractAndSaveJPGs.swift`, `ScanAndExtractJPGs.swift`, `ZoomPreviewHandler.swift`
Sharpness scoring	`FocusMaskEngine+Scoring.swift`, `FocusMaskEngine+MaskGeneration.swift`

Why Blocking Work Matters

Swift’s cooperative thread pool expects async tasks to suspend instead of occupying threads for long periods. ImageIO and CoreImage calls often do not suspend; they block until decode/render work is done.

Repository Git Workflow

Sun, 19 Apr 2026 00:00:00 +0000

Repository Git Workflow: Linear History

This page documents the preferred repository workflow for RawCull documentation changes: branch-based development, signed commits when configured, rebasing onto main, and fast-forward integration without merge commits.

Create a Branch

git checkout -b <new-branch>
git push --set-upstream origin <new-branch>

Daily Workflow

You always work on a dedicated branch. Commit often as you make progress.

Stage changes

git add .

Commit

If you use the Claude CLI, it can generate a Conventional Commits message from the staged diff:

Sony/Nikon MakerNote Parser

Wed, 25 Mar 2026 00:00:00 +0000

Sony/Nikon MakerNote Parser

RawCull reads autofocus points and embedded JPEG locations directly from RAW metadata. This is implemented in RawParserKit, while the app consumes the parser through the vendor-neutral RawFormat protocol.

Source Map

Area	Files
Format protocol/registry	`RawParserKit/Sources/RawParserKit/RawFormat.swift`, `RawFormatRegistry.swift`
Sony parser	`SonyMakerNoteParser.swift`, `SonyRawFormat.swift`
Nikon parser	`NikonMakerNoteParser.swift`, `NikonRawFormat.swift`
Thumbnail/JPEG extractors	`SonyThumbnailExtractor.swift`, `NikonThumbnailExtractor.swift`, `JPGSonyARWExtractor.swift`, `JPGNikonNEFExtractor.swift`
Diagnostics	`RawParserDiagnostics.swift`, app `RawFileDiagnostics.swift`
App scan consumer	`Actors/ScanFiles.swift`
Focus-point normalization	`RawCullCore/Sources/RawCullCore/FocusPointParser.swift`
Tests	`RawParserKit/Tests/RawParserKitTests/`, `RawCullCore/Tests/RawCullCoreTests/FocusPointParserTests.swift`

Vendor-Neutral Contract

App code usually does not call SonyMakerNoteParser or NikonMakerNoteParser directly. It resolves a format first:

Security-Scoped URLs

Thu, 05 Feb 2026 00:00:00 +0000

Security-Scoped URLs

RawCull is a sandboxed macOS app. Any access outside the app container must come from user consent, usually a file/folder picker. RawCull uses two security-scope patterns:

active catalog access for browsing/culling,
persistent bookmarks for the rsync copy source and destination.

Source Map

Area	Files
Active catalog scope	`RawCullViewModel.swift`, `RawCullViewModel+Catalog.swift`, `RawCullApp.swift`
Catalog scan scope	`Actors/ScanFiles.swift`
Copy-folder bookmarks	`Views/CopyFiles/OpencatalogView.swift`, `SourceAndDestinationSection.swift`
rsync runtime scope	`Model/ParametersRsync/ExecuteCopyFiles.swift`
File writes needing scope	`ExtractAndSaveJPGs.swift`, `SaveJPGImage.swift`, rsync workflow

API Basics

The core calls are:

About

Mon, 01 Jan 0001 00:00:00 +0000

About RawCull

Hello!

If you have ideas, feedback, or suggestions about RawCull, please reach out to me at thomeven@gmail.com. I’d love to hear from you.

Background

I started RsyncOSX in August 2016 as a way to learn Swift, and after eight years of releases, it was archived in August 2024. RsyncUI, built with SwiftUI, began in late 2020 and continues to be actively developed. RawCull represents my latest venture into building specialized macOS applications with modern Swift technologies.

Search Results

Mon, 01 Jan 0001 00:00:00 +0000

Website information

Mon, 01 Jan 0001 00:00:00 +0000

Site built with Docsy v0.16.0

Build information

Netlify build information:

Attribute	Value
Netlify built	`true`
Date/time¹	2026-08-12 13:22 UTC²
ID	`6a7c73958fba6e000883586a`
Deploy log	6a7c73958fba6e000883586c
Deploy context	production
Commit	@c736018
Branch	`main`

Approximate build timestamp. ↩︎
In your timezone: unknown. ↩︎