RawCull Tech Documentation on RawCull

Thumbnails and Scan Pipeline

Wed, 15 Jul 2026 00:00:00 +0000

Thumbnails and Scan Pipeline

RawCull separates catalog scanning, thumbnail preloading, UI-driven thumbnail requests, and full-size preview extraction. They share RawParserKit and disk-cache infrastructure, but they have deliberately different memory-admission rules.

The key current invariant is: catalog preloading warms the 200 px grid cache and disk cache, but only UI demand may admit an image to the preview-size RAM cache. This keeps scan order from displacing the images the user is actively viewing.

Memory Cache

Tue, 19 May 2026 00:00:00 +0000

Cache System

RawCull uses several caches because RAW decoding is expensive and because different UI surfaces need different image sizes. The normal thumbnail path is layered RAM -> disk -> RAW extraction. The zoom path has its own disk cache for larger embedded JPEGs. Burst analysis has a separate JSON cache for derived analysis data.

Source Map

Cache	Main files	Stores
Preview memory cache	`Actors/SharedMemoryCache.swift`, `Model/Cache/CachedThumbnail.swift`	Larger `NSImage` thumbnails for detail/loupe use
Grid memory cache	`Actors/SharedMemoryCache.swift`	Small grid thumbnails, downscaled before insertion
Thumbnail disk cache	`Actors/DiskCacheManager.swift`	JPEG thumbnail files under the app cache directory
Full-size JPEG disk cache	`Actors/FullSizeJPGDiskCache.swift`, `Model/Handlers/ZoomPreviewHandler.swift`	Larger embedded JPEG previews for zoom
Burst analysis cache	`Actors/BurstAnalysisCache.swift`	JSON snapshots of embeddings, scores, groups, rankings, and review state
Diagnostics	`Model/Cache/CacheDelegate.swift`, `Model/Cache/CacheStatistics.swift`, `Views/Diagnostics/MemoryDiagnosticsView.swift`	Hit/miss/eviction/pressure counters

Thumbnail Lookup

flowchart TD
 A["Request thumbnail"] --> B{"Preview memory cache?"}
 B -->|"hit"| C["Return NSImage"]
 B -->|"miss"| D{"Disk thumbnail cache?"}
 D -->|"hit"| E["Decode JPEG and refill memory"]
 D -->|"miss"| F["RawParserKit extractThumbnail"]
 F --> G["Create NSImage"]
 G --> H["Store preview memory"]
 G --> I["Store grid memory copy"]
 G --> J["Save disk JPEG"]

The same lookup idea is used by preload and on-demand requests. ScanAndCreateThumbnails warms the cache after a catalog opens. ThumbnailLoader and RequestThumbnail resolve individual thumbnails while the user scrolls.

Memory Pressure

Mon, 01 Jan 0001 00:00:00 +0000

Memory Pressure

RawCull can hold many images in memory while scanning and culling. The memory-pressure code exists to keep the app responsive when macOS reports pressure and to make cache behavior visible during development.

Source Map

File	Role
`Actors/SharedMemoryCache.swift`	Owns cache limits, memory-pressure dispatch source, pressure counters, and cache clearing
`Model/ViewModels/MemoryViewModel.swift`	Samples total system memory, used system memory, app memory, and pressure threshold for the UI
`Model/Cache/CacheConfig.swift`	Contains `CacheRecommendationPolicy` and cache-limit calculation
`Model/ViewModels/MemoryDiagnosticsViewModel.swift`	Writes diagnostics samples for cache/memory review
`Views/Settings/MemoryTab.swift`	Displays memory stats, cache limits, and pressure status
`Views/RawCullSidebarMainView/MemoryWarningLabelView.swift`	Shows the sidebar warning during pressure

Runtime Flow

flowchart TD
 A["SharedMemoryCache.ensureReady"] --> B["startMemoryPressureMonitoring"]
 B --> C["DispatchSourceMemoryPressure"]
 C --> D{"event"}
 D -->|"normal"| E["refreshConfig from settings/adaptive policy"]
 D -->|"warning"| F["shrink cache limits to 60 percent"]
 D -->|"critical"| G["clear memory caches and set 50 MB floor"]
 F --> H["Notify file handler warning"]
 G --> H
 E --> I["Clear warning"]

SharedMemoryCache starts monitoring lazily during ensureReady(). That is called before thumbnail cache use, so the memory-pressure handler is active before large thumbnail work begins.

Concurrency

Sat, 01 Aug 2026 00:00:00 +0000

Concurrency

This page documents concurrency in the RawCullAIModels branch of RawCull and in the first-party Swift packages checked out beside it. The audit covers production source in the RawCull app, DecodeEncodeGeneric, ParseRsyncOutput, PhotoAIKit, PhotoAnalysisKit, RawCullCore, RawParserKit, RsyncArguments, RsyncProcessStreaming, and RsyncAnalyse.

RawCull uses concurrency to keep SwiftUI responsive while it scans folders, reads RAW metadata, extracts and caches previews, calculates sharpness, creates similarity artifacts, groups bursts, runs AI models, downloads model assets, and streams rsync output.

Focus Mask and Sharpness

Tue, 19 May 2026 00:00:00 +0000

Focus Mask and Sharpness

RawCull’s focus system does two related jobs:

produce a visible focus mask for image inspection,
compute a scalar sharpness score for sorting, burst ranking, and saved scoring.

Both jobs use the same core engine, but the UI overlay and the numeric score are not identical. The overlay tries to show useful focus evidence. The score tries to rank files fairly across a catalog or burst.

Detailed Sharpness Scoring

Wed, 15 Jul 2026 00:00:00 +0000

Detailed Sharpness Scoring

This page explains, step by step, where RawCull computes sharpness scores, what code is involved, what the score means, and which factors can move the result up or down.

The short version: RawCull does not ask “is the whole image contrasty?”. It asks “how much reliable edge detail exists in the useful part of the image, especially around the subject or the camera AF point?”. The score is then used for sharpness sorting, burst ranking, saved scoring results, and inspection badges.

Detailed Focus Mask Computation

Mon, 29 Jun 2026 00:00:00 +0000

Detailed Focus Mask Computation

This page explains, step by step, how RawCull computes the visible focus mask: where the work starts in the UI, which model and engine methods run, how the bitmap overlay is produced, what the intermediate values mean, and which factors can change the final result.

The short version: the focus mask is not a camera focus-point display. It is a computed overlay built from local edge energy. RawCull first finds useful regions to inspect, then computes a Laplacian edge-energy image, ranks local patches, thresholds the strongest evidence, colorizes it, clips it to the selected patch areas, and finally draws the result over the photo.

Burst Groups

Wed, 15 Jul 2026 00:00:00 +0000

Burst Groups

Burst analysis turns a flat catalog into groups of adjacent, visually similar frames. It ranks each multi-frame group, presents review queues, and supports culling decisions such as keeping the best frame, keeping the top two, deferring a group, or setting a manual pick.

The implementation separates orchestration, Vision feature-print indexing, pure grouping and ranking engines, cache persistence, and the review UI.

Source Map

Area	Main files
Analysis orchestration and user actions	`sourcecode/RawCull/RawCull/Model/ViewModels/RawCullViewModel+BurstGrouping.swift`
Similarity indexing and grouping state	`sourcecode/RawCull/RawCull/Model/ViewModels/SimilarityScoringModel.swift`
Pure grouping and ranking	`sourcecode/RawCullCore/Sources/RawCullCore/BurstGroupingEngine.swift`, `BurstRankingEngine.swift`
Shared models	`sourcecode/RawCullCore/Sources/RawCullCore/BurstAnalysisModels.swift`, app `BurstAnalysisModels.swift`, `BurstReviewQueueModels.swift`
Cache	`sourcecode/RawCull/RawCull/Actors/BurstAnalysisCache.swift`
Ratings and manual overrides	`CullingModel.swift`, `SavedFiles.swift`
Burst home and review list	`BurstGroupsHomeView.swift`, `SimilarityGridSelectionView.swift`, `CullingGridView.swift`
Single-burst workspace and comparison	`BurstCullingWorkspaceView.swift`, `ComparisonGridView.swift`
Tests	`RawCullCore/Tests/RawCullCoreTests/BurstGroupingEngineTests.swift`, `BurstRankingEngineTests.swift`, app burst/culling tests

End-to-End Flow

flowchart TD
 A["Analyze Bursts"] --> B["RawCullViewModel.analyzeBursts"]
 B --> C{"Valid BurstAnalysisCache?"}
 C -->|"yes"| D["Remap cached IDs and apply snapshot"]
 C -->|"no"| E{"Sharpness scores missing?"}
 E -->|"yes"| F["Calibrate and score target files"]
 E -->|"no"| G["Reuse scores"]
 F --> H{"Similarity embeddings missing?"}
 G --> H
 H -->|"yes"| I["Index Vision feature prints"]
 H -->|"no"| J["Reuse embeddings"]
 I --> K["Group adjacent frames"]
 J --> K
 K --> L["Rank multi-frame groups"]
 L --> M["Apply manual winner overrides"]
 M --> N["Save cache and review-state snapshots"]
 N --> O["Show home dashboard and review queues"]

Every awaited phase is protected by the analysis generation and selected catalog. A cancelled or superseded run cannot publish late results into a newer catalog.

File Read and Write Reference

Mon, 01 Jan 0001 00:00:00 +0000

File Read and Write Reference

This page lists the main places RawCull reads and writes files. Use it before changing sandbox access, cache locations, persistence, or export behavior.

File Map

File/folder	Access	Owner
User-selected catalog folder	Read	`RawCullViewModel`, `ScanFiles`, `DiscoverFiles`, parser package
RAW files (`.arw`, `.nef`)	Read	scan, thumbnails, focus parsing, zoom, export, diagnostics
`focuspoints.json` beside catalog	Read optional	`ScanFiles` fallback
App Support `savedfiles.json`	Read/write	`CullingModel`, `ReadSavedFilesJSON`, `WriteSavedFilesJSON`
App Support burst cache	Read/write/delete	`BurstAnalysisCache`
Thumbnail cache directory	Read/write/delete	`DiskCacheManager`
Full-size JPEG preview cache	Read/write/prune	`FullSizeJPGDiskCache`, `ZoomPreviewHandler`
Extracted `.jpg` sidecars	Write	`ExtractAndSaveJPGs`, `SaveJPGImage`
rsync include files / process output	Write/read process stream	`ExecuteCopyFiles`, `ArgumentsSynchronize`, `PrepareOutputFromRsync`
Security-scoped bookmarks	Read/write UserDefaults	`OpencatalogView`, copy workflow
Settings	Read/write UserDefaults	`SettingsViewModel`

Catalog Reads

The active catalog comes from the sidebar folder selection. RawCullViewModel.startCatalogLoad(for:) starts security-scoped access and then runs the scan.

Synchronous Code

Tue, 19 May 2026 00:00:00 +0000

Synchronous Code

Most RawCull code uses async/await, actors, and task groups. Some Apple framework work is still fundamentally synchronous: ImageIO thumbnail decode, CoreImage RAW rendering, JPEG encoding, and direct binary MakerNote parsing. This page explains where that work lives and how the app keeps it from freezing the UI or starving Swift’s cooperative executor.

Source Map

Area	Files
Cancellation-aware blocking bridge	`RawParserKit/Sources/RawParserKit/CancellableImageIOWork.swift`
Thumbnail extraction	`SonyThumbnailExtractor.swift`, `NikonThumbnailExtractor.swift`
Embedded JPEG extraction	`JPGSonyARWExtractor.swift`, `JPGNikonNEFExtractor.swift`, `SonyRAWJPEGCreator.swift`
MakerNote parsing	`SonyMakerNoteParser.swift`, `NikonMakerNoteParser.swift`
App callers	`RequestThumbnail.swift`, `ScanAndCreateThumbnails.swift`, `ExtractAndSaveJPGs.swift`, `ScanAndExtractJPGs.swift`, `ZoomPreviewHandler.swift`
Sharpness scoring	`FocusMaskEngine+Scoring.swift`, `FocusMaskEngine+MaskGeneration.swift`

Why Blocking Work Matters

Swift’s cooperative thread pool expects async tasks to suspend instead of occupying threads for long periods. ImageIO and CoreImage calls often do not suspend; they block until decode/render work is done.

Repository Git Workflow

Sun, 19 Apr 2026 00:00:00 +0000

Repository Git Workflow: Linear History

This page documents the preferred repository workflow for RawCull documentation changes: branch-based development, signed commits when configured, rebasing onto main, and fast-forward integration without merge commits.

Create a Branch

git checkout -b <new-branch>
git push --set-upstream origin <new-branch>

Daily Workflow

You always work on a dedicated branch. Commit often as you make progress.

Stage changes

git add .

Commit

If you use the Claude CLI, it can generate a Conventional Commits message from the staged diff:

Sony/Nikon MakerNote Parser

Wed, 25 Mar 2026 00:00:00 +0000

Sony/Nikon MakerNote Parser

RawCull reads autofocus points and embedded JPEG locations directly from RAW metadata. This is implemented in RawParserKit, while the app consumes the parser through the vendor-neutral RawFormat protocol.

Source Map

Area	Files
Format protocol/registry	`RawParserKit/Sources/RawParserKit/RawFormat.swift`, `RawFormatRegistry.swift`
Sony parser	`SonyMakerNoteParser.swift`, `SonyRawFormat.swift`
Nikon parser	`NikonMakerNoteParser.swift`, `NikonRawFormat.swift`
Thumbnail/JPEG extractors	`SonyThumbnailExtractor.swift`, `NikonThumbnailExtractor.swift`, `JPGSonyARWExtractor.swift`, `JPGNikonNEFExtractor.swift`
Diagnostics	`RawParserDiagnostics.swift`, app `RawFileDiagnostics.swift`
App scan consumer	`Actors/ScanFiles.swift`
Focus-point normalization	`RawCullCore/Sources/RawCullCore/FocusPointParser.swift`
Tests	`RawParserKit/Tests/RawParserKitTests/`, `RawCullCore/Tests/RawCullCoreTests/FocusPointParserTests.swift`

Vendor-Neutral Contract

App code usually does not call SonyMakerNoteParser or NikonMakerNoteParser directly. It resolves a format first:

Security-Scoped URLs

Thu, 05 Feb 2026 00:00:00 +0000

Security-Scoped URLs

RawCull is a sandboxed macOS app. Any access outside the app container must come from user consent, usually a file/folder picker. RawCull uses two security-scope patterns:

active catalog access for browsing/culling,
persistent bookmarks for the rsync copy source and destination.

Source Map

Area	Files
Active catalog scope	`RawCullViewModel.swift`, `RawCullViewModel+Catalog.swift`, `RawCullApp.swift`
Catalog scan scope	`Actors/ScanFiles.swift`
Copy-folder bookmarks	`Views/CopyFiles/OpencatalogView.swift`, `SourceAndDestinationSection.swift`
rsync runtime scope	`Model/ParametersRsync/ExecuteCopyFiles.swift`
File writes needing scope	`ExtractAndSaveJPGs.swift`, `SaveJPGImage.swift`, rsync workflow

API Basics

The core calls are: