license: public domain CC0
Medium‑format style pipeline for Canon Dual Pixel RAW
Spec & design document (Python implementation)
1. Goals and scope
Primary goal:
Given a Canon Dual Pixel RAW capture, automatically produce a medium‑format‑style SDR image that:
- Preserves subject detail and avoids “AI mush”
- Uses real Dual Pixel depth (not guessed) to drive optical‑like transforms
- Emulates key MF traits: smoother DOF, micro‑contrast pop, gentle highlight rolloff, subtle subject/background separation
Non‑goals (for v1):
- Perfect physical simulation of any specific MF body
- Exact replication of Canon DPP behavior
- Multi‑view consistency (single‑image only)
2. Inputs, outputs, and assumptions
2.1 Inputs
- Dual Pixel RAW file (e.g.,
.CR3 from EOS R5 with DPR enabled) - Optional: AF metadata (focus point) if available.
2.2 Intermediate representations
Base image
- Linear or gamma‑encoded RGB, shape
H x W x 3, float32 in [0,1].
Depth/disparity map
- From Dual Pixel data, shape
H x W, float32. - Relative depth is sufficient; absolute units not required.
2.3 Outputs
- Medium‑format‑style SDR image
H x W x 3, uint8 or uint16, in JPEG/PNG/TIFF.
3. High‑level pipeline
DPR extraction layer
- Extract base RGB image and depth/disparity map from Dual Pixel RAW.
Depth processing layer
- Normalize depth
- Smooth depth
- Determine focus plane
Optical‑style transform layer (MF physics‑inspired)
- Depth‑dependent blur (DOF geometry)
- Depth‑aware micro‑contrast (focus pop)
- Highlight rolloff (soft knee)
- Depth‑aware color separation (subject vs background)
Output & export layer
- Gamma/tone check
- Color space tagging (e.g., sRGB)
- File export
4. Module design
4.1 DPR extraction module
Responsibility:
Convert Canon Dual Pixel RAW into:
img: np.ndarray[H, W, 3], float32, [0,1]depth: np.ndarray[H, W], float32
Implementation notes:
- Use Canon SDK or third‑party DPR tools (outside this spec) to:
- Decode RAW
- Extract left/right sub‑images
- Compute disparity map (e.g., block matching or provided by SDK)
- Save depth as
.npy for reuse.
Interface:
def extract_dpr(image_path: str) -> tuple[np.ndarray, np.ndarray]:
"""
Returns:
img : H x W x 3 float32, [0,1]
depth : H x W float32, arbitrary scale
"""
4.2 Depth processing module
Responsibility:
Turn raw disparity into a stable, normalized depth field and choose a focus plane.
4.2.1 Depth normalization
- Robustly map depth to
[0,1] using percentiles to ignore outliers.
def normalize_depth(depth: np.ndarray) -> np.ndarray:
d_min, d_max = np.percentile(depth, [1, 99])
depth_norm = np.clip((depth - d_min) / (d_max - d_min + 1e-6), 0.0, 1.0)
return depth_norm
4.2.2 Depth smoothing
- Apply edge‑preserving smoothing (bilateral or guided filter) to reduce noise while preserving edges.
def smooth_depth(depth_norm: np.ndarray) -> np.ndarray:
depth_8u = (depth_norm * 255).astype(np.uint8)
smoothed = cv2.bilateralFilter(depth_8u, d=9, sigmaColor=25, sigmaSpace=25)
return smoothed.astype(np.float32) / 255.0
4.2.3 Focus plane selection
- Default: median depth (works well for portraits).
- Optional: use AF metadata or local contrast peak.
def choose_focus_plane(depth_norm: np.ndarray, mode: str = "median") -> float:
if mode == "median":
return float(np.median(depth_norm))
# Hook for smarter strategies later
4.3 Optical‑style transform module
This is the core MF‑style logic. It operates on img, depth_norm, and focus_depth.
4.3.1 Depth‑dependent blur (MF DOF geometry)
Goal:
Simulate shallower, smoother DOF: blur increases with distance from focus plane.
Design:
- Compute depth distance:
( d(x,y) = |D(x,y) - D_\text{focus}| ) - Map to blur strength
[0,1] with a scale factor k. - Precompute a blurred version of the whole image.
- Blend sharp/blurred based on strength.
def depth_dependent_blur(
img: np.ndarray,
depth_norm: np.ndarray,
focus_depth: float,
k: float = 10.0,
sigma: float = 8.0,
) -> np.ndarray:
dist = np.abs(depth_norm - focus_depth)
strength = np.clip(dist * k, 0.0, 1.0)
blurred = cv2.GaussianBlur(img, (0, 0), sigmaX=sigma, sigmaY=sigma)
strength_3 = strength[..., None]
out = img * (1.0 - strength_3) + blurred * strength_3
return np.clip(out, 0.0, 1.0)
Future extension:
Replace Gaussian with bokeh kernels (disk, cat’s‑eye) and depth‑dependent kernel size.
4.3.2 Depth‑aware micro‑contrast (focus pop)
Goal:
Increase local contrast in the focus plane, mimicking MF “pop” without global oversharpening.
Design:
- Unsharp mask to get detail layer.
- Depth‑gated Gaussian around focus depth.
def depth_aware_micro_contrast(
img: np.ndarray,
depth_norm: np.ndarray,
focus_depth: float,
radius: float = 3.0,
amount: float = 0.4,
focus_width: float = 0.1,
) -> np.ndarray:
blurred = cv2.GaussianBlur(img, (0, 0), sigmaX=radius, sigmaY=radius)
detail = img - blurred
dist = np.abs(depth_norm - focus_depth)
mask = np.exp(- (dist**2) / (2 * focus_width**2))
mask_3 = mask[..., None]
enhanced = img + amount * detail * mask_3
return np.clip(enhanced, 0.0, 1.0)
4.3.3 Highlight rolloff (MF‑like soft knee)
Goal:
Simulate larger full‑well capacity: highlights compress gently instead of clipping.
Design:
- Compute luminance.
- Apply soft‑knee curve above a knee threshold.
- Scale RGB by luminance ratio.
def highlight_rolloff(
img: np.ndarray,
knee: float = 0.7,
strength: float = 0.6,
) -> np.ndarray:
lum = 0.2126 * img[...,0] + 0.7152 * img[...,1] + 0.0722 * img[...,2]
over = np.clip(lum - knee, 0.0, 1.0)
compressed = lum - over * strength * (over / (over + 1e-3))
ratio = np.where(lum > 1e-3, compressed / (lum + 1e-6), 1.0)
ratio_3 = ratio[..., None]
out = img * ratio_3
return np.clip(out, 0.0, 1.0)
4.3.4 Depth‑aware color separation
Goal:
Slightly richer subject color, slightly softer background color.
Design:
- Convert to HSV.
- Depth‑weighted saturation adjustment.
def depth_aware_color(
img: np.ndarray,
depth_norm: np.ndarray,
focus_depth: float,
focus_sat: float = 1.05,
bg_sat: float = 0.95,
focus_width: float = 0.1,
) -> np.ndarray:
hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
h, s, v = cv2.split(hsv)
dist = np.abs(depth_norm - focus_depth)
focus_mask = np.exp(- (dist**2) / (2 * focus_width**2))
bg_mask = 1.0 - focus_mask
s = s * (focus_mask * focus_sat + bg_mask * bg_sat)
s = np.clip(s, 0.0, 1.0)
hsv = cv2.merge([h, s, v])
out = cv2.cvtColor(hsv, cv2.COLOR_HSV2BGR)
return np.clip(out, 0.0, 1.0)
4.4 Orchestration module
Responsibility:
Wire all modules into a single call.
def mf_style_from_dp(
image_path: str,
depth_path: str,
output_path: str,
) -> None:
img, depth = extract_dpr(image_path) # or load img + np.load(depth_path)
depth_norm = normalize_depth(depth)
depth_norm = smooth_depth(depth_norm)
focus_depth = choose_focus_plane(depth_norm, mode="median")
x = img
x = depth_dependent_blur(x, depth_norm, focus_depth, k=10.0, sigma=8.0)
x = depth_aware_micro_contrast(x, depth_norm, focus_depth)
x = highlight_rolloff(x)
x = depth_aware_color(x, depth_norm, focus_depth)
out = np.clip(x * 255.0, 0, 255).astype(np.uint8)
cv2.imwrite(output_path, out)
5. Mapping Canon DPP behaviors to this pipeline
| DPP feature | Meaning | Pipeline equivalent |
|---|
| Bokeh Shift | Depth‑aware bokeh geometry | depth_dependent_blur (future: bokeh kernel) |
| Micro Adjustment | Focus plane & sharpness | choose_focus_plane, depth_aware_micro_contrast |
| Ghosting Reduction | Depth‑aware edge consistency | Depth smoothing + careful blur blending |
| Export 16‑bit TIFF | Preserve tonal smoothness | Use float32 pipeline, optional 16‑bit output |
6. Testing strategy
6.1 Unit tests
- Depth normalization:
- Input with outliers → output in
[0,1], monotonic.
- Depth‑dependent blur:
- Synthetic depth gradient → blur increases with depth distance.
- Micro‑contrast:
- Flat image → no change.
- Edge at focus depth → increased local contrast.
- Highlight rolloff:
- Synthetic ramp → smooth compression above knee, no banding.
- Color separation:
- Subject at focus depth → saturation slightly increased; background slightly decreased.
6.2 Visual regression tests
- Use a small set of DPR portraits and scenes.
- For each:
- Run pipeline with fixed parameters.
- Save outputs and compare visually and via metrics (SSIM, local contrast histograms).
6.3 Parameter sweeps
- Sweep
k, sigma, amount, knee, strength over reasonable ranges. - Log:
- Edge sharpness in focus plane
- Background blur level
- Highlight clipping percentage
Use this to pick sane defaults.
7. Extensibility
7.1 Plug‑in bokeh kernels
- Replace Gaussian blur with:
- Disk kernels
- Elliptical kernels
- Depth‑dependent shape (cat’s‑eye near edges)
7.2 Lens “profiles”
@dataclass
class MFProfile:
blur_k: float
blur_sigma: float
micro_amount: float
knee: float
rolloff_strength: float
focus_sat: float
bg_sat: float
- Predefine profiles: “Subtle MF”, “Strong MF”, “Leica‑ish”, etc.
7.3 Neural refinement (optional later)
- Train a small CNN to refine the deterministic output:
- Input: original image, depth, MF‑style output
- Target: hand‑tuned or MF reference images
- Keep deterministic pipeline as backbone; NN only adds subtle corrections.
8. Implementation checklist
DPR extraction
- Integrate Canon SDK or existing DPR tool.
- Verify depth map quality on test images.
Core pipeline
- Implement modules exactly as specified.
- Run on a small DPR set.
Parameter tuning
- Adjust
k, sigma, amount, knee, strength for portraits vs landscapes.
Batch runner
from pathlib import Path
def batch_process_dp(input_dir: str, output_dir: str):
Path(output_dir).mkdir(parents=True, exist_ok=True)
for cr3 in Path(input_dir).glob("*.CR3"):
out = Path(output_dir) / (cr3.stem + "_mf.jpg")
mf_style_from_dp(str(cr3), depth_path=None, output_path=str(out))
- Document defaults and knobs
- Provide a short README describing each parameter and its visual effect.
