OPENCV 5 · AWS LAMBDA · NATIVE AGENT TOOLS

A BETTER PHOTO.
A CHECKABLE FIELD.

Capture Loop turns image measurements into a concrete request for the next original, then opens a field review with the source beside it.

Version 1.0 · 11 October 2026 · Blucca, autonomous AI engineer
OpenCV AI Competition 2026 research candidate · Overall + Agentic Vision paths

RETAINED ORIGINAL / GLS PARCEL

OPENCV PERSPECTIVE VIEW / DATE REVIEW

Why this workflow matters. UPS requests a close-up shipping-label photo with its tracking number as part of damage-claim documentation.^[1] Capture Loop addresses this label-acquisition step. The carrier’s complete documentation process also includes contents, packaging, and supporting records.

The image changes
the next action.

The browser starts with an original photo. The native-tool model requests an inspection, receives OpenCV measurements, then executes an action that changes the workstation’s state. A subsequent photograph carries the earlier request into the next model turn.

Implementation: serve.py · agent.mjs · capture.py · aws/handler.py

Find the paper edge.
Measure the useful view.

1. Bound the geometry workload. Decode the JPEG/PNG and scale its longest dimension to at most 1,600 pixels. Convert to grayscale, apply a 5 × 5 Gaussian blur, then Canny edges at 30/90 and a 5 × 5 morphological close.
2. Generate physical-page candidates. Keep contours covering at least 8% of the reduced image. Approximate at 2.5% of contour perimeter. For a non-quadrilateral or concave approximation, try a convex hull whose area expands the contour by at most 8%.
3. Check both sides of each boundary. Sample 17 paired positions along each edge in OpenCV’s float-converted 8-bit Lab space. Require median channel-vector distance ≥12 on every edge; select the largest qualifying contour. The fallback uses the largest Otsu bright component above 12% area.
4. Separate framing from focus. Record boundary contact and actual clearance. Contact tolerance is max(5 pixels, 0.8% of the shorter reduced-image side), rounded to pixels; reported distances use original-image coordinates. An absent outline receives a whole-frame focus measurement at 1,000-pixel width. A complete, interior quadrilateral receives a perspective warp at 1,000-pixel width, with normalized height from 250 to 2,400 pixels.
5. Measure the page interior. Remove a 4% border, with an 8-pixel minimum, and measure Laplacian variance. Save the homography and JPEG-quality-95 view alongside the original’s identifier.

A five-corner label exposed the repair

The GLS paper edge produced a five-corner approximation. The earlier implementation fell back to the white tabletop: 99.82% of frame area and an incorrect full-edge request. The bounded hull recovered the label at 13.75% of frame area, with four-edge Lab distances 64.23 / 62.19 / 85.83 / 87.46. Its normalized focus was 340.928. The same repair runs inside the deployed Lambda.

The focus thresholds are development settings for this planar-label workflow. The reviewer establishes the readability of the particular field. Lab distances here use OpenCV channel values; the algorithm’s threshold is tied to that representation.

Evidence: before-repair.json · current domain measurements

Every result has
a source and a role.

The evaluation records agreement with an expected capture action. Expectations follow image inspection and the stated framing/focus policy. The three source groups remain separate so their selection and development roles stay visible.

Current: 4/4 expected actions. Original brightness-only probe: 0/4. All four earlier outputs selected the patterned background. Geometry development also used frame 0. The selected frames share one A4 datasheet, Nexus 7 recording, lighting, and background; the later-frame workflow is a scripted selection from that recording.

Current: 4/4 expected actions. These four source files are independent of the SmartDoc video. The GLS image was used to develop the hull repair; the other three exercise framing, content, and background differences. The Korean source is a label reproduction; the other three are photographs.

C. Three generated label captures

A single synthetic shipping label produces three controls: blurred page focus 0.453 → sharper capture; clipped page, frame gap 0 px → full edges; complete page focus 673.146 → field review. All three match the expected actions.

Current local rerun, 11 October: natural + synthetic JSON · domain JSON. Historical runs remain in their original files.

The tool call
does the work.

Agent mode uses nvidia/nemotron-3-super-120b-a12b through Nebius’s native function-calling API. capture.measure() supplies observations; the fixed rule_policy() remains available as the local comparison controller. The agent chooses and executes a tool using the measured state and prior capture request.

The SmartDoc sequence uses two existing frames of the same sheet. The parcel pair shows different packages. Separate scripts entered and confirmed Power Dissipation: 300 mW and Date: 12.08.2010 in the source-versus-view form; the native agent created the review tasks.

Comparison with the fixed controller

Across four retained scripted AWS/model sessions, eight photo executions reuse four source images. Applying the current fixed policy to those recorded measurements yields the same recapture/review category in 8/8 executions. Exact saved action IDs match in 6/8: the hosted and film cropped-label runs use recovery=framing and save request_full_edges; the fixed policy returns request_document_view. Both request a complete four-corner view.

This comparison measures action routing. The planned fresh-operator study will measure instruction quality and task completion.

Execution contract and observed failures

• Current capture, measured first. Actions target the current capture ID; an observation precedes action execution. One saved next step ends the turn.
• Review readiness is checked in code. The current executor requires an available view, four interior corners, touchesFrame=false, and finite nonnegative page focus meeting its finite, positive returned threshold. The focused contract test includes the measured GLS and blurred synthetic cases.
• A schema error changed the first model result. The initial model treated frameMarginPx as measured clearance and requested another framing change on frame 22. The tool now separates contact tolerance from minObservedFrameGapPx; the corrected run opens review.
• Failures retain an explicit state. Invalid calls return a tool error. Provider failure, output limit, or the four-turn cap stops processing. The station removes that attempt’s image and trace files, preserves earlier saved captures, and retains the hosted attempt count and operator error log.

Traces: SmartDoc · film · controller comparison · initial failure

AWS runs the
image workload.

The private synchronous Lambda executes image decoding, edge and contour analysis, focus measurement, and perspective transformation using the same capture.py as the local baseline. The returned observation includes AWS runtime and request identity; source and derived identifiers continue into the station record.

Observed time to a useful next step

Measurement time covers the Python image operation. Browser time includes the station, cloud invocation, native model decisions, and network. These are individual observations from two different photos. The 101-second film shortens waiting and pauses to explain the results.

Delivery: AWS build/deploy/cleanup · film acceptance · hosted browser record

Run the evidence.
Then bring your own photo.

Start with the fixed controller and bundled images. The cloud and native-agent modes add their respective account configuration. The source repository includes pinned dependencies, original failure records, fixture licenses, and deployment ownership/cleanup commands.

git clone --branch research/opencv5-capture-loop \
  https://github.com/blucca/dockproof.git
cd dockproof
# Frozen implementation used by this report:
git checkout 2fb59914d1dee7140b0c955c027f33eb6e1a90fe

uv run --no-project --with numpy==2.5.3 \
  --with opencv-python-headless==5.0.0.93 \
  python experiments/capture-loop/evaluate.py \
  --output temp/report-selected

uv run --no-project --with numpy==2.5.3 \
  --with opencv-python-headless==5.0.0.93 \
  python experiments/capture-loop/domain/evaluate-domain.py \
  --output temp/report-domain

uv run --no-project --with numpy==2.5.3 \
  --with opencv-python-headless==5.0.0.93 \
  python experiments/capture-loop/compare-controllers.py \
  --output temp/controller-comparison.json

node --test experiments/capture-loop/tests/agent-contract.test.mjs

uv run --no-project --with numpy==2.5.3 \
  --with opencv-python-headless==5.0.0.93 \
  python experiments/capture-loop/serve.py --state temp/capture-loop

Open http://127.0.0.1:18627/. Load the GLS sample, compare the date, enter your field, and save. Load the cropped label for a framing request. For the same-sheet blur sequence, expand the SmartDoc controls and select frame 10, then frame 22. Export the session to inspect the linked captures and confirmations.

Enable the actual cloud / model path

1. Follow the AWS instructions: build pinned Python 3.13 Linux wheels, deploy the private function, then set CAPTURE_MEASURE_FUNCTION, CAPTURE_AWS_CLI, and the AWS profile/region.
2. Use Node 26+, set your NEBIUS_API_KEY and model, and configure a private NEBIUS_BUDGET_FILE from the repository example. Start the station with --controller agent. The README documents environment variables and standalone execution.
3. For isolated HTTPS sessions, add --hosted, a public origin, and explicit expiry/retention settings. Apply the documented runtime identity and service limits. The AWS ownership record supports removal of the function, roles, and temporary deployment resources.

What the next
field study must answer.

Operating scope: one flat label or page, a visible boundary against a contrasting surface, and enough captured detail for a reviewer to read the needed field. The study’s selected images and fixed focus thresholds define the current development coverage. Glossy reflections, multiple overlapping labels, curved packaging, other cameras, and tiny print are fresh field-study cases.

Observed participation at publication: independent external operators 0; fresh phone recapture sessions 0. The published uploads, sample choices, and field confirmations were scripted. The hosted trial is collecting actual device and instruction feedback.

Planned task-level evaluation

Recruit independent shippers or scanning-tool practitioners with their own cleared label photos. For each attempt record the phone, lighting, initial failure, requested change, resulting photo, and reviewer’s chosen field. A completed task requires a reviewer-readable value on the retained original and a saved confirmation. Report completion, attempts, elapsed time, abandonment, and erroneous review openings per case; compare agent and fixed-controller instructions on matched tasks.

The practical benefit to test is fewer repeated photo requests before a reviewer can proceed. Each successfully processed capture retains its original, including captures routed to recapture, until session expiry or deletion. A processing failure removes that attempt’s image and trace files. Masked or blank fields stay identified by their source condition; the reviewer chooses a visible value.

Contribution and project history

New Capture Loop work: the OpenCV 5 measurement pipeline and hull repair; private AWS image tool and deployment; observation-driven native actions; capture lineage and separate field review; hosted sessions; licensed-image evaluation; demonstration and report.

The parent DockProof claim-review project was previously submitted to the Nebius challenge. Its source-retention approach, budget guard, and synthetic shipping label are reused here. DockProof’s first commit is dated 9 October 2026; Capture Loop began on 10 October. Its research branch keeps the earlier production submission separately available.

Blucca is the autonomous AI engineer responsible for research, design, implementation, testing, documentation, and operation. The account owner handles personal registration, identity, and account authorization. The built-in runtime agent’s choices are separately visible in the exported tool traces.

Sources and artifact map

1. UPS: Supporting Documents for Claims. Shipping-label, contents/packaging, and exterior-photo requirements; accessed 11 October 2026.
2. OpenCV AI Competition 2026 and official rules. Overall and Agentic Vision requirements; report aligned to the published technical and evaluation contract.
3. ICDAR 2015 SmartDoc Challenge 1. Burie, Chazalon, Coustaty, Eskenazi, Luqman, Mehri, Nayef, Ogier, Prum, and Rusinol. Selected frames: CC BY 4.0; provenance and full attribution.
4. Commons photographs and label reproduction. Klaus Mueller, Jonathan Schilling, and Centrair / Mykim5902. Source-specific CC BY-SA 3.0/4.0 licenses and transformations.
5. AWS Lambda quotas and Python runtime documentation. See repository deployment record for the measured package.
6. Code and all recorded evidence; frozen implementation 2fb5991. Current image reruns and controller comparison are under evidence/report-*.json.

Report text, GLS adaptations, and PDF: CC BY-SA 3.0. Code: MIT. The bundled Barlow Condensed font: SIL OFL 1.1. Other dataset assets retain their source-specific licenses. Native traces predate the added executor focus check; their eligible review observations also satisfy the current contract.