AI Crack & Pothole Detection System

Project Overview

This project is a production-grade, offline desktop application purpose-built for automated road surface inspection. It analyses drive-through video footage, identifies six distinct categories of pavement distress, measures each defect, grades its severity, and produces a complete set of engineering-quality reports — all without requiring an internet connection or cloud infrastructure.

The problem it solves is both practical and economically significant. Manual road condition surveys are slow, inconsistent between inspectors, dangerous when conducted near live traffic, and expensive to repeat at the cadence modern asset-management programs demand. Highway authorities, municipal works departments, and infrastructure maintenance contractors need objective, repeatable data on where cracks are forming, how severe they are, and how distress is distributed along a route. Existing commercial systems typically require proprietary hardware rigs or cloud-based processing pipelines with recurring per-kilometre fees. This application replaces that workflow with a single installable tool that runs on any modern Windows workstation equipped with a consumer GPU, ingests ordinary dashcam or smartphone video, and produces survey-grade output the same day the footage is captured.

From a business value perspective, the system compresses a labour-intensive field-and-office workflow into a largely automated pipeline: an operator loads video, optionally marks the region of road to analyse, presses start, and collects deliverables. The economic impact is a meaningful reduction in unit cost per kilometre surveyed, faster turn-around on condition reports, and standardised outputs that feed directly into pavement management systems and maintenance tendering processes.

Key Features & Functionality

The application is organised around a clean, tab-based workflow that mirrors how an inspection engineer actually works. Users can queue individual videos or batch-load many at once, and jobs run to completion with auto-resume so a long overnight processing run survives interruptions. A visual editor lets the operator draw either rectangular or polygonal regions of interest directly on a video frame, constraining the detector to the travelled lane and suppressing noise from road furniture, vegetation, and the sky.

During processing, the system delivers a live preview of the annotated video stream with per-frame detection overlays, a heads-up display of throughput in frames per second, estimated time to completion, and a running count of unique cracks identified. Once processing completes, a review interface allows QA staff to step through every detection, verify or correct classifications, flag false positives, and promote or demote severity ratings before any report is finalised — a critical feature for regulated infrastructure reporting where auditability matters.

Deliverables are comprehensive: a deduplicated crack inventory in CSV, a segment-wise summary aggregating defects along chainage intervals, a formatted Excel workbook complete with charts and conditional formatting, a presentation-ready PDF report, an annotated MP4 with legends and metadata burned into the video, and a complete JSON results package that preserves every measurement for downstream analysis or reopening in review mode. Six distress categories are supported — longitudinal cracks, transverse cracks, alligator (fatigue) cracking, block cracks, edge cracks, and potholes — each rendered in a distinct colour and graded on low, medium, or high severity using class-specific thresholds.

Technology Stack

The application is built in Python 3.10+ and is genuinely cross-platform-capable, with Windows as the primary deployment target. Computer vision is powered by the Ultralytics YOLOv8 family of segmentation models, supported by OpenCV for low-level image and video operations. Object tracking uses the ByteTrack algorithm (with BoT-SORT as an alternative) to link detections across frames. The desktop interface is built on PySide6, Qt's official Python binding, which provides the polished native-look user experience expected of professional engineering tooling.

Numerical processing relies on NumPy, SciPy, and pandas. Reporting outputs are generated with openpyxl for Excel workbooks and ReportLab for PDF documents. Configuration is expressed in YAML for easy operator tuning. Optional zero-shot auto-annotation tooling integrates Hugging Face Transformers with IDEA-Research's Grounding DINO model to bootstrap new training datasets, and Roboflow integration streamlines dataset management. PyTorch with CUDA acceleration powers inference, with automatic fallback to CPU where no GPU is available.

Technical Complexity & Challenges

Road-defect detection from in-vehicle video is deceptively difficult, and several facets of the problem drove significant engineering investment.

The most fundamental challenge is unique-defect counting. A single crack may appear in hundreds of consecutive frames as the vehicle drives past, and naïve frame-by-frame detection dramatically over-counts. The system resolves this through a spatiotemporal deduplication layer that combines multi-object tracking identifiers with configurable spatial and temporal windows and intersection-over-union matching, so a given physical defect contributes exactly one record to the final inventory even if its detections momentarily drop out or jitter between frames. Multiple observations of the same defect aggregate confidence rather than creating duplicates, improving both recall and precision compared to a single-frame decision.

A second area of complexity is real-world measurement under monocular, uncalibrated video. Raw pixel dimensions are not directly useful to a road engineer — defect length needs to be reported in metres, and severity bands must correspond to engineering practice. The system combines configurable camera calibration parameters, speed-based chainage tracking, and per-class severity policies to translate pixel geometry into units that fit existing pavement-distress specifications. Class-specific overrides — for example, larger area thresholds for potholes than for hairline cracks — preserve the nuance that a one-size-fits-all rule would erase.

The segmentation-then-geometry pipeline itself is non-trivial. Pixel masks from YOLOv8 are converted into defect contours, filtered to remove noise and artefacts, then analysed for area, major-axis length, and representative width. Severity is classified against configurable thresholds, and results are attached to the appropriate chainage segment so density metrics can be produced.

Architecturally, the application follows a strict separation of concerns. A core pipeline module orchestrates video ingestion and frame processing, while detection, tracking, metrics, reporting, and UI responsibilities live in independent subsystems with well-defined interfaces. Long-running processing runs on background threads and communicates with the UI through a worker abstraction, ensuring the interface stays responsive even while the GPU is fully loaded. A job manager supervises the queue, handling cancellation, pause/resume, parallelism limits, and state persistence. This modular decomposition was a deliberate decision: it keeps the machine-learning layer swappable and makes the codebase tractable for a small team to maintain.

Performance was another critical axis. The application is designed to run comfortably on consumer-grade NVIDIA GPUs, with fallbacks to CPU. Model presets (fast, balanced, accurate) let operators trade throughput against accuracy for the job at hand, and imagery is batched and resized intelligently to keep GPU memory in bounds on long videos. Live preview rendering is throttled independently from the detection loop so UI refresh never becomes the bottleneck.

Design & User Experience

The interface is built around a dark, focused theme designed for the long viewing sessions that typical inspection workflows demand. Navigation is tab-based — Dashboard, ROI Editor, Review, and Settings — so the operator is never more than one click from any major function. The design philosophy prioritises progressive disclosure: the default path to a finished report is a three-click workflow, while advanced detection parameters, class-specific severity overrides, and tracking tunables are available to power users through the settings dialog.

Video interaction is a first-class concern. A custom video canvas supports smooth zoom and pan, frame-accurate seeking, and direct drawing of regions of interest with both rectangle and polygon tools. During processing, the canvas doubles as a live preview window so operators can visually confirm the detector is behaving sensibly before committing to a long run. The review interface presents each detection alongside the frame it was captured in, with one-click accept/reject controls and keyboard navigation so QA can proceed at pace.

Visual language is consistent throughout: each crack class owns a colour that appears identically in the live preview, the annotated video output, the Excel charts, and the PDF report, so readers of any deliverable can mentally link defects across media. Legends, metadata banners, and chainage indicators are composited into the annotated video so output clips remain self-describing even when separated from the main report.

Scale & Scope

The primary application consists of roughly 35 Python modules organised into seven subsystems and totalling approximately 11,500 lines of production code, with the largest single modules — the video annotator, the PDF generator, the dashboard, the settings dialog, and the processing pipeline — each running in the 500-to-650-line range. Around this core sits an additional auto-annotation utility for dataset bootstrapping, a comprehensive YAML configuration surface exposing dozens of tunable parameters, and a structured logging subsystem with rotating file output.

The codebase integrates more than a dozen external libraries spanning deep learning, computer vision, desktop UI, numerical computing, reporting, and data-annotation tooling. It supports multiple pre-trained model presets, multiple tracker algorithms, configurable device targeting, and a reporting subsystem that produces six distinct output artefacts per video. Operators can process multiple jobs in parallel, with job state persisted between sessions so in-flight batches survive restarts. A curated training dataset of several hundred annotated images accompanies the application, along with custom-fine-tuned YOLOv8 segmentation weights targeting the specific distress classes the system reports on.

Business Impact & Use Case

The primary audience is the civil infrastructure sector: national and state highway authorities, municipal public-works departments, road-construction contractors operating under performance-based maintenance contracts, and specialist pavement-condition-survey firms. Secondary audiences include airport pavement managers, large facility owners with extensive private road networks (industrial estates, campuses, logistics parks), and academic researchers working on pavement deterioration models.

The value delivered is measurable. Survey throughput increases dramatically because what previously required a two-person crew walking or slow-driving a route can now be captured at traffic speed by a single driver and analysed in the office. Consistency improves because the same model and the same thresholds score every kilometre identically, eliminating inter-rater variability. Deliverable quality improves because every reported defect is backed by a timestamped video frame and a measurable mask — an auditable evidence trail that manual clipboard surveys cannot match. And because the system runs entirely offline on the client's own hardware, there are no per-kilometre cloud-processing fees and no data-sovereignty concerns when dealing with government road networks.

Development Approach

The project follows a configuration-driven, modular architecture. A central YAML file exposes every meaningful parameter — model selection, detection thresholds, tracking behaviour, deduplication windows, severity bands, calibration, chainage, output formatting, and job-queue limits — so new deployments can be adapted to local conditions without code changes. This emphasis on externalised configuration is a deliberate choice aimed at maintainability: as engineering practice or client requirements evolve, adjustments are data-only.

Each subsystem presents a narrow, well-documented interface. Detection code knows nothing about reporting; reporting code knows nothing about GPU management; the UI layer communicates with the processing pipeline exclusively through worker-thread signals and dataclass result objects. Dataclasses are used throughout to model domain entities — crack records, pipeline results, progress updates, segment summaries — giving the code the clarity of typed contracts without the ceremony of heavier frameworks.

Operational robustness was a first-class concern. Structured logging with rotating file output captures the entire processing history, dependency checks run at startup to fail fast on misconfigured environments, configuration is validated on load, and long-running jobs persist progress so work is not lost to a crash or a power cut. A command-line interface with --version, --config, and --debug flags makes the tool friendly to both end users and support engineers diagnosing field issues. The result is a system that balances the research energy of modern computer-vision tooling with the reliability expectations of production infrastructure software — a tool designed not just to demonstrate capability, but to be used, day after day, on real jobs.