Autonomous Vehicles

How abstraction enables engineering

Contents

    Tip: Use the toggle at top-right to switch between slide view and article mode.

    Clubs at U-M for autonomy

    • Michigan Autonomous Aerial Vehicles (MAAV) — autonomous drones and perception. Site: maavumich.org
    • Michigan Mars Rover (MRover) — autonomous rover: perception, planning, manipulation. Site: mrover.org
    • UM::Autonomy — RoboBoat autonomous surface vehicle. Site: umautonomy.com
    • Michigan Robotic Submarine (Robosub) — underwater autonomy and perception. Site: www.michiganrobosub.com

    Tip: These teams welcome newcomers; all majors are encouraged. Look for software (perception, SLAM, planning), embedded systems, and controls roles.

    What is autonomy (discussion)

    SAE levels of driving automation (J3016)

    SAE J3016 levels of driving automation infographic

    Graphic: SAE J3016 levels of driving automation. See the SAE blog update for details and definitions: sae.org/blog/sae-j3016-update.

    Abstraction: from physics to decisions

    1. Electromagnetics & mechanics (photons, forces)
    2. Transducers & signals (voltages, point clouds)
    3. Features & objects (lanes, vehicles, pedestrians)
    4. Scene understanding (semantics, intentions)
    5. Prediction & planning (what, when, where)
    6. Control & actuation (torque, brake pressure)

    Each layer hides complexity and exposes interfaces that software can reason about. Classic references: NVIDIA DRIVE, Waymo Research.

    The AV software stack

    • Sensing: cameras, lidar, radar, IMU, GPS
    • Perception: detection, segmentation, tracking
    • Localization & mapping: SLAM, HD maps
    • Prediction: motion forecasting
    • Planning: behavior + trajectory generation
    • Control: MPC / PID tracking
    • Compute: accelerators, safety, redundancy

    Overviews: Mobileye: the self‑driving stack, NVIDIA DRIVE SDK.

    Autonomous driving in extreme weather

    Video: Autonomous vehicles in extreme weather. Consider sensor robustness: cameras vs. lidar vs. radar in rain/fog/snow.

    Interactive room: click to sample distance and update estimate

    Click inside the room to sample a wall distance and update a running estimate. Measurement model: \(z_t = d + \varepsilon,\; \varepsilon \sim \mathcal{N}(0, \sigma^2)\). We smooth noisy readings with an exponential moving average (EMA): \(x_t = (1-\alpha)\,x_{t-1} + \alpha\,z_t\). Smaller \(\alpha\) smooths; larger \(\alpha\) reacts faster.

    Major players and approaches

    Torc Robotics — heavy‑duty trucks, safety‑first deployment Tesla — vision‑centric, end‑to‑end planning Waymo — lidar‑rich, HD‑maps, redundancy Cruise — urban robotaxis Aurora — trucking focus, Aurora Driver Mobileye — EyeQ chips, REM maps, RSS safety Zoox — bespoke robotaxi platform Nuro — last‑mile delivery May Mobility — autonomous shuttles, AV transit NVIDIA — platforms & toolchains Baidu Apollo — China robotaxi; HD maps + V2X Pony.ai — robotaxi pilots in US/China WeRide — robotaxi and robus (China, ME) Wayve — end‑to‑end, AV2.0, vision‑language Oxa (Oxbotica) — autonomy platform for fleets Kodiak Robotics — autonomous trucking Gatik — fixed‑route middle‑mile Plus — driver‑in supervised trucking Einride — electric, teleoperated freight pods Motional — robotaxi JV (Hyundai‑Aptiv)

    Tesla AI days (vision stack): tesla.com/AI. Waymo tech: Waymo Engineering Blog.

    Latest tech

    Video: network demo

    Tesla neural network example. Autoplay is muted so it works in most browsers. Use controls to pause or unmute.

    Waymax: stopping at traffic

    Waymo Waymax driving: stopping at traffic

    Image from Waymo's Waymax project: waymo.com/research/waymax.

    • Emerging approaches leverage modern networks and compute.
    • Accelerators enable real‑time perception and planning.
    • Connectivity and data pipelines drive model improvements.

    Autonomous simulation

    Research repos

    Related open-source simulators

    Open research and simulation tools for autonomous driving.

    Reasoning about complex scenes

    Perception abstracts millions of pixels into objects, lanes, and free space. Prediction abstracts motion into probabilistic trajectories. Planning reduces choices to feasible, safe maneuvers.

    Image: Shibuya Crossing (2018). Background complexity motivates layered abstractions. Read about Shibuya Crossing.

    Behavior abstraction as a finite‑state machine

    Interactive FSM

    Open the interactive FSM in a dedicated page:

    Open interactive FSM (new tab)

    This opens a standalone page where you can drag nodes, create/change edges (Shift‑drag), and step/reset the simulation.

    • Engineers abstract continuous complexity into discrete modes (states).
    • Transitions encode policy and safety constraints.
    • Works with planners (e.g., trajectory optimizer) downstream.

    Waymo papers.

    EE abstraction: RC circuit — physics, math, and model

    Breadboard with LED, resistor, and battery
    Breadboard with LED, resistor, and battery (static image).

    Interactive circuit (Falstad CircuitJS)

    Direct link: Open RC circuit (step response).

    Steady-state Vout and I (9 V → 1 kΩ → diode)

    • Assumptions: Vs = 9 V battery, R = 1 kΩ series, nominal diode with VD,on ≈ 1.7 V. Vout is the node at the diode anode.
    • Steady state (DC): KVL ⇒ Vs = I · R + VD ⇒ Iss = (Vs − VD)/R.
    • Output voltage: Vout,ss ≈ VD,on ≈ 1.7 V (typical LED forward drop).
    • Edge case: if Vs ≤ VD,on, the diode is off ⇒ I = 0, Vout ≈ 0 V.

    Numeric result: Iss = (9 − 1.7)/1000 A ≈ 7.3 mA; Vout,ss ≈ 1.7 V. Note: LEDs often drop ~1.8–2.2 V (red) or higher; using VD,on = 2.0 V would give I ≈ 7.0 mA and Vout ≈ 2.0 V.

    From abstraction to automation

    • Define interfaces between layers (contracts).
    • Choose representations that simplify reasoning.
    • Quantify uncertainty and safety (RSS, verification).
    • Automate with data, simulation, and closed‑loop testing.

    Further reading: A Survey of Motion Planning and Control Techniques for Self-driving Urban Vehicles.

    Where things break down: why AV failures occur

    • Sensor degradation: glare, low light, rain/fog; occlusions and dirty lenses.
    • Perception ambiguity: unusual objects, adversarial appearances, poor calibration.
    • Prediction edge cases: rare human behaviors, multi-agent interactions, distribution shift.
    • Planning/control limits: poor cost tuning, unmodeled dynamics, actuation saturation.
    • System/UX: handoff timing, alerts ignored, map inaccuracies, software bugs.
    • Long-tail risk: rare combinations not seen in training or testing.

    Use layered safety: sensing redundancy, robust perception, conservative planning, fallback maneuvers, and rigorous testing.

    Video: Bay Bridge accident. Discuss what failed and how to mitigate.

    Case study: Waymo recall

    Video: Waymo recall. Discuss root causes, corrective actions, and implications for safety cases and validation.

    Credits and links