⬅️ Back to agenda

Performance-Based Metrics and Planning

September 16, 2025

08:30 AM – 10:00 AM at Thomas H. Swain Room

Planning for a better future? You Betcha!!

This session presents practical applications of travel demand models in performance measurement, project planning, and scenario evaluation. Case studies from California and Ohio demonstrate how regional models inform project-level analysis, forecast safety outcomes, and address coordination challenges between regional and state agencies. The session also explores the use of performance metrics in congestion pricing. Together, these presentations underscore the evolving role of travel models in addressing uncertainty, supporting decision-making, and improving transportation system outcomes.

5 Sub-sessions:
Regional Collision Forecasting for Project and Plan-Level Analysis in the Sacramento, CA Region

Title

Regional Collision Forecasting for Project and Plan-Level Analysis in the Sacramento, CA region

Topics

Road safety, collision forecasting, travel model application

Abstract Background

Highway capacity expansion projects often highlight collision reduction as a benefit, traditionally quantified using crash modification factors (CMFs) based on specific improvements. However, this approach does not capture the potential increase in collisions resulting from induced demand – the phenomenon where increased road capacity leads to more driving and, consequently, greater exposure to crash risk. To address this gap, the Sacramento Area Council of Governments (SACOG) developed a collision prediction model (CPM) to estimate how many collisions a project or plan scenario may create or prevent based on how the scenario affects factors such as vehicle-miles traveled (VMT), transit activity, and land use.

Description of Abstract

SACOG developed a collision prediction model (CPM) using its regional travel demand model and observed collision data to estimate how many collisions a project or plan scenario may create or prevent based on how the project affects key factors that influence collision risk such as vehicle-miles traveled (VMT), transit activity, and land use.

SACOG’s CPM predicts collisions at a zonal level for 15,498 256-acre hexagonal zones covering the SACOG region using a Zero-Inflated Negative Binomial (ZINB) model. Key explanatory variables include VMT, job and population density, transit service density, miles of multi-lane roads, and number of high-speed intersections. These inputs all come from SACOG’s SACSIM travel demand model. Notably, the CPM is comprised of 11 submodels whose variables and coefficients are specific to different road types (freeways versus arterials) and different built environment types (e.g., urban, suburban, rural, etc.). To train and test the CPM, we used six years (2014-2019) of geocoded collision data from UC Berkeley’s Traffic Injury Management System (TIMS). We trained the model on years 2014-2016, then tested the model’s predictions against years 2017-2019. 

The resulting CPM has been used on several test projects and demonstrates its usefulness as a complement to more traditional CMF-based analysis, giving practitioners insight into whether a project will have a net safety cost or benefit, as well as how costs and benefits are distributed across different communities.

Statement on Why Abstract is Noteworthy

SACOG’s CPM contributions to planning practice include:

  • Enabling more comprehensive safety analysis that considers safety effects both at a project location and regional level.
  • Adding an equity component to safety analysis by comparing changes in collisions for environmental justice (EJ) areas versus non-EJ areas.
  • Enabling identification of “hot spot” zones where observed collision frequency is much higher than the CPM predicts.
  • Inspiring California DOT (Caltrans) to incorporate a rubric-based “crash exposure factor ” into its Caltrans System Investment Strategy (CSIS) safety metric. The factor lowers a project’s safety score if it induces a significant amount of VMT, and raises its safety score if it significantly reduces VMT.

Project is in progress: model is complete and has been used on test projects, but not for final analysis or planning decisions.

Enhancing Project-Level ABM Application by Asking the Right Questions

Background to Problem

Activity-based travel demand models (ABMs) provide a sophisticated framework for simulating individual travel behaviors and forecasting transportation demand. While ABMs are valuable for regional policy analysis, their application at the project level requires additional due diligence. The complexity of ABMs poses challenges in ensuring model results are appropriately scoped, interpreted, and refined for local-scale decision-making. Practitioners must navigate model selection, input refinement, and post-processing considerations to generate meaningful, reliable insights for transportation projects.

Description of Application

This presentation explores a structured approach to applying SACSIM19, an ABM developed by the Sacramento Area Council of Governments (SACOG), at the project level. Through case studies, it identifies key questions practitioners should ask to enhance ABM effectiveness. The first consideration is whether SACSIM19 is the appropriate tool, depending on project type, size, study area characteristics, and desired forecasting outputs. The study provides examples where SACSIM19’s detailed structure added value and cases where simpler models were more appropriate to answer the questions being asked.

Effective use of SACSIM19 depends on proper scoping of the analysis. Decisions on subarea validation, full model versus assignment-only runs, the number of model runs, and parameter settings can significantly influence outcomes. Case studies illustrate how subarea validation affects volumes and how different model parameters such as convergence criteria and assignment iterations can impact vehicle miles traveled (VMT). The presentation explores how running the model multiple times reduces model noise due to stochastic elements variability.

Refining model inputs is essential for reliable forecasting. This presentation explores how SACSIM19’s population synthesis process, which includes multi-resolution land use inputs, are affected by the buffering of accessibility metrics. External gateway travel estimates require additional care to consider how visitor travel behavior is modeled in an ABM structure.

Post-processing is key for extracting actionable insights. ABMs provide a substantial amount of data that can be used for project level analysis, such as intersection-level volumes, but require post-processing (such as volume difference method) and reasonability checks to ensure appropriate use.

By understanding the model’s capabilities, limitations, and necessary adjustments, this study shows how practitioners can ensure ABMs deliver valuable insights to support informed decision-making.

Statement on Why Application is Noteworthy

This study offers a practical framework for applying ABMs at a project level, helping transportation planners refine model use for localized decision-making. The insights from SACSIM19 case studies provide a replicable methodology for improving project-level ABM applications, making this study valuable for agencies, model practitioners, and transportation planners looking to maximize the utility of ABMs.

Project Status

All example projects concluded within 2021-2024 timeframe 

Modeling at the Crossroads: Practices and Challenges for Applying MPO TDMs to Caltrans Projects

Metropolitan Planning Organizations (MPOs) and some Regional Transportation Planning Agency (RTPAs) in California develop and maintain regional travel demand models (TDMs) for long-range transportation planning and air quality conformity analysis. While these TDMs are valuable tools for policy evaluation, they are also frequently used for project-level analysis, including vehicle miles traveled (VMT) assessments, congestion evaluations, and CEQA-required environmental reviews. However, significant challenges arise when adapting regional models for project-level applications, including misalignment of base years, limited scenario flexibility, and inadequate sensitivity testing for localized impacts. This presentation provides a framework for assessing the suitability, limitations, and recommended practices for applying MPO models in Caltrans project-level analysis in California.

This presentation evaluates MPO TDMs in California, focusing on their practical application for project-level analysis. To investigate potential challenges, a statewide survey was conducted, gathering responses from 25 out of 44 MPOs and RTPAs regarding model ownership, data availability, and calibration practices. Additionally, interviews with 14 MPOs and RTPAs were conducted to examine model structures, post-pandemic adjustments, induced VMT methodologies, and validation procedures. A technical review of six major MPO models assessed model documentation, calibration, transparency, and usability for project-level applications. Lastly, a project-level review of four Caltrans projects was conducted, incorporating feedback from Caltrans project managers to identify project-level implementation challenges.

Findings indicate that while MPO models offer broad application potential, the “off-the-shelf” models are often not directly suitable for project-level analysis without significant modifications. Key issues include inconsistencies in scenario development, stochastic variability in activity-based models (ABMs), and insufficient model sensitivity to project-specific changes. While most regional models undergo calibration and validation, many lack standardized methodologies for sensitivity testing, benchmark validation, and induced VMT estimation. ABMs, though well-suited for analyzing regional policy impacts, often require extensive adjustments for project-level applications, making them less practical without additional due diligence. Additionally, most models lack sufficient feedback mechanisms to generate reasonable induced VMT estimates for project-level anaysis. The review of Caltrans projects revealed limited documentation of project-specific modeling considerations and highlighted several concerns related to model applicability.

Through interviews with MPOs, RTPAs, and Caltrans project managers, this study examines the challenges of applying TDMs for project-level analysis from multiple perspectives. By identifying these gaps and challenges, this project recommends establishing standardized modeling guidelines in collaboration with Caltrans, MPOs, and RTPAs. Developing a rigorous model review process, including a structured checklist for project applications, will improve transparency and consistency. Providing technical guidance on project documentation and scenario development will help ensure that model modifications and assumptions are properly documented and reviewed.

Ohio Planning Applications: Information-Forward

Abstract Background

The development of a fully customized travel demand model designed to address specific applications can be both costly and time consuming. The Ohio Department of Transportation (ODOT) recently conducted two separate and distinct planning studies that managed to pivot from the existing statewide model: planning for the 2024 eclipse and a strategic statewide transportation analysis. By focusing on information delivery, highly valuable support can be delivered to decision-makers using existing resources at reduced costs.

Description of Abstract

Two major planning efforts have recently relied on the Ohio Statewide Travel Demand Model: a statewide transportation analysis, and the planning for the April 8, 2024 total solar eclipse, which occurred with a path of totality spanning from Maine to Texas. To better prepare for the atypical traffic conditions expected to occur on the eclipse day, the Ohio Department of Transportation developed an eclipse event tool using ODOT’s statewide transportation model. To make the changes to the statewide model that were necessary to forecast eclipse-day travel demand, ODOT relied on observed data from the 2017 eclipse, which crossed over Kentucky and Tennessee, thus providing an opportunity to observe how travelers altered their travel patterns in an area not too far from Ohio. Traffic exiting these areas saw atypical increases at times-of-day inconsistent with normal peaking characteristics leading to high levels of congestion that disrupted traffic at many locations. This presentation will discuss how the observed data were used to adjust the Ohio statewide model to reflect anticipated eclipse day traffic.  Additionally, accounting for the adjustments to trip lengths to reflect Ohio’s particular travel patterns and the methods used to assign traffic at hourly intervals will be discussed.  Emphasis will be given to how agency partners throughout Ohio were introduced to the outputs generated by the Eclipse tool.  For the Strategic Transportation and Development Analysis, the state developed three population forecasts and the statewide model was used to determine what highway facilities would potentially be stressed in the future, either from inherent population growth, or if a major economic development initiative occurred in one of the sites throughout the state specifically identified for its ability to accommodate development.  Each economic region of the state was analyzed to determine where the highest uncertainties were in regard to the resiliency of the transportation system and its ability to meet future needs. 

Statement of Why Abstract is Noteworthy

This work is notable because it demonstrates how two very different sets of tools that pivot from a central model and focus on information delivery were developed to create highly relevant information to decision makers in a short amount of time. Once the tools were in place, we were able to evaluate and create layers of information to inform STDA priorities in a data driven method and for Eclipse able to answer questions on how to prepare and move people for a once in a lifetime event. 

Congestion Pricing Analysis and Performance Metrics

Background to Problem

Following the recent implementation of congestion pricing in New York City, various other cities that have explored congestion pricing strategies have shown renewed interest. Notably, congestion pricing is one of the various strategies adopted in Southern California and the San Francisco Bay Area to help achieve regional sustainability goals. The impact of these strategies on local and regional trip making, vehicle-miles traveled and greenhouse gas reductions, among other impacts, have been assessed with regional travel demand models (RTDMs). This presentation will outline an essential modification required to obtain useful measures of user benefits from a RTDM, based on our experience in analyzing such projects.

Description of Application

When evaluating alternatives in a congestion pricing study, it is important to consider not only basic metrics such as VMT changes, VHT changes, and delay changes but also the differential impact of tolls and travel time improvements on various population segments. While dynamic traffic assignment (DTA) tools are useful to simulate the traffic dynamics under a road pricing regime, it can be challenging to use them at a regional scale to understand the cost and benefits at an individual (traveler) level. For this reason, RTDMs are still used widely to study congestion pricing projects. Off-the shelf RTDMS typically generate one set of average skims for each time period and vehicle class. While these average skims are sufficient for predicting demand, they are not particularly useful for reporting costs and benefits at an individual level. These skims typically underrepresent the congestion reduction benefits. Moreover, using these average skims to report differences in the level-of-service for different population segments suggests nearly identical costs and benefits to different segments of the society, which need not be the case as individuals with lower value-of-time (VOT) can divert to other routes, modes and/or time periods  to avoid/reduce their travel cost. For this project we post-processed the standard RTDM model outputs to create skims that were segmented by VOT values, and used those skims to compute metrics such as travel time improvements, toll amount paid, changes in job accessibility for specific individuals, and other metrics. Once these individual metrics are appropriately computed based on these VOT skims, the results are summarized at a higher level of aggregation, such as by household income groupings or equity focus community areas. In this presentation, we will showcase the various metrics and insights produced using this approach. In addition to this, we will also present the details of the other modifications made to the model to use distributed VOT in mode choice and assignment, as has been recommended by the Strategic Highway Research Program.

Statement on Why Abstract is Noteworthy

With growing interest in congestion pricing as an areawide or regional strategy, RTDMs will continue to be used to assess user benefits and impacts, at least until dynamic traffic assignment tools can be deployed at the scale of medium and large regions and integrated with demand models. This presentation describes a methodology for specifying RTDMs for use in congestion pricing applications that extends SHRP guidance on this topic.  

Project is complete