Definition of Performance Measures
Performance measures (PMs) are an important aspect of the DM process and are central to gauging the degree to which goals and objectives are achieved. During the planning stage, PMs are used to screen and select a preferred solution from among the possible alternatives. Once a solution has been implemented, PMs provide a method to evaluate the level of success that was attained in achieving intended goals.
With the passage of the latest transportation authorization (PL 112-141), the Moving Ahead for Progress in the 21st Century Act, called MAP-21, performance measurement has become a popular topic in transportation planning. Although MAP-21 requires the undertaking of systemic performance measurements, when this Guide refers to measuring performance, it particularly refers to measuring specific outcomes of interventions taken. These could range from a modeling result to more tangible data points such as safety, parking, use of alternative fuels, or reliability.
PMs can be defined in numerous ways, but practice shows that they work best when they are: (1) directly related to a single objective; (2) easily quantifiable; (3) able to gauge the entire range of levels of achievement (a PM that is defined as a continuous variable is better than one that takes only two values, like “achieved” or “not achieved”). Chapter 3 in the FHWA Desk Reference also provides a detailed list of PMs that can be used for various objectives (Federal Highway Administration 2012c).
Tasks involved in defining PMs are:
- Stakeholder outreach and agency coordination:
- Different stakeholders are likely to have different ideas about what PMs should be used, and how to measure them. For example, the delivery costs paid by receivers may be a good metric to measure the objective of “increasing the competitiveness of downtown.” However, freight carriers may argue that delivery costs do not account for the full cost of a delivery given that carriers, typically, absorb parking fines and tolls due to the competitive pressures of the market.
- Respecting the confidential nature of commercially sensitive data is crucial. Many useful PMs—such as the full cost of delivery just mentioned—could require the use of data that carriers may refuse to share, such as driver wages, indirect costs, and fringe benefits. Engaging private-sector associations and trade groups could enable the public sector to create solid cost estimates for use as input to the PMs. Gaining stakeholder support in the process of defining the PMs, and securing the corresponding input data, are essential.
- Data collection:
- PMs are by definition quantitative, and thus require data on the existing or base conditions and/or, in the case of planning efforts, estimates of their future values. Producing such estimates requires the use of planning models and/or simulations. It is suggested that freight planning staff work closely with the modelers at the MPO/state DOT to ensure that the available models can produce the desired PMs. If the models are not capable of providing the necessary PMs, either the PMs must be redefined to suit what the models can provide, or the models must be modified to provide the desired PMs. Careful consideration is needed to determine whether adjusting the PMs or adjusting the models will yield the most applicable and useful data.
- Freight PMs may require data from all modes of transportation, and may include analysis of safety, mobility, system conditions, pavement conditions, travel times, congestion, accessibility, parking, or environmental conditions related to freight movements.
- Freight data availability often is an issue in defining PMs. Engaging stakeholders in the definition of PMs and, at the same time, securing their support to get the necessary input data, can mitigate the data availability issue considerably.
- Assessment and analysis:
- PMs are used at several steps in the management and planning processes, such as to assess the base case conditions surrounding a freight issue, and to compare the results of the assessment to conditions in other jurisdictions. Such comparisons provide context to PMs that may otherwise be difficult to interpret.
- PM analyses must account for such important factors as the variability of the input data used; the time it takes to collect the data and update the PMs; and the sensitivity (or lack thereof) of the PM to changes in the input variables. For example, PMs that use highly variable data (e.g., travel times), need to be analyzed with caution to ensure the robustness of the results. A PM that relies on data collected every 2 or 3 years will fail to capture rapidly changing conditions, whereas a PM that is too sensitive, or too insensitive, may be difficult to analyze. All of these factors need to be taken into account. Adjustments may be needed to the definitions of the PMs and the necessary input data to ensure that the PMs adequately fulfill their roles.
- Generation of outputs:
- Outputs from this task are:
- A set of PMs that assess the degree to which goals and objectives are achieved at different points in time.
- A data collection and/or modeling plan to assess the PMs.
- A collaboration agreement that outlines the role and responsibility of each of the various stakeholders in providing data as needed to estimate the PMs.
- Outputs from this task are: