Based on an assessment of the various freight data challenges drawn from the literature, a set of broad freight data needs were developed. These needs cut across multiple planning and operational functions such as, long-range freight planning, system performance monitoring, modal-shift analysis, last-mile deliveries, e-commerce, truck idling, truck parking, and land-use. 

Leadership regarding freight data collection, organization, analysis, and standards

Currently, there is a general lack of leadership regarding how to collect, organize, and analyze data for different use cases in freight transportation planning and operations decision-making. Stakeholders in the freight data industry include the U.S. Census Bureau, FHWA, BTS, AASHTO, state departments of transportation, MPOs, and private sector agencies. Each entity collects freight data in a variety of formats, sample size, and frequency. Also, data preprocessing and delivery varies considerably by the type of data collected and the purpose and use of the data. 

The BTS Freight Data Program aggregates and publishes freight-related data from various sources and modes of transport. This includes the Transborder Freight data, FAF, Transportation Services Index, Hazardous Materials Shipments, amongst others. Despite its extensive freight data catalog, the BTS Freight Program is currently not the authoritative entity for guiding the use of all freight data sources, specifically, data from the private sector. Its mandate is mainly limited to datasets published through the FHWA. 

The Freight Transportation Data Committee at TRB (ABJ 90) brings together public and private sector practitioners to discuss the role, sources, methods, and use of freight data for various transportation functions. As a voluntary organization, members are unable to commit sufficient time and resources in the development and advancement of concrete policies, goals, methods, and guidance on the use of freight data. 

The AASHTO Special Committee on Freight also develops policies regarding legislation, regulation, and other matters related to the safe and reliable movement of goods. This Special Committee includes representation from all 52 member DOTs, providing technical expertise and training for members and other freight-related agencies. As part of its 2018 mandate, the committee cites “Goal 2: Improve freight transportation planning practices through the enhancement of freight data and analytical methods and the dissemination of best practices.” 1

There have been efforts to organize data at the federal level. This includes a 2003 framework for the development of freight transportation data as part of a national freight data program. This comprehensive plan recognized the need for public/private partnerships to be represented within the framework and for the resulting data sets to be compatible across geographical and functional aggregations. 2 However, the interview with Heather Monteiro revealed that not only was this framework no longer actively being pursued, but to her knowledge, there were no other efforts at the federal level that have resulted in accessible and meaningfully used data for public agency freight planning.

As illustrated in the above examples, there are entities conducting tasks related to freight data used for planning and operations. Unfortunately, none of these entities can be recognized as the go-to leader for guidance and policy-making regarding best practices for the collection, accessing, and publishing of freight data. Public and private-sector agencies continue to work in silos despite occasional collaboration amongst the various parties at conferences such as TRB. There is a need for concerted, well-funded, and intentional effort to encourage various stakeholders involved in freight data acquisition and delivery to develop standards, best practices, and guidance regarding the use of freight data. 

Continual communication of the benefits of freight data investments to agency leadership

Freight transportation planners and data practitioners need to continually communicate to leadership needs/reasons about investing in robust freight data for planning and transportation investment decision-making. An example cited in the literature is the use of “telling the freight story” narrative to engage agency leadership and demonstrate how informed freight investments support regional and state economic development. 3

Partnerships amongst agencies involved in freight planning and operations

SHRP2 C20 cites the need for improved communication and partnerships among local, regional, and state agencies as well as the private industry to ensure consistency of how data is acquired and used. Improved communication fosters awareness of existing data acquisitions, minimizes data duplications efforts, and leverages shared human and financial resources. 4 Partnerships provide opportunities for state DOTs to purchase data licensing agreements at scale for different geographical locations to support local and regional planning agencies within the state. Before purchasing datasets from private vendors, enforceable cost and data sharing agreements should be in place to address issues relating to restrictions and limitations regarding the use of the data. An example of a data-sharing partnership cited in the literature is the I-95 Corridor Coalition, which procures certain data sets on behalf of its members and makes the data available at a reduced cost or as part of coalition membership. 5

Coordinated data-sharing and exchange file formats

There is also a lack of a coordinated data-sharing and exchange file format for freight data users. Data exchange formats such as TransXML, LandXML, Geographic Information Framework Data Standard, amongst others, cannot be utilized to represent commodity flow movements and freight flows in general. 6 The International Organization for Standardization (ISO) 14825:2011 Geographic Data Files was developed for intelligent transportation systems and focused on the road and road-related information for ITS applications. It also centered on other services such as in-vehicle or portable navigation systems, traffic management centers, or services linked with road management systems (e.g. public transport systems). 7 8  Suggested freight data exchange formats similar to the General Transit Feed Specification (GTFS) 9 or Data Catalog Vocabulary (DCAT) will be beneficial to the industry. GTFS is a common data exchange format for public transportation schedules and associated geographic information. It allows public transit agencies publish transit data for developers to use in applications. It is composed of a series of text files with each containing specific transit information. For example, the location of stops, routes, trips, agency name, stop times, transfers, frequencies, amongst others. DCAT, on the other hand, facilitates interoperability of data published on the Web by organizations, researchers, and agencies. 10   DCAT provides Resource Description Framework (RDF) classes and properties to allow datasets and data services to be described and included in a catalog. It facilitates the consumption and aggregation of metadata from multiple catalogs, to increase the discoverability of datasets and data services, and allows for federated searches for datasets across catalogs in multiple sites. 

Freight data collection standards

Data collected by agencies is typically for a specific purpose to serve specific needs. This results in the following differences as cited in Retrieving Information from Heterogeneous Freight Data Sources to Answer Natural Language Queries: 11

  • Differences in sample sizes, data pre-processing, and estimation techniques 
  • Differences in data quality control processes 
  • Differences in the level of disaggregation and accuracy of the data being reported
  • Differences in file storage formats such as tabulated text files, relational databases, spreadsheets, GIS, web pages, and other web standard-based file formats 
  • Differences in data element definitions and scope for data elements with similar names 
  • Differences in commodity, industry, and land use classifications systems 
  • Differences in vehicle classification systems and modes of transport
  • Differences in the frequency at which the data is collected and reported

There is a need for guidance and best practices for data collection. This guidance can be developed for various activities such as on-site surveys, web surveys, LBS data, GPS data, ELD data, amongst others. Guidance on these data collection activities would address issues relating to data sharing partnerships, data aggregation, and data obfuscation. The FHWA Traffic Monitoring Guide 12 is an example of guidance for state DOTS with regards to policies, standards, procedures, and equipment typically used in a traffic monitoring program. Similar guidance for freight data is needed. The guidance can cover best practices for data security and privacy guarantees for the private industry (to encourage data sharing).

Centralized data repository

NCFRP Report 35: Implementing the Freight Transportation Data Architecture: Data Element Dictionary 13 cites the need for a central data collection repository. This would allow locally-collected or project-specific data to be stored or shared with other data users in the transportation community. These project-specific data sources could complement currently available freight data sources as well as provide additional opportunities to test or validate freight-related models. With knowledge of existing challenges associated with freight data use and privacy concerns, implementing a centralized data collection repository may be a challenge. However, there is an opportunity to implement a data repository focused on publishing data elements and guidance for the integration and use of freight data. An example of such a repository was developed as part of the NCFRP Report 35 project, which was called the Freight Data Dictionary. Now housed with the National Transportation Library (NTL), the Freight Data Dictionary is currently outdated and needs to include new private and big data sources. Without a central leadership of freight data experts, users, and practitioners, maintaining the library will pose an ongoing challenge for the NTL. 

Funding to support a collaborative freight data program

As agencies continue to remain siloed in terms of operations, management, and funding sources, there is a need for a pooled fund program to support the proposed needs listed above. Specifically, for an organization or entity to provide leadership, be responsible for developing data standards, and create a platform for public and private sector data exchange, it needs to be well funded. An example of such a program is the World Wide Web Consortium. 10 W3C is an international community of member organizations, full-time staff, and the general public working together to develop Web standards such as HTML, CSS, PNG images, Web scripting, and dynamic content (e.g. audio and video). Despite its reliance on volunteers and industry-experts, W3C’s full-time paid staff ensures the day to day operations of the consortium. A similar model could be adopted by state agencies, TRB, and private sector organizations for developing standards regarding the collection, processing, sharing, and use of freight data for planning and operations purposes.

Referencing Page:

Freight Data
Type
Challenges & Needs