Based on an assessment of the various freight data challenges drawn from the literature, a set of broad freight data needs were developed. These needs cut across multiple planning and operational functions such as, long-range freight planning, system performance monitoring, modal-shift analysis, last-mile deliveries, e-commerce, truck idling, truck parking, and land-use.
Leadership regarding freight data collection, organization, analysis, and standards
Currently, there is a general lack of leadership regarding how to collect, organize, and analyze data for different use cases in freight transportation planning and operations decision-making. Stakeholders in the freight data industry include the U.S. Census Bureau, FHWA, BTS, AASHTO, state departments of transportation, MPOs, and private sector agencies. Each entity collects freight data in a variety of formats, sample size, and frequency. Also, data preprocessing and delivery varies considerably by the type of data collected and the purpose and use of the data.
The BTS Freight Data Program aggregates and publishes freight-related data from various sources and modes of transport. This includes the Transborder Freight data, FAF, Transportation Services Index, Hazardous Materials Shipments, amongst others. Despite its extensive freight data catalog, the BTS Freight Program is currently not the authoritative entity for guiding the use of all freight data sources, specifically, data from the private sector. Its mandate is mainly limited to datasets published through the FHWA.
The Freight Transportation Data Committee at TRB (ABJ 90) brings together public and private sector practitioners to discuss the role, sources, methods, and use of freight data for various transportation functions. As a voluntary organization, members are unable to commit sufficient time and resources in the development and advancement of concrete policies, goals, methods, and guidance on the use of freight data.
The AASHTO Special Committee on Freight also develops policies regarding legislation, regulation, and other matters related to the safe and reliable movement of goods. This Special Committee includes representation from all 52 member DOTs, providing technical expertise and training for members and other freight-related agencies. As part of its 2018 mandate, the committee cites “Goal 2: Improve freight transportation planning practices through the enhancement of freight data and analytical methods and the dissemination of best practices.” 1
There have been efforts to organize data at the federal level. This includes a 2003 framework for the development of freight transportation data as part of a national freight data program. This comprehensive plan recognized the need for public/private partnerships to be represented within the framework and for the resulting data sets to be compatible across geographical and functional aggregations. 2 However, the interview with Heather Monteiro revealed that not only was this framework no longer actively being pursued, but to her knowledge, there were no other efforts at the federal level that have resulted in accessible and meaningfully used data for public agency freight planning.
As illustrated in the above examples, there are entities conducting tasks related to freight data used for planning and operations. Unfortunately, none of these entities can be recognized as the go-to leader for guidance and policy-making regarding best practices for the collection, accessing, and publishing of freight data. Public and private-sector agencies continue to work in silos despite occasional collaboration amongst the various parties at conferences such as TRB. There is a need for concerted, well-funded, and intentional effort to encourage various stakeholders involved in freight data acquisition and delivery to develop standards, best practices, and guidance regarding the use of freight data.
Continual communication of the benefits of freight data investments to agency leadership
Freight transportation planners and data practitioners need to continually communicate to leadership needs/reasons about investing in robust freight data for planning and transportation investment decision-making. An example cited in the literature is the use of “telling the freight story” narrative to engage agency leadership and demonstrate how informed freight investments support regional and state economic development. 3
Partnerships amongst agencies involved in freight planning and operations
SHRP2 C20 cites the need for improved communication and partnerships among local, regional, and state agencies as well as the private industry to ensure consistency of how data is acquired and used. Improved communication fosters awareness of existing data acquisitions, minimizes data duplications efforts, and leverages shared human and financial resources. 4 Partnerships provide opportunities for state DOTs to purchase data licensing agreements at scale for different geographical locations to support local and regional planning agencies within the state. Before purchasing datasets from private vendors, enforceable cost and data sharing agreements should be in place to address issues relating to restrictions and limitations regarding the use of the data. An example of a data-sharing partnership cited in the literature is the I-95 Corridor Coalition, which procures certain data sets on behalf of its members and makes the data available at a reduced cost or as part of coalition membership. 5
Coordinated data-sharing and exchange file formats
There is also a lack of a coordinated data-sharing and exchange file format for freight data users. Data exchange formats such as TransXML, LandXML, Geographic Information Framework Data Standard, amongst others, cannot be utilized to represent commodity flow movements and freight flows in general. 6 The International Organization for Standardization (ISO) 14825:2011 Geographic Data Files was developed for intelligent transportation systems and focused on the road and road-related information for ITS applications. It also centered on other services such as in-vehicle or portable navigation systems, traffic management centers, or services linked with road management systems (e.g. public transport systems). 7 8 Suggested freight data exchange formats similar to the General Transit Feed Specification (GTFS) 9 or Data Catalog Vocabulary (DCAT) will be beneficial to the industry. GTFS is a common data exchange format for public transportation schedules and associated geographic information. It allows public transit agencies publish transit data for developers to use in applications. It is composed of a series of text files with each containing specific transit information. For example, the location of stops, routes, trips, agency name, stop times, transfers, frequencies, amongst others. DCAT, on the other hand, facilitates interoperability of data published on the Web by organizations, researchers, and agencies. 10 DCAT provides Resource Description Framework (RDF) classes and properties to allow datasets and data services to be described and included in a catalog. It facilitates the consumption and aggregation of metadata from multiple catalogs, to increase the discoverability of datasets and data services, and allows for federated searches for datasets across catalogs in multiple sites.
Freight data collection standards
Data collected by agencies is typically for a specific purpose to serve specific needs. This results in the following differences as cited in Retrieving Information from Heterogeneous Freight Data Sources to Answer Natural Language Queries: 11
- Differences in sample sizes, data pre-processing, and estimation techniques
- Differences in data quality control processes
- Differences in the level of disaggregation and accuracy of the data being reported
- Differences in file storage formats such as tabulated text files, relational databases, spreadsheets, GIS, web pages, and other web standard-based file formats
- Differences in data element definitions and scope for data elements with similar names
- Differences in commodity, industry, and land use classifications systems
- Differences in vehicle classification systems and modes of transport
- Differences in the frequency at which the data is collected and reported
There is a need for guidance and best practices for data collection. This guidance can be developed for various activities such as on-site surveys, web surveys, LBS data, GPS data, ELD data, amongst others. Guidance on these data collection activities would address issues relating to data sharing partnerships, data aggregation, and data obfuscation. The FHWA Traffic Monitoring Guide 12 is an example of guidance for state DOTS with regards to policies, standards, procedures, and equipment typically used in a traffic monitoring program. Similar guidance for freight data is needed. The guidance can cover best practices for data security and privacy guarantees for the private industry (to encourage data sharing).
Centralized data repository
NCFRP Report 35: Implementing the Freight Transportation Data Architecture: Data Element Dictionary 13 cites the need for a central data collection repository. This would allow locally-collected or project-specific data to be stored or shared with other data users in the transportation community. These project-specific data sources could complement currently available freight data sources as well as provide additional opportunities to test or validate freight-related models. With knowledge of existing challenges associated with freight data use and privacy concerns, implementing a centralized data collection repository may be a challenge. However, there is an opportunity to implement a data repository focused on publishing data elements and guidance for the integration and use of freight data. An example of such a repository was developed as part of the NCFRP Report 35 project, which was called the Freight Data Dictionary. Now housed with the National Transportation Library (NTL), the Freight Data Dictionary is currently outdated and needs to include new private and big data sources. Without a central leadership of freight data experts, users, and practitioners, maintaining the library will pose an ongoing challenge for the NTL.
Funding to support a collaborative freight data program
As agencies continue to remain siloed in terms of operations, management, and funding sources, there is a need for a pooled fund program to support the proposed needs listed above. Specifically, for an organization or entity to provide leadership, be responsible for developing data standards, and create a platform for public and private sector data exchange, it needs to be well funded. An example of such a program is the World Wide Web Consortium. 10 W3C is an international community of member organizations, full-time staff, and the general public working together to develop Web standards such as HTML, CSS, PNG images, Web scripting, and dynamic content (e.g. audio and video). Despite its reliance on volunteers and industry-experts, W3C’s full-time paid staff ensures the day to day operations of the consortium. A similar model could be adopted by state agencies, TRB, and private sector organizations for developing standards regarding the collection, processing, sharing, and use of freight data for planning and operations purposes.
- 1(AASHTO, 2018) - AASHTO. (2018). Strategic_ Plan for the AASHTO Special Committee on Freight. AASHTO. Retrieved from https://www.transportation.org/wp-content/uploads/2019/03/Freight-Strategic-Plan-2018-Approved.pdf
- 2Development, N. R. (2003). A Concept for a National Freight Data Program. Transportation Research Board of the National Academies.
- 3FHWA. (2020, April 8). Freight Demand Modeling and Data Improvement: A Strategic Roadmap for Making Better Freight Investments. Retrieved from FHWA Freight Management and Operations: https://ops.fhwa.dot.gov/freight/freight_analysis/fdmdi/index.htm
- 4USDOT. (2017). Freight Demand Modeling and Data Improvement Handbook. Washington, DC: Federal Highway Administration Office of Operations.
- 5USDOT. (2017). Freight Demand Modeling and Data Improvement Handbook. Washington, DC: Federal Highway Administration Office of Operations.
- 6Seedah, D. P., Sankaran, B., & O'Brien, W. J. (2015). Approach to classifying freight data elements across multiple data sources. Transportation Research Record 2529 No. 1, 56-65.
- 7Oosterom, P. v., & Zlatanova, S. (2008). Creating Spatial Information Infrastructures: Towards the Spatial Semantic Web. CRC Press - Taylor and Francis Group.
- 8International Organization for Standardization. (2011). Intelligent Transport Systems Geographic Data Files 14825. Geneva: ISO. https://www.iso.org/obp/ui/#iso:std:iso:14825:ed-2:v1:en
- 9Google. (2019, October 4). Google Developers. Retrieved July 24, 2020, from GTFS Static Overview: https://developers.google.com/transit/gtfs
- 10 a b World Wide Web Consortium. (2020, February 4). Data Catalog Vocabulary (DCAT) - Version 2. Retrieved from World Wide Web Consortium: https://www.w3.org/TR/vocab-dcat-2/
- 11Seedah, D. (2014). Retrieving Information from Heterogeneous Freight Data Sources to Answer Natural Language Queries. Retrieved June 2021: http://hdl.handle.net/2152/28341
- 12USDOT. (October 2016). Traffic Monitoring Guide. Retrieved June 2021 from https://www.fhwa.dot.gov/policyinformation/tmguide/tmg_fhwa_pl_17_003.pdf
- 13Walton, C. M., Seedah, D. P., Choubassi, C., Wu, H., Ehlert, A., Harrison, R., & Loftus-Otway, L. (2015). NCFRP 35: Implementing the Freight Transportation Data Architecture: Data Element Dictionary. Washington, DC: National Academies of Science. Retrieved June 4, 2020, from https://www.nap.edu/catalog/21910/implementing-the-freight-transportation-data-architecture-data-element-dictionary