Section 7.4
Data Governance and Management
This section provides an overview of data governance and management practices that are essential for achieving data quality, consistency and integration. These practices are applicable to all kinds of agency data, and are best implemented at the agency-wide level. After introducing general concepts and principles, the section highlights specific governance and management applications for TAM. It concludes with a synthesis of available tools for assessing data governance and management maturity levels.
Data Governance and Management
This section provides an overview of data governance and management practices that are essential for achieving data quality, consistency and integration. These practices are applicable to all kinds of agency data, and are best implemented at the agency-wide level. After introducing general concepts and principles, the section highlights specific governance and management applications for TAM. It concludes with a synthesis of available tools for assessing data governance and management maturity levels.
7.4.1
Fundamental Concepts and Principles
Data governance and management practices are essential for achieving reliable, consistent, integrated and accessible data that is of value for decision-making. Several definitions, concepts and principles are important to understand before embarking on a data governance initiative.
Data Management and the TAM Data Assistant
Data governance and data management are interrelated but distinct practices.
Data management includes activities such as data quality management, data documentation, metadata management, security and access controls, data integration, and data archiving.
Data governance is a policy making and oversight function for data management. Implementing data governance involves forming and chartering decision making bodies, defining roles and responsibilities, establishing policies that set expectations for behavior, and setting up standard processes for things like approving data standards, resolving data issues, and acquiring new types of data. Data governance is generally implemented in a hierarchical fashion, with an executive body at the top, a data council or board in the middle, and several more focused groups oriented around specific systems, business processes, organizational units or functions.
Data stewardship is closely related to data management and governance. It refers to established responsibilities and accountabilities for managing data. In general parlance, a steward is someone who is entrusted with the responsibility for taking care of someone else’s property. Similarly, a data steward is someone who takes care of data on behalf of their agency. Different types of stewardship roles can be defined and formalized within an agency data governance policy. Data stewardship can be viewed as the way to operationalize data governance policies, processes and standards.
Data governance can be implemented to:
- Improve quality and consistency of data
- Ensure coordination across different business units
- Maximize efficiency in data collection and management processes
- Enable data integration and shared solutions to make the most of available IT resources
- Ensure there is a solid business case for new data collection
- Ensure that data will be maintained once it is collected
Agencies may be motivated to establish a formal data governance function as they try to move from a siloed approach to collecting and managing data to one that is more coordinated and centralized.
For example, implementing a reporting system that takes data from multiple sources within the agency creates the need for standardization, documentation, and agreed-upon update cycles. It is important to get agreement on standard data definitions, formats and code lists from different business units to achieve consistency. It is also important to clarify who is responsible for fixing errors and the process for error correction in the event that errors occur.
Data governance is a means to an end. It is important to clearly define and communicate why an agency needs to strengthen data governance: what is happening now that the agency may want to avoid (such as data duplication)? What is not happening now that the agency may want to achieve (such as standardized data)? The effort involved in putting data governance in place should not be underestimated, since it involves changes in how decisions are made and changes in behavior. A full scale agency data governance model can take years to mature. However, data governance can be rolled out incrementally to focus on short term objectives. It is a good idea to adopt a set of principles to provide the foundation for data governance policies and practices. The AASHTO Data Principles (see callout box) can be used as a model.
TIP
Data itself should be viewed as an asset to be managed.
Florida DOT
Florida Department of Transportation (FDOT) launched a statewide initiative to better manage and integrate agency data. This effort combines the resources, goals, and objectives of Florida’s Technology and Operation Divisions into the initiative known as ROADS, which stands for:
- R—Reliable, accurate, authoritative, accessible data
- O—Organized data that produces actionable information
- A—Accurate governance-produced data
- D—Data and technology integration
- S—Shared agency data to perform cross-functional analysis
The agency has created processes, procedures, and guidelines so that all data (financial, safety, project, program, assets, etc.) are organized and accessible. Florida’s steering committee, known as RET (ROADS Executive Team), is led by the agency’s Chief of Transportation Technology and Civil Integrated Management Officer. The committee, which includes district secretaries, financial and planning executives, and operational directors, is charged with governance leadership and instituting processes that will change the culture of the agency by converting data to knowledge.
ROADS is being implemented incrementally, through a series of 6-month initiatives. One initiative related to asset management is to standardize inventory attributes for 120 different classes of infrastructure assets and the agency’s approximately 170 enterprise software applications. Part of this effort is to determine specific authoritative source data to include in a new data warehouse. The data warehouse will provide a single authoritative site for sharing the accurate data.
Through the ROADS initiatives, Florida DOT has created a strategic direction for data integration covering data stewards, division responsibilities, asset inventory, business system integration, and an implementation roadmap. By coordinating its efforts, the agency is able to maximize the value of its data while streamlining processes for data collection, management, and dissemination.
Florida DOT Enterprise Information Management
Source: Florida DOT. 2019
7.4.2
Data Governance Practices Supporting TAM
Data governance practices can be implemented to support development of a valuable, reliable base of integrated information for TAM decision making.
A first step in data governance is to identify key decision points to be governed. These may include:
- Adopting common data definitions or standard code lists
- Adopting location referencing standards
- Adopting standard tools for field data collection
- Collecting new asset data to be included within an integrated asset management system
- Archiving or deleting existing data
- Modifying data elements for an existing TAM data source
- Adding new data layers to an enterprise GIS repository
- Adding new data marts to a data warehouse
- Adding new reports or controls to a BI environment
- Responding to an external request for data
It is best to take an incremental approach to setting up governance processes, starting with a few high impact areas that are aligned with what the agency is trying to achieve. For each of the selected decisions to be governed, think both about the criteria or guidelines to be followed as well as all the people who should be consulted or involved in making the decision.
TIP
Data governance practices should involve stakeholders responsible for collecting and analyzing the data, as well as those who will be using the data in decision making.
- Criteria and Guidelines: Developing guidelines for key decisions is a good way to institutionalize practices that reflect the agency’s goals for data. For example, some agencies have established “readiness checklists” that need to be completed before data can be added to an enterprise repository. These ensure (among other things) that a data owner or point of contact has been identified, that necessary metadata is provided, that a refresh cycle has been specified, and that the authoritative source system of record has been identified.
- Decision Making Process: Consider who should be involved in each of these decisions – who is responsible for making technical recommendations, who should be consulted, who has approval authority, and who needs to be informed about the decision. Define a process for resolving issues and conflicts; and a process for granting exceptions to established standards.
Agency data governance bodies can be responsible for adopting both guidelines and process flows impacting decisions that impact multiple business functions. If there are no existing governance bodies or if decisions to be governed are specific to TAM, a separate TAM data governance group can be established.
Keep in mind that the function of governance bodies is to make decisions. Use technical advisory groups, working groups or communities of interest to do the collaborative work required to develop standards or make recommendations about changes to data and systems.
Connecticut DOT
CTDOT established a data governance structure with an initial focus on creating a Transportation Enterprise Database (TED). The vision for the TED is to:
“Create an accessible transportation safety and asset data enterprise system where authoritative data sets are managed by data stewards and formatted for consumption and analysis in a manner that allows stakeholders to use tools that are both effective and meet their business needs.”
CTDOT’s Data Governance Structure is made up of:
- An Executive Oversight Committee, chaired by the agency Chief of Staff, with membership consisting of the agency’s bureau chiefs.
- A Data Governance Council, with members representing key agency functions including Policy and Planning, Asset Management, Engineering and Construction, Maintenance, Traffic, Safety Management, Finance and Administration, Public Transportation, and Information Technology. CTDOT uses consultants to facilitate.
- Data owners and stewards.
The initial charge of the Data Governance Council was to “Prioritize safety and asset data governance solutions to provide the foundational tools necessary to expand enterprise data participation across all disciplines within the agency.” The Data Council is responsible for:
- Identifying data being collected and maintained agency wide.
- Documenting data standards and coordinate development of new standards.
- Developing guidance for data dictionaries, user manuals, and training programs.
- Establishing quality control/quality assurance (QC/QA) processes.
- Facilitating the integration and interoperability of information between authoritative roadway inventory databases and the Department’s enterprise wide data system.
- Informing the Executive Committee of emerging data priorities and how they best might be addressed.
- Reporting to the Executive Oversight Committee as needed to make recommendations regarding data governance challenges or technology opportunities.
Data Owners:
- Have supervisory, administrative, and technical control over a dataset.
- Are responsible for the oversight of the collection, storage, maintenance, and implementation of business rules / managing its use including rules for how data will be exposed for general public consumption.
- Ensure access to the data asset is authorized and controlled.
Data Stewards are responsible for the management of data assets on a day to day basis in terms of content, update and data extract processes, data migration to TED and for the development of metadata. They ensure that:
- There are documents highlighting the origin and sources of authoritative data and completes each metadata element.
- Data has a collection and a maintenance cycle defined.
- Data quality processes are in place.
- Data is protected against unauthorized access or change.
Ohio DOT
Ohio DOT has established a standard process for adding a new asset to their inventory. As illustrated in the flowchart below, the process has three stages – (1) Asset Overview, where the request is submitted, evaluated, and approved, (2) Requirements, in which business and technical requirements for collecting and managing the new data are documented, and (3) Application Development, where the technology solution is developed either in-house (using standard tools), via contract (for custom development) or through acquisition of a commercial off-the-shelf (COTS) package.
As part of the TAM Audit Group workflow shown in the figure, ODOT has introduced over 693,000 active ancillary assets into their inventory.
Ohio DOT TAM Audit Group Workflow Diagram
Source: Ohio DOT. 2019
7.4.3
Assessing Data Management and Governance Maturity
Data management and governance implementation can be viewed as a long term process of maturation. Several models and assessment tools are available to help agencies identify their current state, set goals for where they want to be, and create plans for moving up the maturity scale.
There are several different assessment tools tailored to DOT data programs that can be used or adapted as needed. In addition, several DOTs have created their own tools. Most of these tools are based on a maturity model.
A typical maturity model could include the following levels:
- Level 1-Initial
- Level 2-Repeatable processes
- Level 3-Defined and documented processes
- Level 4-Measured and managed processes
- Level 5-Optimizing processes (continuous improvement)
TIP
Use a maturity model to identify gaps, prioritize initiatives and track progress over time.
For TAM information and systems, maturity levels can be assigned to different aspects of data management and governance. Assessments can also be conducted at different levels of the organization – from the agency-wide level, to the level of individual information systems (or even data elements).
Figure 7.5 Example Maturity Model
Table 7.6 shows the data management and information system-related assessment elements from the TAM Gap Analysis Tool, developed under NCHRP Project 08-90.
Table 7.6 - TAM Analysis Tool Assessment Elements
| Element | Sub-element | Sample Assessment Criteria |
|---|---|---|
Data Management| Asset Inventory |
| |
| Asset Condition and Performance |
|
|
| Data Governance |
|
|
| Information Systems | System Technology and Integration |
|
| Decision-Support Tools |
|
|
| System Features |
|
Figure 7.6 illustrates the data assessment guidance created under NCHRP 08-92. This process is suitable for application either at the agency-wide level, for an individual data program, or for a business process. It goes into greater depth than the TAM Gap Analysis Tool.
Figure 7.6 Folio Describing the Transportation Agency Data Self-Assessment Process
Iowa DOT
Iowa DOT conducted a detailed data maturity assessment for over 180 data systems. Assessments were based on a standardized questionnaire administered to data stewards and custodians. The questions covered data quality, availability of metadata, whether a data retention plan was in place, the degree to which data collection was automated, and several other factors. Charts were produced showing maturity scores for each system, with roll-ups at the division level. This tool helps the agency track their progress over time and identify specific data improvements to pursue.
Sample Data Assessment Summary Radar Chart
Source: Iowa DOT. 2019
AASHTO
AASHTO Digital TAM Guide Update Hackathon — Kickoff Meeting
Florida DOT
Florida Department of Transportation (FDOT) launched a statewide initiative to better manage and integrate agency data. This effort combines the resources, goals, and objectives of Florida’s Technology and Operation Divisions into the initiative known as ROADS, which stands for:
- R—Reliable, accurate, authoritative, accessible data
- O—Organized data that produces actionable information
- A—Accurate governance-produced data
- D—Data and technology integration
- S—Shared agency data to perform cross-functional analysis
The agency has created processes, procedures, and guidelines so that all data (financial, safety, project, program, assets, etc.) are organized and accessible. Florida’s steering committee, known as RET (ROADS Executive Team), is led by the agency’s Chief of Transportation Technology and Civil Integrated Management Officer. The committee, which includes district secretaries, financial and planning executives, and operational directors, is charged with governance leadership and instituting processes that will change the culture of the agency by converting data to knowledge.
ROADS is being implemented incrementally, through a series of 6-month initiatives. One initiative related to asset management is to standardize inventory attributes for 120 different classes of infrastructure assets and the agency’s approximately 170 enterprise software applications. Part of this effort is to determine specific authoritative source data to include in a new data warehouse. The data warehouse will provide a single authoritative site for sharing the accurate data.
Through the ROADS initiatives, Florida DOT has created a strategic direction for data integration covering data stewards, division responsibilities, asset inventory, business system integration, and an implementation roadmap. By coordinating its efforts, the agency is able to maximize the value of its data while streamlining processes for data collection, management, and dissemination.
Florida DOT Enterprise Information Management
Source: Florida DOT. 2019
Connecticut DOT
CTDOT established a data governance structure with an initial focus on creating a Transportation Enterprise Database (TED). The vision for the TED is to:
“Create an accessible transportation safety and asset data enterprise system where authoritative data sets are managed by data stewards and formatted for consumption and analysis in a manner that allows stakeholders to use tools that are both effective and meet their business needs.”
CTDOT’s Data Governance Structure is made up of:
- An Executive Oversight Committee, chaired by the agency Chief of Staff, with membership consisting of the agency’s bureau chiefs.
- A Data Governance Council, with members representing key agency functions including Policy and Planning, Asset Management, Engineering and Construction, Maintenance, Traffic, Safety Management, Finance and Administration, Public Transportation, and Information Technology. CTDOT uses consultants to facilitate.
- Data owners and stewards.
The initial charge of the Data Governance Council was to “Prioritize safety and asset data governance solutions to provide the foundational tools necessary to expand enterprise data participation across all disciplines within the agency.” The Data Council is responsible for:
- Identifying data being collected and maintained agency wide.
- Documenting data standards and coordinate development of new standards.
- Developing guidance for data dictionaries, user manuals, and training programs.
- Establishing quality control/quality assurance (QC/QA) processes.
- Facilitating the integration and interoperability of information between authoritative roadway inventory databases and the Department’s enterprise wide data system.
- Informing the Executive Committee of emerging data priorities and how they best might be addressed.
- Reporting to the Executive Oversight Committee as needed to make recommendations regarding data governance challenges or technology opportunities.
Data Owners:
- Have supervisory, administrative, and technical control over a dataset.
- Are responsible for the oversight of the collection, storage, maintenance, and implementation of business rules / managing its use including rules for how data will be exposed for general public consumption.
- Ensure access to the data asset is authorized and controlled.
Data Stewards are responsible for the management of data assets on a day to day basis in terms of content, update and data extract processes, data migration to TED and for the development of metadata. They ensure that:
- There are documents highlighting the origin and sources of authoritative data and completes each metadata element.
- Data has a collection and a maintenance cycle defined.
- Data quality processes are in place.
- Data is protected against unauthorized access or change.
Ohio DOT
Ohio DOT has established a standard process for adding a new asset to their inventory. As illustrated in the flowchart below, the process has three stages – (1) Asset Overview, where the request is submitted, evaluated, and approved, (2) Requirements, in which business and technical requirements for collecting and managing the new data are documented, and (3) Application Development, where the technology solution is developed either in-house (using standard tools), via contract (for custom development) or through acquisition of a commercial off-the-shelf (COTS) package.
As part of the TAM Audit Group workflow shown in the figure, ODOT has introduced over 693,000 active ancillary assets into their inventory.
Ohio DOT TAM Audit Group Workflow Diagram
Source: Ohio DOT. 2019
Iowa DOT
Iowa DOT conducted a detailed data maturity assessment for over 180 data systems. Assessments were based on a standardized questionnaire administered to data stewards and custodians. The questions covered data quality, availability of metadata, whether a data retention plan was in place, the degree to which data collection was automated, and several other factors. Charts were produced showing maturity scores for each system, with roll-ups at the division level. This tool helps the agency track their progress over time and identify specific data improvements to pursue.
Sample Data Assessment Summary Radar Chart
Source: Iowa DOT. 2019
AASHTO
AASHTO Digital TAM Guide Update Hackathon — Kickoff Meeting