Governance Layers: Making Data Discoverable, Trustworthy, and Secure

Data governance is the set of policies, processes, organizational structures, and technical capabilities that ensure data assets are managed as a strategic resource. A governance layer within a data architecture encompasses the systems and practices that make data discoverable — so analysts can find what they need — trustworthy — so consumers can rely on the data they find — and secure — so access is appropriately controlled and auditable.

Governance is not a project with a completion date. It is an ongoing operational discipline that requires sustained investment in tooling, process, and organizational accountability. Organizations that treat governance as a one-time compliance exercise typically find that their data environments degrade in quality and trustworthiness over time.

Data Cataloging

A data catalog is a centralized registry of data assets — datasets, tables, columns, pipelines, reports, and APIs — enriched with metadata that describes the content, quality, ownership, and usage of each asset. Effective catalogs reduce the time analysts spend searching for data and increase confidence in the data they find.

Technical Metadata

Technical metadata is extracted automatically from source systems and includes schema information (table names, column names, data types), statistics (row counts, cardinality, null rates), and physical location (database, schema, table path in object storage). Automated metadata extraction through catalog connectors enables continuous synchronization between the catalog and the actual state of data assets.

Business Metadata

Business metadata is authored by humans and includes dataset descriptions, column-level annotations explaining what a field represents, business glossary terms, and data classification labels (confidential, PII, internal). Business metadata requires active contribution from data owners and stewards and typically accumulates gradually through a combination of manual authoring and annotation workflows.

Enterprise catalog platforms commonly deployed in Canadian organizations include Alation, Collibra, Atlan, AWS Glue Data Catalog, and Microsoft Purview (formerly Azure Purview). Each platform makes different trade-offs between automated metadata extraction, search experience, glossary management, and integration depth with specific cloud platforms.

Data Lineage

Data lineage tracks the flow of data from its origin through every transformation and aggregation step to its final destination in reports or applications. Lineage serves multiple governance functions: it enables impact analysis (which downstream consumers will be affected by a change to a source?), root cause analysis (which upstream transformation introduced this data quality issue?), and regulatory compliance documentation (where did this reported figure originate?).

Lineage is captured at two granularities: column-level lineage, which tracks individual field transformations through SQL queries and pipeline logic, and dataset-level lineage, which records which datasets were read and written by each pipeline job. Column-level lineage is more operationally useful but significantly harder to capture automatically, particularly for complex transformations or transformations expressed in general-purpose programming languages rather than SQL.

The OpenLineage specification provides an open standard for capturing and emitting lineage events from data pipelines. Integration with orchestration platforms (Airflow with the OpenLineage provider, Dagster's asset-based lineage model) and transformation tools (dbt's lineage graph) enables automatic lineage population without manual documentation effort.

Access Control Architecture

Access control in a data platform governs who can read, write, and modify which data assets. The appropriate access control model depends on the sensitivity of the data, the organizational structure, and the regulatory context.

Role-Based Access Control (RBAC) assigns permissions to roles rather than individual users, and then assigns roles to users. This approach scales to large organizations where individual permission management would be impractical. Common roles in a data platform include analyst (read access to curated datasets), engineer (read-write access to pipeline staging areas), and administrator (full platform access).

Attribute-Based Access Control (ABAC) extends RBAC by evaluating access decisions against a combination of user attributes, resource attributes, and environmental conditions. ABAC enables fine-grained access policies such as restricting access to records belonging to a specific business unit or geographic region based on the requesting user's organizational attributes.

Column-level security restricts visibility of specific columns (such as columns containing personal information or salary data) to authorized roles. Row-level security applies filter predicates automatically based on the user's identity, so a regional manager querying a sales table sees only records for their assigned region. Both capabilities are supported natively in major cloud warehouse platforms.

Data Quality Management

Data quality management encompasses the processes and technical controls that detect, remediate, and prevent data quality issues. The dimensions of data quality commonly evaluated include completeness (required fields are populated), accuracy (values are correct), consistency (values are consistent across systems), timeliness (data is available within expected windows), and uniqueness (no duplicate records where uniqueness is expected).

Quality rules are implemented as assertions evaluated against datasets — either within the pipeline (as pre- and post-transformation checks) or as separate monitoring jobs. Tools such as dbt tests, Great Expectations, and Soda Core provide frameworks for defining, executing, and reporting on data quality assertions. Results are typically surfaced in dashboards and trigger alerts when thresholds are breached.

Regulatory Compliance in Canada

Canadian organizations managing personal information are subject to PIPEDA federally and provincial privacy laws in Quebec, Alberta, and British Columbia. Quebec's Law 25 introduced requirements for Privacy Impact Assessments for new data processing activities, mandatory breach notification within 72 hours for incidents involving significant risk, and enhanced individual rights including the right to data portability and erasure. These obligations have direct implications for data architecture, including the need for systems capable of locating and deleting individual records across distributed data stores.

Financial institutions regulated by OSFI are subject to Guideline B-10 on Third-Party Risk Management and Guideline E-21 on Operational Risk and Resilience, both of which include requirements for data management practices, including governance documentation and vendor oversight for cloud data platforms. The governance layer of an enterprise data architecture serves as a key control environment for satisfying these regulatory obligations.