Skip to main content

Metabase

Overview​

Metabase is a business intelligence and analytics platform. Learn more in the official Metabase documentation.

The DataHub integration for Metabase covers BI entities such as dashboards, charts, datasets, and related ownership context. It also captures table-level lineage and stateful deletion detection.

Concept Mapping​

Source ConceptDataHub ConceptNotes
"Metabase"Data Platform
DashboardDashboard
Card/QuestionChart
ModelDatasetSubTypes ["Metabase Model"]; transforming models also get "View"
CollectionTag and ContainerTags optional; nested collections keep parent containers
Database TableDatasetFrom connected database
UserUser (a.k.a CorpUser)Ownership information

Module metabase​

GA

Important Capabilities​

CapabilityStatusNotes
Detect Deleted Entities✅Enabled by default via stateful ingestion.
Platform Instance✅Enabled by default.
Table-Level Lineage✅Supported by default for charts and dashboards.

Overview​

The metabase module ingests metadata from Metabase into DataHub. It is intended for production ingestion workflows and module-specific capabilities are documented below.

Prerequisites​

To use this connector, you'll need:

  • Metabase version v0.41+ (Models require v0.41+)
  • Authentication credentials (either username/password or API key — API key is recommended)
  • Appropriate permissions to access the Metabase API

Authentication​

DataHub supports two authentication methods:

  1. API Key (Recommended) — more secure, no password management required. Generate one under Account Settings → API Keys in your Metabase instance.
  2. Username/Password

Install the Plugin​

pip install 'acryl-datahub[metabase]'

Starter Recipe​

Check out the following recipe to get started with ingestion! See below for full configuration options.

For general pointers on writing and running a recipe, see our main recipe guide.

source:
  type: metabase
  config:
    # Coordinates
    connect_uri: https://metabase.company.com
    
    # Credentials (API key recommended)
    api_key: "${METABASE_API_KEY}"
    
    # Alternative: Username/Password authentication
    # username: "${METABASE_USERNAME}"
    # password: "${METABASE_PASSWORD}"
    
    # Optional: Custom display URI (if connect_uri is only for ingestion)
    # display_uri: https://metabase.company.com
    
    # Feature flags
    extract_collections_as_tags: true
    extract_models: true
    exclude_other_user_collections: false
    
    # Optional: Custom platform mappings
    # engine_platform_map:
    #   athena: glue
    #   sparksql: spark
    
    # Optional: Database name overrides
    # database_alias_map:
    #   postgres: my_postgres_db
    
    # Optional: Platform instance mappings
    # database_id_to_instance_map:
    #   "42": my_platform_instance
    # platform_instance_map:
    #   clickhouse: my_clickhouse_cluster
    
    # Default schema for SQL parsing
    default_schema: public

sink:
  # sink configs

Config Details​

Note that a . is used to denote nested fields in the YAML recipe.

FieldDescription
api_key
One of string(password), null
Metabase API key. If provided, the username and password will be ignored. Recommended method.
Default: None
connect_uri
string
Metabase host URL.
Default: http://localhost:3000
convert_lineage_urns_to_lowercase
boolean
Whether to convert dataset (table) names to lowercase when creating lineage URNs. Column names always preserve their original case to match upstream source connectors. Most DataHub connectors (including Postgres, ClickHouse, BigQuery) preserve the original case from the database. Only set to true if your upstream connector explicitly lowercases table names (rare).
Default: False
convert_urns_to_lowercase
boolean
Whether to convert dataset urns to lowercase. This value is part of each dataset's URN identity, so it must stay fixed for the life of a deployment. Changing it after data has been ingested re-keys every dataset (e.g. MyDb.MyTable becomes mydb.mytable); with stateful ingestion enabled the old-cased URNs are then soft-deleted as stale while the new-cased ones are created, producing duplicate or orphaned entities. Pick one value before the first run and leave it unchanged.
Default: False
database_alias_map
One of string, null
Database name map to use when constructing dataset URN.
Default: None
database_id_to_instance_map
One of string, null
Custom mappings between metabase database id and DataHub platform instance
Default: None
default_schema
string
Default schema name to use when schema is not provided in an SQL query
Default: public
display_uri
One of string, null
optional URL to use in links (if connect_uri is only for ingestion)
Default: None
engine_platform_map
One of string, null
Custom mappings between metabase database engines and DataHub platforms
Default: None
exclude_other_user_collections
boolean
Flag that if true, exclude other user collections
Default: False
extract_collections_as_tags
boolean
Extract Metabase collections as tags on dashboards and charts
Default: True
extract_models
boolean
Extract Metabase models (saved questions used as data sources) as datasets. Off by default: enabling it ingests models as dataset entities (and dashboards link to them as dataset edges) instead of charts, which changes their URNs.
Default: False
password
One of string(password), null
Metabase password, used when an API key is not provided.
Default: None
platform_instance_map
One of string, null
A holder for platform -> platform_instance mappings to generate correct dataset urns
Default: None
request_timeout_sec
number
Timeout in seconds for each HTTP request to the Metabase API. Prevents ingestion from hanging indefinitely on an unresponsive server.
Default: 30.0
username
One of string, null
Metabase username, used when an API key is not provided.
Default: None
env
string
The environment that all assets produced by this connector belong to
Default: PROD
stateful_ingestion
One of StatefulStaleMetadataRemovalConfig, null
Default: None
stateful_ingestion.enabled
boolean
Whether or not to enable stateful ingest. Default: True if a pipeline_name is set and either a datahub-rest sink or datahub_api is specified, otherwise False
Default: False
stateful_ingestion.fail_safe_threshold
number
Prevents large amount of soft deletes & the state from committing from accidental changes to the source configuration if the relative change percent in entities compared to the previous state is above the 'fail_safe_threshold'.
Default: 75.0
stateful_ingestion.remove_stale_metadata
boolean
Soft-deletes the entities present in the last successful run but missing in the current run with stateful_ingestion enabled.
Default: True

Capabilities​

Lineage​

The connector extracts lineage across all Metabase query types:

Native SQL​

SQL queries are parsed with DataHub's SQLGlot-based parser to extract table references, including those inside JOIN clauses and subqueries. Metabase template variables ({{variable}}, [[WHERE ...]]) are stripped before parsing.

Query Builder (MBQL)​

Questions and models built with Metabase's visual query builder store their logic as MBQL — a structured JSON representation. The connector resolves MBQL to upstream database tables and, for models, also produces column-level lineage:

  • Table-level: The source-table field and all joins[].source-table entries are resolved to DataHub dataset URNs, covering multi-table join scenarios.
  • Column-level (models only): result_metadata[].field_ref records which MBQL expression produced each output column. The connector resolves these refs to upstream field URNs via /api/field/{id}:
    • ["field", id, ...] — direct pass-through column
    • ["expression", name] — calculated column; traces back through query.expressions
    • ["aggregation", index] — metric column; traces back through query.aggregation. COUNT(*) with no explicit field fans-in all resolved upstream columns (matching Tableau lineage behaviour).
Nested Queries​

Charts or models that reference other cards (source-table: "card__456") are recursively resolved to their ultimate source tables (max depth: 5, to guard against circular references).

Dashboard Lineage​

Table dependencies from all charts in a dashboard are rolled up into direct table-to-dashboard lineage edges, deduplicating tables referenced by multiple charts.

Collection Tags​

Metabase Collections are mapped to DataHub tags:

  • Tag format: metabase_collection_{sanitized_name} (e.g. "Sales & Marketing" → metabase_collection_sales_marketing)
  • Tags are applied to dashboards, charts, and models within that collection
  • Disable with extract_collections_as_tags: false

Database and Platform Mapping​

Metabase databases are mapped to a DataHub platform based on the engine field returned by /api/database. Override with engine_platform_map:

engine_platform_map:
  athena: glue

DataHub determines the database name from the same API response. Override with database_alias_map:

database_alias_map:
  postgres: my_custom_db_name

Platform Instance Mapping​

When multiple instances of the same platform exist in DataHub (for example, two ClickHouse clusters), map Metabase database IDs to platform instances with database_id_to_instance_map:

database_id_to_instance_map:
  "42": platform_instance_in_datahub

The key must be a string, not an integer.

If database_id_to_instance_map is not set, platform_instance_map is used as a fallback. If neither is set, platform instance is omitted from dataset URNs.

Filtering Collections​

To exclude collections owned by other users:

exclude_other_user_collections: true

Limitations​

  • Column-level lineage is only available for cards saved as a Model (type "model"). Regular MBQL questions do not expose reliable field_ref data.
  • Template variables in native SQL queries ({{variable}}, [[optional clause]]) are stripped before parsing, so lineage based on dynamic table references may be incomplete.
  • Circular card references are cut off at depth 5; deeply nested card chains beyond that limit will not have lineage extracted.

Troubleshooting​

If ingestion fails, validate credentials, permissions, connectivity, and scope filters first. Then review ingestion logs for source-specific errors and adjust configuration accordingly.

  • Auth failure: ensure the API key or username/password is correct and the account has access to the relevant collections.
  • Missing lineage: check that extract_models: true is set and that the cards are saved as Models in Metabase.
  • Unknown platform warnings: add an entry to engine_platform_map for the unrecognised engine name.

Code Coordinates​

  • Class Name: datahub.ingestion.source.metabase.source.MetabaseSource
  • Browse on GitHub
Questions?

If you've got any questions on configuring ingestion for Metabase, feel free to ping us on our Slack.

💡 Contributing to this documentation

This page is auto-generated from the underlying source code. To make changes, please edit the relevant source files in the metadata-ingestion directory.

Tip: For quick typo fixes or documentation updates, you can click the ✏️ Edit icon directly in the GitHub UI to open a Pull Request. For larger changes and PR naming conventions, please refer to our Contributing Guide.