Timbr’s native integration with Databricks lets teams define ontology-based semantic models directly within familiar Databricks workflows. Instead of stopping at cleaned tables and pipelines, teams can add the business layer that makes data easier to consume: concepts, relationships, measures, and rules.
Databricks excels at scalable data processing, enabling teams to build reliable pipelines across large datasets. Delta Live Tables further supports this process by providing a framework for maintainable data pipelines, data quality checks, orchestration, and Delta Lake reliability.
Timbr extends this workflow by adding semantic modeling as a governed layer above the data pipeline. Business context, relationships, measures, and rules can be defined declaratively, making the output of Databricks pipelines easier to query, reuse, and understand across notebooks, SQL endpoints, BI tools, applications, and AI agents.
This integrated approach helps teams:
Define business meaning once: Concepts, relationships, measures, and rules can be modeled once and reused across Databricks workflows and downstream tools.
Reduce duplicated logic: Business rules can be updated in the semantic model without rebuilding every notebook, dashboard, or application.
Preserve context across the pipeline: Cleaned and transformed data remains connected to the business definitions and relationships that explain how it should be used.
Govern consumption: Semantic definitions can align with Databricks Unity Catalog, helping teams keep access, permissions, and policy controls consistent.
With Timbr, data engineers can extend Databricks pipelines with governed semantic models, connecting cleaned data to business concepts, relationships, measures, and rules that downstream users and AI agents can reuse.
- Intuitive visual UI (no-code).
- SQL-based declarative modeling capabilities.
For users that prefer SQL coding, Timbr offers SQL extensions for modeling complex structures like Knowledge Graphs (ontologies) and multi-dimensional data (OLAP Cubes).
SQL extensions examples:
CREATE CONCEPT in Timbr is an extension of CREATE TABLE statement.
CREATE MAPPING in Timbr is an extension of CREATE VIEW statement.
RELATIONSHIP CONSTRAINT in Timbr is an extension of FOREIGN KEY definition to represent relationships.
Timbr’s support of SQL modeling enables Databricks users to integrate data modeling into ETL pipelines written in SQL or directly in Databricks, ensuring consistency and efficiency in data processing and semantic integration.
The following example shows how to build, deploy, and run Delta Live Tables in Databricks notebooks and create semantic models as part of the pipeline.
We shall use the “Retail Sales” from Delta Live Tables Example Notebooks
First, we create two base streaming tables:
- Customers table from CSV files
- Sales orders raw table from JSON files
Streaming tables in Databricks enable real-time data processing by ingesting and transforming streaming data into Delta tables. They provide automatic handling of data consistency, schema evolution, and incremental data processing, ensuring reliable and up-to-date data for analytics and reporting.
We also need to clean and transform the Sales orders raw table so it will be optimized for analysis:
- Date data type casting from Unix time of order-to-order date and order timestamp.
- Use EXPLODE function to UNNEST the order products data as it was originally nested in the JSON:[{“curr”:”USD”,”id”:”AVpfuJ4pilAPnD_xhDyM”,”name”:”Rony LBT-GPX555 Mini-System with Bluetooth and NFC”,”price”:993,”promotion_info”:null,”qty”:3,”unit”:”pcs”}]
We can validate that the tables definition is correct and run the job in Databricks to create the tables to be accessible in Unity Catalog:
Once we have the two tables cleaned, standardized and optimized for queries (In Delta format and not CSV/JSON) we can start modeling in Timbr.
Using Timbr’s native integration with Databricks, we can perform the data modeling directly from the Databricks notebook.
The first step is to create the business concepts that represent the tables we just created:
We created two concepts: customers and sales orders in Timbr SQL DDL statements. You can find additional information on Timbr SQL reference
Sales orders:
CREATE OR REPLACE CONCEPT `sales_orders` (
`customer_id` string,
`customer_name` string,
`number_of_line_items` string,
`ordered_product` string,
`order_date` date,
`order_datetime` timestamp,
`order_number` bigint ,
PRIMARY KEY(`order_number`)) INHERITS (`thing`);
Customers:
CREATE OR REPLACE CONCEPT `customers` (
`city` string,
`customer_id` string,
`customer_name` string,
`district` string,
`loyalty_segment` string,
`postcode` string,
`region` string,
`ship_to_address` string,
`state` string,
`street` string,
CONSTRAINT `has_sales_orders` FOREIGN KEY (`customer_id`) REFERENCES `sales_orders` (`customer_id`) INVERSEOF `of_customers`,
PRIMARY KEY(`customer_id`), LABEL(`customer_name`)) INHERITS (`thing`);
The CREATE CONCEPT statement is an extension of a CREATE TABLE as it supports both column definition to model properties, and foreign key constraints to define relationships. This makes it natural for SQL users who are familiar with CREATE TABLE statements.
In addition. we created classification for orders based on the city of the customer:
CREATE OR REPLACE CONCEPT sales_orders_in_la INHERITS (sales_orders) from dtimbr.sales_orders WHERE of_customers[customers].city = 'Los Angeles';
CREATE OR REPLACE CONCEPT sales_orders_in_chicago INHERITS (sales_orders) from dtimbr.sales_orders WHERE of_customers[customers].city = 'Chicago';
By leveraging the relationship, we created between customers concept (has_sales_orders) and sales_orders (of_customers) we can easily traverse the data model without writing a single JOIN.
Inheritance in Timbr allows teams to define business rules and classifications as part of the semantic model, making business terms reusable across Databricks workflows.
The last part is to map the data to the concepts (this can also be done automatically in the Timbr UI if you already have tables defined in Unity):
CREATE OR REPLACE MAPPING `map_customers` INTO `customers` AS SELECT * FROM `retail_demo`.`default`.`customers`;
CREATE OR REPLACE MAPPING `map_sales_orders` INTO `sales_orders` AS SELECT * FROM `retail_demo`.`default`.`sales_orders`
The Timbr CREATE MAPPING statement is an extension of SQL CREATE VIEW as it allows you to define a query to map a source table to a target concept.
Users can map multiple tables to the same concept from different schemas, catalogs, databases, or connected systems. Timbr handles the required UNION logic and fills null values where needed.
We can now explore our data model in the Timbr UI or directly in our Databricks notebooks using Unity catalog:
Querying the Semantic Model
Timbr creates virtual schemas to represent the ontology (semantic model) in a relational way.
The SQL extensions of Timbr only apply for modeling. Standard SQL is used for querying the semantic model. When querying the virtual schemas created by Timbr, Timbr pushes down the query in the SQL dialect and functions of the underlying DB connected to Timbr (in our case Databricks SQL dialect and functions).
When querying the virtual tables created by Timbr, users don’t need to explicitly write JOINs or UNION statements anymore as Timbr automatically infers which JOINs or UNIONs are needed based on the semantic model, thus reducing the SQL complexity significantly.
Users can still write JOINs and utilize any advanced SQL capability of Databricks (For example: PIVOT/Window functions). This is possible as Timbr translates the query from the virtual tables to the real tables in Databricks defined in the Timbr mappings.
This allows users to query the semantic data model directly from the Databricks notebook:
The Timbr query leveraged the relationship between sales_orders and customers so no JOIN was needed.
Behind the scenes, Timbr generates a query on the cleaned tables with a JOIN and sends it to be executed on Databricks directly.
Summary
This end-to-end workflow shows how Databricks pipelines and Timbr semantic modeling work together. Databricks provides the scalable foundation for ingestion, transformation, and processing. Timbr adds the ontology-based semantic layer that turns cleaned data into reusable business concepts, relationships, measures, and rules.
By defining semantic models directly within Databricks workflows, teams can move beyond raw tables and technical schemas. Analysts, engineers, BI users, applications, and AI agents can query governed business concepts instead of manually reconstructing joins, rules, and metric logic for every use case.
The result is a more reusable and governed path from ETL to business meaning – with Databricks powering the data pipeline and Timbr making the output easier to understand, query, and consume.
