Enterprise Scale | |
Permissions |
|
Platform(s) |
|
Related reading |
HyperDB is a powerful feature that allows Enterprise Scale customers to work with large datasets (up to 100M rows) in a new storage layer outside of a base. HyperDB tables can be populated from CSV files or connected directly to other cloud sources, making it easy to bring your important data into Airtable. Builders can access the data stored in HyperDB by using published data sets in the Data Library. New features like on-demand sync allow your organization to operationalize massive datasets across your complex workflows.
HyperDB prerequisites
Before using HyperDB, ensure you have:
An Enterprise Scale plan
Enterprise Admin or Integration Admin permissions
Data in CSV format or access to Snowflake, Salesforce, or other supported integrations (if using external integrations)
A primary key field that is unique per row in your dataset
Data with no more than 100M rows
HyperDB is not yet available for our European data residency customers
Getting Started with HyperDB
To create a HyperDB table:
Open your admin panel.
Click HyperDB.
Click Add table.
Select the org unit you want to use.
Uploading CSVs:
Upload your CSV.
Create the schema for your table.
The system will attempt to detect appropriate field types, but you can modify these as needed.
Important: Once you choose field types, you cannot change them in HyperDB, so ensure you’ve gotten the schema correct
Select a primary column (unique ID field).
Select a secondary search column (this should be human-readable and will enable users to perform friendly name searches when linking records via on-demand sync).
Choose a name and description for your HyperDB table.
Wait for your dataset to process. You can return to the specific HyperDB table’s page to check the status of your import. This may take some time depending on the size of your table.
Publishing data sets to your data library
To create data sets:
Click Published data sets.
Click Create data set.
Choose which fields you want included in the data set
Optional: Define filters to scope your data set to a specific slice of your data. You can also review how large the data set is with the filters. You can publish a data set of any size, but we tell you if the data set is larger than what can be directly synced into a base.
Choose if you want to allow downstream syncs of the data.
If you choose not to allow downstream syncs, end users will not be able to daisy-chain syncs to other bases.
Review the preview of the data set to make sure it’s what you expect.
Choose a name and description for the data set.
To view or manage published data sets:
Open your admin panel.
Click HyperDB.
All published data sets from a HyperDB table are displayed here.
Click More details to see which bases are syncing your published data.
(Optional): Click the … icon at the end the data data you modify.
Using HyperDB in a base
After creating a HyperDB table and published a data set, collaborators can access it through the data library in their bases.
To use a HyperDB table in a base:
Open your Airtable homepage.
Open the base where you want to use HyperDB data.
Click + Add or import.
Builders who are in the admin's defined audience can find published data sets in the data library.
Locate and select your HyperDB data located under the “Add from data library” section.
After selecting your data, a preview window opens.
Click Add verified data.
There are two primary ways to use a HyperDB data set in a base:
Option 1: Builder-based filtered sync allow collaborators to:
Pull in the entire dataset—with builder-defined filters—if the existing dataset is too large or if there's a more relevant slice of data.
Create complex filters to sync the specific data set relevant to your application.
Sync data into Airtable based on your filter criteria when data is updated in the HyperDB table.
Adjust filter settings by modifying the "Sync configuration" in the table settings.
Option 2: On-demand sync
Open your admin panel.
Click Data sets.
Select the … icon next to the table’s name you want to use.
You can choose to update the existing table, or overwrite the entire table.
Upload your new CSV file.
A preview window opens, allowing you to confirm
Click Next.
Adding and removing fields via the HyperDB in admin panel
You can add or remove fields from a HyperDB table by choosing the “Edit fields” dropdown. You can choose what type of field to add and the new name of the field on the next page.
Right-clicking on a field in the table preview will allow you to delete or rename the field.
When deleting fields you will see a warning of which bases are syncing this data into their bases. You might want to consider contacting the owner of those bases before making changes to the data set
Updating the table via CSV will allow you to provide a superset of fields for your table. Any new fields present in the updated CSV will be added to the table. In addition, any field re-ordering present in the updated CSV will be respected by the updated table.
HyperDB limitation and dependencies
HyperDB tables require the Enterprise Scale plan
Data must be under 100M rows
Cannot change field types after table creation
Single select fields are limited to 1,000 options
Cannot remove single select options after creating a HyperDB table
FAQs
What is the limit to the number of HyperDB tables I can create?
5 HyperDB tables are included with Enterprise Scale plans. Additional tables are available as a paid add-on via our sales team.
Can I update my HyperDB table automatically?
Currently, there's no automated way to keep CSV-based HyperDB tables up to date. However, you can use the API to update data or connect directly to Snowflake or Salesforce for regular refreshes. For CSV-based tables, you can replace all data or upload a differential update.
Can I access HyperDB data through the API?
Yes, HyperDB tables can be accessed through the Airtable API.
I’m having an issue with HyperDB, how do I contact support?
Contact Airtable Support by clickingon the lower right corner.
Within the body of your request, please include the HyperDB Table ID which is helpful for our team to troubleshoot potential issues. This ID can be found by following these steps:
Navigate to your admin panel on a browser.
Click the HyperDB tab.
Search or browse for the table where you are experiencing issues and click the table name to view more details.
In your browser’s address bar copy the last portion of the URL from
edt
on so that you have the full ID that will look like thisedtxxxxxxxxxxxxxx
.Include that ID in your message to support.