Getting started with selective sync in HyperDB

  • Updated on Jan 8, 2025
  • Published on Jan 7, 2025
  • 4 minute(s) read

Plan availability

Enterprise Scale only

Permissions

Owners/Creators - To create or update synced tables, you'll need to have creator permissions in the base where you are setting up the sync.

Platform(s)

Web/Browser, Mac app, and Windows app

Related reading

  • Getting started with Airtable sync - If you haven’t set up a sync before, we recommend first reading this article to become familiarized with how the Airtable Sync feature generally works.

  • Airtable Sync FAQs - Answers to common questions that might help unblock you if you are encountering issues.

Understanding selective syncing

Selective sync is a new way to link records on an as-needed basis, to a massive dataset powered by Airtable’s HyperDB feature. You can search over millions of records without slowing down performance.

This is done by syncing each record to your table as it is linked, instead of the typical way that sync works, which would bring in all records from a synced data set.

A GIF showing the process of linking users in a "User info" field that is selectively synced to a local table.

For example:

  1. To get started, you need to have an existing table that you want to link to a larger set of data. In the diagram below, Zelos Inc. has an existing “Support tickets” table, and wants to link the “Requester” field to the HyperDB-backed data set “All users.”
    A diagram showing an example of how selective sync pulls in data from HyperDB.

  2. They would then add the data set “All users” as a new table named “User info” by choosing the “All users” data set from the data library.

  3. During the setup process, they will use selective sync to bring in the user data, choosing to link it to the “Support requests” table and “User info” linked record field.
    An image showing the setup process of using selective sync to connect records from HyperDB to a table in a base via a linked record field.

  4. Once the “All users” data set has been connected to the selectively synced table, you can search for any record with the linked record picker. Support tickets can then be associated with any user in the “All users” dataset. The selectively synced “User info” table will only display a subset of users that have been actively linked to tickets.

    An image showing a local table connected via a linked record field to a selectively synced table.

Note

Once a record has been selectively synced into the base, it will remain there, even if the linked record is removed from the specific support ticket.

Using selective syncing versus filtering

There are two ways you can add a HyperDB data set to a table: selective sync and filtering.  Selective sync is the recommended way to add HyperDB data sets to a table, but here are examples of when each would be relevant.

Selective sync

Filtering

Best for

Performance - Only add selected records to a base to prevent bogging down your base’s performance with too many (unneeded) records.

Completeness - Having all of the data in the table. This may help for referencing information, but can also slow down base performance significantly.

Linked record relationships

Linking a data set to other tables in the base is important.

I don’t need linked records, I just need to see all the records.

Updates

Data will get dynamically updated from the HyperDB data set.

Data will get dynamically updated from the HyperDB data set.

Upon creation

A new blank table will be created. As records are linked to the HyperDB data set, they will populate in this new table.

A new table will be created with all of the filtered records.

General example

I want to bring in only information about “Ratatouille”, “Cars”, and “Up” from my “Movies” dataset.

I want to bring in all the Pixar movies information between 2000-2010 from my “Movies” dataset.

FAQs

What fields can I search by using the linked record picker?

HyperDB requires that each record have a unique primary key. Because that primary key is often not easy to read, we’ve introduced the concept of a “search field” for HyperDB. This is configured by the admin and published as part of the data set.

An image showing a data set with various customers' information.

If the search field is visible in the linked record picker (the field cannot be hidden), the linked record picker will allow you to search by either the primary field or the search field.

An image showing several customers named "Emily" and their information from the backend hyperDB data set.

As records are added with the linked record picker, they’ll be added to the selectively synced table.

How can I use automations to help with linking?

You can use an automation to link to any record in a selectively synced table as long as you know the unique ID for the record. You can do so by triggering an Update record action in the local table that links to the selectively synced dataset. In this way, if you update a specific linked record field with the unique ID from the larger dataset, then it will get synced into the base automatically.An image showing the configuration of an update record action in an automation where a linked record field is connected via a dynamic token.

Can I edit records in a selectively synced table?

No, records can not be altered from how they appear in the original data set.

Will I have to follow base limits using filtering?

Yes, before a data set can be added into a base, it must be filtered down to our records limit of 250,000 records.  

Can I search through any column in the linked record picker using selective sync?

No, you’ll be able to search through two columns (primary key and an additional search key) in order to find records.

Will I be able to preview the larger data set a selective sync pulls from?

You can preview the data set using the linked record picker to scroll through records, otherwise, there is no way to “preview” the entire data set.