This application runs at start up for Airbyte. It is responsible for making sure that the environment is upgraded and in a good state. e.g. It makes sure the database has been migrated to the correct version.
The given Java code represents a class called
Bootloader, which is responsible for performing various bootstrapping operations for the Airbyte environment. Let’s go through the code step by step:
- The class is annotated with
@Slf4j, indicating that it is a Micronaut component and uses the Simple Logging Facade for Java (SLF4J) for logging.
load()method is the main entry point of the class and performs the following bootstrapping operations:
Initializes the databases.
Airbyte has two main databases the config and the jobs database which have several tables. We’re talk more about them in next session.
Checks migration compatibility.
The previous breaking version was 0.32.0-alpha described in our docs
Checks protocol version compatibility.
Checks the compatibility of the protocol version used by Airbyte connectors. It validates the protocol version constraints, and if there are any issues, it throws an exception.
The Airbyte Protocol currently all connectors are in the latest version of the protocol.
The minimal version configured today is
0.0.0(first release) and maximum is
0.3.0. This check make sure whatever you upgrade the connector follows the protocol Airbyte uses so it won’t break after you upgrade.
Runs database migrations.
Checks if upgrading from the current Airbyte version to a future version is a legal upgrade. It reads the initial Airbyte database version, compares it with the future version, and throws an exception if the upgrade is not allowed.
Airbyte uses Flyway to make the migrations whatever there are changes in the
airbyte-db. Flyway make the migration easy using code and testable SQL changes.
Each migration is versioned in Github you check all migrations since v0.30.22 here
Creates a default workspace if none exists.
This will be your user unique id needed to make API calls using the Configuration API
The Workspace ID it’s an random UUID.
Creates a default deployment if none exists.
Any time Airbyte is started up with new volumes or persistence, it will be assigned a new deployment id. This is different from the lifecycle of the rest of the data layer which may be persisted across deployments
Sets the Airbyte version.
.envfile and update the
Executes post-load tasks.
it’s used to release new features, currently there isn’t any
Logs the completion of bootstrapping.
Wooho it’s done!
That’s a high-level overview of the given Java code. It represents a bootloader component responsible for initializing and configuring various aspects of the Airbyte environment during startup.
- airbyte-bootloader Github code
- The table
airbyte_jobs_migrationsstores all Flyway migration history and additional info
- The table
airbyte_metadatakeep history of each version of Airbyte and when it was upgraded. Almost the same as the
airbyte_jobs_migrationsbut in higher level.