Track Metadata in the Cloud
Use a MySQL database to track your metadata non-locally
Track your Metadata in the Cloud
While the local SQLite-based Metadata Store is a great default to get you started quickly, you will eventually need to move towards a deployed, shared and scalable database, for example once you switch to remote orchestration. This database will ideally be accessible both from your local machine, your orchestrator and the individual worker nodes of your orchestrator. A deployed MySQL Database can tick all these boxes.
Registering the MySQL Metadata Store
For the purpose of these docs it is assumed that the MySQL database is set up already. In order to connect to your MySQL instance there are a few parameters that you will need to register a Metadata Store stack component.
In all cases you will need to set these three fields:
host - The host IP address is the public IP address of your MySQL instance
port - The port at which to reach the MySQL instance (default is 3306)
database - The name of the database that will be used
For authentication, you'll have the choice of setting username and password directly to a stack component (not recommended for production setting), or you'll be able to register a secret in your secrets manager that allows you to also supply SSL certificates for more secure connections.
This is not recommended for production settings as the password is easily accessible on your machine in clear text and communication with the database is unencrypted.
Last updated