Normally, we store the sky config yaml on disk, in ~/.sky/config.yaml. But when using Postgres, we store the sky config yaml in the DB (config_yaml table) instead.

When profiling, I noticed that we are wasting 1 RTT to the DB to try creating the skypilot_config table on the DB. We do this for every request, as this is done in reload_config().

Instead, we should just be making sure that all the tables are up to date once when the server starts. As part of this, we should bring back alembic for sky config, as it makes things easier to manage too. We had tried moving skypilot config table to use alembic, but reverted it (#6320) as we encountered some race conditions.

This PR brings back alembic for skypilot config. There shouldn't be any race conditions this time, as we are only initializing the db once, at module import (and thus at server startup too) time, not lazily inside another function.

Benchmarked with multitime -n 50 sky status:

• Local API server
• GCP Cloud SQL Postgres, in us-west1 (oregon)
• 2k clusters, 100 users

Before:

            Mean        Std.Dev.    Min         Median      Max
real        4.958       0.152       4.658       4.916       5.327

After:

            Mean        Std.Dev.    Min         Median      Max
real        4.450       0.251       4.148       4.409       5.479

TODO: try to make a smoke test with postgres + some config editing (?)

Tested (run the relevant ones):

• Code formatting: install pre-commit (auto-check on commit) or bash format.sh
• Any manual or new tests for this PR (please specify below)
• All smoke tests: /smoke-test (CI) or pytest tests/test_smoke.py (local)
• Relevant individual tests: /smoke-test -k test_name (CI) or pytest tests/test_smoke.py::test_name (local)
• Backward compatibility: /quicktest-core (CI) or pytest tests/smoke_tests/test_backward_compat.py (local)