Fixes #7159.

The new managed jobs controller code uses the local API server in consolidation mode, but if you are using SSO or basic auth, it somehow needs to authenticate, even though it's running on the same host, just in different processes.

This PR fixes this by bypassing the oauth2 and basic auth middleware, if consolidation mode is enabled AND the request comes from a loopback address. We also handle the case where there is a proxy running in the same host (in which case the request will come from a loopback address), by checking for common headers added by proxies such as X-Forwarded-For, Forwarded, etc.

We also considered having the API server open a Unix domain socket, and similarly have requests to the socket be treated as internal (no auth needed), however it doesn't seem like there is a good way to do this in uvicorn while being able to share worker processes with the existing server.

Tested (run the relevant ones):

• Code formatting: install pre-commit (auto-check on commit) or bash format.sh
• Any manual or new tests for this PR (please specify below)
  • New smoke test test_loopback_access_with_basic_auth
  • A few unit tests
  • Tested locally using a local oauth2 proxy and a local API server with SKYPILOT_AUTH_OAUTH2_PROXY_ENABLED=true
  • Tested manually on a remote API server with okta oauth login
• All smoke tests: /smoke-test (CI) or pytest tests/test_smoke.py (local)
• Relevant individual tests: /smoke-test -k test_name (CI) or pytest tests/test_smoke.py::test_name (local)
• Backward compatibility: /quicktest-core (CI) or pytest tests/smoke_tests/test_backward_compat.py (local)