Redo #6889 and fix the race issue, the fix is included in the 9a68691 commit

Test:

1. Manual test passed:

$ sky launch -c c1 -y --cpus 2+ --cloud kubernetes
$ python test.py
$ cat test.py
from sky.client import sdk
import threading

threads = []
for i in range(30):
    thread = threading.Thread(target=sdk.tail_logs, args=('c1', None, True), daemon=True)
    threads.append(thread)
    thread.start()

for thread in threads:
    thread.join()

2. Unit test case:

   skypilot/tests/unit_tests/test_sky/server/requests/test_requests.py

   Line 486 in 9a68691

   async def test_get_request_async_race_condition(isolated_database):

Tested (run the relevant ones):

• Code formatting: install pre-commit (auto-check on commit) or bash format.sh
• Any manual or new tests for this PR (please specify below)
• All smoke tests: /smoke-test (CI) or pytest tests/test_smoke.py (local)
• Relevant individual tests: /smoke-test -k test_name (CI) or pytest tests/test_smoke.py::test_name (local)
• Backward compatibility: /quicktest-core (CI) or pytest tests/smoke_tests/test_backward_compat.py (local)