-
Notifications
You must be signed in to change notification settings - Fork 920
[logs] show a better error when server logs are not available #6855
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
| if log_path == constants.API_SERVER_LOGS: | ||
| resolved_log_path = pathlib.Path( | ||
| constants.API_SERVER_LOGS).expanduser() | ||
| if not resolved_log_path.exists(): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would it be possible for us to | tee the output to the api srever log path in our helm's sky api start, so people can always get the server log with sky api logs --server-logs?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not easily. Currently we use exec so that the main process in the container is the skypilot API server process. To use tee we would not be able to use exec, but that could cause some issues with tini.
| # 3. Exec ensures the process is a direct child of tini, enables correct signal handling. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see, I was just a bit worried that people may not have k8s credential locally if they are using a SkyPilot client, but better to have the message printed.
For the message itself, a quick edit:
'Server log file does not exist. The API server may '
'have been started with `--foreground` - check '
'the stdout of API server process, such as: kubectl logs -n api-server-namespace api-server-pod-name'
|
/smoke-test --remote-server |
Michaelvll
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @cg505! Left a minor comment.
| if log_path == constants.API_SERVER_LOGS: | ||
| resolved_log_path = pathlib.Path( | ||
| constants.API_SERVER_LOGS).expanduser() | ||
| if not resolved_log_path.exists(): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see, I was just a bit worried that people may not have k8s credential locally if they are using a SkyPilot client, but better to have the message printed.
For the message itself, a quick edit:
'Server log file does not exist. The API server may '
'have been started with `--foreground` - check '
'the stdout of API server process, such as: kubectl logs -n api-server-namespace api-server-pod-name'
If you start the API server with
sky api start --foreground, as in our Helm chart,sky api logs --server-logswill not work (TODO: make it work).Before this change,
sky api logs --server-logswould silently return nothing. Now, it should return a somewhat more helpful error message.Tested (run the relevant ones):
bash format.sh/smoke-test(CI) orpytest tests/test_smoke.py(local)/smoke-test -k test_name(CI) orpytest tests/test_smoke.py::test_name(local)/quicktest-core(CI) orpytest tests/smoke_tests/test_backward_compat.py(local)