diff --git a/docs/docs/features/ml-hardware-acceleration.md b/docs/docs/features/ml-hardware-acceleration.md index bd4fe49e96..5ad0bcd11f 100644 --- a/docs/docs/features/ml-hardware-acceleration.md +++ b/docs/docs/features/ml-hardware-acceleration.md @@ -47,6 +47,7 @@ You do not need to redo any machine learning jobs after enabling hardware accele #### ROCm +- On Linux, The [AMDGPU driver module](https://rocm.docs.amd.com/projects/install-on-linux/en/latest/how-to/docker.html) needs to be installed on the server and, if secure boot is used, the signing key of DKMS [needs to be enrolled in UEFI BIOS](https://wiki.debian.org/SecureBoot) - The GPU must be supported by ROCm. If it isn't officially supported, you can attempt to use the `HSA_OVERRIDE_GFX_VERSION` environmental variable: `HSA_OVERRIDE_GFX_VERSION=`. If this doesn't work, you might need to also set `HSA_USE_SVM=0`. - The ROCm image is quite large and requires at least 35GiB of free disk space. However, pulling later updates to the service through Docker will generally only amount to a few hundred megabytes as the rest will be cached. - This backend is new and may experience some issues. For example, GPU power consumption can be higher than usual after running inference, even if the machine learning service is idle. In this case, it will only go back to normal after being idle for 5 minutes (configurable with the [MACHINE_LEARNING_MODEL_TTL](/install/environment-variables) setting).