Purge everything first, then install latest driver from distro repo, last the toolkit from nvidia repo.
This is why cuda-12 doesn't work with podman 3.4.4 on ubuntu 22.04 I think:
- Rootless configuration for nvidia container runtime
- Setup missing hook for nvidia container runtime
- Increase memlock and stack ulimits