Skip to content

coredump with RTX 4090 #33

@cunfate

Description

@cunfate

OS:

PRETTY_NAME="Ubuntu 22.04.5 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.5 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

Kernel:

Linux 5.15.0-136-generic

Driver:

nvidia-smi
Wed Nov 19 10:49:58 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 570.153.02             Driver Version: 570.153.02     CUDA Version: 12.8     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 4090        On  |   00000000:67:00.0 Off |                  Off |
| 30%   34C    P8             22W /  450W |       1MiB /  24564MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
|   1  NVIDIA GeForce RTX 4090        On  |   00000000:68:00.0 Off |                  Off |
| 30%   36C    P8             29W /  450W |       1MiB /  24564MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
|   2  NVIDIA GeForce RTX 4090        On  |   00000000:6C:00.0 Off |                  Off |
| 30%   35C    P8             31W /  450W |       1MiB /  24564MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
|   3  NVIDIA GeForce RTX 4090        On  |   00000000:6D:00.0 Off |                  Off |
| 30%   35C    P8             26W /  450W |       1MiB /  24564MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
|   4  NVIDIA GeForce RTX 4090        On  |   00000000:E5:00.0 Off |                  Off |
| 30%   39C    P8             29W /  450W |       1MiB /  24564MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
|   5  NVIDIA GeForce RTX 4090        On  |   00000000:E6:00.0 Off |                  Off |
| 31%   36C    P8             27W /  450W |       1MiB /  24564MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
|   6  NVIDIA GeForce RTX 4090        On  |   00000000:E7:00.0 Off |                  Off |
| 30%   35C    P8             23W /  450W |       1MiB /  24564MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
|   7  NVIDIA GeForce RTX 4090        On  |   00000000:E8:00.0 Off |                  Off |
| 30%   36C    P8             21W /  450W |       1MiB /  24564MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+

coredump with command python legged_gym/scripts/train.py --task g1_ground --run_name test_g1 --headless

(cuda-gdb) target cudacore core_1763432930_i-57e8ydfp6v26xx7w2h682507e_3391243.nvcudmp
Opening GPU coredump: core_1763432930_i-57e8ydfp6v26xx7w2h682507e_3391243.nvcudmp

CUDA Exception: Warp Illegal Address
The exception was triggered at PC 0x7fdcf8dc3920
[Current focus set to CUDA kernel 0, grid 25714637, block (46,1,0), thread (0,1,0), device 0, sm 93, warp 1, lane 0]
#0  0x00007fdcf8dc3960 in artiPropagateVelocityTGS2<<<(64,2,1),(32,2,1)>>> ()
(cuda-gdb) bt
#0  0x00007fdcf8dc3960 in artiPropagateVelocityTGS2<<<(64,2,1),(32,2,1)>>> ()
(cuda-gdb)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions